The Frameworks that Google, DeepMind, Microsoft and Uber Use to Train Deep Learning Models at Scale by @jrdothoughts via @Medium

GPipe, Horovod, TF-Replicator and DeepSpeed combine cutting edge aspects of deep learning research and infrastructure to scale the training of deep learning models.

I found this fascinating.  I really hadn't quite connected all the dots in my mind to connect the frameworks up like this.