Big improvements in distributed training of #AI models.

Researchers at Google figured out how to lower bandwidth requirements by 100x and still produce a billion-parameter scale model: https://arxiv.org/abs/2501.18512v1