Training a 1T-Parameter MoE on EU public infrastructure with PyTorch and TorchTitan

A 1-trillion-parameter Mixture-of-Experts (MoE) model was successfully trained on public infrastructure in the European Union using PyTorch and the TorchTitan framework. This achievement overcomes prior limitations related to architectural complexity and interconnect bottlenecks that made such scaling on shared resources seem unfeasible.