Deploying large models requires end-to-end toolchain collaboration spanning training, operators, inference, orchestration, and safe distribution. The PyTorch Foundation has established a two-tier matrix of "hosted projects" and "ecosystem projects," connecting over 100 projects that cover every stage of AI development. In this talk, a PyTorch Ambassador begins by explaining the foundation’s operating model and governance logic. Then, following the LLM development lifecycle, the talk introduces each hosted project under the PyTorch Foundation: PyTorch provides the core framework from eager mode to torch.compile; DeepSpeed handles distributed training and inference optimization at the scale of hundreds of billions of parameters; Helion allows writing high-performance operators in native PyTorch syntax with automatic tuning; vLLM achieves high-throughput inference via PagedAttention; Ray enables elastic orchestration and serving, smoothly scaling inference engines to production clusters; and Safetensors stores model weights in a secure, pickle-free format. The talk covers the entire PyTorch ecosystem landscape and the collaborative relationships among these projects, giving the audience a clear AI development roadmap.
PyTorch Ecosystem in LLM Era
KeynoteAbstract
Centered around the PyTorch ecosystem in the large-model era and starting with the fundamentals of PyTorch training, this presentation sequentially introduces how Helion accelerates low-level operators, how DeepSpeed powers distributed training, how Safetensors stores and distributes model weights, how vLLM boosts the throughput of large-model inference, and how Ray orchestrates distributed tasks. It sorts out the full-stack open-source infrastructure maintained by the PyTorch Foundation, spanning from model training to production systems.