PyTorch

At the PyTorch Conference North America in San Jose, the foundation's projects announced several efforts to run PyTorch natively on more hardware. TorchTPU, a Google and Meta collaboration, gives Google TPUs a native backend supporting eager execution and torch.compile; TorchNeuron for AWS Trainium shipped as an Apache 2.0 public beta that runs PyTorch workloads with a single configuration change. Workload migration can require only a device change, while peak performance can require additional changes to shapes, sharding and operations.

At the PyTorch Conference North America in San Jose, the foundation's projects announced several efforts to run PyTorch natively on more hardware. TorchTPU, a Google and Meta collaboration, gives Google TPUs a native backend supporting eager execution and torch.compile; TorchNeuron for AWS Trainium shipped as an Apache 2.0 public beta that runs PyTorch workloads with a single configuration change. Workload migration can require only a device change, while peak performance can require additional changes to shapes, sharding and operations.

PyTorch 2.15 makes the rewritten NCCL backend, previewed as opt-in nccl2 in 2.14, the default for every nccl process group, keeping TORCH_DIST_USE_NCCL2=0 and the nccl-legacy name as rollback paths. At cluster scale a single stalled GPU could halt a whole training run and the cause was often reproduced by hand; a backend-agnostic flight recorder now writes a post-mortem to disk when a collective hangs, and a timeout on any rank triggers dumps from every rank.

PyTorch 2.15 makes the rewritten NCCL backend, previewed as opt-in nccl2 in 2.14, the default for every nccl process group, keeping TORCH_DIST_USE_NCCL2=0 and the nccl-legacy name as rollback paths. At cluster scale a single stalled GPU could halt a whole training run and the cause was often reproduced by hand; a backend-agnostic flight recorder now writes a post-mortem to disk when a collective hangs, and a timeout on any rank triggers dumps from every rank.

PyTorch users on AMD MI350-series GPUs can now reach FlyDSL kernels through existing operator APIs for dense and grouped GEMM, MXFP8/MXFP4 scaled GEMM, RMSNorm, TopK, and attention via AOTriton. Those operators previously ran on ATen, Triton, or AITER ASM implementations. On MI355X operator suites, FlyDSL gives a 1.10x geometric-mean speedup over the faster ATen/Triton baseline at each of 15 BF16 dense-GEMM shapes, and 1.58x and 1.68x over ATen for MXFP8 and MXFP4. End-to-end vLLM whole-request speedups reach 1.13x for BF16, 2.04x for MXFP8, and 3.01x for MXFP4, depending on model and concurrency.

PyTorch users on AMD MI350-series GPUs can now reach FlyDSL kernels through existing operator APIs for dense and grouped GEMM, MXFP8/MXFP4 scaled GEMM, RMSNorm, TopK, and attention via AOTriton. Those operators previously ran on ATen, Triton, or AITER ASM implementations. On MI355X operator suites, FlyDSL gives a 1.10x geometric-mean speedup over the faster ATen/Triton baseline at each of 15 BF16 dense-GEMM shapes, and 1.58x and 1.68x over ATen for MXFP8 and MXFP4. End-to-end vLLM whole-request speedups reach 1.13x for BF16, 2.04x for MXFP8, and 3.01x for MXFP4, depending on model and concurrency.

TorchTitan's annual update adds PyTorch-native training support for open-weight models including Muse Spark Open, Qwen3.5, Kimi K3 and DeepSeek-V4, and introduces TitanRL, which keeps one model from pretraining through post-training to inside vLLM so training and inference no longer run two copies that can quietly disagree. In supported on-policy configurations the trainer and generator produce bitwise identical log probabilities, leaving exactly zero gap at every step. The update also reports DeepSeek V3 671B throughput on the MLPerf training benchmark and infrastructure work around FlexShard and torch_checkpointing.

TorchTitan's annual update adds PyTorch-native training support for open-weight models including Muse Spark Open, Qwen3.5, Kimi K3 and DeepSeek-V4, and introduces TitanRL, which keeps one model from pretraining through post-training to inside vLLM so training and inference no longer run two copies that can quietly disagree. In supported on-policy configurations the trainer and generator produce bitwise identical log probabilities, leaving exactly zero gap at every step. The update also reports DeepSeek V3 671B throughput on the MLPerf training benchmark and infrastructure work around FlexShard and torch_checkpointing.