PyTorch 2.15 on the Mac: MoE decoding 5–6.5× faster and video VAE memory down from 270 GiB to 8 GiB
Synopsis
PyTorch 2.15 brings FP8 tensors, a native grouped matmul and a batch of dedicated Metal kernels to the Mac's MPS backend, making local LLM decoding and video generation faster and lighter on memory. Previously many MPS operators went through MPSGraph, MoE decoding copied group offsets to the CPU and looped over groups, and tiled video VAE decoding pushed driver memory to 270 GiB. On an M3 Ultra, Qwen3.6-35B-A3B decoding rises from 9.4 to 61.3 tokens/s, and Wan2.2 VAE decoding peaks near 8 GiB.
Interpretation
PyTorch models on the Mac gain FP8 storage, faster language-model decoding and leaner video decoding in 2.15, as MPS moves the most expensive decoding and generation steps onto dedicated Metal kernels. Previously even small models on MPS often required working around unsupported operations, performance cliffs and unexpected memory growth; F.linear still went through MPSGraph, and grouped matmul copied group offsets to the CPU and looped over groups. On an M3 Ultra with torch.compile, Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s and LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s; Wan2.2 tiled VAE decoding drops its peak driver memory from about 270 GiB to 8 GiB.
During decoding each new token passes through the model as a single row, so large matrix multiplications become matrix-vector products; 2.15 connects existing GEMV kernels to F.linear and adds a native grouped matmul for mixture-of-experts layers. Grouped matmul in 2.14 copied group offsets to the CPU and looped over groups, while the new Metal kernels read offsets directly on the GPU and give small groups their own kernel. Across 96 BF16 layer shapes from real models, F.linear reaches a 1.23× geometric-mean speedup, and FP32 softmax over a [1, 128256] row of logits falls from 57 µs to 7 µs.
Perspective
The release targets PyTorch users running the MPS backend on Apple silicon Macs, covering language-model decoding, linear algebra, convolutions and padding, softmax and reductions, MPS streams and several new operators. MPS now requires macOS 15 or later, and PyTorch 2.15 requires Python 3.11 or later. The release notes say 2.16 will run attention entirely on Metal and continue moving operators off MPSGraph.
The release notes state that some 2.16 changes have landed in nightlies while others are still in review, so details may change before release. logspace does not yet support integer or boolean outputs with more than one element. The speedup numbers come from the specified hardware and torch.compile settings, and behavior on other configurations is not reported.