# Grouped matmul

PyTorch 2.15 brings FP8 tensors, a native grouped matmul and a batch of dedicated Metal kernels to the Mac's MPS backend, making local LLM decoding and video generation faster and lighter on memory. Previously many MPS operators went through MPSGraph, MoE decoding copied group offsets to the CPU and looped over groups, and tiled video VAE decoding pushed driver memory to 270 GiB. On an M3 Ultra, Qwen3.6-35B-A3B decoding rises from 9.4 to 61.3 tokens/s, and Wan2.2 VAE decoding peaks near 8 GiB.

PyTorch 2.15 brings FP8 tensors, a native grouped matmul and a batch of dedicated Metal kernels to the Mac's MPS backend, making local LLM decoding and video generation faster and lighter on memory. Previously many MPS operators went through MPSGraph, MoE decoding copied group offsets to the CPU and looped over groups, and tiled video VAE decoding pushed driver memory to 270 GiB. On an M3 Ultra, Qwen3.6-35B-A3B decoding rises from 9.4 to 61.3 tokens/s, and Wan2.2 VAE decoding peaks near 8 GiB.