MiniMax-M3.1-Flash lands in vLLM on day one: NVFP4 cache lifts 256K-token concurrency from 28 to 76 sequences per deployment

MiniMax-M3.1-Flash treats inference efficiency as an architectural target rather than a deployment-side tuning exercise: both the KV cache and the routed expert weights are stored in the low-precision NVFP4 format. The previous MiniMax-M3 stored expert weights in MXFP8 and kept its KV cache in FP8, so both the checkpoint and the per-token cache footprint were larger. The checkpoint shrinks from 451.5 GB (MiniMax-M3-MXFP8) to 250.1 GB, 44.6% smaller, and the NVFP4 KV cache needs 34,560 bytes per token instead of 61,440 bytes in FP8, a 43.75% saving.

On the same hardware, the smaller cache converts directly into higher concurrent serving capacity. M3 with an FP8 KV cache fits only 28 sequences in the same configuration. On four NVIDIA B300 GPUs with DEP4 deployment and 256K-token sequences, M3.1 fits 76 concurrent sequences, about 2.7 times as many as M3.

vLLM rewrote the key sparse-attention kernels for the NVFP4 cache so the low-precision format actually pays off during decode. The KV cache is NVFP4 while the query stays FP8, so native NVFP4 tensor core instructions cannot be used and the cache must first be dequantized to FP8 for the matrix multiply. The Blackwell kernels use the QMUL4 instruction new in CUDA 13.4 to scale four NVFP4 values into four FP8 values at once, giving up to 1.7x faster indexer QK decode and 1.3x faster sparse attention decode.

End-to-end reinforcement learning on the NVFP4 checkpoint works on day one with veRL, training only rank-32 LoRA adapters. The trainer quantizes the base model the same way the engine serves it and syncs only the adapters after each step instead of the full weights. On DAPO-Math-17K with 16 GB300 GPUs and 200 GRPO steps, AIME-2024 accuracy rises from 36.3% to 67.5% and training reward from -0.06 to 0.67.

The report covers vLLM's day-one support for MiniMax-M3.1-Flash on NVIDIA Blackwell, NVIDIA Rubin and AMD MI350-series GPUs, and every performance figure comes from the specific deployment configurations and benchmark settings stated in the source. Prebuilt images are available for both vendors, while the PyPI nightly wheel becomes available only after the code is reviewed and merged upstream.

The reported speedups each correspond to a specific kernel, GPU and context length, for example the 3.3x indexer DCP figure measured at 512K context on 4x GB300. The AMD indexer DCP figure of 2.8x comes from 45 requests of 30K-210K tokens on 4x MI355X. The reinforcement-learning result comes only from a 200-step GRPO run on DAPO-Math-17K.