# NVFP4

Thinking Machines' Inkling 2 Small gained vLLM support on its release day. It is a text-and-image-in, text-out mixture-of-experts model with 594B total and 17B active parameters, up to 1M tokens of context, and a bundled DSpark speculative-decoding drafter. vLLM fits it with kernel fusion, programmatic dependent launch, sequence parallelism and a Gluon attention kernel rewritten for the 6:1 GQA ratio, reaching about 630 tok/s for a single user at TP4 on GB200 with acceptance length 3.5, and also supports LoRA and reinforcement learning via SkyRL.

Thinking Machines' Inkling 2 Small gained vLLM support on its release day. It is a text-and-image-in, text-out mixture-of-experts model with 594B total and 17B active parameters, up to 1M tokens of context, and a bundled DSpark speculative-decoding drafter. vLLM fits it with kernel fusion, programmatic dependent launch, sequence parallelism and a Gluon attention kernel rewritten for the 6:1 GQA ratio, reaching about 630 tok/s for a single user at TP4 on GB200 with acceptance length 3.5, and also supports LoRA and reinforcement learning via SkyRL.

SGLang adds NVFP4 storage to its paged KV cache, cutting per-token key-value bytes to about 56% of FP8. On Qwen3.8-27B at 32K, 160K and 1M input lengths with matched decode concurrency, peak-batch decode throughput rises about 26–30%; on agentic load beyond concurrency 12, FP8 throughput collapses while NVFP4 keeps climbing. Accuracy gaps vary by task, reaching 1.60 points on SWE-bench Verified.

SGLang adds NVFP4 storage to its paged KV cache, cutting per-token key-value bytes to about 56% of FP8. On Qwen3.8-27B at 32K, 160K and 1M input lengths with matched decode concurrency, peak-batch decode throughput rises about 26–30%; on agentic load beyond concurrency 12, FP8 throughput collapses while NVFP4 keeps climbing. Accuracy gaps vary by task, reaching 1.60 points on SWE-bench Verified.

MiniMax open-sourced MiniMax-M3.1-Flash, and vLLM supports it on day one on NVIDIA Blackwell, NVIDIA Rubin and AMD MI350-series GPUs. The model is quantization-aware trained for an NVFP4 KV cache and stores its routed MoE weights in NVFP4, shrinking the checkpoint from 451.5 GB to 250.1 GB and the KV cache from 61,440 to 34,560 bytes per token. On four NVIDIA B300 GPUs with DEP4 deployment and 256K-token sequences, M3.1 fits 76 concurrent sequences where M3 with an FP8 KV cache fits 28.

MiniMax open-sourced MiniMax-M3.1-Flash, and vLLM supports it on day one on NVIDIA Blackwell, NVIDIA Rubin and AMD MI350-series GPUs. The model is quantization-aware trained for an NVFP4 KV cache and stores its routed MoE weights in NVFP4, shrinking the checkpoint from 451.5 GB to 250.1 GB and the KV cache from 61,440 to 34,560 bytes per token. On four NVIDIA B300 GPUs with DEP4 deployment and 256K-token sequences, M3.1 fits 76 concurrent sequences where M3 with an FP8 KV cache fits 28.

NVIDIA released Nemotron 3.5 Super VL, an open, customizable multimodal model for long enterprise agent workflows that need reasoning, tool use, and text, image, document, and video understanding. It uses a 120B-parameter mixture-of-experts architecture that activates 12B parameters per token, and NVIDIA reports up to 5x higher throughput on agentic tasks than other open models in its size class on a single GPU. Teams can post-train it with NeMo on their own data, tools, and workflows, and NVIDIA also released agentic tool-use, finance reasoning, and 40 vision-language datasets.

NVIDIA released Nemotron 3.5 Super VL, an open, customizable multimodal model for long enterprise agent workflows that need reasoning, tool use, and text, image, document, and video understanding. It uses a 120B-parameter mixture-of-experts architecture that activates 12B parameters per token, and NVIDIA reports up to 5x higher throughput on agentic tasks than other open models in its size class on a single GPU. Teams can post-train it with NeMo on their own data, tools, and workflows, and NVIDIA also released agentic tool-use, finance reasoning, and 40 vision-language datasets.

Meta released the weights of Muse Spark Open, an 816B-parameter omnimodal mixture-of-experts model with 42.7B active at inference, co-trained with Muse Code for coding-agent workflows. A compressed MoE block and a hybrid global/local attention schedule cut inter-GPU traffic and long-session KV cache memory, and the model supports a 1M-token context with text, image, video, and audio input. On Meta-reported benchmarks it is competitive with frontier-lab models on agentic coding and knowledge work, though each model was tested with its own lab's coding agent rather than a shared harness.

Meta released the weights of Muse Spark Open, an 816B-parameter omnimodal mixture-of-experts model with 42.7B active at inference, co-trained with Muse Code for coding-agent workflows. A compressed MoE block and a hybrid global/local attention schedule cut inter-GPU traffic and long-session KV cache memory, and the model supports a 1M-token context with text, image, video, and audio input. On Meta-reported benchmarks it is competitive with frontier-lab models on agentic coding and knowledge work, though each model was tested with its own lab's coding agent rather than a shared harness.