NVIDIA's open multimodal model lands on SGLang day one: 120B parameters, 12B active, 243.9 tok/s on one GPU
Synopsis
NVIDIA released Nemotron 3.5 Super VL, an open multimodal model that takes text, images, documents and video and returns text for always-on agents that read screens, documents and tools. SGLang supports deployment from day one, serving it with continuous batching, prefix caching and speculative decoding. The mixture-of-experts model has 120 billion total parameters and activates 12 billion per token, with built-in multi-token prediction; it reaches 243.9 tok/s on a single RTX 6000 PRO.
Interpretation
Always-on AI agents now have an open model built specifically for multimodal execution, handling images, documents and video while emitting text an agent can act on. Such agent workflows previously leaned on general models to handle visual understanding and text generation separately, whereas Nemotron 3.5 Super VL is positioned as the high-accuracy multimodal execution model in the Nemotron 3 family, alongside Nemotron 3.5 Lightning and Nemotron 3 Ultra. Weights are downloadable from Hugging Face in BF16 and NVFP4 formats, and SGLang ships a matching deployment path.
The serving problem is fitting a large model into an affordable memory and throughput budget by running only a slice of its parameters per step. The mixture-of-experts design gives the model 120 billion total parameters while activating 12 billion per token, keeping per-token compute down without shrinking model capacity. SGLang carries the model with continuous batching, prefix caching, speculative decoding and an OpenAI-compatible API, and provides a container image and launch command.
The model's built-in multi-token prediction heads supply draft tokens for speculative decoding, proposing several future tokens that are then verified during decoding. Plain token-by-token decoding accumulates latency on long agent tasks, while multi-token prediction proposes subsequent tokens in parallel at the decoding stage. On a single RTX 6000 PRO with SGLang's speculative decoding settings the model reaches 243.9 tok/s, and on a Baseten endpoint it exceeds 600 TPS on Artificial Analysis Intelligence Index agentic tasks.
Perspective
The report covers the model release and the SGLang deployment path: a 120-billion-parameter mixture-of-experts model activating 12 billion parameters per token, text, image, document and video input with text output, and optimizations including C-RADIOv4 vision encoding, a hybrid Mamba-Transformer architecture, multi-token prediction and NVFP4 quantization. Deployment targets include DGX Spark, 1x GB10, B200, B300, RTX PRO 6000, H200 and 2x H100, and the model runs locally on a single DGX Spark and RTX 6000 PRO. Domain adaptation is done by post-training with NVIDIA NeMo.
The throughput comparison uses other open models in its size category as the reference, without listing the comparison models or test conditions in the text. The 5x throughput and 4x faster task completion claims correspond to two figures whose measurement details require the original figures. Multi-token prediction is described as accelerating while preserving the model's output quality target, but the text does not quantify that target.