# Multi-Token Prediction

Mellum2.1 is the next version of Mellum2, with the architecture unchanged: still a 12B mixture-of-experts model with 2.5B active parameters under Apache 2.0, with the changes concentrated after pre-training. Mellum2 was fast but could not work inside a repository at the level its developers wanted; this version, trained with reinforcement learning in real environments across thousands of environments and millions of sandboxed runs, can explore a codebase, edit files and check its own changes. Under one shared evaluation setup it is compared with Mellum2 and with the similar open models Qwen3.5-9B and Gemma 4 E4B, and the largest gain is in agentic coding, with improvements also reported in coding, competitive programming, math, tool calling and general knowledge.

Mellum2.1 is the next version of Mellum2, with the architecture unchanged: still a 12B mixture-of-experts model with 2.5B active parameters under Apache 2.0, with the changes concentrated after pre-training. Mellum2 was fast but could not work inside a repository at the level its developers wanted; this version, trained with reinforcement learning in real environments across thousands of environments and millions of sandboxed runs, can explore a codebase, edit files and check its own changes. Under one shared evaluation setup it is compared with Mellum2 and with the similar open models Qwen3.5-9B and Gemma 4 E4B, and the largest gain is in agentic coding, with improvements also reported in coding, competitive programming, math, tool calling and general knowledge.

NVIDIA released Nemotron 3.5 Super VL, an open multimodal model that takes text, images, documents and video and returns text for always-on agents that read screens, documents and tools. SGLang supports deployment from day one, serving it with continuous batching, prefix caching and speculative decoding. The mixture-of-experts model has 120 billion total parameters and activates 12 billion per token, with built-in multi-token prediction; it reaches 243.9 tok/s on a single RTX 6000 PRO.

NVIDIA released Nemotron 3.5 Super VL, an open multimodal model that takes text, images, documents and video and returns text for always-on agents that read screens, documents and tools. SGLang supports deployment from day one, serving it with continuous batching, prefix caching and speculative decoding. The mixture-of-experts model has 120 billion total parameters and activates 12 billion per token, with built-in multi-token prediction; it reaches 243.9 tok/s on a single RTX 6000 PRO.

NVIDIA released Nemotron 3.5 Super VL, an open multimodal model for long-running agents that folds document, image, and video understanding together with reasoning, instruction following, and tool use in one model, reducing the need to deploy separate text and vision models. It uses a 120-billion-parameter mixture-of-experts design with 12 billion active parameters, combined with a hybrid Mamba-Transformer architecture, multi-token prediction, and NVFP4 quantization, and can be post-trained with NVIDIA NeMo on an organization's own data, tools, policies, and workflows. The stated comparison is up to 5x faster throughput on agentic tasks than other open models in its size category on a single GPU, and 4x faster task completion.

NVIDIA released Nemotron 3.5 Super VL, an open multimodal model for long-running agents that folds document, image, and video understanding together with reasoning, instruction following, and tool use in one model, reducing the need to deploy separate text and vision models. It uses a 120-billion-parameter mixture-of-experts design with 12 billion active parameters, combined with a hybrid Mamba-Transformer architecture, multi-token prediction, and NVFP4 quantization, and can be post-trained with NVIDIA NeMo on an organization's own data, tools, policies, and workflows. The stated comparison is up to 5x faster throughput on agentic tasks than other open models in its size category on a single GPU, and 4x faster task completion.