vLLM

Reflection's Beam is a 501B-parameter mixture-of-experts model with 23B active parameters per token, and vLLM supports it from day one with reasoning, tool calling and structured outputs working out of the box. Its first implementation was almost pure PyTorch; coding agents profiled the model, wrote kernels and benchmarked each change, cutting batch-1 decode on TP8 from 8.95 to 3.30 milliseconds per step and an 8×2K prefill from 328 to 176 milliseconds. With its DFlash2 draft head, a single request reaches 708 tokens per second on TP8 at an acceptance length of 3.7.

Reflection's Beam is a 501B-parameter mixture-of-experts model with 23B active parameters per token, and vLLM supports it from day one with reasoning, tool calling and structured outputs working out of the box. Its first implementation was almost pure PyTorch; coding agents profiled the model, wrote kernels and benchmarked each change, cutting batch-1 decode on TP8 from 8.95 to 3.30 milliseconds per step and an 8×2K prefill from 328 to 176 milliseconds. With its DFlash2 draft head, a single request reaches 708 tokens per second on TP8 at an acceptance length of 3.7.

NVIDIA released Nemotron 3.5 Super VL, an open multimodal model for long-running agents that folds document, image, and video understanding together with reasoning, instruction following, and tool use in one model, reducing the need to deploy separate text and vision models. It uses a 120-billion-parameter mixture-of-experts design with 12 billion active parameters, combined with a hybrid Mamba-Transformer architecture, multi-token prediction, and NVFP4 quantization, and can be post-trained with NVIDIA NeMo on an organization's own data, tools, policies, and workflows. The stated comparison is up to 5x faster throughput on agentic tasks than other open models in its size category on a single GPU, and 4x faster task completion.

NVIDIA released Nemotron 3.5 Super VL, an open multimodal model for long-running agents that folds document, image, and video understanding together with reasoning, instruction following, and tool use in one model, reducing the need to deploy separate text and vision models. It uses a 120-billion-parameter mixture-of-experts design with 12 billion active parameters, combined with a hybrid Mamba-Transformer architecture, multi-token prediction, and NVFP4 quantization, and can be post-trained with NVIDIA NeMo on an organization's own data, tools, policies, and workflows. The stated comparison is up to 5x faster throughput on agentic tasks than other open models in its size category on a single GPU, and 4x faster task completion.

vLLM shipped inference support for Muse Spark Open on release day, a mixture-of-experts model with 816B total parameters and 42B active per token, handling a 1M context window plus native image, video and audio input. Such models usually wait weeks for community ports; here Nvidia and AMD images arrived together with a trained DSpark speculative draft model. On TPU with speculative decoding, four v7 chips exceed 650 tokens/sec and eight exceed 960 tokens/sec.

vLLM shipped inference support for Muse Spark Open on release day, a mixture-of-experts model with 816B total parameters and 42B active per token, handling a 1M context window plus native image, video and audio input. Such models usually wait weeks for community ports; here Nvidia and AMD images arrived together with a trained DSpark speculative draft model. On TPU with speculative decoding, four v7 chips exceed 650 tokens/sec and eight exceed 960 tokens/sec.