Vera Rubin NVL72's first agentic-serving results: up to about 5.7x GB300's throughput per megawatt

NVIDIA's rack-scale Vera Rubin NVL72 delivers higher total throughput per megawatt of measured GPU power than the previous-generation GB300 NVL72 on coding-agent workloads, and the lead widens as per-user output speed rises. AA-AgentPerf previously had no Vera Rubin results, with GB300 NVL72 as the prior-generation comparison system on this workload. On DeepSeek V4 Pro, Vera Rubin serves 132M tokens/s per MW at about 190 tok/s per user versus about 23M for GB300, roughly 5.7x; the lead is about 5.4x on DeepSeek V4.1 Flash at about 580 tok/s, about 3.0x on GLM-5.2 at about 165 tok/s and about 2.8x on Kimi K3 at about 67 tok/s.

The serving stack keeps each agent session's repeated context in cache, prefilling only the tokens each turn adds, and routes requests by session to the worker already holding that cache. A coding agent resends the whole conversation each turn, so context grows as it reads files, runs tests and makes edits, while replies are often just short tool calls. AA-AgentPerf replays real coding-agent trajectories with up to 132K input tokens per request and 97% of input tokens repeated on average; most Vera Rubin configs offload KV cache to CPU memory over NVLink-C2C, and the DeepSeek V4 Pro prefill workers have 120 to 200 GiB of host memory configured for KV cache.

The results cover serving configs for four models on the AA-AgentPerf replay workload, with throughput counted against measured GPU power and excluding CPUs, networking and cooling; per-user speed is the 25th percentile across requests and excludes time to first token.

These are first results on Vera Rubin with early Rubin-specific kernels and tuning, and NVIDIA expects performance to keep improving as further optimizations land; on DeepSeek V4.1 Flash, Vera Rubin's replies were capped at 2,048 output tokens, about 2% of replies reached the cap and the average reply was about a tenth shorter, modestly favoring its throughput; measured against end-to-end speed including time to first token, GB300 matches or beats Vera Rubin's throughput per MW at most speeds on DeepSeek V4 Pro, while on GLM-5.2 Vera Rubin still leads by 1.1x to 1.8x.