An 816B open-weight multimodal model runs on day one: SGLang and Miles wire up inference and RL backends together

An open-weight model can get inference support on launch day across several accelerators, covering NVIDIA Blackwell and H200, AMD MI350X and MI355X, and Intel Xeon 6 with P-core. Support for models of this size has typically lagged the weight release, leaving deployers to wait for backend adaptation. On eight B300 GPUs with TP8/EP8 in an OpenHands multi-turn agentic coding replay, the NVFP4 checkpoint reaches 14.8K tokens/s per GPU at 543 tokens/s per user at concurrency 1, and 80.3K tokens/s per GPU at 61 tokens/s per user at concurrency 128.

Long-prompt prefill latency shrinks by splitting activations along the token dimension across tensor-parallel ranks, so pooling, unpooling and normalization each handle only their own share of tokens. With tensor parallelism alone, the pooler, unpooler and boundary layer normalization process the same full token batch on every rank. Across seven concurrency levels from 1 to 64 on four GB300 GPUs with TP4/EP4 and 16K prefill chunks, sequence parallelism plus NVFP4 MegaMoE cuts p90 time to first token by 28.3% to 41.9%, from 29.03 s to 19.47 s at concurrency 64.

The training side supports full-parameter updates and keeps multi-turn rollout token sequences identical to what the model actually sampled. Re-rendering the chat template during multi-turn training can alter the existing prefix, making the training sequence diverge from the original trajectory. Full-language-model GRPO training on DAPO-Math-17K on 64 H200 and 32 B300 GPUs gives a train-rollout log-probability difference of 0.0044 to 0.0071 with rollout routing replay, against 0.015 to 0.020 without it.

The report covers Muse Spark Open support in SGLang and Miles: inference paths on NVIDIA Blackwell and H200, AMD MI350X and MI355X, and Intel Xeon 6 with P-core, plus full-parameter reinforcement-learning training through Megatron-Core. Performance numbers come from OpenHands and AgentX replays on eight B300 GPUs, a prefill sweep on four GB300 GPUs, and training runs on 64 H200 and 32 B300 GPUs. TPU recipes and access to SGLang-TorchTPU are described as coming soon.

Speculative decoding acceptance length in the replays is a fixed simulated value: 4.5 for the OpenHands sweep and 4.16 for the AgentX sweep, the latter measured on 80 real coding first-turn prompts, so decode cost reflects that acceptance length rather than the draft model's measured quality. AgentX keeps the token counts and prefix structure of recorded traces but generates synthetic prompt text. Some reference values in the accuracy table come from the model card and published SGLang test results.