vLLM runs an 816B sparse model on day one: 1M context at a quarter of the KV cache, over 960 tokens/sec on eight TPU v7 chips
vLLM shipped inference support for Muse Spark Open on release day, a mixture-of-experts model with 816B total parameters and 42B active per token, handling a 1M context window plus native image, video and audio input. Such models usually wait weeks for community ports; here Nvidia and AMD images arrived together with a trained DSpark speculative draft model. On TPU with speculative decoding, four v7 chips exceed 650 tokens/sec and eight exceed 960 tokens/sec.