Skip to main content
Search

# Day 0 support for Muse Spark Open on vLLM

1 related articles

Subscribe to updates
  1. vLLM Blog

    vLLM runs an 816B sparse model on day one: 1M context at a quarter of the KV cache, over 960 tokens/sec on eight TPU v7 chips

    vLLM shipped inference support for Muse Spark Open on release day, a mixture-of-experts model with 816B total parameters and 42B active per token, handling a 1M context window plus native image, video and audio input. Such models usually wait weeks for community ports; here Nvidia and AMD images arrived together with a trained DSpark speculative draft model. On TPU with speculative decoding, four v7 chips exceed 650 tokens/sec and eight exceed 960 tokens/sec.

You’re all caught up