# Hybrid global/local attention schedule

Meta released the weights of Muse Spark Open, an 816B-parameter omnimodal mixture-of-experts model with 42.7B active at inference, co-trained with Muse Code for coding-agent workflows. A compressed MoE block and a hybrid global/local attention schedule cut inter-GPU traffic and long-session KV cache memory, and the model supports a 1M-token context with text, image, video, and audio input. On Meta-reported benchmarks it is competitive with frontier-lab models on agentic coding and knowledge work, though each model was tested with its own lab's coding agent rather than a shared harness.

Meta released the weights of Muse Spark Open, an 816B-parameter omnimodal mixture-of-experts model with 42.7B active at inference, co-trained with Muse Code for coding-agent workflows. A compressed MoE block and a hybrid global/local attention schedule cut inter-GPU traffic and long-session KV cache memory, and the model supports a 1M-token context with text, image, video, and audio input. On Meta-reported benchmarks it is competitive with frontier-lab models on agentic coding and knowledge work, though each model was tested with its own lab's coding agent rather than a shared harness.