Meta open-sources Muse Spark Open: an 816B coding model that holds a whole project in 1M tokens

Coding models are being designed for sustained multistep work rather than single-shot completion. The model plans, acts, observes, and self-corrects, supporting multistep task execution with tool use, output evaluation, and iterative refinement. General coding models were optimized for single-shot completions such as generating a function, writing a unit test, or suggesting an inline fix, excelling at discrete, well-scoped tasks with small context and immediate answers. The model was trained on NVIDIA GB300 NVL72, and the release describes higher first-attempt accuracy, reliable tool calling, and deep codebase understanding across repositories.

Long sessions are sustained without breaking tasks into pieces, using goal conditioning to hold direction and context compaction to retain the knowledge needed to keep progressing. General coding models typically require splitting a task into pieces as context grows. The 1M token context window holds an entire project, including complex dependencies, legacy code, and thousands of files.

Multimodal perception is folded into a single reasoning pass instead of separate pipelines per modality. A 1.8B perception encoder natively processes interleaved text, images, video frames, and audio. The common approach maintains a separate pipeline for each modality. The design targets developer workflows spanning screenshots, diagrams, error logs, and video walkthroughs within the same agent session.

The report covers Meta's release of Muse Spark Open and its training and deployment configuration on NVIDIA GB300 NVL72, aimed at developers building coding agents that need long context and multistep tool calling. The stated deployment paths include downloading NVFP4 weights from Hugging Face, using the NVIDIA NIM container, and configuring serving directly through SGLang, vLLM, or TensorRT LLM.

The material gives no head-to-head benchmark numbers against other models; higher first-attempt accuracy, reliable tool calling, and throughput gains are described qualitatively. Real sparse-MoE throughput depends on the 130 TB/s communication bandwidth of the 72-GPU NVLink domain, which the material says determines how much of the theoretical throughput a deployment realizes.