# Sparse GQA

Step 5 Preview treats agentic workloads as the starting point for model design: it uses a narrow-but-deep stack of 92 transformer blocks, interleaves efficient and sparse full attention at 3:1, and extends the full-attention path to contexts up to 1M tokens with Sparse GQA. Its total parameter count is under one-quarter and its active count under one-third of trillion-scale models such as Qwen3.8-Max and Kimi K3, and it reports long-horizon agentic capability comparable to those counterparts. Agent-native data synthesis and long-rollout RL infrastructure accompany the model, yielding million-level verifiable environments.

Step 5 Preview treats agentic workloads as the starting point for model design: it uses a narrow-but-deep stack of 92 transformer blocks, interleaves efficient and sparse full attention at 3:1, and extends the full-attention path to contexts up to 1M tokens with Sparse GQA. Its total parameter count is under one-quarter and its active count under one-third of trillion-scale models such as Qwen3.8-Max and Kimi K3, and it reports long-horizon agentic capability comparable to those counterparts. Agent-native data synthesis and long-rollout RL infrastructure accompany the model, yielding million-level verifiable environments.