# Last-Loop KV Caching

Dory1.0-4B is a 4-billion-parameter looped language model whose single checkpoint varies how many times a shared recurrent block runs at inference, letting one model serve different compute budgets. It was pretrained on 6 trillion tokens, the first 4.5 trillion in NVFP4, then extended to a 256K context and post-trained with SFT and RLVR. On the equal-weight average of 12 benchmarks it rises from 35.2 at one loop to 39.7 at two and 40.6 at four, and with two or more loops it beats Qwen3.5-9B, Gemma 4 12B and Gemma 4 26B-A4B on that average.

Dory1.0-4B is a 4-billion-parameter looped language model whose single checkpoint varies how many times a shared recurrent block runs at inference, letting one model serve different compute budgets. It was pretrained on 6 trillion tokens, the first 4.5 trillion in NVFP4, then extended to a 256K context and post-trained with SFT and RLVR. On the equal-weight average of 12 benchmarks it rises from 35.2 at one loop to 39.7 at two and 40.6 at four, and with two or more loops it beats Qwen3.5-9B, Gemma 4 12B and Gemma 4 26B-A4B on that average.