Zhe Qian

The authors introduce LoopVL, extending looped Transformers to vision-language models: the language backbone is pretrained from scratch on HRM-Text's open-source framework and data, using nested Module-Loop and Model-Loop computation over shared L/H parameter stacks, with the default H2L3 configuration executing 128 Transformer-layer calls per forward pass; under the same 0.14T-token training budget, LoopVL outperforms a same-depth Transformer-VL 1B baseline on MMStar, RealWorldQA, and ChartQA and approaches or surpasses larger 4B dense baselines, while the authors report cross-cycle visual attention reallocation they call a Visual Aha Moment.

The authors introduce LoopVL, extending looped Transformers to vision-language models: the language backbone is pretrained from scratch on HRM-Text's open-source framework and data, using nested Module-Loop and Model-Loop computation over shared L/H parameter stacks, with the default H2L3 configuration executing 128 Transformer-layer calls per forward pass; under the same 0.14T-token training budget, LoopVL outperforms a same-depth Transformer-VL 1B baseline on MMStar, RealWorldQA, and ChartQA and approaches or surpasses larger 4B dense baselines, while the authors report cross-cycle visual attention reallocation they call a Visual Aha Moment.