# InfiLoop

Looped Transformers deepen reasoning by re-running a shared block at test time, but the usual carry-last rule lets later iterations overwrite correct intermediate results. InfiLoop replaces that rule with learned attention over recurrent outputs: a shared pseudo-query scores each candidate once, a learned exponential decay discounts older ones, and the aggregation has an exact streaming recurrence needing only a weighted-state vector and a normalization scalar per token. On Sudoku-Extreme a 7M-parameter model reaches 97.9% exact accuracy versus 94.2% for FPRM, and hits 90.9% at 1,872 layers where FPRM needs 3,744 layers for 90.1%.

Looped Transformers deepen reasoning by re-running a shared block at test time, but the usual carry-last rule lets later iterations overwrite correct intermediate results. InfiLoop replaces that rule with learned attention over recurrent outputs: a shared pseudo-query scores each candidate once, a learned exponential decay discounts older ones, and the aggregation has an exact streaming recurrence needing only a weighted-state vector and a normalization scalar per token. On Sudoku-Extreme a 7M-parameter model reaches 97.9% exact accuracy versus 94.2% for FPRM, and hits 90.9% at 1,872 layers where FPRM needs 3,744 layers for 90.1%.