A decision model that loops its own layers eight times: 1.4B parameters close in on a single-pass model three times its size
Synopsis
Typed decision models used to return option probabilities in a single forward pass, giving every decision a fixed amount of computation. SanSi turns a language model pre-trained to loop into a typed decision model, reading the option probabilities after every loop and training each with a proper scoring rule, so one model serves every budget from one loop to eight. On 10,027 test decisions from 59 sources it reaches 72.0% accuracy, 13.5 points above a non-looped model of the same shape trained with the same recipe and 1.8 points below a single-pass model with three times its parameters, at 7.7 times the GPU time of one pass.
Interpretation
Applying the same layers repeatedly lets a 1.4B-parameter decision model close much of the gap to a single-pass model with three times its parameters. Typed decision models previously made one forward pass, so every decision received a fixed amount of computation and callers had to add parameters or switch to generated reasoning for harder decisions. On 10,027 test decisions, SanSi read after eight loops reaches 72.0% accuracy, 13.5 points above SmolLM2-1.7B, a non-looped model of the same shape trained with the same data, recipe and seeds, and 1.8 points below Qwen3.5-4B, which has three times the parameters.
The gain from looping concentrates on decisions that must bring several pieces of information together, not on classification that one pass already handles. A single pass is already enough for classification, where the gain is smallest, while multi-step reasoning, long documents and knowledge questions are where the single-pass model is weakest. By item type the gain is 3.1 points on classification, 15.1 on multi-step reasoning, 16.7 on long documents, 14.2 on sentence pairs and 17.6 on knowledge questions, which occur only in the test set; by distance from training it is 10.5 points in distribution, 10.0 on near transfer and 15.8 on far transfer.
Reading and training every loop makes one model usable at any computation budget rather than only at a fixed depth. Putting the loss on the last loop only collapses the early loops, so the model cannot be read early. With the loss on the last loop only, loop 1 falls from 58.4% to 35.6% and loop 8 is 1.1 points lower; SanSi matches the one-loop model at loop 1, adds 8.5 points at loop 2 and 3.5 at loop 3, and three loops already deliver 88% of the gain from loop 1 to loop 8.
The loops make the model better at noticing that the evidence for a decision is missing, while confidence keeps rising after the answers settle. A single-pass model still gives a hard answer when the key evidence is removed, and its confidence does not separate items with evidence from those without. On the 382 items whose key evidence was removed, the hard-answer rate falls from 27.1% at loop 1 to 17.5% at loop 8 and the evidence AUROC rises from 0.835 to 0.935; calibration error is lowest at loop 3 (0.082) and climbs back to 0.093 at loop 8.
Perspective
The findings hold for backbones pre-trained to loop, mainly Ouro-1.4B and also Ouro-2.6B; adding a loop to a same-shape model not pre-trained that way, with the same recipe, brings no gain. Looping trades computation for accuracy: a decision with eight loops takes about 3.3 times the GPU time of Qwen3.5-4B, about 1.6 times with four loops, and about 6.3 times for SanSi-2.6B. The reasoning-depth analysis rests on two program-generated tasks, and the verifier case study uses one dataset and one generator. The suite is in English and assembled from existing public datasets, and its mix of item types is the authors' choice, so results are also reported by type and by test group.
The gain relies on a backbone pre-trained to loop, and only two sizes of that family are studied, so how far it carries to other and larger looped backbones remains open. The authors read a fixed number of loops and leave rules that stop early on easy items to future work. Calibration improves only up to the third loop, confidence keeps rising after the answers settle, and running more loops than were trained lowers accuracy. The verifier case study uses one dataset and one generator and illustrates one use of the loops rather than evaluating SanSi as a reward model in general. The authors also cannot exclude that a backbone has seen some of these public datasets during pre-training.
