Option order stops steering the answer: iterated cyclic reordering drives flip rate to zero across five benchmarks and five models
Synopsis
System One decision models answer a choice question in one forward pass, but the answer shifts with the order in which options are listed, and the probabilities carry a prior over answer labels. IJev starts from an order that ignores the input order, repeatedly reorders options by a prior-corrected cyclic average, and averages the round distributions instead of returning the last round. Across five benchmarks and five models, every cell has flip rate zero; the default uniform round average lowers negative log-likelihood in 24 of 25 cells while accuracy stays comparable to a single cyclic average.
Interpretation
A System One decision model that answers in one forward pass changes its choice with the order in which options are listed, and the dependence grows with the number of options; in an agent that order often comes from a tool registry or a page layout, so a change there can silently change the prediction. The earlier AnyJev-L0 read all cyclic shifts of the input order and divided out a label prior, but the listings it averaged were rotations of the input order, so that order still decided which listings entered the average, and those listings form a single cyclic class. On five benchmarks and five models, the raw listwise readout has flip rates from 7.9% to 59.7%, and AnyJev-L0 reduces them to 1.4%–39.4% but never to zero.
IJev alternates reordering with the prior-corrected cyclic average, and each round depends only on the previous invariant order, so every iterate and the round average are independent of the input order. An invariant ordering operator is a fixed function that returns the same ordered list for every input order of a fixed option set; sorting by option text and sorting by pointwise yes/no scores are two examples. Across the 25 model–benchmark cells, IJev has flip rate 0.0 everywhere, as the invariance proposition guarantees; the 20-round runs were not repeated under a shuffle.
The trajectory is still moving at the 20-round budget, so returning the last round would be an arbitrary sample; the default instead averages the distributions of the rounds that were run. A single cyclic average covers only the one cyclic class selected by the initial operator, different invariant operators select different classes, and a model that is not rotation-symmetric can assign them different distributions. Relative to AnyJev-L0, the 20-round uniform average lowers ECE in all 25 cells, Brier in 23, and NLL in 24; the misses are on Typed Decisions for Qwen3-4B and Phi-3.5-mini.
The reliable gain of the default rule is calibration, while accuracy is comparable or slightly higher. At the pre-specified 20-round budget, the uniform average matches or exceeds AnyJev-L0 in accuracy averaged over the five benchmarks for every model. The average accuracy margins are 1.8, 0.4, 0.0, 1.8 and 1.0 points for Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Mistral-7B and Phi-3.5-mini; individual cells move both ways, and the largest gain is Qwen3-1.7B on Typed Decisions, from 49.0 to 57.7.
Perspective
The demonstrated setting is five single-label choice benchmarks and five instruction-tuned models from 1.7B to 8B parameters, all run in fp16 on a single RTX 4090 D, with every prior estimated without labels from the evaluation batch.
The invariance proof assumes a deterministic model and canonically sorted prompts, so an interface returning nondeterministic logits can still flip; the batch prior assumes a representative evaluation batch and couples items inside the batch; and the pointwise initialization carries little information when its scores are nearly flat, so later rounds can lose accuracy.