A network first differentiates, then recoheres: after the data-constraint graph connects, the learned state keeps reorganizing

A network in training does more than fit its labels: the Boolean function it has learned passes through two reorganization stages that run in opposite directions. Earlier accounts of learning mostly tracked accuracy, yet models with equal accuracy can make different predictions on unobserved inputs, so accuracy does not pin down the learned function. On the 11-bit parity task, across 24 training trajectories the effective Schmidt rank peaks near normalized progress 0.6 and falls again by generalization onset, with both directions present in all 24 seeds.

Reading the training set as a signed bipartite graph gives a definite criterion for which relative signs the data fix, and this graph's connectivity only grows with data. A duplicated data point repeats an occupied edge and leaves the graph unchanged; only a previously unseen input occupies a new edge and can merge connected components. Across 10,000 random data-addition sequences, the probability that all 32 left substrings fall in one component reaches one half near 128 added data points, while full connectivity of all 96 vertices arrives near 256.

On an already connected fixed graph, the learned state reorganizes in a fixed order: correlation magnitude changes first, target alignment follows, and predictive accuracy comes last. Nonmonotonic test error during training has been attributed to modes learned at different rates; here the learned function itself is tracked directly. In the branching study over 25 starting models, the purity contrast's error bars first become entirely positive at update 5, target-mode population at update 200, and the block-accuracy contrast at update 400.

On the same fixed graph, a low-rank, target-selective organization that has already been learned can be lost and restored, but the recovered sign table is not identical to the earlier one. Across 24 first-retreat events, mean effective Schmidt rank rises from about 1.5 at last-high to about 3.5 at retreat, and falls back to about 1.5 in the 20 events with a saved recovery checkpoint. Over 80 recovered output states from 20 recovery events, the squared function fidelity averages about 0.9 and stays strictly below one in every case.

The results concern the parity function on binary strings of finite length, learned by a Transformer, and averaged over long-range output positions at a fixed cut. The authors expect the crossover from differentiation to recoherence to shift with string length as the connectivity thresholds of the constraint graph do, which longer binary strings can test; varying the architecture and the target function probes whether these dynamics persist in broader learning settings.

The authors note that whether human learning shares the same mechanism remains an open question. Recovery restores a similar low-rank, target-selective organization, but every recovered sign table retains residual differences, with squared function fidelity always below one. The results come from a finite system, and extension to larger systems is only expected.