Finite experience can pin a rule for any length: spatial memory built as a universal computer holds to 500 symbols after short-string training
Synopsis
A theory treats spatial memory as a universal computer: a learner that reads only coincidences, whether two quantities are equal, rather than coordinates such as position or count that keep taking new values, has its rule fixed by finite experience at every length. The resulting machine pairs coprime-period rings, called clocks, with a controller that never reads them; trained on strings of up to 40 symbols, it holds its rule to length 500 on nine of fifteen benchmark tasks, five of which no LSTM or Transformer solves, and after training with new periods it reverses strings of 15,015 symbols without error before failing on the predicted curve.
Interpretation
Whether a learner generalizes from short to long inputs depends on what it reads when it decides: reading a coordinate such as a position or a count, which keeps taking values never seen in training, meets unseen situations on long inputs, while reading only coincidences, which have the same two answers at every size, lets finite experience fix the rule. LSTMs and Transformers trained on strings of up to 40 symbols fail at lengths 41 to 500 on tasks as plain as duplicating a string, and a Transformer that reverses 40 symbols without error is at chance by length 50. The distinction is stated as a theorem: a learner that reads a coordinate has, for every training size, a twin that acts identically on training situations but answers differently on some longer input, whereas a learner whose situations hold only coincidences, a finite state and finite records is fixed once it is correct up to some size.
A memory that meets this requirement must grow with the input while the learner meets only finitely many situations in it; such a memory looks the same from every place and is therefore made of cycles, and a few cycles whose periods share no factor cover the most moments with the fewest units. These cycles correspond to the modules of the entorhinal grid code, cells that detect their return to place cells, the hippocampal index files records under the readings, and a prefrontal controller moves the cycles without ever reading them. In a network of neurons, turning every ring by whole wells is a symmetry of the dynamics; under a weak bias, suitable pulses and adaptation below a threshold, each ring has a stable bump in every well, a pulse carries every bump whole wells on, and on these states the network is the clock.
Reading only coincidences costs no computational power: what finite experience fixes in such a learner is exactly what multitape Turing machines compute in step with their input, so a circuit that evolved to find the way home can learn arithmetic and grammar. Earlier universality constructions kept the unbounded part of a computation in a precision or an address that the controller reads, a coordinate; here the unbounded part sits in the positions of clocks the controller never reads. A Turing machine with work tapes that reads one input symbol per step is, step for step, a machine with unbounded clocks, and conversely a machine whose clocks move by at most one reading per step under moves fixed by the controller's state is simulated in real time by a multitape Turing machine.
In experiments the memory is built from the theory and only the controller is learned: trained on strings of up to 40 symbols, the machine holds its rule to length 500 on nine of fifteen benchmark tasks, five of which no LSTM or Transformer solves. On those five tasks the best published runs reach 60.9% on reversal, 59.1% on stack manipulation and at most 75.1% on duplication, odds first and binary addition; the two networks that extrapolate on them were both given a memory, a stack for the Stack-RNN and a tape for the Tape-RNN, and a tape is a clock. Of 30 runs that fit their training lengths, 27 extrapolate to length 500; after training, clocks of periods 3, 5, 7 and 11 leave three of three reversal and two of three duplication machines correct on every input up to length 15,015, 375 times their longest training input, and beyond the range reversal follows the parameter-free predicted curve to within 0.27 percentage points on average.
Perspective
The demonstrated scope is training strings of up to 40 symbols and test lengths 41 to 500 on fifteen benchmark tasks, clock-exchange experiments with periods changed after training, reversal up to length 15,015, and ring attractor networks with a few neurons per well plus compiled Turing machines; the authors state that no human participants or animal experiments were involved.
Six of the fifteen tasks are unsolved: bracketed arithmetic and equation solving fit no run within six attempts because their procedures meet all their situations only at lengths 29 and 49, one cycle-navigation seed loses accuracy steadily with length, no procedure that keeps pace with the input is known for multiplication or square root, and bucket sort needs a fourth clock. The authors also note that a place cell requiring every module to agree fires once within range, and that the variation of field sizes within one cell is left unexplained.