Skip to main content
Search

# Length Generalization

1 related articles

Subscribe to updates
  1. arXiv

    Finite experience can pin a rule for any length: spatial memory built as a universal computer holds to 500 symbols after short-string training

    A theory treats spatial memory as a universal computer: a learner that reads only coincidences, whether two quantities are equal, rather than coordinates such as position or count that keep taking new values, has its rule fixed by finite experience at every length. The resulting machine pairs coprime-period rings, called clocks, with a controller that never reads them; trained on strings of up to 40 symbols, it holds its rule to length 500 on nine of fifteen benchmark tasks, five of which no LSTM or Transformer solves, and after training with new periods it reverses strings of 15,015 symbols without error before failing on the predicted curve.

You’re all caught up