Discrete logarithms grok too: a 427k-parameter model memorizes first, then generalizes on prime-order groups

Grokking also occurs on an inverse map such as the finite-field discrete logarithm, where a model stays in a memorizing regime long before it suddenly generalizes. Earlier grokking work focused on forward algebraic operations such as modular addition and multiplication, whose target map is directly specified by a simple group operation; the discrete logarithm is an inverse map on a cyclic group with no simple local input-output rule. For the anchor model at p = 113 with a 30% training fraction, train accuracy passes 99% at epoch 90, test accuracy stays near chance for about 22k epochs, and only reaches 99.8% by about epoch 37k, a grokking delay ratio of about 298.

Grokking is not confined to one model scale: the capacity sweep finds it recurring at multiple neural scales, with an intermediate non-monotonic region whose outcome depends on data. The sweep covers 12 models from 90k to 14.4M parameters with training fractions varying from 35% to 60%, so it is an empirical capacity-data map rather than a fixed one-variable capacity law. The 427k-parameter model reaches 96.2% at 33% training data, while the 657k and 1.09M models reach only 25.7% and 28.5% at the same fraction; the 657k model groks to 99.6% when given 40% training data, and models from 1.64M upward all reach at least 99.2% under the staged data schedule.

During generalization, hidden representations become increasingly aligned with multiplicative-character structure, the Fourier basis parameterized by discrete-log coordinates. Group-structure probes show composition consistency and cyclic-shift accuracy rising in tandem with test accuracy, indicating the model is discovering the algebraic structure of the discrete logarithm rather than only memorizing input-output pairs. At the pre-unembedding layer, algebra-aware symbolic regression preferentially selects cos(2πx/q) and sin(2πx/q), reaching R² = 0.42 on dimension 54 versus 0.04 for free regression on the same dimension; free regression independently recovers angular frequencies near π/2 and π, matching valid Fourier harmonics for q = 112.

Grokking is not restricted to smooth composite-order groups or to generators seen during training. At p = 113 the group order is 112 = 16 × 7, which is smooth and could invite subgroup-decomposition shortcuts; the prime-order control rules that out. In the order-113 subgroup of F∗227, all six experiments grok, including three seeds of M04 and a disjoint-generator out-of-distribution split that reaches 100% test accuracy in 326k epochs; that prime-order model has a spectral participation ratio of 96.0 versus 28.9 for the composite-order M04.

The demonstrated scope is learning dynamics on small finite fields: the primary experiments use p = 113 with a single extension check at p = 521, and the author states these settings do not support claims about practical cryptanalysis. The capacity sweep covers 12 models from 90k to 14.4M parameters with training fractions varying from 35% to 60%, so it demonstrates multi-scale grokking and a data-dependent non-monotonic region rather than a controlled one-variable scaling law. The author treats matched sample-complexity estimation as a separate confirmatory study and notes that the integer-token vocabulary grows with field size, so the p = 521 extension is not used to claim fixed-capacity problem-size scaling.

The author lists partial entanglement of capacity and data as a limitation: training fraction changes across M07 to M12, so the sweep is not a controlled one-variable law. Most capacity points are single-seed, and multi-seed replication is needed to estimate grokking probabilities near the empirical boundary. Symbolic-regression fits to individual hidden dimensions remain moderate, and the author argues stronger mechanistic claims require multi-dimensional symbolic models, exact full-domain verification and causal intervention. The anchor result also depends on a progressive weight-decay schedule, and additional optimizer and regularization controls would be needed to establish which training ingredients are causal rather than merely associated; the author further observes 18 slingshot events accompanying the transition but explicitly does not claim slingshots are necessary for grokking.