# Model Growth

Using a unified prelude–core–coda looped-transformer family, this work fits a separate compute-optimal recipe for each architecture and trains scaling ladders to about 10²⁰ FLOPs on FineWeb, finding that model growth (raising the number of core passes mid-training) and a boundary operator (normalizing the residual stream and re-injecting the prelude output) can change pre-training scaling exponents so that compute-efficiency gains widen with scale, while untying weights moves only the constant; in data-constrained multi-epoch training the optimal loop count grows with compute and scaling loops is more compute-efficient than scaling model size.

Using a unified prelude–core–coda looped-transformer family, this work fits a separate compute-optimal recipe for each architecture and trains scaling ladders to about 10²⁰ FLOPs on FineWeb, finding that model growth (raising the number of core passes mid-training) and a boundary operator (normalizing the residual stream and re-injecting the prelude output) can change pre-training scaling exponents so that compute-efficiency gains widen with scale, while untying weights moves only the constant; in data-constrained multi-epoch training the optimal loop count grows with compute and scaling loops is more compute-efficient than scaling model size.