# Neural Scaling Laws

SIM replaces clinical raw k-space with training samples generated from multi-coil MRI physics, then scales data and model capacity together in a generic Transformer. Across 14 public and in-house datasets spanning seven centers, five vendors, 0.3 to 7 T and more than ten protocols, its zero-shot reconstruction beats the second-best method by 1.06 to 3.95 dB PSNR; in a 150-scan blinded reader study, SIM reached 100% diagnostic usability at an acceleration factor the source does not state.

SIM replaces clinical raw k-space with training samples generated from multi-coil MRI physics, then scales data and model capacity together in a generic Transformer. Across 14 public and in-house datasets spanning seven centers, five vendors, 0.3 to 7 T and more than ten protocols, its zero-shot reconstruction beats the second-best method by 1.06 to 3.95 dB PSNR; in a 150-scan blinded reader study, SIM reached 100% diagnostic usability at an acceleration factor the source does not state.

Classical neural scaling laws write excess risk as a power law in capacity and data and assume the compute-optimal allocation exponent stays fixed across scales. This analysis uses Mellin–Barnes contour integration to split finite-scale departures along the capacity axis into a three-term expansion, reduces multi-domain mixing to scalar water-filling via KKT duality, and shows the allocation exponent drifts in proportion to capacity and data exponent variances. Across 45 operator spectra, Pythia, Chinchilla's 245 runs and RegMix's 17 domains, the estimators give holdout error and cut high-compute allocation regret.

Classical neural scaling laws write excess risk as a power law in capacity and data and assume the compute-optimal allocation exponent stays fixed across scales. This analysis uses Mellin–Barnes contour integration to split finite-scale departures along the capacity axis into a three-term expansion, reduces multi-domain mixing to scalar water-filling via KKT duality, and shows the allocation exponent drifts in proportion to capacity and data exponent variances. Across 45 operator spectra, Pythia, Chinchilla's 245 runs and RegMix's 17 domains, the estimators give holdout error and cut high-compute allocation regret.

In controlled in-context learning tasks comparing linear and nonlinear attention, the authors find that nonlinear attention can selectively focus on relevant tokens, letting strong and weak spectral directions be learned in parallel and producing inverse-depth loss decay, whereas linear attention's depth scaling depends on the data spectrum and quickly plateaus.

In controlled in-context learning tasks comparing linear and nonlinear attention, the authors find that nonlinear attention can selectively focus on relevant tokens, letting strong and weak spectral directions be learned in parallel and producing inverse-depth loss decay, whereas linear attention's depth scaling depends on the data spectrum and quickly plateaus.

Using a unified prelude–core–coda looped-transformer family, this work fits a separate compute-optimal recipe for each architecture and trains scaling ladders to about 10²⁰ FLOPs on FineWeb, finding that model growth (raising the number of core passes mid-training) and a boundary operator (normalizing the residual stream and re-injecting the prelude output) can change pre-training scaling exponents so that compute-efficiency gains widen with scale, while untying weights moves only the constant; in data-constrained multi-epoch training the optimal loop count grows with compute and scaling loops is more compute-efficient than scaling model size.

Using a unified prelude–core–coda looped-transformer family, this work fits a separate compute-optimal recipe for each architecture and trains scaling ladders to about 10²⁰ FLOPs on FineWeb, finding that model growth (raising the number of core passes mid-training) and a boundary operator (normalizing the residual stream and re-injecting the prelude output) can change pre-training scaling exponents so that compute-efficiency gains widen with scale, while untying weights moves only the constant; in data-constrained multi-epoch training the optimal loop count grows with compute and scaling loops is more compute-efficient than scaling model size.