# Stochastic Rounding

At the low learning rates common in reinforcement learning with verifiable rewards, mixed-precision BF16 training converges more slowly than FP16 because small updates to FP32 shadow weights are often erased when those weights are rounded to BF16, leaving the weights unchanged for several steps. Replacing round-to-nearest with stochastic rounding in that quantization step makes BF16 follow FP16's reward trajectory on 1.5B, 14B and 30B-A3B models across two math datasets, while drifting less from the reference model than BF16 at a raised learning rate and keeping BF16's dynamic range without loss scaling.

At the low learning rates common in reinforcement learning with verifiable rewards, mixed-precision BF16 training converges more slowly than FP16 because small updates to FP32 shadow weights are often erased when those weights are rounded to BF16, leaving the weights unchanged for several steps. Replacing round-to-nearest with stochastic rounding in that quantization step makes BF16 follow FP16's reward trajectory on 1.5B, 14B and 30B-A3B models across two math datasets, while drifting less from the reference model than BF16 at a raised learning rate and keeping BF16's dynamic range without loss scaling.

The work extends the PRISM vectorized rounding library to arbitrary virtual precision via a variable-precision stochastic rounding (VPSR) algorithm and, holding the numerical format fixed while varying only the rounding rule at individual operation sites, emulates DistilGPT-2; it supports the site-level trade-off with a probabilistic forward-error bound for linear projections and a second-order decomposition of expected cross-entropy at the softmax, finding that at t=6 significand bits SR in the MLP gives 1.15x the full-precision reference perplexity versus 2.21x for RN, that the ordering reverses at the language-model head, and that a mixed-precision configuration (MLP output at t=6, SR in the MLP, RN at the head) brings perplexity within 1.10x, a 28% reduction over matched-bit RN.

The work extends the PRISM vectorized rounding library to arbitrary virtual precision via a variable-precision stochastic rounding (VPSR) algorithm and, holding the numerical format fixed while varying only the rounding rule at individual operation sites, emulates DistilGPT-2; it supports the site-level trade-off with a probabilistic forward-error bound for linear projections and a second-order decomposition of expected cross-entropy at the softmax, finding that at t=6 significand bits SR in the MLP gives 1.15x the full-precision reference perplexity versus 2.21x for RN, that the ordering reverses at the language-model head, and that a mixed-precision configuration (MLP output at t=6, SR in the MLP, RN at the head) brings perplexity within 1.10x, a 28% reduction over matched-bit RN.

The authors introduce Clean and its low-precision variant Q-Clean: randomized Nyström approximations replace SOAP's explicit left and right preconditioners, reducing optimizer-state memory from quadratic to linear in layer dimensions, while four orthogonal blocks (Main, A, B, C) reintegrate off-subspace components into the update; pre-training LLaMA-350M and 1.3B on C4, Clean reaches perplexity close to SOAP with less optimizer memory than AdamW, Q-Clean cuts optimizer memory by over 50% versus Muon, Clean reaches AdamW's final performance 26% faster in wall-clock time, and the methods uniquely enable pre-training a 13B model on a single 80GB H100.

The authors introduce Clean and its low-precision variant Q-Clean: randomized Nyström approximations replace SOAP's explicit left and right preconditioners, reducing optimizer-state memory from quadratic to linear in layer dimensions, while four orthogonal blocks (Main, A, B, C) reintegrate off-subspace components into the update; pre-training LLaMA-350M and 1.3B on C4, Clean reaches perplexity close to SOAP with less optimizer memory than AdamW, Q-Clean cuts optimizer memory by over 50% versus Muon, Clean reaches AdamW's final performance 26% faster in wall-clock time, and the methods uniquely enable pre-training a 13B model on a single 80GB H100.