Checkpoints stop costing bits: reference exponents predict displacement length, cutting storage up to 27%

Checkpoint trajectories now have a usable conditioning signal for lossless compression: the exponent of a reference weight predicts the bit length of its displacement without side information. Existing lossless codecs either operate on XOR bit patterns or ignore trajectory-specific conditional structure. Across bfloat16, float16, and float32 trajectories, ULPack reduces storage by up to 27% against ZipNN, 21% against zstd-19, and 11.5% against FM-Delta.

The codec treats the difference between adjacent checkpoints as an integer displacement in ordered floating-point space, then lets entropy models encode those residuals conditioned on the reference exponent. This conditioning exploits the dependence between ordered-integer residuals and the reference exponent instead of compressing XOR bit patterns directly. On two dense trajectories, compression improves as updates shrink and reference conditioning strengthens, while near-uniform low-order bits set the remaining compression floor and earlier checkpoints add little information.

The report covers lossless compression of checkpoint trajectories, for researchers and engineers who keep training snapshots under storage limits. The reported results are limited to bfloat16, float16, and float32 trajectories, plus compression behavior observed on two dense trajectories.

Near-uniform low-order bits set the remaining compression floor, and earlier checkpoints add little information; the improvement that accompanies shrinking updates and stronger reference conditioning was observed on two dense trajectories.