RL weight updates stop shipping whole checkpoints: ARCv3 cuts BF16 deltas to about 0.19% of original size, keeping cross-region training closer to policy
Synopsis
Fireworks released ARCv3, a compressor that turns the BF16 weight updates a trainer sends to rollout machines into compact deltas. Cross-region reinforcement learning previously meant shipping full checkpoints or relying on compressors such as ARCv2; ARCv3 encodes the bit-level structure of weight changes, sending mostly mantissa bits and packing rare exponent and sign changes separately. Across 1,000 weight-update deltas from production RL workloads, ARCv3 produced an average payload of about 0.19% of the original BF16 weight size, down from 0.36% for ARCv2, with bit-for-bit reconstruction of the trainer's weights.
Interpretation
In reinforcement learning training, the trainer continuously pushes updated model weights to the fleet of machines generating rollouts, and that step can now move a smaller delta instead of a full checkpoint. The prior path sent the full checkpoint to every rollout machine or used compressors such as ARCv2; in the RL workloads Fireworks examined, only about 2% of BF16 weights changed between consecutive checkpoints. Across 1,000 weight-update deltas drawn from production RL workloads, ARCv3 produced an average delta payload of around 0.19% of the original BF16 weight size, versus 0.36% for ARCv2, with both compressing losslessly so the comparison is bytes needed to describe the same update.
The compression exploits how BF16 weights change between training steps: the overwhelming majority of updates shift only the mantissa, while the exponent and sign stay put. ARCv3 encodes that asymmetry directly, omitting unchanged values, contributing only mantissa bits for mantissa-only changes, and packing the rarer exponent shifts and sign flips separately with each update's changed fields as a single block. The rollout side applies both streams to the previous checkpoint and checksums the result against the trainer's original to verify the reconstruction matches; for each tensor, ARCv3 also runs several general-purpose compression algorithms in parallel and keeps the smallest output, at extra CPU cost.
Smaller deltas let a cross-region rollout fleet update faster and stay closer to the current policy, making training across three or four regions more practical. Bigger deltas can let the trainer outrun the rollouts, so the fleet drifts out of date and the multi-region setup loses its edge over a single mega-cluster. On the same 1,000 BF16 weight-update deltas, ARCv3 shrank the compressed delta to roughly half the size of the next-best approach, compared against ARCv2, PULSESync and the delta-compression implementation in Hugging Face TRL.
Perspective
The demonstrated setting is BF16 weights, 1,000 weight-update deltas drawn from production RL workloads, and Fireworks' own cross-region synchronization between trainer and rollouts; ARCv3 is already the default when using the Fireworks Trainer SDK for rollouts, and teams bringing their own trainer can integrate it through the fireworks-delta-compression package.
The compression is described as lossless, with reconstruction bit-for-bit identical to the trainer's weights; the roughly 2% change rate and mantissa-dominated update distribution were observed in the RL workloads Fireworks examined, and the text does not say whether that distribution holds for other models or training stages.