Zephon data loader achieves elastic determinism for online stateful pipelines with competitive throughput on text and vision-language workloads
Zephon is a data loader for foundation models that provides elastic determinism for online, stateful pipelines (such as online tokenization, packing, and mixing of samples), meaning a deterministic global training batch sequence despite changes in GPU topology, frequent checkpoint-resume cycles, and different data processing backends, while checkpointing only bounded in-flight state so recovery cost does not grow with training progress; it achieves competitive throughput on text and vision-language workloads and offers a combination of guarantees no existing loader provides for online, stateful pipelines.