# ZeroSM communication

In sharded data-parallel training, the packed low-precision layouts that compute kernels want and the update units that optimizers need have been tied together; SS-HSDP separates runtime parameters from update parameters and uses shared data views and hierarchical sharding to avoid redundant movement. On MXFP8 video and language-model workloads at 128 GPUs, MFU rises by up to 21.7% over the fastest baseline, with peak memory comparable to or lower than the lowest-memory baseline.

In sharded data-parallel training, the packed low-precision layouts that compute kernels want and the update units that optimizers need have been tied together; SS-HSDP separates runtime parameters from update parameters and uses shared data views and hierarchical sharding to avoid redundant movement. On MXFP8 video and language-model workloads at 128 GPUs, MFU rises by up to 21.7% over the fastest baseline, with peak memory comparable to or lower than the lowest-memory baseline.