Skip to main content
Search

# FSDP2

1 related articles

Subscribe to updates
  1. PyTorch

    PyTorch 2.15 makes its rewritten NCCL backend the default, and a hang now dumps every rank's state to disk

    PyTorch 2.15 makes the rewritten NCCL backend, previewed as opt-in nccl2 in 2.14, the default for every nccl process group, keeping TORCH_DIST_USE_NCCL2=0 and the nccl-legacy name as rollback paths. At cluster scale a single stalled GPU could halt a whole training run and the cause was often reproduced by hand; a backend-agnostic flight recorder now writes a post-mortem to disk when a collective hangs, and a timeout on any rank triggers dumps from every rank.

You’re all caught up