# Policy Distillation

The authors formalize staggered participation (SP), a cooperative MARL setting where agents contribute to one objective over temporally offset participation windows and earlier participants leave task-relevant information for later ones, and propose SPL, a training-time augmentation combining prospective acquisition supervision for earlier agents with outcome-supervised receiver learning for later agents; across 60 MPE/RWARE backbone comparisons SPL shows higher observed mean task completion in every case, averaging a 14.1% difference, with gains extending to eight-agent teams and a physics-based UAV–UGV setting in Isaac Lab.

The authors formalize staggered participation (SP), a cooperative MARL setting where agents contribute to one objective over temporally offset participation windows and earlier participants leave task-relevant information for later ones, and propose SPL, a training-time augmentation combining prospective acquisition supervision for earlier agents with outcome-supervised receiver learning for later agents; across 60 MPE/RWARE backbone comparisons SPL shows higher observed mean task completion in every case, averaging a 14.1% difference, with gains extending to eight-agent teams and a physics-based UAV–UGV setting in Isaac Lab.

The work identifies a teacher-training failure mode in which standard flow-based teachers pair noise with replay targets independently, routing nearby noise samples toward conflicting coordination modes and producing samples between valid modes that the distillation loss propagates to students; it proposes MoSDOT, which summarizes multimodal replay into a finite mode support with prescribed capacities and uses conditional semi-discrete optimal transport to assign each noise sample to a single mode before teacher training, improving endpoint quality and routing consistency on controlled diagnostics and offline MARL benchmarks, and studies a shared-randomness variant to expose the residual gap intrinsic to strict-product execution.

The work identifies a teacher-training failure mode in which standard flow-based teachers pair noise with replay targets independently, routing nearby noise samples toward conflicting coordination modes and producing samples between valid modes that the distillation loss propagates to students; it proposes MoSDOT, which summarizes multimodal replay into a finite mode support with prescribed capacities and uses conditional semi-discrete optimal transport to assign each noise sample to a single mode before teacher training, improving endpoint quality and routing consistency on controlled diagnostics and offline MARL benchmarks, and studies a shared-randomness variant to expose the residual gap intrinsic to strict-product execution.