# Offline Multi-Agent Reinforcement Learning

The work identifies a teacher-training failure mode in which standard flow-based teachers pair noise with replay targets independently, routing nearby noise samples toward conflicting coordination modes and producing samples between valid modes that the distillation loss propagates to students; it proposes MoSDOT, which summarizes multimodal replay into a finite mode support with prescribed capacities and uses conditional semi-discrete optimal transport to assign each noise sample to a single mode before teacher training, improving endpoint quality and routing consistency on controlled diagnostics and offline MARL benchmarks, and studies a shared-randomness variant to expose the residual gap intrinsic to strict-product execution.

The work identifies a teacher-training failure mode in which standard flow-based teachers pair noise with replay targets independently, routing nearby noise samples toward conflicting coordination modes and producing samples between valid modes that the distillation loss propagates to students; it proposes MoSDOT, which summarizes multimodal replay into a finite mode support with prescribed capacities and uses conditional semi-discrete optimal transport to assign each noise sample to a single mode before teacher training, improving endpoint quality and routing consistency on controlled diagnostics and offline MARL benchmarks, and studies a shared-randomness variant to expose the residual gap intrinsic to strict-product execution.

The work proposes SCOUT, an offline multi-agent reinforcement learning framework that separately trains a flow-matching behavioral prior and a decomposed value function, then at test time transports behavioral samples toward high-value regions via Stein variational gradient descent, using the number of transport steps in place of a fixed regularization coefficient; under the individual-global-max (IGM) principle it proves a single-term KL bound on the joint soft-value gap that vanishes as transport converges, with an irreducible additive residual proportional to the IGM violation; empirically it achieves the best average performance across discrete and continuous offline MARL benchmarks and yields improvements in all offline-to-online configurations.

The work proposes SCOUT, an offline multi-agent reinforcement learning framework that separately trains a flow-matching behavioral prior and a decomposed value function, then at test time transports behavioral samples toward high-value regions via Stein variational gradient descent, using the number of transport steps in place of a fixed regularization coefficient; under the individual-global-max (IGM) principle it proves a single-term KL bound on the joint soft-value gap that vanishes as transport converges, with an irreducible additive residual proportional to the IGM violation; empirically it achieves the best average performance across discrete and continuous offline MARL benchmarks and yields improvements in all offline-to-online configurations.