OMAF replaces diffusion policies with one-step flow policies, reaching up to 3.4x returns and 10.5x sample efficiency on multi-agent tasks

OMAF combines expressive generative policies with one-step action generation, using a Transformer-based flow policy to capture complex multi-agent coordination behaviors. Prior generative policies in online multi-agent settings were mainly diffusion-based, whose iterative sampling is costly; OMAF switches to one-step flow generation, removing iterative sampling while retaining the ability to model multimodal action distributions. The abstract states the method composition and design motivation, supported by experiments on 10 standard tasks from MPE and MAMuJoCo; network scale, training steps, and statistical testing details are not given in the abstract.

OMAF introduces an approximate path score surrogate that provides a principled route to synchronized flow policy optimization. The surrogate connects the flow model's approximate path score to the policy optimization objective, allowing the flow policy to be updated online rather than serving only as an offline generator or one requiring multi-step sampling. The abstract describes it as a core method component and states its role in supporting synchronized flow policy optimization; no error bounds or ablation results for the approximation appear in the abstract.

OMAF develops a joint optimization scheme coupling softmax Q-value estimation with a joint flow policy objective for stable and sample-efficient learning. The joint objective places value estimation and flow policy learning within a single optimization framework, targeting the balance between stability and sample efficiency in online multi-agent learning. The abstract states the scheme is used for stable and sample-efficient learning and points to comparisons across 10 tasks as effect evidence; variance, number of random seeds, and significance analyses are not reported in the abstract.

Across 10 standard tasks from MPE and MAMuJoCo, OMAF achieves up to 3.4x higher returns and 10.5x sample efficiency improvement compared with baseline methods. These numbers directly compare the one-step flow policy against existing baselines on returns and sample efficiency, indicating that removing iterative sampling did not sacrifice policy expressiveness. Evidence comes from the 10 standard-task experiments described in the abstract; the abstract does not list the baselines, per-task breakdowns, or confidence intervals, so the improvements should be read as reported values under that experimental setting.

The work targets online multi-agent reinforcement learning settings where cooperative tasks must balance multimodal action-distribution modeling with training and execution efficiency, and its validation scope is 10 standard tasks from MPE and MAMuJoCo. For researchers and practitioners seeking to reduce the sampling overhead of generative policies, OMAF offers a reusable direction of replacing iterative sampling with one-step flow generation, including separately reusable components: the Transformer flow policy, the approximate path score surrogate, and the joint optimization with softmax Q-value estimation.

The abstract does not provide the baseline list, per-task result breakdowns, number of random seeds, or variance, so the specific comparison conditions behind the 3.4x return and 10.5x sample efficiency improvements still need confirmation in the full text. How the approximation error of the path score surrogate affects policy optimization stability, how the joint optimization scheme behaves at different task scales, and how the one-step flow policy performs beyond MPE and MAMuJoCo are questions a careful reader can continue to watch.