Multiple objectives stop being extra baggage: each agent learns its own preference, and the dynamic variant tops the Pareto front on four cooperative tasks
Synopsis
Multi-objective multi-agent reinforcement learning has treated multiple objectives as an extra optimisation dimension stacked on the multi-agent problem. MIRL instead uses that objective structure to guide cooperation: a team preference fixes the system-level trade-off while each agent carries a learnable individual preference, adjusted by meta-gradient according to subsequent team performance. On four multi-objective particle-environment tasks, MIRL-S and MIRL-D improve Pareto-front quality over five baselines, with the dynamic variant MIRL-D strongest overall.
Interpretation
The multi-objective structure becomes a learning signal for cooperation rather than one more optimisation dimension on top of the multi-agent problem. Existing work treats multi-agent cooperation and multi-objective optimisation as largely independent components of a learning problem, which is very costly outside trivial cases. On the Spread, Formation, Line and Reference cooperative tasks, MIRL-S and MIRL-D reach higher hypervolume and expected utility than all five baselines: DTDE-PPO, CTDE-PPO-V, CTDE-PPO-Q, MO-COMA and MO-MIX.
Beyond the fixed team preference, each agent holds a learnable individual preference that decides how its per-objective contributions are combined into its own policy-update signal. The individual preferences are adjusted by meta-gradient according to the subsequent team-level performance evaluated under the team preference after that policy update. Experiments use MPE extended to two objectives, the second being interaction energy; Spread, Formation and Line have three agents, Reference has two agents plus communication, and each environment uses three random seeds.
Perspective
The demonstrated scope is MPE extended to two-objective cooperative tasks with two or three agents, with all methods trained under the same joint environment-step budget.
The benchmark tasks are not designed to cleanly separate distinct behavioural tendencies or cooperative roles, so relating changes in learned preferences to specific behaviour changes remains difficult; evaluation is also limited to two-objective cooperative tasks with relatively small agent populations.