PCOR stops concatenating peer cues with observations: conditioning lifts POO success on unseen peers from 30.2% to 51.1%

How peer information enters the policy is now designed separately: the observation is first encoded into a base feature for the current interaction state, and the peer representation no longer arrives as a parallel input but determines how those features are read. Earlier opponent-modeling methods mainly studied what peer representation to learn, then concatenated or jointly encoded it with the current observation as policy input, leaving the relation between the two sources to be discovered implicitly during policy learning. On the partially observable Overcooked and Predator-Prey with Watchtowers benchmarks, PCOR attains higher final evaluation performance than the reported baselines on unseen peers; POO evaluation success is 0.511±0.080 and PPW evaluation return is −3.72±0.07, each a mean±s.d. over five random seeds.

In the implementation, the current observation is encoded into a feature, the peer representation produces feature-wise scaling coefficients, and the gated-scaled feature is passed to the policy network. This conditioning is a gated scaling form of FiLM-style conditional transformation rather than a full affine transform; the peer representation comes from a transformer-based interaction-history encoder trained jointly with a peer-identification objective. In ablation, the Weak-Z variant keeps the same architecture but scales the conditioning coefficient to 10⁻⁶, and POO success drops from 51.1% to 12.0%; removing the peer representation (Pure Obs) or keeping only it (Pure Z) is clearly worse, with Pure Z at 0% success on POO.

Mechanistic analysis shows that as interaction history accumulates, PCOR's policy representations become increasingly peer-discriminative and its observation-to-representation mapping more consistent for the same peer. On POO, PACE has higher linear-probe accuracy and silhouette score for peer representations early in adaptation, while PCOR catches up and is comparable or slightly higher by the end; both methods keep high same-peer and low cross-peer similarity. On unseen peers, PCOR reaches 45.7% and 47.6% success in games 4 and 5 versus 38.0% and 35.7% for PACE; PACE is higher in the early games 1 and 2, an early-versus-late trade-off.

The conclusions rest on two partially observable benchmarks with peers drawn from predefined rule-based policy pools and a five-episode adaptation horizon per peer in the main protocol; evaluation reports seen peers, unseen peers and their union. Mechanistic analysis is conducted only on POO, with PCOR and PACE evaluated on the same unseen peers, initial configurations and adaptation sessions.

The authors note it remains unclear whether the advantage persists for larger behavioral diversity, longer interaction horizons, learning or non-stationary peers, or additional multi-agent domains; when only one or two episodes of interaction are available, methods with stronger early peer identification such as PACE can still do better, and averaged Eval20 scores mask this trade-off. Training uses a peer-identification loss over discrete peer identities; labels are not needed at execution time, but this supervision may not extend to unlabeled or open-ended peer populations. The mechanistic analyses reveal correlations only, do not establish causality, and do not show whether the same representation-level patterns hold on PPW.