# Causal Collaboration Effectiveness

The authors built AgentWorld, a benchmark of 100 human-annotated tasks (plus 100 LLM-augmented variants) on the open-source MMORPG engine Kaetram, requiring 3–20 agents with asymmetric roles to coordinate over 25–55 rounds under a blackbox setting, and proposed Causal Collaboration Effectiveness (CCE), a graph-based metric over causal action graphs; testing Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B, the best model reaches only 52.0% task success and a CCE of 0.320, with failure modes centered on communication breakdowns, role confusion, and inability to maintain shared plans across rounds.

The authors built AgentWorld, a benchmark of 100 human-annotated tasks (plus 100 LLM-augmented variants) on the open-source MMORPG engine Kaetram, requiring 3–20 agents with asymmetric roles to coordinate over 25–55 rounds under a blackbox setting, and proposed Causal Collaboration Effectiveness (CCE), a graph-based metric over causal action graphs; testing Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B, the best model reaches only 52.0% task success and a CCE of 0.320, with failure modes centered on communication breakdowns, role confusion, and inability to maintain shared plans across rounds.