# LLM-Based Multi-Agent Systems

9 related articles

Subscribe to updates

arXiv Nov 9, 2026
Deleting the strongest message is not enough: collective dependence analysis cuts controlled multi-agent violations from 93% to 6%
Individually compliant language model agents can still jointly support a policy violation through messages that are harmless one by one. Collective Dependence Analysis (CDA) measures each message's necessity and sufficiency by removing and retaining it, cutting violations from 93% to 6% in controlled conversations and from 98% to 9% in agent-generated operational ones. It returns both an explanation and a minimal intervention set, though jointly required groups resist exact recovery and removals can also disrupt correct decisions.
Individually compliant language model agents can still jointly support a policy violation through messages that are harmless one by one. Collective Dependence Analysis (CDA) measures each message's necessity and sufficiency by removing and retaining it, cutting violations from 93% to 6% in controlled conversations and from 98% to 9% in agent-generated operational ones. It returns both an explanation and a minimal intervention set, though jointly required groups resist exact recovery and removals can also disrupt correct decisions.
Edoardo Chidichimo（First author, Corresponding author）
Collective Dependence Analysis Collective Misalignment LLM-Based Multi-Agent Systems
arXiv Oct 12, 2026
Multi-agent teams stop adding agents by default: estimate the backbone's competence first, and a strong model saves 36% of tokens
Multi-agent collaboration is now sized by how competent the backbone is on the current query: 93.81% average across six benchmarks, and on higher-competence backbones average accuracy rises from 87.22 to 89.97 while token cost per query drops 36.26%.
Hong Kong Polytechnic University
Songmiao Wang（First author）
Competence-Aware Orchestration Capability Fingerprinting Role Activation Communication Topology Generation LLM-Based Multi-Agent Systems
arXiv Oct 9, 2026
EMFA models error propagation to locate decisive errors in multi-agent failures, raising Who&When step-level attribution by 3.45 and 4.40 points
The work proposes EMFA, which reconstructs failed multi-agent trajectories into structured dependency representations, explicitly models cascading error propagation and persistent interaction loops, and combines dual-branch candidate screening with counterfactual verification to identify the decisive error—the agent–step pair whose correction would recover the failed execution; on the Who&When benchmark it achieves state-of-the-art step-level attribution accuracy, improving the previous best by 3.45 and 4.40 percentage points on the Hand-Crafted and Algorithm-Generated subsets, with the best agent-level accuracy on Hand-Crafted and competitive results on Algorithm-Generated.
The work proposes EMFA, which reconstructs failed multi-agent trajectories into structured dependency representations, explicitly models cascading error propagation and persistent interaction loops, and combines dual-branch candidate screening with counterfactual verification to identify the decisive error—the agent–step pair whose correction would recover the failed execution; on the Who&When benchmark it achieves state-of-the-art step-level attribution accuracy, improving the previous best by 3.45 and 4.40 percentage points on the Hand-Crafted and Algorithm-Generated subsets, with the best agent-level accuracy on Hand-Crafted and competitive results on Algorithm-Generated.
Tianjin University National University of Defense Technology
Jiaqi Liao（First author）
EMFA Error-Propagation Modeling Failure Attribution LLM-Based Multi-Agent Systems Who&When Benchmark
arXiv Oct 9, 2026
ReCast turns frozen-LLM hidden states into attribution-oriented step representations, achieving the best Hit@1 on four failure-attribution benchmarks
After quantifying that text embeddings and hidden states separate root-cause steps only weakly, the work proposes ReCast: an attribution-aware probe selects layers, a sparse Johnson–Lindenstrauss projection builds complementary pattern and deviation features, and a bidirectional encoder trained with contrastive and ranking objectives converts frozen-LLM hidden states into attribution-oriented step representations; it also releases the ReCast-2K training set and attains the best Hit@1 on Who&When, TraceElephant, AFTraj-2K, and Who&When Pro.
After quantifying that text embeddings and hidden states separate root-cause steps only weakly, the work proposes ReCast: an attribution-aware probe selects layers, a sparse Johnson–Lindenstrauss projection builds complementary pattern and deviation features, and a bidirectional encoder trained with contrastive and ranking objectives converts frozen-LLM hidden states into attribution-oriented step representations; it also releases the ReCast-2K training set and attains the best Hit@1 on Who&When, TraceElephant, AFTraj-2K, and Who&When Pro.
Peking University Tsinghua University Future Academy
Weilin Jin（First author）
ReCast Failure Attribution ReCast-2K Representation Learning LLM-Based Multi-Agent Systems
arXiv Oct 8, 2026
SAUCE estimates uncertainty in parallel multi-agent reasoning via filtering-style sequential inference, improving misclassification detection, selective prediction, and calibration across five backbones and two protocols
The work proposes SAUCE, a training-free uncertainty estimator that casts uncertainty in parallel multi-agent reasoning as sequential inference over a latent system-level belief: each round's cross-agent agreement serves as evidence and its generation uncertainty sets the evidence weight, updated through a filtering-style recursion, improving misclassification detection, selective prediction, and calibration across five backbones, five benchmarks, and the Debate and DyLAN protocols.
The work proposes SAUCE, a training-free uncertainty estimator that casts uncertainty in parallel multi-agent reasoning as sequential inference over a latent system-level belief: each round's cross-agent agreement serves as evidence and its generation uncertainty sets the evidence weight, updated through a filtering-style recursion, improving misclassification detection, selective prediction, and calibration across five backbones, five benchmarks, and the Debate and DyLAN protocols.
Tunyu Zhang（First author）
SAUCE Uncertainty Estimation LLM-Based Multi-Agent Systems Bayesian Filtering Selective Prediction
arXiv May 7, 2026
An ensemble-based information-theoretic task selection framework makes LLM multi-agent communication-structure optimization beat random training in both benign and adversarial settings
The work proposes an ensemble-based information-theoretic task selection framework that estimates task informativeness by how much a candidate task changes the distribution over graph parameters, using ensemble Kalman inversion as a derivative-free approximation of the Bayesian update, and combines embedding-based representative selection, surrogate modeling, and batch Thompson sampling for scalability; validated across benign and adversarial settings and multiple task formats, it consistently outperforms random training with more effective task selection and greater overall cost efficiency.
The work proposes an ensemble-based information-theoretic task selection framework that estimates task informativeness by how much a candidate task changes the distribution over graph parameters, using ensemble Kalman inversion as a derivative-free approximation of the Bayesian update, and combines embedding-based representative selection, surrogate modeling, and batch Thompson sampling for scalability; validated across benign and adversarial settings and multiple task formats, it consistently outperforms random training with more effective task selection and greater overall cost efficiency.
University of Wisconsin–Madison
Huchen Yang（First author）
Active Learning LLM-Based Multi-Agent Systems Communication Structure Optimization Ensemble Kalman Inversion Thompson Sampling
arXiv Oct 1, 2026
Freezing agents and training only latent links: benign link training alone raises harmful compliance above text communication, and an RL attack lifts the mean from 27.9 to 76.9
This work studies the safety of latent communication in multi-agent systems, where lightweight trainable links replace text messages, and finds that with all agent parameters frozen, benign link training alone increases harmful compliance relative to text communication, that an attacker can amplify this via supervised optimization on harmful query-response pairs, poisoning benign training data, or a reward-guided reinforcement-learning attack that needs no harmful target responses, raising the mean harmful-compliance score from 27.9 with benignly trained links to 76.9 across three topologies and four safety benchmarks, and that adapting the rewards toward safer behavior repairs compromised links by updating only the links.
This work studies the safety of latent communication in multi-agent systems, where lightweight trainable links replace text messages, and finds that with all agent parameters frozen, benign link training alone increases harmful compliance relative to text communication, that an attacker can amplify this via supervised optimization on harmful query-response pairs, poisoning benign training data, or a reward-guided reinforcement-learning attack that needs no harmful target responses, raising the mean harmful-compliance score from 27.9 with benignly trained links to 76.9 across three topologies and four safety benchmarks, and that adapting the rewards toward safer behavior repairs compromised links by updating only the links.
Muhammad Huzaifa（First author）
Latent Communication Safety Alignment GRPO Reward-guided link repair LLM-Based Multi-Agent Systems
arXiv Sep 30, 2026
REST adds four differentiable losses to latent thoughts, lifting accuracy by up to 7.5 points across 7 benchmarks
The work shows that training latent recursive LLM systems with only final-answer cross-entropy leaves the thought representation subject to four failures, and introduces REST, which turns causality, minimality, separability and stability into differentiable losses added to cross-entropy, improving accuracy over the CE-only baseline by an average of 3.5 points in the multi-agent setting and 3.3 in the single-agent setting, up to 7.5 and 6.5 points at best, and raising the rate of convergence on a final answer from 73% to 95%, across 7 benchmarks in mathematics, science, medicine and code generation under matched training data and latent budget.
The work shows that training latent recursive LLM systems with only final-answer cross-entropy leaves the thought representation subject to four failures, and introduces REST, which turns causality, minimality, separability and stability into differentiable losses added to cross-entropy, improving accuracy over the CE-only baseline by an average of 3.5 points in the multi-agent setting and 3.3 in the single-agent setting, up to 7.5 and 6.5 points at best, and raising the rate of convergence on a final answer from 73% to 95%, across 7 benchmarks in mathematics, science, medicine and code generation under matched training data and latent budget.
Fahd Seddik（First author）
REST (REpresentation-Supervised Thoughts)Latent Recursive LLM Systems Latent Reasoning LLM-Based Multi-Agent Systems
arXiv Sep 26, 2026
MeshHeal uses two-timescale peer review to reach 0.839 degraded-phase accuracy at 51k tokens per task on BBH, MATH, and MMLU-Pro, versus Symphony's 0.807 at 115k
The work introduces MeshHeal, a fully decentralized self-healing framework that couples ability-matched peer review across two timescales to address gray failures in decentralized LLM-based multi-agent systems: at the fast timescale, an adaptive hierarchy escalates uncertain or low-scoring outputs from repeated single-reviewer evaluation to committee deliberation and, when needed, correction before use; at the slow timescale, a task- and ability-conditioned peer-relative detector aggregates scores to distinguish persistent degradation from ordinary output variation, trigger mandatory committee review, and eventually exclude degraded agents from ordinary routing, with recovery probes providing fresh evidence for reintegration; the authors also introduce Model-Backed MAS Evaluation, which ti
The work introduces MeshHeal, a fully decentralized self-healing framework that couples ability-matched peer review across two timescales to address gray failures in decentralized LLM-based multi-agent systems: at the fast timescale, an adaptive hierarchy escalates uncertain or low-scoring outputs from repeated single-reviewer evaluation to committee deliberation and, when needed, correction before use; at the slow timescale, a task- and ability-conditioned peer-relative detector aggregates scores to distinguish persistent degradation from ordinary output variation, trigger mandatory committee review, and eventually exclude degraded agents from ordinary routing, with recovery probes providing fresh evidence for reintegration; the authors also introduce Model-Backed MAS Evaluation, which ti
The Ohio State University Johns Hopkins University Arizona State University University of Houston
Keru Chen（First author）
MeshHeal self-healing for gray failures in decentralized LLM agent networks Model-Backed MAS Evaluation LLM-Based Multi-Agent Systems

You’re all caught up