Agent swarms learn adaptive prompt injection: even the strongest defender, Gemini 4 Argon, resists only 66% of attacks

Prompt injection is not solved just because public benchmarks have saturated: a more capable automated attacker can still make frontier models follow third-party instructions embedded in tool output. Earlier static attack payloads stopped working against frontier models around mid-2025, human red-teaming in the GraySwan Arena was nearing saturation by fall 2026, and the only reproducible open-source adaptive attacker, Siren, had a very low success rate. Across 100 synthetic tool-use scenarios, the strongest attacker, Gemini 4 Argon, left the most robust defender, Gemini 4 Argon, resisting only 66% of attacks at the full budget and 87% at a 150-query budget.

Attacker capability is itself a major driver of exploit success: the same defender models resist noticeably less when the attacker is stronger. With Gemini Flash 3.6 as the attacker, the mean resistance rate across defenders was 42.1%; with Gemini 4 Argon as the attacker, the mean fell to 23.1%. Evaluating the same defender set against four Gemini attackers of increasing capability, Gemini 4 Argon as defender resisted 100% against the Flash 3.6, 3.7 and 3.8 attackers and 87% against the 4 Argon attacker.

The attack search is carried out by a swarm of agents working in parallel and sharing leaderboards and whiteboard notes, rather than by a single model rewriting payloads. Earlier LLM-guided optimizers relied on a single language model to iteratively propose new payloads from candidate injections and defender trajectories. Each agent can call tools to evaluate candidates, read team and global leaderboards, and read and post whiteboard notes; total evaluation calls are capped in all experiments, and each team is assigned a natural-language optimization strategy.

In a comparison with the reproducible baseline Prompt Siren, this attacker found more successful injections on the same scenarios. Prompt Siren's mini-goat attack broke only 2 of 80 AgentDojo scenarios, and increasing attacker turns did not improve it. With GLM-5.3 as attacker and Gemini Flash 3.6 as defender, the adaptive optimizer broke 6 of 80 AgentDojo scenarios within a modest budget of 25 defender queries.

Evaluation uses 100 synthetic tool-use scenarios whose attack goal is a random tool call unrelated to the user task; Gemini models ran with safety classifiers and out-of-model prompt injection defenses disabled in both attacker and defender roles, and open-weight models were accessed through hosted inference endpoints. The comparison with prior work covers only 80 AgentDojo scenarios, with GLM-5.3 as attacker and Gemini Flash 3.6 as defender.

The authors note that comparison with prior adaptive attacks is generally difficult because most do not release code or attacker models, so the comparison used only the reproducible Prompt Siren baseline. They also state that the link between attacker capability and other cybersecurity abilities, such as exploit capability measured by ExploitBench, is an observed inverse correlation, with further analysis left to future work.