DeepMind Agent Swarms Break Prompt Injection Defenses on Six of Seven Models

Google DeepMind published research showing that an automated attacker built from swarms of LLM agents drove attack resistance to 10% or less on six of seven models tested, including models that score as robust on public benchmarks.

Christopher Ort
By Christopher Ort · 2 min read
Share
A swarm of drones flying in coordinated formation over a research facility at dusk
Image: AI-generated, i10X
On this page
  1. Quick Take
  2. In plain English
  3. What happened
  4. Who is affected
  5. What it means
  6. Sources

Google DeepMind published research on November 9, 2026, demonstrating that automated swarms of LLM agents can systematically break prompt injection defenses on frontier models. The study found that six of seven tested models resisted 10% or fewer attack scenarios when the attacker used its full query budget.

i10X Quick Take

Quick Take

  1. What: Automated agent swarms found working prompt injections against models rated as robust on public benchmarks
  2. Who is affected: Seven models tested: Gemini 3.6 Flash (1% resistance), Gemini 3.7 Flash (6%), Gemini 3.8 Flash (10%), Gemini 4 Argon (66%), Kimi K3 (1%), GLM-5.3 (1%), and DeepSeek-V4.1-Flash (1%)
  3. Research team: Alexey Kurakin, Vishal Karande, Alex Bie, Chongyang Shi, Shuang Song at Google DeepMind
  4. Key finding: Only Gemini 4 Argon maintained 66% resistance, all others fell to 10% or below
  5. What to do: Do not rely on model hardening alone, implement defense in depth

In plain English

Google researchers built an automated system that uses multiple AI agents working together to find ways to trick other AI models into ignoring their instructions. They tested seven different AI models, and six of them failed to resist these attacks more than 90% of the time. Only one model, Gemini 4 Argon, held up well. The researchers say AI companies need multiple layers of protection, not just harder-to-trick models.

What you should do: If you are an ordinary user of AI chatbots, you do not need to do anything, but this shows the AI systems you use may be vulnerable to sophisticated attacks.

What happened

Google DeepMind published “Scaling Adaptive Prompt Injection Attacks with Agent Swarms” on arXiv on November 9, 2026. The research team, with Alexey Kurakin as corresponding author, built an automated attacker that runs many LLM agents in parallel to search for prompt injections. The system then measured how seven defender models held up against these attacks.

The attacker treats prompt injection as a search problem, spending a query budget probing the defender and adapting each attempt based on what the previous one returned. Attack goals were random non-malicious actions rather than harmful ones, which separates how well a model holds its instruction hierarchy from how well it recognizes danger.

The paper measures Attack Resistance Rate, the share of scenarios where the attack failed. A lower number means a stronger attack. At the full query budget of 150 defender queries (320 calls in total), six of the seven models resisted 10% or fewer of the attack scenarios.

Who is affected

Seven models served as defenders with thinking on and chain of thought hidden:

  • Gemini 3.6 Flash: 1% attack resistance
  • Gemini 3.7 Flash: 6% attack resistance
  • Gemini 3.8 Flash: 10% attack resistance
  • Gemini 4 Argon: 66% attack resistance (strongest defender)
  • Kimi K3: 1% attack resistance
  • GLM-5.3: 1% attack resistance
  • DeepSeek-V4.1-Flash: 1% attack resistance

Gemini 4 Argon was both the strongest defender and the strongest attacker in the study. The attacker broke 6 of 80 AgentDojo scenarios in 25 queries, compared to 2 for the prior Prompt Siren technique.

What it means

The research demonstrates that models scoring as robust on public benchmarks can still be vulnerable to adaptive, automated attacks. The authors argue that model hardening should be part of a full stack with defense in depth, not relied upon alone.

The authors also state that frontier defenses may need proactive, automated red-teaming to stay resilient against more capable adversaries. The study shows that as attackers gain access to more sophisticated tools, including agent swarms, the gap between benchmark performance and real-world resilience may widen.

Sources

Christopher Ort

Christopher Ort

Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). He is the author of “The Style Penalty”, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.

Editor's picks

  1. Microsoft Teams Lets AI Agents Join as Digital Teammates With Their Own Identity
  2. Azure Cobalt 200 VMs Reach General Availability With Up to 50% More CPU Performance
  3. Microsoft Launches AI Transparency Initiative With NIST Standards Work and 40 Partners
  4. Agent 365 Gets Business Analytics and FinOps for AI as Nadella Cites 50 Million Agents
  5. Microsoft Perception Is Generally Available, With Red, Blue and Green Security Agents