BlogLLM Benchmark Scores as Published, Side by Side
Gray Swan Indirect Prompt Injection (IPI): Results as Published
Every result the catalog holds for Gray Swan Indirect Prompt Injection (IPI): 18 models from 7 developers, read from 5 pages. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area Safety and AlignmentMaintainer Gray Swan with the UK AI Security Institute, the US Center for AI Standards and Innovation, and model developersIntroduced 2026Data version 2026103001, as of 2026-10-30
Probability that an attacker finds a successful indirect prompt injection after k attempts, across coding, tool use and GUI computer use.
Not checked against a maintainer's page.
Gray Swan IPI (attack success rate after k attempts)
attack success rate after k attempts, lower is better
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-6.1 SolOpenAI, released 2026-09-29 | 8.6% | Google, Gemini 4 Carbon: our next frontier modelchecked 2026-10-29 | - |
| GPT-6 SolOpenAI, released 2026-09-22 | 10.1% | Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19 | - |
| GPT-6 AstraOpenAI, released 2026-09-03 | 8.5% | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19; printed on 2 more pages | - |
| GPT-5.6 SolOpenAI, released 2026-07-09 | 27% | Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19 | - |
| Claude Haiku 5.5Anthropic, released 2026-10-07 | 4% | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19 | Setup as reported: Gray Swan indirect prompt injection, k=15 attempts (Figure 5.2.1.A); tool use |
| 0.2% | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19 | Setup as reported: Gray Swan indirect prompt injection, k=15 attempts (Figure 5.2.1.A); coding | |
| 24.4% | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19 | Setup as reported: Gray Swan indirect prompt injection, k=15 attempts (Figure 5.2.1.A); GUI | |
| 7.1% | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19 | Setup as reported: Gray Swan indirect prompt injection, k=15 attempts (Figure 5.2.1.A) | |
| Claude Sonnet 5.5Anthropic, released 2026-09-28 | 12.5% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more page | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, GUI computer-use surface |
| 0.7% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, coding surface | |
| 3.4% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15 | |
| 0.4% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=1 | |
| 2.7% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=10 | |
| 1% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, tool-use surface | |
| Claude Opus 5.5Anthropic, released 2026-09-22 | 1% | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19; printed on 2 more pages | - |
| Claude Fable 5.1Anthropic, released 2026-09-01 | 1% | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19; printed on 2 more pages | - |
| Claude Opus 5Anthropic, released 2026-07-24 | 4.6% | Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19; printed on 1 more page | - |
| Claude Sonnet 5Anthropic, released 2026-06-30 | 17.2% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, GUI computer-use surface |
| 6.7% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15 | |
| 0.7% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=1 | |
| 5.1% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=10 | |
| 0.9% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, coding surface | |
| 6% | Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19 | Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, tool-use surface | |
| Claude Haiku 4.5Anthropic, released 2025-10-15 | 83.2% | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19 | - |
| Gemini 4 CarbonGoogle, released 2026-10-28 | 0% | Google, Gemini 4 Carbon: our next frontier modelchecked 2026-10-29 | - |
| Gemini 4 ArgonGoogle, released 2026-09-30 | 0.7% | Google, Gemini 4 Argon: Built to solve your hardest problemschecked 2026-10-25; printed on 3 more pages | Attempt budget: 15 adaptive attempts |
| Gemini 3.8 FlashGoogle, released 2026-09-02 | 5.5% | Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19; printed on 3 more pages | - |
| Grok 4.6xAI, released 2026-08-12 | 51.8% | Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19; printed on 1 more page | - |
| Muse Spark 1.3Meta | 15.9% | Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19; printed on 1 more page | - |
| Kimi K3Moonshot AI, released 2026-07-16 | 52.7% | Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19; printed on 1 more page | - |
| GLM-5.3Z.ai, released 2026-08-18 | 31.5% | Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19; printed on 1 more page | - |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.