BlogLLM Benchmark Scores as Published, Side by Side
ExploitGym: Results as Published
Every result the catalog holds for ExploitGym: 27 models from 8 developers, read from 9 pages. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area CybersecurityMaintainer Academic author team led by UC Berkeley researchersIntroduced 2026Data version 2026103001, as of 2026-10-30
ExploitGym measures whether AI agents can take a program input that already triggers a software vulnerability and turn it step by step into a working attack. The benchmark has 898 instances in three areas: ordinary user programs, a JavaScript engine, and the Linux kernel. Protections are varied from task to task to see how they change agent success, and every task runs in a reproducible container.
ExploitGym
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-6.1 SolOpenAI, released 2026-09-29 | 35.1% | OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page | - |
| GPT-6 LunaOpenAI, released 2026-09-22 | 11.6% | OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19 | - |
| GPT-6 SolOpenAI, released 2026-09-22 | 22.1% | OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19 | - |
| GPT-6 AstraOpenAI, released 2026-09-03 | 42.4% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 4 more pages | - |
| GPT-5.6 LunaOpenAI, released 2026-07-09 | 12.4% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| GPT-5.6 SolOpenAI, released 2026-07-09 | 33.7% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 4 more pages | - |
| GPT-5.6 TerraOpenAI, released 2026-07-09 | 23.2% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| GPT-5.5OpenAI, released 2026-04-23 | 15.1% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| Claude Opus 5.5Anthropic, released 2026-09-22 | 34.5% | Google, Gemini 4 Carbonchecked 2026-10-29; printed on 2 more pages | - |
| Claude Fable 5.1Anthropic, released 2026-09-01 | 30.4% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24; printed on 1 more page | Effort: max; Fallback: with fallback |
| Claude Mythos 5.1Anthropic, released 2026-09-01 | 30.4% | Google, Gemini 4 Carbon: our next frontier modelchecked 2026-10-29 | Attempt budget: time cap per task: 6h |
| Claude Opus 5Anthropic, released 2026-07-24 | 22.1% | DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 2 more pages | - |
| Claude Fable 5Anthropic, released 2026-06-09 | 28.4% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 1 more page | - |
| Gemini 4 CarbonGoogle, released 2026-10-28 | 54.8% | Google, Gemini 4 Carbonchecked 2026-10-29; printed on 1 more page | Harness: Antigravity |
| Gemini 4 ArgonGoogle, released 2026-09-30 | 42% | Google, Gemini 4 Carbonchecked 2026-10-29; printed on 1 more page | Harness: Antigravity |
| DeepSeek-V4.1-FlashDeepSeek, released 2026-09-10 | 15.3% | DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15 | Effort: max |
| DeepSeek-V4-Flash-0731DeepSeek | 1.8% | DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15 | Effort: max |
| DeepSeek-V4-Pro-0813DeepSeek | 5.4% | DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15 | Effort: max |
| Kimi K3.1 ProMoonshot AI, released 2026-10-23 | 20.7% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Harness: Kimi Code; Attempt budget: 6-hour wall-clock budget per task |
| GLM-5.3Z.ai, released 2026-08-18 | 15% | DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 2 more pages | - |
| MiMo-V2.6-FlashXiaomi, released 2026-09-21 | 6% | Xiaomi, Xiaomi MiMo-V2.6-Pro-RL model cardchecked 2026-10-19 | - |
| MiMo-V2.6-ProXiaomi, released 2026-09-21 | 17.8% | Xiaomi, Xiaomi MiMo-V2.6-Pro-RL model cardchecked 2026-10-19; printed on 1 more page | - |
| MiMo-V2.5-ProXiaomi, released 2026-04-27 | 0.2% | Xiaomi, Xiaomi MiMo-V2.6-Pro-RL model cardchecked 2026-10-19 | - |
ExploitGym (2h)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.6 SolOpenAI, released 2026-07-09 | 216 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead) |
| Claude Fable 5Anthropic, released 2026-06-09 | 181 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead); Fallback: w/ fallback |
| Claude Opus 4.8Anthropic, released 2026-05-28 | 80 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead) |
| Qwen3.8-MaxAlibaba, released 2026-08-03 | 14 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Effort: max; Harness: Claude Code 2.1.207; Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead) |
| Kimi K3Moonshot AI, released 2026-07-16 | 36 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Effort: max; Harness: Claude Code 2.1.207; Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead) |
| GLM-5.3Z.ai, released 2026-08-18 | 105 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Effort: max; Harness: Claude Code 2.1.207; Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead) |
| GLM-5.2Z.ai, released 2026-06-16 | 29 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead) |
ExploitGym (6h)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.6 SolOpenAI, released 2026-07-09 | 293 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead) |
| Claude Fable 5Anthropic, released 2026-06-09 | 247 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead); Fallback: w/ fallback |
| Claude Opus 4.8Anthropic, released 2026-05-28 | 120 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead) |
| Qwen3.8-MaxAlibaba, released 2026-08-03 | 26 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Effort: max; Harness: Claude Code 2.1.207; Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead) |
| Kimi K3Moonshot AI, released 2026-07-16 | 70 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Effort: max; Harness: Claude Code 2.1.207; Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead) |
| GLM-5.3Z.ai, released 2026-08-18 | 130 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Effort: max; Harness: Claude Code 2.1.207; Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead) |
| GLM-5.2Z.ai, released 2026-06-16 | 39 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead) |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.