BlogLLM Benchmark Scores as Published, Side by Side
Agents’ Last Exam: Results as Published
Every result the catalog holds for Agents’ Last Exam: 38 models from 9 developers, read from 20 pages. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area Tool Use and WorkflowsMaintainer UC Berkeley Center for Responsible, Decentralized IntelligenceIntroduced 2026Data version 2026103001, as of 2026-10-30
Agents' Last Exam tests AI agents on long, realistic professional tasks with checkable outcomes, covering 55 sub-industries of the non-physical economy. Agents get only a task description and work in full operating-system sandboxes, and outputs are graded against hidden, deterministic reference answers. The public release (ALE-V1) has 147 tasks drawn from a larger corpus of more than 1,500.
Agents' Last Exam (pass rate)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-6 AstraOpenAI, released 2026-09-03 | 34.2% | Google, Gemini 4 Argonchecked 2026-10-29; printed on 1 more page | - |
| GPT-5.6 SolOpenAI, released 2026-07-09 | 30.6% | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 6 more pages | Effort: max |
| GPT-5.6 TerraOpenAI, released 2026-07-09 | 28% | Google, Gemini 3.7 Flash model cardchecked 2026-10-15; printed on 1 more page | - |
| GPT-5.5OpenAI, released 2026-04-23 | 26.6% | Moonshot AI, Kimi K3 model cardchecked 2026-10-15 | Effort: xhigh; Harness: Codex |
| Claude Opus 5.5Anthropic, released 2026-09-22 | 38.2% | Google, Gemini 4 Argonchecked 2026-10-29; printed on 1 more page | - |
| Claude Opus 5Anthropic, released 2026-07-24 | 28.6% | DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 3 more pages | - |
| Claude Sonnet 5Anthropic, released 2026-06-30 | 33.3% | Google, Gemini 3.7 Flash model cardchecked 2026-10-15 | - |
| Claude Fable 5Anthropic, released 2026-06-09 | 25.7% | Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 2 more pages | Effort: xhigh; Harness: Claude Code; Fallback: w/ fallback; 40% of tasks annotated as downgraded (dagger on the card) |
| Claude Opus 4.8Anthropic, released 2026-05-28 | 27% | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 5 more pages | - |
| Gemini 4 ArgonGoogle, released 2026-09-30 | 39.5% | Google, Gemini 4 Argonchecked 2026-10-29 | - |
| Gemini 3.7 FlashGoogle, released 2026-08-13 | 26.3% | Google, Gemini 3.7 Flash model cardchecked 2026-10-15 | - |
| Gemini 3.6 FlashGoogle, released 2026-07-21 | 24.2% | Google, Gemini 3.7 Flash model cardchecked 2026-10-15 | - |
| DeepSeek-V4.1-Flash-1017DeepSeek, released 2026-10-17 | 35.4% | DeepSeek, DeepSeek-V4.1-Flash-1017 model cardchecked 2026-10-18; printed on 1 more page | - |
| DeepSeek-V4.1-FlashDeepSeek, released 2026-09-10 | 31.8% | DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 1 more page | Effort: max; Harness: official scaffold |
| DeepSeek-V4-Flash (Preview)DeepSeek, released 2026-04-24 | 15.8% | DeepSeek, DeepSeek-V4-Flash-0731 model cardchecked 2026-10-15; printed on 1 more page | - |
| DeepSeek-V4-Pro (Preview)DeepSeek, released 2026-04-24 | 16.5% | DeepSeek, DeepSeek-V4-Flash-0731 model cardchecked 2026-10-15; printed on 1 more page | - |
| DeepSeek-V4-Flash-0731DeepSeek | 25.2% | DeepSeek, DeepSeek-V4-Flash-0731 model cardchecked 2026-10-15; printed on 3 more pages | - |
| DeepSeek-V4-Pro-0813DeepSeek | 25.7% | DeepSeek, DeepSeek-V4-Pro-0813 model cardchecked 2026-10-15; printed on 2 more pages | - |
| DeepSeek-V4-Vision-ExpDeepSeek | 27.3% | Z.ai, Z.ai GLM-5.3-Flash model cardchecked 2026-10-15 | - |
| Qwen3.8-MaxAlibaba, released 2026-08-03 | 27% | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 1 more page | - |
| Qwen3.7-PlusAlibaba, released 2026-06-01 | 13.2% | Alibaba, Qwen3.8-27B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.6-27BAlibaba, released 2026-04-22 | 10.6% | Alibaba, Qwen3.8-27B model cardchecked 2026-10-15 | - |
| Qwen3.7-MaxAlibaba | 11.8% | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| Qwen3.8-27BAlibaba | 20.4% | Alibaba, Qwen3.8-27B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.8-Flash-NextAlibaba | 24.3% | Alibaba, Qwen3.8-Flash-Next model cardchecked 2026-10-15 | - |
| Kimi K3.1 ProMoonshot AI, released 2026-10-23 | 43.2% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Harness: Kimi Code; Dataset revision: v1.1 |
| Kimi K3Moonshot AI, released 2026-07-16 | 28.3% | Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 5 more pages | Effort: max; Harness: Kimi Code |
| GLM-5.3-FlashZ.ai, released 2026-08-26 | 26.3% | Z.ai, Z.ai GLM-5.3-Flash model cardchecked 2026-10-15 | Effort: max |
| GLM-5.3Z.ai, released 2026-08-18 | 28.5% | Z.ai, GLM-5.3 model cardchecked 2026-10-15; printed on 3 more pages | Effort: max; Harness: Claude Code; Context: 1000000 |
| GLM-5.2Z.ai, released 2026-06-16 | 23.8% | Z.ai, GLM-5.3 model cardchecked 2026-10-15; printed on 4 more pages | - |
| MiniMax-M3.1-FlashMiniMax, released 2026-10-19 | 29.6% | MiniMax, MiniMax-M3.1-Flash model cardchecked 2026-10-25; printed on 1 more page | - |
| MiMo-V2.6-FlashXiaomi, released 2026-09-21 | 27.6% | Xiaomi, Xiaomi MiMo-V2.6-Pro-RL model cardchecked 2026-10-19 | - |
| MiMo-V2.6-ProXiaomi, released 2026-09-21 | 31.6% | Xiaomi, Xiaomi MiMo-V2.6-Pro-RL model cardchecked 2026-10-19 | - |
| MiMo-V2.5-ProXiaomi, released 2026-04-27 | 13.2% | Xiaomi, Xiaomi MiMo-V2.6-Pro-RL model cardchecked 2026-10-19 | - |
Agents' Last Exam (score)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-6 SolOpenAI, released 2026-09-22 | 56.4% | OpenAI, Introducing GPT-6 Sol and GPT-6 Lunachecked 2026-10-19 | Effort: max |
| GPT-6 AstraOpenAI, released 2026-09-03 | 59.3% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| GPT-5.6 LunaOpenAI, released 2026-07-09 | 50.3% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| GPT-5.6 SolOpenAI, released 2026-07-09 | 52.7% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 2 more pages | - |
| GPT-5.6 TerraOpenAI, released 2026-07-09 | 50.4% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| GPT-5.5OpenAI, released 2026-04-23 | 46.9% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| Claude Opus 5Anthropic, released 2026-07-24 | 55.5% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| Claude Fable 5Anthropic, released 2026-06-09 | 40.5% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more page | - |
| Claude Opus 4.8Anthropic, released 2026-05-28 | 45.1% | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 1 more page | - |
| Gemini 3.1 Pro PreviewGoogle, released 2026-02-19 | 32.1% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| Qwen3.8-MaxAlibaba, released 2026-08-03 | 52.4% | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| Qwen3.7-PlusAlibaba, released 2026-06-01 | 33.6% | Alibaba, Qwen3.8-27B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.6-27BAlibaba, released 2026-04-22 | 27.3% | Alibaba, Qwen3.8-27B model cardchecked 2026-10-15 | - |
| Qwen3.7-MaxAlibaba | 31.1% | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| Qwen3.8-27BAlibaba | 42.9% | Alibaba, Qwen3.8-27B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.8-Flash-NextAlibaba | 51.2% | Alibaba, Qwen3.8-Flash-Next model cardchecked 2026-10-15 | - |
Agents' Last Exam (CLI)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-6 AstraOpenAI, released 2026-09-03 | 38.8% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Dataset revision: v1.1; Fallback: with fallback |
| Claude Opus 5.5Anthropic, released 2026-09-22 | 40.7% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Dataset revision: v1.1; Fallback: with fallback |
| DeepSeek-V4.1-Flash-1017DeepSeek, released 2026-10-17 | 36% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Harness: Claude Code; Dataset revision: v1.1 |
| Kimi K3.1 ProMoonshot AI, released 2026-10-23 | 44.2% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Harness: Kimi Code; Dataset revision: v1.1 |
| GLM-5.3Z.ai, released 2026-08-18 | 29.4% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Dataset revision: v1.1 |
Agents' Last Exam v1.1 (score)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-6 AstraOpenAI, released 2026-09-03 | 73.1% | Google, Gemini 4 Carbonchecked 2026-10-29 | - |
| Claude Opus 5.5Anthropic, released 2026-09-22 | 71.6% | Google, Gemini 4 Carbonchecked 2026-10-29 | - |
| Gemini 4 CarbonGoogle, released 2026-10-28 | 77.4% | Google, Gemini 4 Carbonchecked 2026-10-29 | Harness: ALE-Claw |
| Gemini 4 ArgonGoogle, released 2026-09-30 | 69% | Google, Gemini 4 Carbonchecked 2026-10-29 | Harness: ALE-Claw |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.