BlogLLM Benchmark Scores as Published, Side by Side
Claude Opus 4.8: Benchmark Scores as Published
Every result the catalog holds for Claude Opus 4.8: 117 tests in 17 areas, from 21 pages. 29 printed by Anthropic itself, the rest by the company named beside each number. Each number is as printed; nothing is ranked or rescored.
Developer AnthropicReleased 2026-05-28Context window 1MMax output 128KInput Text, imageOutput TextWeights ClosedData version 2026103001, as of 2026-10-30
Knowledge Work
Agentic Coding
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| SWE-Bench Pro | 69.2% | Anthropic, Claude Opus 4.8checked 2026-10-15; printed on 6 more pages | - |
| Terminal-Bench 2.1 | 74.6% | Anthropic, Claude Opus 4.8checked 2026-10-15; printed on 11 more pages | Harness: Terminus-2 |
| DeepSWE v1.1 | 59% | Anthropic, Claude Opus 5checked 2026-10-15; printed on 9 more pages | - |
| NL2Repo-Bench | 69.4 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 4 more pages | - |
| ProgramBench Almost@1 | 15.5% | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | - |
| SWE-Marathon | 40% | Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 2 more pages | Effort: max; Harness: Claude Code; Dataset revision: H20-calibrated branch of the official tasks as of July 9, 2026, prior to the final v1.1 release |
| ProgramBench | 63.8% | Moonshot AI, Kimi K2.7 Code model cardchecked 2026-10-15; printed on 2 more pages | Effort: xhigh; Context: 262144 |
| FrontierCode v1.1 Main | 46.5% | Anthropic, Claude Opus 5checked 2026-10-15 | - |
| Terminal-Bench 3.0 | 21.1% | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | - |
| FrontierSWE (dominance score) | 70 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 3 more pages | - |
| Kimi Code Bench 2.0 | 67.4% | Moonshot AI, Kimi K2.7 Code model cardchecked 2026-10-15; printed on 1 more page | Effort: xhigh; Context: 262144 |
| SWE-Marathon v1.1 (resolution rate) | 48.8% | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | - |
| QwenSWEBench | 84 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| DSBench-FullStack | 71.6% | DeepSeek, DeepSeek-V4-Flash-0731 model cardchecked 2026-10-15; printed on 1 more page | - |
| DSBench-Hard | 71.7% | DeepSeek, DeepSeek-V4-Flash-0731 model cardchecked 2026-10-15; printed on 1 more page | - |
| Artificial Analysis Coding Agent Index v1.1 | 72.5 | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| AndroidBench | 69.8 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| DeepSWE v1.0 | 55.75% | xAI, Grok 4.5checked 2026-10-15 | Effort: max; Harness: each provider's own harness (Artificial Analysis run) |
| QwenQoderBench | 62.7 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| Frontier-Bench v0.1 | 21.1% | Anthropic, Claude Opus 5checked 2026-10-15 | Harness: mini-SWE-agent (Anthropic internal run) |
| Z.ai Code Bench | 29.5% | Z.ai, GLM-5.3 postchecked 2026-10-15 | Effort: High |
| FrontierCode (Diamond) | 13.4% | Anthropic, Claude Fable 5 and Claude Mythos 5checked 2026-10-15 | Effort: xhigh |
ML Engineering
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| PostTrainBench v1.1 | 34.1% | Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 2 more pages | Effort: max |
| PaperBench | 80.3 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| MLS-Bench-Lite | 42.8 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 2 more pages | - |
Science and Math
Long Context
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| AA-LCR | 67.7% | Moonshot AI, Kimi K3 model cardchecked 2026-10-15 | Effort: max |
| LongBench v2 | 69.1% | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| GraphWalks BFS 1M (F1) | 68.1% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | Context: 1M |
| GraphWalks BFS 256k (F1) | 85.9% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | Context: 256k |
| MRCR v2 8-needle 256K | 83.2 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
Computer Use
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| OSWorld-Verified | 83.4% | Anthropic, Claude Opus 4.8checked 2026-10-15; printed on 3 more pages | - |
| OSWorld 2.0 | 55.7% | Anthropic, Claude Opus 5checked 2026-10-15; printed on 2 more pages | - |
Multimodal Understanding
Cybersecurity
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| CyberGym | 83.1% | DeepSeek, DeepSeek-V4-Flash-0731 model cardchecked 2026-10-15; printed on 2 more pages | - |
| ExploitBench | 40% | Anthropic, Claude Fable 5 and Claude Mythos 5checked 2026-10-15; printed on 2 more pages | - |
| ExploitGym (2h) | 80 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead) |
| ExploitGym (6h) | 120 | Z.ai, GLM-5.3 model cardchecked 2026-10-15 | Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead) |
General Knowledge and Reasoning
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| Humanity's Last Exam (no tools) | 49.8% | Anthropic, Claude Opus 4.8checked 2026-10-15; printed on 7 more pages | Tools: none |
| Humanity's Last Exam (with tools) | 57.9% | Anthropic, Claude Opus 4.8checked 2026-10-15; printed on 9 more pages | Tools: enabled; exact tools in source |
| ARC-AGI-3 | 1.5% | Anthropic, Claude Opus 5checked 2026-10-15; printed on 1 more page | - |
Code Generation
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| QwenReactBench | 1694 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| QwenSVGBench | 1648 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
Tool Use and Workflows
Web Search and Research
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| BrowseComp | 84.3% | Anthropic, Claude Opus 5checked 2026-10-15; printed on 2 more pages | - |
| WideSearch (metric not stated) | 72.9 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| DeepSearchQA (F1) | 93.1% | Moonshot AI, Kimi K3 model cardchecked 2026-10-15 | Effort: max |
| ResearchRubrics | 73.5% | Moonshot AI, Kimi K3 model cardchecked 2026-10-15 | Effort: max |
Instruction Following
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| IFBench | 62.2 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
Healthcare
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| HealthBench | 52.4 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| HealthBench Professional | 56.9% | Anthropic, Claude Fable 5 and Claude Mythos 5checked 2026-10-15; printed on 2 more pages | - |
Safety and Alignment
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| Self-preference bias, Claude-identity system prompt (0-9 scale)lower is better | -0.05 | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19 | - |
| Self-preference bias, no system prompt (0-9 scale)lower is better | -0.04 | Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19 | - |
Engineering and Design
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| BenchCAD | 27.3% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
| Blueprint-Bench 2 (normalized score) | 14.5% | Anthropic, Claude Fable 5 and Claude Mythos 5checked 2026-10-15 | - |
| BenchCAD (python tool) | 51.8% | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | Tools: python tool |
Preference and Composite Rankings
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| Artificial Analysis Intelligence Index v4.1 | 55.7 | OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15 | - |
Price
API list prices in US dollars per one million tokens, as printed on the developer's pricing page on the day checked. Cost per task is measured by Vals AI.
| Price | Value | Printed by | Setup as printed |
|---|---|---|---|
| Input | $5 | Anthropic, Anthropic API pricingchecked 2026-10-15 | - |
| Cached input | $0.5 | Anthropic, Anthropic API pricingchecked 2026-10-15 | - |
| Cache write | $6.25 | Anthropic, Anthropic API pricingchecked 2026-10-15 | Cache: 5 minute cache |
| Cache write, 1 hour | $10 | Anthropic, Anthropic API pricingchecked 2026-10-15 | - |
| Output | $25 | Anthropic, Anthropic API pricingchecked 2026-10-15 | - |
| Input, batch | $2.5 | Anthropic, Anthropic API pricingchecked 2026-10-15 | - |
| Output, batch | $12.5 | Anthropic, Anthropic API pricingchecked 2026-10-15 | - |
| Input, fast | $10 | Anthropic, Anthropic API pricingchecked 2026-10-15 | - |
| Output, fast | $50 | Anthropic, Anthropic API pricingchecked 2026-10-15 | - |
| Cost per taskVals AI, USD, lower is better | $26.88 | Vals AI, Vibe Code Bench v1.1 leaderboardchecked 2026-10-27 | Harness: OpenHands |
More from Anthropic
- Claude Haiku 5.5
- Claude Sonnet 5.5
- Claude Opus 5.5
- Claude Fable 5.1
- Claude Mythos 5.1
- Claude Opus 5
- Claude Sonnet 5
- Claude Fable 5
- Claude Mythos 5
- Claude Opus 4.7
- Claude Mythos Preview
- Claude Sonnet 4.6
- Claude Opus 4.6
- Claude Opus 4.5
- Claude Haiku 4.5
- Claude Sonnet 4.5
- Claude Opus 4.1
- Claude Opus 4
- Claude Sonnet 4
- Claude 3.7 Sonnet
- Claude 3.5 Sonnet
Every number here is read from the same catalog file as the benchmark table, data version 2026103001. Where one page printed the model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.