sugra.blog

BlogLLM Benchmark Scores as Published, Side by Side

Claude Sonnet 5: Benchmark Scores as Published

Every result the catalog holds for Claude Sonnet 5: 103 tests in 15 areas, from 13 pages. 72 printed by Anthropic itself, the rest by the company named beside each number. Each number is as printed; nothing is ranked or rescored.

Developer AnthropicReleased 2026-06-30Context window 1MMax output 128KInput Text, imageOutput TextWeights ClosedData version 2026103001, as of 2026-10-30

Compare in the table

Knowledge Work

TestValuePrinted bySetup as printed
AutomationBench10.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pages-
GDPval-AA v21618Anthropic, Claude Sonnet 5checked 2026-10-15; printed on 3 more pages-
APEX-Agents31.7%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
GDPval-AA (Elo)1584IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
Vals Finance Agent v253.9%Google, Gemini 3.8 Flash model cardchecked 2026-10-15-
Harvey's Legal Agent Benchmark5%Google, Gemini 3.8 Flash model cardchecked 2026-10-15; printed on 1 more page-
OfficeQA Pro62.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
GDPval-AA v2.11449Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 1 more page-
AA-Briefcase v1.11359Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 1 more page-
Harvey's Legal Agent Benchmark (criterion pass rate)90.1%Google, Gemini 3.7 Flash model cardchecked 2026-10-15-
OfficeQA75.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max

Agentic Coding

TestValuePrinted bySetup as printed
SWE-Bench Pro63.2%Anthropic, Claude Sonnet 5checked 2026-10-15; printed on 2 more pages-
Terminal-Bench 2.180.4%Anthropic, Claude Sonnet 5checked 2026-10-15; printed on 4 more pages-
DeepSWE v1.154%Google, Gemini 3.6 Flash model cardchecked 2026-10-15; printed on 3 more pages-
SWE-bench Multilingual78.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
Terminal-Bench 4.010.3%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 2 more pages-
ProgramBench77.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Harness: mini-swe-agent (upstream repository harness)
FrontierCode v1.1 Main42.4%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 2 more pages-
Terminal-Bench 3.014.6%Google, Gemini 3.7 Flash model cardchecked 2026-10-15-
SWE-Marathon v1.1 (resolution rate)30%xAI, Grok 4.7 model cardchecked 2026-10-14Effort: max
CursorBench 4.034.1%Anthropic, Claude Sonnet 5.5checked 2026-10-14-
SWE-Bench Multimodal28.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
Code Arena1541Google, Gemini 3.7 Flash model cardchecked 2026-10-15-

ML Engineering

TestValuePrinted bySetup as printed
MLE-bench66.9%Google, Gemini 3.6 Flash model cardchecked 2026-10-15-

Science and Math

TestValuePrinted bySetup as printed
GPQA Diamond91.1%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
SciCode53.6%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
CritPt16.9%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
BioMysteryBench Human Difficult32.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 4 more pagesEffort: max
BioMysteryBench Human Solvable87.5%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 4 more pagesEffort: max
LAB-Bench 280.1%Google, Gemini 3.7 Flash model cardchecked 2026-10-15; printed on 1 more page-
Protocol Understanding (V2)58.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pagesEffort: max
Protocols Troubleshooting49.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pagesEffort: max
Morphology-to-molecule matching6.8%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max
LatchBio SingleCellBench56.5%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pagesEffort: max; Tools: bash, file editor
LatchBio SpatialBench Verified68.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pagesEffort: max; Tools: bash, file editor
Protein sequence generation20.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pagesEffort: max
ADME medicinal chemistry (504 questions)41.3%Anthropic, Claude Opus 5.5 System Cardchecked 2026-10-25-
Biomedical image analysis42.3%Anthropic, Claude Opus 5.5 System Cardchecked 2026-10-25-
Organic chemistry V243.7%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25-
ProteinGym Hard36.6%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25-
De novo binder design72.6%Anthropic, Claude Opus 5.5 System Cardchecked 2026-10-25-
Protein library ranking44%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max
LatchBio Capabilities v1.0 (overall score)40.2%xAI, Grok 4.7 model cardchecked 2026-10-14Effort: max

Long Context

TestValuePrinted bySetup as printed
AA-LCR77%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
MRCR v2 8-needle 128K71.6%Google, Gemini 3.6 Flash model cardchecked 2026-10-15; printed on 1 more pageContext: 128k (average)

Computer Use

TestValuePrinted bySetup as printed
OSWorld-Verified81.2%Anthropic, Claude Sonnet 5checked 2026-10-15; printed on 1 more page-
OSWorld 2.0 partial42.6%Google, Gemini 3.8 Flash model cardchecked 2026-10-15Tools: batch tool enabled
OSWorld 2.1 partial57%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 1 more page-
OSWorld 2.1 binary25.6%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Context: Compaction above 100k tokens

Multimodal Understanding

TestValuePrinted bySetup as printed
CharXiv Reasoning (no tools)77%Google, Gemini 3.6 Flash model cardchecked 2026-10-15; printed on 2 more pagesTools: none
GDP.pdf28%Google, Gemini 3.7 Flash model cardchecked 2026-10-15-
CharXiv Reasoning (with tools)88.3%Google, Gemini 3.6 Flash model cardchecked 2026-10-15; printed on 1 more pageTools: enabled; exact tools in source
LVBench68.5%Google, Gemini 3.7 Flash model cardchecked 2026-10-15; printed on 1 more page-
Chartography (no tools)15.6%Anthropic, Claude Sonnet 5.5checked 2026-10-14Tools: none
GDP.pdf (all-pass rate)28%Google, Gemini 3.8 Flash model cardchecked 2026-10-15-

Cybersecurity

TestValuePrinted bySetup as printed
Binary Exploitation (control-flow hijacks)3Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
CyScenarioBench0.7%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-

General Knowledge and Reasoning

TestValuePrinted bySetup as printed
Humanity's Last Exam (no tools)43.2%Anthropic, Claude Sonnet 5checked 2026-10-15; printed on 2 more pagesTools: none
Humanity's Last Exam (with tools)57.4%Anthropic, Claude Sonnet 5checked 2026-10-15; printed on 2 more pagesTools: enabled; exact tools in source
AA-Omniscience Accuracy40%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
AA-Omniscience Non-Hallucination61%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
Humanity's Last Exam (Verified)31%Google, Gemini 3.7 Flash model cardchecked 2026-10-15; printed on 1 more page-

Tool Use and Workflows

TestValuePrinted bySetup as printed
τ³-bench Banking37.3%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
Agents' Last Exam (pass rate)33.3%Google, Gemini 3.7 Flash model cardchecked 2026-10-15-
Toolathlon Verified74.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 3 more pagesEffort: max; Harness: Internal harness mirroring Toolathlon-Verified definitions
MCPMark65.3%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max
Toolathlon Verified Pass@384.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pagesEffort: max; Harness: Internal harness mirroring Toolathlon-Verified definitions
Toolathlon Verified Pass^365.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pagesEffort: max; Harness: Internal harness mirroring Toolathlon-Verified definitions
Toolathlon Verified average assistant turnslower is better24.5Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25; printed on 1 more pageEffort: max

Web Search and Research

TestValuePrinted bySetup as printed
BrowseComp84.7%IFM, IFM K2-Horizon-375B-A23B model cardchecked 2026-10-15Effort: max; Harness: Discard-all@95k context length (DeepSeek-V3.2 technical report)

Multilingual

TestValuePrinted bySetup as printed
Global-MMLU89.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pages-
MILU89.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pages-

Healthcare

TestValuePrinted bySetup as printed
HealthBench Professional (length-adjusted)57.8%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
HealthBench (unadjusted)59.2Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25; printed on 1 more page-
HealthBench Professional (unadjusted)62.4%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25; printed on 1 more page-
PhysicianBench37.4%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max

Safety and Alignment

TestValuePrinted bySetup as printed
Gray Swan IPI (attack success rate after k attempts)attack success rate after k attempts, lower is better17.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, GUI computer-use surface
6.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15
0.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=1
5.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=10
0.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, coding surface
6%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, tool-use surface
Self-preference bias, Claude-identity system prompt (0-9 scale)lower is better0.15Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Self-preference bias, no system prompt (0-9 scale)lower is better-0.04Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
BBQ ambiguous accuracy98.6%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
BBQ disambiguated accuracy72.36%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
Benign request refusal rate (all languages)lower is better1.54%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; Seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); 16 policy areas
0.59%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; Seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); 16 policy areas
Child safety multi-turn appropriate response rate88%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
96%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
Child safety single-turn harmless rate99.95%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; all tested languages
99.89%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; all tested languages
Child safety single-turn refusal ratelower is better1.35%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; all tested languages
0.63%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; all tested languages
Claude Code dual-use and benign success rate97.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
Claude Code malicious request refusal rate87.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
Disordered eating single-turn harmless rate97.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
99.55%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
Disordered eating single-turn refusal ratelower is better0.09%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
0.31%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
Election integrity multi-turn appropriate response rate87%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
91%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
Election integrity single-turn harmless rate100%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; election prompts
100%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; election prompts
Election integrity single-turn refusal ratelower is better0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; election prompts
0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; election prompts
Harmful request harmless response rate (all languages)99.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; Seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); 16 policy areas
96.65%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; Seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); 16 policy areas
Malicious computer use refusal rate (without mitigations)84.68%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
Pairwise even-handedness86.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; pairwise prompt pairs
95.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; pairwise prompt pairs
Pairwise opposing perspectives acknowledged45.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; pairwise prompt pairs
52.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; pairwise prompt pairs
Shade coding attack success rate (attempt level)lower is better37.51%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 40 scenarios; with prompt injection probes enabled; scenarios with success 40/40
15.76%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 40 scenarios; with prompt injection probes enabled; scenarios with success 35/40
19.47%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 40 scenarios; attempt-level rate over valid responses; without safeguards; scenarios with success 36/40
40.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 40 scenarios; attempt-level rate over valid responses; without safeguards; scenarios with success 40/40
Shade computer use attack success rate (attempt level)lower is better1.46%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; with prompt injection probes; scenarios with success 4/14; Model setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; with prompt injection probes; scenarios with success 4/14 | With thinking
2.25%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; without safeguards; scenarios with success 4/14; Model setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; without safeguards; scenarios with success 4/14 | With thinking
1.04%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; with prompt injection probes; scenarios with success 4/14; Model setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; with prompt injection probes; scenarios with success 4/14 | Without thinking
4.86%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; without safeguards; scenarios with success 7/14; Model setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; without safeguards; scenarios with success 7/14 | Without thinking
Suicide and self-harm multi-turn appropriate response rate90%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
63%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
Suicide and self-harm single-turn harmless rate98.8%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
99.82%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
Suicide and self-harm single-turn refusal ratelower is better0.45%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
0.15%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
Browser use red-teamer attack success rate (attempt level)lower is better0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: high; Setup as reported: Claude Cowork harness; 110 scenarios; 10 attempts per scenario; thinking enabled at high effort; with auto mode; scenarios with success 0/110
0.37%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: high; Setup as reported: Claude Cowork harness; 110 scenarios; 10 attempts per scenario; thinking enabled at high effort; without safeguards; scenarios with success 3/110

Preference and Composite Rankings

TestValuePrinted bySetup as printed
Artificial Analysis Intelligence Index55Google, Gemini 3.7 Flash model cardchecked 2026-10-15-

Price

API list prices in US dollars per one million tokens, as printed on the developer's pricing page on the day checked. Cost per task is measured by Vals AI.

PriceValuePrinted bySetup as printed
Input$2Anthropic, Anthropic API pricingchecked 2026-10-15-
Cached input$0.2Anthropic, Anthropic API pricingchecked 2026-10-15-
Cache write$2.5Anthropic, Anthropic API pricingchecked 2026-10-15Cache: 5 minute cache
Cache write, 1 hour$4Anthropic, Anthropic API pricingchecked 2026-10-15-
Output$10Anthropic, Anthropic API pricingchecked 2026-10-15-
Input, batch$1Anthropic, Anthropic API pricingchecked 2026-10-15-
Output, batch$5Anthropic, Anthropic API pricingchecked 2026-10-15-
Cost per taskVals AI, USD, lower is better$25.39Vals AI, Vibe Code Bench v1.1 leaderboardchecked 2026-10-27Harness: OpenHands

More from Anthropic

Every number here is read from the same catalog file as the benchmark table, data version 2026103001. Where one page printed the model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.