BlogLLM Benchmark Scores as Published, Side by Side

Claude Sonnet 5.5: Benchmark Scores as Published

Every result the catalog holds for Claude Sonnet 5.5: 136 tests in 12 areas, from 6 pages. All were printed by Anthropic itself. Each number is as printed; nothing is ranked or rescored.

Developer AnthropicReleased 2026-09-28Context window 1MMax output 128KInput Text, imageOutput TextWeights ClosedData version 2026103001, as of 2026-10-30

Knowledge Work

TestValuePrinted bySetup as printed
AutomationBench44.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Fallback: API default fallbacks enabled
44.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
OfficeQA Pro65.6%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max
GDPval-AA v2.11844Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 3 more pages-
AA-Briefcase v1.11811Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 3 more pages-
Harvey's Legal Agent Benchmark (criterion pass rate)92.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: high; Harness: Artificial Analysis harness
93.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Harness: Artificial Analysis harness
OfficeQA76.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max
Harvey LAB-AA v1.1 (tasks meeting every criterion)10%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Harness: Artificial Analysis harness
11.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: high; Harness: Artificial Analysis harness

Agentic Coding

TestValuePrinted bySetup as printed
SWE-Bench Pro81.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max
DeepSWE v1.171%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19-
SWE-bench Multilingual90.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max
Terminal-Bench 4.070.6%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 3 more pages-
ProgramBench79.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageHarness: mini-swe-agent (upstream repository harness); Context: Up to 1M tokens
FrontierCode v1.1 Main46.2%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 3 more pagesEffort: max
52.1%Anthropic, Claude Sonnet 5.5checked 2026-10-14Effort: xhigh
FrontierSWE v261.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Harness: Proximal agent harness
FrontierCode v1.1 Extended59.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max; Harness: Claude Code (per Cognition)
64.4%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: xhigh; Harness: Claude Code (per Cognition)
CursorBench 4.055.5%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 1 more page-
SWE-Bench Multimodal54.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max

Science and Math

TestValuePrinted bySetup as printed
BioMysteryBench Human Difficult44.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
Terminal-Bench Science 0.159.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max; Harness: Claude Code --bare; Fallback: No fallback fired on any request or trial
BioMysteryBench Human Solvable89.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
Protocol Understanding (V2)66.6%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
Protocols Troubleshooting67.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
Morphology-to-molecule matching25%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
LatchBio SingleCellBench59.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Tools: bash, file editor
LatchBio SpatialBench Verified72.5%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Tools: bash, file editor
Protein sequence generation51%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
ADME medicinal chemistry (504 questions)65.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
Biomedical image analysis72.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
ArXivMath (August 2026), with tools95.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Tools: Code execution sandbox, no internet access
De novo binder design82.3%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Context: Agent campaign, 24 hours per target
Protein library ranking54.8%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
ArXivMath (August 2026), no tools86.8%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max

Computer Use

TestValuePrinted bySetup as printed
OSWorld 2.1 partial80.1%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 1 more page-
OSWorld 2.1 offline subset83.9%Anthropic, Claude Haiku 5.5checked 2026-10-15; printed on 1 more page-
OSWorld 2.1 binary43.5%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max; Context: Compaction above 100k tokens

Multimodal Understanding

TestValuePrinted bySetup as printed
Chartography (no tools)61.6%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 3 more pagesTools: none
Chartography (with tools)90.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max; Tools: Container with the image and standard libraries, image cropping tool

Cybersecurity

General Knowledge and Reasoning

TestValuePrinted bySetup as printed
Humanity's Last Exam (no tools)56.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 2 more pagesEffort: auto; Tools: none; Attempt budget: 980k tokens per attempt
56.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
Humanity's Last Exam (with tools)64.5%Anthropic, Claude Sonnet 5.5checked 2026-10-14; printed on 3 more pagesTools: enabled; exact tools in source

Tool Use and Workflows

TestValuePrinted bySetup as printed
Toolathlon Verified77.8%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Harness: Internal harness mirroring Toolathlon-Verified definitions
Toolathlon Verified Pass@385.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Harness: Internal harness mirroring Toolathlon-Verified definitions
Toolathlon Verified Pass^368.5%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Harness: Internal harness mirroring Toolathlon-Verified definitions

Multilingual

TestValuePrinted bySetup as printed
Global-MMLU92.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more page-
MILU91.6%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more page-

Healthcare

TestValuePrinted bySetup as printed
HealthBench Professional (length-adjusted)69.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max
HealthBench Professional77.1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
PhysicianBench63.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: max

Safety and Alignment

TestValuePrinted bySetup as printed
Gray Swan IPI (attack success rate after k attempts)attack success rate after k attempts, lower is better12.5%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, GUI computer-use surface
0.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, coding surface
3.4%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15
0.4%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=1
2.7%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=10
1%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: IPI benchmark, probability that an attacker succeeds after k attempts; all models use extended thinking; no injection protections; k=15, tool-use surface
Self-preference bias, Claude-identity system prompt (0-9 scale)lower is better0.16Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Self-preference bias, no system prompt (0-9 scale)lower is better0.06Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
BBQ ambiguous accuracy99.97%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more page-
BBQ disambiguated accuracy80.01%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more page-
Benign request refusal rate (all languages)lower is better0.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt; Seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); 16 policy areas
0.02%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; Seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); 16 policy areas
Child safety multi-turn appropriate response rate98%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt
86%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
Child safety single-turn harmless rate99.69%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt; all tested languages
99.57%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; all tested languages
Child safety single-turn refusal ratelower is better0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt; all tested languages
0.01%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; all tested languages
Claude Code dual-use and benign success rate98.4%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more page-
Claude Code malicious request refusal rate85.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more page-
Disordered eating single-turn harmless rate98.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt
95.29%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
Disordered eating single-turn refusal ratelower is better0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt
0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
Election integrity multi-turn appropriate response rate86%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: API, without a system prompt
86%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
Election integrity single-turn harmless rate100%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: API, without a system prompt; election prompts
100%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt; election prompts
Election integrity single-turn refusal ratelower is better0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt; election prompts
0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; election prompts
Harmful request harmless response rate (all languages)99.4%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt; Seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); 16 policy areas
95.61%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; Seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); 16 policy areas
Malicious computer use refusal rate (without mitigations)79.46%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more page-
Pairwise even-handedness99%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt; pairwise prompt pairs
97.9%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; pairwise prompt pairs
Pairwise opposing perspectives acknowledged60.2%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt; pairwise prompt pairs
41.4%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt; pairwise prompt pairs
Shade coding attack success rate (attempt level)lower is better2.63%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: Shade attacker, 200 attempts per scenario, 40 scenarios; with prompt injection probes enabled; scenarios with success 13/40
3.01%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 40 scenarios; attempt-level rate over valid responses; without safeguards; scenarios with success 16/40
Shade computer use attack success rate (attempt level)lower is better0.07%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; with prompt injection probes; scenarios with success 1/14
0.07%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: Shade attacker, 200 attempts per scenario, 14 scenarios; attempt-level rate over valid responses; without safeguards; scenarios with success 1/14
Suicide and self-harm multi-turn appropriate response rate60%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: API, without a system prompt
100%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
Suicide and self-harm single-turn harmless rate99.07%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: API, without a system prompt
99.81%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: claude.ai, with the default system prompt
Suicide and self-harm single-turn refusal ratelower is better0.41%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageSetup as reported: claude.ai, with the default system prompt
0.04%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Setup as reported: API, without a system prompt
Browser use red-teamer attack success rate (attempt level)lower is better0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19; printed on 1 more pageEffort: high; Setup as reported: Claude Cowork harness; 110 scenarios; 10 attempts per scenario; thinking enabled at high effort; with auto mode; scenarios with success 0/110
0%Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: high; Setup as reported: Claude Cowork harness; 110 scenarios; 10 attempts per scenario; thinking enabled at high effort; without safeguards; scenarios with success 0/110
Behavioral audit: Accepting unverifiable authorizationlower is better2.8Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Admirable behavior6.59Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Behavior consistency6.66Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Biological weapons upliftlower is better1.016Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Causal decision theory adherencelower is better3.41Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Character driftlower is better1.073Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Chemical weapons upliftlower is better1.018Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Circumventing constraints or controlslower is better1.7Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Coherence between actions and views6.64Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Complying with prompt injectionslower is better1.47Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Condescension toward the userlower is better1.42Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Controlled substance synthesis upliftlower is better1.051Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Creative mastery6.82Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Disallowed cyberoffenselower is better1.125Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Encouragement of user delusionlower is better1.03Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Evasiveness on controversial topicslower is better1.26Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Explosive weapons upliftlower is better1.022Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Failure to disclose bad or lazy behaviorlower is better1.57Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: False completion claimslower is better1.6Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Fraudlower is better1.133Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Fun or funny behavior2.76Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Good for the user6.99Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Harmful sexual contentlower is better1.06Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Ignoring explicit constraintslower is better1.99Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Illegible thinkinglower is better2.74Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Important omissionslower is better1.82Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Input hallucinationlower is better1.44Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Intellectual depth7.52Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Military-grade weaponslower is better1.053Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Overrefusallower is better1.707Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Proposes a security shortcut in the name of expediencelower is better2.12Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Radiological or nuclear weapons upliftlower is better1.004Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Reckless tool uselower is better2.23Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Supporting user autonomy7.33Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Sycophancylower is better1.79Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Undermining liberal democracylower is better1.21Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Unfaithful thinkinglower is better1.8Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: User deceptionlower is better1.24Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Verbalized evaluation awarenesslower is better2.31Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Warmth6.27Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Behavioral audit: Wet blanketlower is better1.833Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Multi-turn appropriate response rate: Bio weapons83%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, claude.ai (hatched bars), Figure 4.1.3.A
78%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, API (solid bars), Figure 4.1.3.A
Multi-turn appropriate response rate: Cyberattacks96%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, API (solid bars), Figure 4.1.3.A
99%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, claude.ai (hatched bars), Figure 4.1.3.A
Multi-turn appropriate response rate: Deadly weapons83%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, claude.ai (hatched bars), Figure 4.1.3.A
80%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, API (solid bars), Figure 4.1.3.A
Multi-turn appropriate response rate: Hate and discrimination89%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, API (solid bars), Figure 4.1.3.A
85%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, claude.ai (hatched bars), Figure 4.1.3.A
Multi-turn appropriate response rate: Influence operations57%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, API (solid bars), Figure 4.1.3.A
59%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, claude.ai (hatched bars), Figure 4.1.3.A
Multi-turn appropriate response rate: Romance scams96%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, API (solid bars), Figure 4.1.3.A
94%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, claude.ai (hatched bars), Figure 4.1.3.A
Multi-turn appropriate response rate: Tracking and surveillance65%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, API (solid bars), Figure 4.1.3.A
65%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, claude.ai (hatched bars), Figure 4.1.3.A
Multi-turn appropriate response rate: Violent extremism75%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, API (solid bars), Figure 4.1.3.A
85%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Multi-turn, claude.ai (hatched bars), Figure 4.1.3.A
Political refusals (Pairwise political refusal rate)Pairwise political refusal rate, lower is better4.9%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Pairwise political bias evaluation (Figure 4.4.1.A-C); the card prints the values as bar labels, claude.ai
2.7%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Setup as reported: Pairwise political bias evaluation (Figure 4.4.1.A-C); the card prints the values as bar labels, API
Rate of reaching outside the sandboxlower is better22%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Sandbox escape attempted crossings, all severitieslower is better6.9%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Sandbox escape attempted crossings, low severitylower is better4.3%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Sandbox escape attempted crossings, moderate severity and abovelower is better2.6%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-
Rate of using credentials or files outside the sandboxlower is better5.3%Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19-

Engineering and Design

TestValuePrinted bySetup as printed
BenchCAD Vision2Code, no tools (voxel IoU)0.747Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max
BenchCAD Vision2Code, with tools (voxel IoU)0.963Anthropic, Claude Sonnet 5.5 System Cardchecked 2026-10-19Effort: max; Tools: Container with the image files and standard libraries, image cropping tool
BenchCAD Vision2Code voxel IoU0.747Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Tools: none
0.963Anthropic, Claude Haiku 5.5 System Cardchecked 2026-10-19Tools: enabled

Price

API list prices in US dollars per one million tokens, as printed on the developer's pricing page on the day checked. Cost per task is measured by Vals AI.

PriceValuePrinted bySetup as printed
Input$2Anthropic, Anthropic API pricingchecked 2026-10-15-
Cached input$0.1Anthropic, Anthropic API pricingchecked 2026-10-15-
Cache write$2.5Anthropic, Anthropic API pricingchecked 2026-10-15Cache: 5 minute cache
Cache write, 1 hour$4Anthropic, Anthropic API pricingchecked 2026-10-15-
Output$10Anthropic, Anthropic API pricingchecked 2026-10-15-
Input, batch$1Anthropic, Anthropic API pricingchecked 2026-10-15-
Output, batch$5Anthropic, Anthropic API pricingchecked 2026-10-15-
Cost per taskVals AI, USD, lower is better$31.25Vals AI, Vibe Code Bench v1.1 leaderboardchecked 2026-10-27Harness: OpenHands

More from Anthropic

Every number here is read from the same catalog file as the benchmark table, data version 2026103001. Where one page printed the model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.