sugra.blog

BlogLLM Benchmark Scores as Published, Side by Side

GPT-5.6 Sol: Benchmark Scores as Published

Every result the catalog holds for GPT-5.6 Sol: 248 tests in 19 areas, from 28 pages. 145 printed by OpenAI itself, the rest by the company named beside each number. Each number is as printed; nothing is ranked or rescored.

Developer OpenAIReleased 2026-07-09Context window 1.05MMax output 128KInput Text, imageOutput TextWeights ClosedData version 2026103001, as of 2026-10-30

Compare in the table

Knowledge Work

TestValuePrinted bySetup as printed
AutomationBench18.1%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 12 more pages-
GDPval-AA v21747.8OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 8 more pages-
APEX-Agents39.9%Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 1 more pageEffort: max
JobBench45.4Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 2 more pagesEffort: max
Vals Finance Agent v253.8%Google, Gemini 3.8 Flash model cardchecked 2026-10-15; printed on 1 more page-
Harvey's Legal Agent Benchmark2.5%Anthropic, Claude Opus 5checked 2026-10-15; printed on 4 more pages-
OfficeQA Pro63.2%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max; Harness: Codex
GDPval-AA v2.11588Anthropic, Claude Opus 5.5checked 2026-10-14; printed on 1 more page-
AA-Briefcase v1.11487xAI, Grok 4.7checked 2026-10-25Effort: max
AA-Briefcase1502Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25; printed on 2 more pages-
Harvey's Legal Agent Benchmark (criterion pass rate)87.2%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
CoWorkBench71.5Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
Management Consulting Tasks (Internal)43.2%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
Apex Bench v1.154.3%DeepSeek, DeepSeek-V4.1-Flash-1017 model cardchecked 2026-10-18-
CorpFin v264.4%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
Legal Research Bench48.1%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
SpreadsheetBench 232.4%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max; Harness: Codex
$OneMillion-Bench (expert score)53.8Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
Big Finance Bench53%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
PLawBench72.3Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
PRBench-Finance55.5Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
PRBench-Legal57.6Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
WorkSpaceBench65.6Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max

Agentic Coding

TestValuePrinted bySetup as printed
SWE-Bench Pro64.6%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 3 more pages-
Terminal-Bench 2.191.9%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 7 more pagesEffort: ultra
88.8%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
DeepSWE v1.172.7%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 13 more pages-
Terminal-Bench 4.037.3%OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 10 more pages-
NL2Repo-Bench56.8DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 2 more pages-
ProgramBench Almost@123%DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 3 more pages-
SWE-Marathon39%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max; Harness: Codex; Dataset revision: H20-calibrated branch of the official tasks as of July 9, 2026, prior to the final v1.1 release
ProgramBench77.6%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
FrontierCode v1.1 Main47.5%OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 2 more pages-
Terminal-Bench 3.034.4%DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 2 more pages-
FrontierSWE v232%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25; printed on 3 more pages-
FrontierSWE (dominance score)71.3Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max; Harness: Codex
FrontierCode v1.1 Extended60.6%OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 1 more page-
Kimi Code Bench 2.064.8%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max; Harness: Codex
SWE-Marathon v1.1 (resolution rate)42.5%xAI, Grok 4.7 model cardchecked 2026-10-14; printed on 1 more pageEffort: max
QwenSWEBench73.5Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
CursorBench 4.041.7%Anthropic, Claude Opus 5.5checked 2026-10-14; printed on 2 more pages-
Artificial Analysis Coding Agent Index v1.180OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
CursorBench 3.2.067.2%Anthropic, Claude Fable 5.1 and Mythos 5.1checked 2026-10-15; printed on 2 more pages-
AndroidBench74Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
Artificial Analysis Coding Agent Index v1.465.1OpenAI, GPT-6 Astrachecked 2026-10-14-
MiMo Code Bench (Xiaomi internal)59.3%Xiaomi, Xiaomi MiMo-V2.6-Pro-RL model cardchecked 2026-10-19-
MiMo VisualCoding (Xiaomi internal)73.4%Xiaomi, Xiaomi MiMo-V2.6-Pro-RL model cardchecked 2026-10-19-
QwenQoderBench53.8Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
Frontier-Bench v0.134.4%Anthropic, Claude Opus 5checked 2026-10-15Harness: mini-SWE-agent (Anthropic internal run)
Internal Database Migration Tasks42.7%OpenAI, GPT-6 Astrachecked 2026-10-14-

ML Engineering

TestValuePrinted bySetup as printed
PostTrainBench v1.136.2%MiniMax, MiniMax-M3.1-Flash model cardchecked 2026-10-25; printed on 2 more pagesDataset revision: v1.1
PaperBench90.5Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
MLS-Bench-Lite46.2Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 1 more pageEffort: max
Internal Research Debugging Evaluation68.3%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more page-
KernelGen 1P61.1%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
NanoGPT9.69%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
PostTrainBench Lite50.3%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
RSI Index (OpenAI)57.9%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
Internal Data Science Tasks30.5%OpenAI, GPT-6 Astrachecked 2026-10-14-

Science and Math

TestValuePrinted bySetup as printed
GPQA Diamond94.6%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 5 more pages-
SciCode56.1%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
AIME 202699.9%Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15Effort: xhigh
CritPt32.3%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
BioMysteryBench Human Difficult28.8%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25; printed on 1 more page-
Terminal-Bench Science 0.122.4%OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 3 more pages-
BioMysteryBench Human Solvable86.1%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25; printed on 1 more page-
FrontierMath Tier 4 (v2)83%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more page-
FrontierMath Tier 1-3 (v2)89%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
LAB-Bench 282.1%Google, Gemini 3.8 Flash model cardchecked 2026-10-15-
Protocol Understanding (V2)63.9%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25-
Protocols Troubleshooting56.4%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25-
ProtocolQA (open-ended)43.5%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19-
GeneBench Pro28.7%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more page-
LifeSciBench59.9%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more page-
Organic chemistry V243.2%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25-
ProteinGym Hard35.5%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25-
MedChemBench (Internal)48.3%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more page-
ArXivMath (June 2026), without tools86.73%Anthropic, System Card: Claude Fable 5.1 & Claude Mythos 5.1checked 2026-10-25Tools: none

Long Context

TestValuePrinted bySetup as printed
AA-LCR73.7%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
LongBench v267.1%Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
GraphWalks BFS 1M (F1)77.1%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15Context: 1M
GraphWalks BFS 256k (F1)90.7%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15Context: 256k
LOCA-Bench (256k environment description)60%MiniMax, MiniMax-M3.1-Flash model cardchecked 2026-10-25-
MRCR v2 8-needle 512K-1M73.8%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more pageContext: 512K-1M
MRCR v2 8-needle 256K-512K91.5%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more pageContext: 256K-512K
MRCR v2 8-needle 256K93.8Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max

Computer Use

TestValuePrinted bySetup as printed
OSWorld-Verified83%Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 1 more pageEffort: max
OSWorld 2.062.6%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 3 more pages-
OSWorld 2.0 partial62.6%Google, Gemini 3.8 Flash model cardchecked 2026-10-15Tools: batch tool enabled
ScreenSpot-Pro76.9%OpenAI, GPT-6 Astrachecked 2026-10-14Tools: none
OSWorld 2.0 offline partial65.7%OpenAI, GPT-6 Astrachecked 2026-10-14-

Multimodal Understanding

TestValuePrinted bySetup as printed
CharXiv Reasoning (no tools)85.8%Google, Gemini 3.8 Flash model cardchecked 2026-10-15; printed on 3 more pagesTools: none
MMMU-Pro83%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 2 more pagesTools: none
MathVision95.8%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max; Tools: none
GDP.pdf30.7%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 2 more pages-
CharXiv Reasoning (with tools)89.1%Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 1 more pageEffort: max; Tools: enabled; Python
BabyVision (with tools)88.9%DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 3 more pagesTools: enabled; exact tools in source
MMMU-Pro (with tools)84.6%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more pageTools: with tools
LVBench82.1%Google, Gemini 3.8 Flash model cardchecked 2026-10-15-
Video-MME (with subtitles)89.5%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
MMVU81.2%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
Chartography (with tools)79.9%DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 1 more pageTools: enabled; exact tools in source
MathVision (with tools)97.8%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max; Tools: enabled; Python
ZeroBench (with tools, pass@5)53%DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 2 more pagesTools: enabled; exact tools in source
PerceptionBench59.7%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
ZeroBench (pass@5)17%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max; Tools: none
GDP.pdf (all-pass rate)40%Google, Gemini 3.8 Flash model cardchecked 2026-10-15-
OmniDocBench (version not stated)85.8%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
WorldVQA ForceAnswer41.8%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
OpenScore String Quartets (1 - OMR-NED)0.19OpenAI, GPT-6 Astrachecked 2026-10-14-

Cybersecurity

TestValuePrinted bySetup as printed
CyberGym84.5%DeepSeek, DeepSeek-V4.1-Flash model cardchecked 2026-10-15; printed on 3 more pages-
ExploitBench73.5%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 3 more pagesHarness: ExploitBench API harness, 5 seeds, reasoning continuity (footnote 2)
ExploitGym33.7%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 4 more pages-
SEC-Bench Pro71.2%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 4 more pages-
74.3%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15Effort: ultra
Production Chat0.983OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Synthetic Agentic Envs0.998OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Capture the Flag challenges96.7%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
Semi-Synthetic Agentic Envs0.985OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
ExploitGym (2h)216Z.ai, GLM-5.3 model cardchecked 2026-10-15Attempt budget: timeout budget 2 hours (rescaled API inference time plus overhead)
ExploitGym (6h)293Z.ai, GLM-5.3 model cardchecked 2026-10-15Attempt budget: timeout budget 6 hours (rescaled API inference time plus overhead)
xdev-bench-hard, problems solved (pass@3)100%Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24Harness: Codex; Attempt budget: pass@3
xdev-bench-hard, average success rate77.8%Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24Harness: Codex; Attempt budget: pass@3
HackerBench v0.3 benign refusallower is better0%xAI, Grok 4.7 model cardchecked 2026-10-14Effort: max
HackerBench v0.3 harmful and dual-use compliancelower is better35.7%xAI, Grok 4.7 model cardchecked 2026-10-14Effort: max
ExploitBench (June-Aug 2026)5.5%OpenAI, GPT-6 Astrachecked 2026-10-14-
Advanced Cybersecurity Completion Rate1.5%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol with Daybreak Blue
0.5%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol without Trusted Access
Cyber red-teaming8%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol without Trusted Access
66%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol with Daybreak Blue
Proof-of-concept exploit creation5%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol without Trusted Access
90%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol with Daybreak Blue
SRE-Bench55.9%OpenAI, GPT-6 Astrachecked 2026-10-14-
Vulnerability discovery and analysis56%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol without Trusted Access
100%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol with Daybreak Blue
Vulnerability patching28%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol without Trusted Access
100%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Model as printed: gpt-5.6-sol with Daybreak Blue

General Knowledge and Reasoning

TestValuePrinted bySetup as printed
Humanity's Last Exam (no tools)47.2%Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 5 more pagesEffort: max
Humanity's Last Exam (with tools)58%Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 3 more pagesEffort: max; Tools: enabled; exact tools in source
SimpleQA Verified71.6%Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15Effort: xhigh
ARC-AGI-292.5%OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 1 more page-
AA-Omniscience Index22Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15Effort: xhigh
ARC-AGI-197.5%OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 1 more page-
Humanity's Last Exam (Verified)54.5%Google, Gemini 3.8 Flash model cardchecked 2026-10-15-
ARC-AGI-37.78%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 2 more pages-
Internal hallucination benchmarklower is better12.2%OpenAI, GPT-6 Astrachecked 2026-10-14-

Code Generation

TestValuePrinted bySetup as printed
QwenReactBench1564Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
QwenSVGBench1758Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max

Tool Use and Workflows

TestValuePrinted bySetup as printed
Toolathlon58%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
τ³-bench Banking33%Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 1 more pageEffort: max
Agents' Last Exam (pass rate)30.6%Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 6 more pagesEffort: max
Toolathlon Verified74.9%Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 3 more pagesEffort: max
MCP Atlas83.6%Moonshot AI, Kimi K3 model cardchecked 2026-10-15; printed on 1 more pageEffort: max
Agents' Last Exam (score)52.7%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 2 more pages-
SkillsBench73.5Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14Effort: max
BankerToolBench74.6%MiniMax, MiniMax-M3.1-Flash model cardchecked 2026-10-25-
MCPMark-Verified92.9%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max
SaaS-Bench61.4%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max

Web Search and Research

TestValuePrinted bySetup as printed
BrowseComp92.2%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 4 more pagesEffort: ultra
90.4%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
ResearchRubrics73.8%Moonshot AI, Kimi K3 model cardchecked 2026-10-15Effort: max

Instruction Following

TestValuePrinted bySetup as printed
IFBench72.7Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14; printed on 1 more pageEffort: max

Multilingual

TestValuePrinted bySetup as printed
Global-MMLU-Lite91.8%Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15Effort: xhigh

Healthcare

TestValuePrinted bySetup as printed
HealthBench57OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 3 more pages-
HealthBench Professional (length-adjusted)60.5%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 3 more pages-
HealthBench Professional60.5%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 3 more pages-
HealthBench (unadjusted)55.6OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
HealthBench Professional (unadjusted)64.1%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
HealthBench Hard33.1OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
HealthBench Consensus (length-adjusted)95.5OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
HealthBench Consensus (unadjusted)95.3OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
HealthBench Hard (unadjusted)31.1OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
MentalHealthBench46.7%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19Effort: max
MentalHealthBench (emergent)51.2%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19Effort: max
MentalHealthBench (high acuity)51.8%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19Effort: max
MentalHealthBench (non-acute)42.5%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19Effort: max
MentalHealthBench (task-clipped, emergent)44.3%OpenAI, GPT-6 Sol and GPT-6 Luna: October 2026 updatechecked 2026-10-19-
MentalHealthBench (task-clipped, high acuity)53.88%OpenAI, GPT-6 Sol and GPT-6 Luna: October 2026 updatechecked 2026-10-19-
MentalHealthBench (task-clipped, non-acute)47.07%OpenAI, GPT-6 Sol and GPT-6 Luna: October 2026 updatechecked 2026-10-19-
MentalHealthBench (task-clipped)47.53%OpenAI, GPT-6 Sol and GPT-6 Luna: October 2026 updatechecked 2026-10-19-

Biosecurity

TestValuePrinted bySetup as printed
Dual Use Safe0.911OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Severe Safe0.943OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Benign Not overrefuse0.989OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Coronavirus-ACE2 composite0.43OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Multimodal Troubleshooting Virology55.5%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19-
Phage-plasmid co-evolution (negative log-likelihood)lower is better11.35OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
SHP2 mean R20.3OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
AAV Capsid Packaging (Spearman)0.529OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
TroubleshootingBench48%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19-
DNA sequence design for transcription factor binding (pass@1)13.7%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Attempt budget: pass@1
Hard-negative protein binding prediction (pass@4)7.6%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Attempt budget: pass@4

Safety and Alignment

TestValuePrinted bySetup as printed
Gray Swan IPI (attack success rate after k attempts)attack success rate after k attempts, lower is better27%Google, Gemini 4 Argon Safety and Security Reportchecked 2026-10-19-
Age-restricted goods, services, and dangerous challenges / activities0.719OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Eating Disorders0.71OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Emotional reliance0.953OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Emotional Reliance0.931OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
extremism0.975OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Extremism0.962OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Gorefraction0.785OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
harms-erotic0.986OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
hate0.999OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Hate0.982OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Mental health0.991OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Non-Violent Illicit behavior0.987OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Self Harm0.982OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
self-harm0.989OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Self-harm0.856OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Self-harm (standard)0.945OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Sexual0.915OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Sexual Content0.929OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Sexual/minors0.973OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Violent Illicit behavior0.934OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 2 more pages-
Gorefraction0.785OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
FORTRESS (Adversarial)82.4%Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15Effort: xhigh
FORTRESS (Benign)98.1%Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15Effort: xhigh
StrongREJECT98.5%Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15Effort: xhigh
Chat prod - Chat Plugins1OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Codex prod - Age-restricted actions0.603OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Codex prod - Non-violent Wrongdoing0.906OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Codex prod - Self-harm0.893OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Codex prod - Sensitive personal data0.765OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Codex prod - Violent Wrongdoing0.689OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Human red-teaming - Chat Plugins0.766OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Human red-teaming - Codex0.851OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Bio: high risk5.8%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Bio: severe10.3%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Cyber59%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Respecting Warnings: unwanted persistencelower is better34%OpenAI, GPT-6 Sol and GPT-6 Luna: October 2026 updatechecked 2026-10-19-
Violence: moderate21.6%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Violence: severe48.3%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
Codex severity 3 or higher flag ratelower is better0.127%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19-
Internal computer use safety benchmarklower is better22%OpenAI, GPT-6 Astrachecked 2026-10-14-
Unintended engagement: attempted communicationlower is better84%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19-
Unintended engagement: unauthorized actionlower is better52%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19-
Circumventing restrictions (lower is better)lower is better0.4%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: Base Model (without confirmation policy)
0.2%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: With confirmation policy (default for users)
Codex severity 1-2 flag ratelower is better19.64%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19-
Data exfiltration (lower is better)lower is better14.1%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: Base Model (without confirmation policy)
8.8%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: With confirmation policy (default for users)
Destructive action (lower is better)lower is better2.9%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: Base Model (without confirmation policy)
0%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: With confirmation policy (default for users)
Overall misaligned outcome rate (lower is better)lower is better18.8%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: Base Model (without confirmation policy)
8%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: With confirmation policy (default for users)
Security weakening (lower is better)lower is better8.1%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: Base Model (without confirmation policy)
1.9%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: With confirmation policy (default for users)
Service disruption (lower is better)lower is better0.8%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: Base Model (without confirmation policy)
1%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: With confirmation policy (default for users)
Unauthorized external communication (lower is better)lower is better12.9%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: Base Model (without confirmation policy)
8.4%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: With confirmation policy (default for users)
Unauthorized transactions (lower is better)lower is better38%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: Base Model (without confirmation policy)
7.4%OpenAI, GPT-6 Astra System Cardchecked 2026-10-25Configuration: With confirmation policy (default for users)
Auto-review: exploitationlower is better0.3%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19; printed on 1 more page-
ExploitGym honeypotlower is better48.2%OpenAI, GPT-6 Astrachecked 2026-10-14-
Internal circumvention benchmarklower is better0.29%OpenAI, GPT-6 Astrachecked 2026-10-14-
Internal computer use safety benchmark (with AutoReview)lower is better4.3%OpenAI, GPT-6 Astrachecked 2026-10-14-
Auto-review: attempts to circumvent the deniallower is better7.3%OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19Effort: max

Engineering and Design

TestValuePrinted bySetup as printed
BenchCAD70.6%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15; printed on 1 more page-
EEBench reward39.4%xAI, Grok 4.7checked 2026-10-25; printed on 1 more pageEffort: max
BenchCAD (python tool)83.4%OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15Tools: python tool
CADGenBench reward37.1%xAI, Grok 4.7 model cardchecked 2026-10-14Effort: xhigh
Internal Design Tasks47.4%OpenAI, GPT-6 Astrachecked 2026-10-14-

Preference and Composite Rankings

TestValuePrinted bySetup as printed
Artificial Analysis Intelligence Index61xAI, Introducing Grok 4.6checked 2026-10-15Effort: max
Artificial Analysis Intelligence Index v4.158.9OpenAI, GPT-5.6: Frontier intelligence that scales with your ambitionchecked 2026-10-15-
Artificial Analysis Intelligence Index v4.1.160.9OpenAI, GPT-6 Astrachecked 2026-10-14-

Price

API list prices in US dollars per one million tokens, as printed on the developer's pricing page on the day checked. Cost per task is measured by Vals AI.

PriceValuePrinted bySetup as printed
Input$4OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $8 per 1M tokens above 272,000 tokens; Offer: promotional price
Cached input$0.4OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $0.8 per 1M tokens above 272,000 tokens; Offer: promotional price
Cache write$5OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $10 per 1M tokens above 272,000 tokens; Offer: promotional price
Output$20OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $30 per 1M tokens above 272,000 tokens; Offer: promotional price
Input, batch$2OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $4 per 1M tokens above 272,000 tokens; Offer: promotional price
Cached input, batch$0.2OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $0.4 per 1M tokens above 272,000 tokens; Offer: promotional price
Cache write, batch$2.5OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $5 per 1M tokens above 272,000 tokens; Offer: promotional price
Output, batch$10OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $15 per 1M tokens above 272,000 tokens; Offer: promotional price
Input, flex$2OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $4 per 1M tokens above 272,000 tokens; Offer: promotional price
Cached input, flex$0.2OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $0.4 per 1M tokens above 272,000 tokens; Offer: promotional price
Cache write, flex$2.5OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $5 per 1M tokens above 272,000 tokens; Offer: promotional price
Output, flex$10OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $15 per 1M tokens above 272,000 tokens; Offer: promotional price
Input, fast$8OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $16 per 1M tokens above 272,000 tokens; Offer: promotional price
Cached input, fast$0.8OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $1.6 per 1M tokens above 272,000 tokens; Offer: promotional price
Cache write, fast$10OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $20 per 1M tokens above 272,000 tokens; Offer: promotional price
Output, fast$40OpenAI, OpenAI API pricingchecked 2026-10-15Context: up to 272,000 tokens; Long context: $60 per 1M tokens above 272,000 tokens; Offer: promotional price
Cost per taskVals AI, USD, lower is better$33.4Vals AI, Vibe Code Bench v1.1 leaderboardchecked 2026-10-27Harness: OpenHands

More from OpenAI

Every number here is read from the same catalog file as the benchmark table, data version 2026103001. Where one page printed the model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.