BlogLLM Benchmark Scores as Published, Side by Side
GPT-6 Astra: Benchmark Scores as Published
Every result the catalog holds for GPT-6 Astra: 202 tests in 16 areas, from 22 pages. 122 printed by OpenAI itself, the rest by the company named beside each number. Each number is as printed; nothing is ranked or rescored.
Developer OpenAIReleased 2026-09-03Context window 1.05MMax output 128KInput Text, imageOutput TextWeights ClosedData version 2026103001, as of 2026-10-30
Knowledge Work
Agentic Coding
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| DeepSWE v1.1 | 74.1% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 3 more pages | - |
| Terminal-Bench 4.0 | 57.9% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 7 more pages | - |
| ProgramBench Almost@1 | 52.5% | Meta, Meta Muse Wonder model cardchecked 2026-10-19; printed on 1 more page | Effort: max; Harness: mini-swe-agent |
| SWE-Marathon | 42.5% | Meta, Meta Muse Wonder model cardchecked 2026-10-19; printed on 1 more page | Effort: max |
| FrontierCode v1.1 Main | 53.3% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 2 more pages | - |
| FrontierSWE v2 | 65.5% | Anthropic, Claude Opus 5.5 System Cardchecked 2026-10-25; printed on 3 more pages | - |
| FrontierCode v1.1 Extended | 64.5% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 1 more page | - |
| Kimi Code Bench 3.0 Long-horizon | 66.5% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Fallback: with fallback |
| Artificial Analysis Coding Agent Index v1.4 | 67 | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| SWE-sweep | 9.5% | Google, Gemini 4 Carbonchecked 2026-10-29 | - |
| Internal Database Migration Tasks | 63.9% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| Vibe Code Bench | 89.6% | Google, Gemini 4 Argonchecked 2026-10-29 | - |
ML Engineering
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| PostTrainBench v1.1 | 44.3% | Google, Gemini 4 Argonchecked 2026-10-29 | - |
| Internal Research Debugging Evaluation | 78.05% | OpenAI, GPT-6.1 Sol system card addendumchecked 2026-10-19 | - |
| PostTrainBench v1.2 | 41.9% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Dataset revision: v1.2; Fallback: with fallback |
| Internal Data Science Tasks | 40.9% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
Science and Math
Long Context
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| GraphWalks BFS up to 128k (F1) | 98.7% | Google, Gemini 4 Argonchecked 2026-10-29 | - |
| MRCR v2 8-needle 512K-1M | 96.3% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| MRCR v2 8-needle 256K-512K | 100% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| GraphWalks BFS 256k to 1M (F1) | 71.8% | Google, Gemini 4 Argonchecked 2026-10-29; printed on 1 more page | - |
| MemoryBench (Meta internal) | 81.4% | Meta, Meta Muse Wonder model cardchecked 2026-10-19 | Effort: max |
Computer Use
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| ScreenSpot-Pro | 92.7% | OpenAI, GPT-6 Astrachecked 2026-10-14 | Tools: none |
| OSWorld 2.1 partial | 75.8% | Meta, Meta Muse Wonder model cardchecked 2026-10-19; printed on 1 more page | Effort: max |
| OSWorld 2.1 offline subset | 80.9% | Google, Gemini 4 Carbonchecked 2026-10-29 | - |
| OSWorld 2.1 binary | 34.3% | Meta, Meta Muse Wonder model cardchecked 2026-10-19 | Effort: max |
| OSWorld 2.0 offline partial | 72.6% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 1 more page | - |
Multimodal Understanding
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| CharXiv Reasoning (no tools) | 90% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: none; Fallback: with fallback |
| MMMU-Pro | 86.9% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: none; Fallback: with fallback |
| GDP.pdf | 34.2% | Meta, Meta Muse Wonder model cardchecked 2026-10-19; printed on 1 more page | Effort: max |
| CharXiv Reasoning (with tools) | 92.8% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: python; Fallback: with fallback |
| BabyVision (with tools) | 96.6% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: python; Fallback: with fallback |
| MMMU-Pro (with tools) | 87.8% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: python; Fallback: with fallback |
| LVBench | 87.5% | Google, Gemini 4 Argonchecked 2026-10-29 | - |
| Chartography (no tools) | 71% | Google, Gemini 4 Argonchecked 2026-10-29; printed on 1 more page | Tools: none |
| Chartography (with tools) | 91% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: python; Fallback: with fallback |
| ZeroBench (with tools, pass@5) | 58% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: python; Attempt budget: pass@5; Fallback: with fallback |
| PerceptionBench | 77.6% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: none; Fallback: with fallback |
| ZeroBench (pass@5) | 52% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: none; Fallback: with fallback |
| ChartQA Pro | 65.1% | Mistral AI, Mistral Large 4 (preview)checked 2026-10-15 | - |
| ZeroBench_main (avg@5) | 43.2% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: none; Fallback: with fallback |
| Dense200 (bbox) | 41.5% | Mistral AI, Mistral Large 4 (preview)checked 2026-10-15 | - |
| OpenScore String Quartets (1 - OMR-NED) | 0.84 | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
Cybersecurity
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| ExploitBench | 100% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 2 more pages | - |
| ExploitGym | 42.4% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 4 more pages | - |
| SEC-Bench Pro | 85.4% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 4 more pages | - |
| CWE-bench v1 | 68% | Google, Gemini 4 Argonchecked 2026-10-29; printed on 2 more pages | - |
| Production Chat | 0.97 | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25; printed on 1 more page | - |
| Synthetic Agentic Envs | 0.992 | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25; printed on 1 more page | - |
| Artificial Analysis Cyber Index (successes) | 33 | Mistral AI, Mistral Large 4 (preview)checked 2026-10-15 | - |
| Semi-Synthetic Agentic Envs | 0.997 | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25; printed on 1 more page | - |
| ExploitBench (June-Aug 2026) | 39% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| Advanced Cybersecurity Completion Rate | 3.5% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | - |
| Cyber red-teaming | 7.4% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | Model as printed: gpt-6 Astra without Trusted Access |
| 76.9% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | Model as printed: gpt-6 Astra with Daybreak Blue | |
| Proof-of-concept exploit creation | 92% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | Model as printed: gpt-6 Astra with Daybreak Blue |
| 2.4% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | Model as printed: gpt-6 Astra without Trusted Access | |
| SRE-Bench | 88% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| Vulnerability discovery and analysis | 100% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | Model as printed: gpt-6 Astra with Daybreak Blue |
| 66.7% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | Model as printed: gpt-6 Astra without Trusted Access | |
| Vulnerability patching | 44.4% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | Model as printed: gpt-6 Astra without Trusted Access |
| 100% | OpenAI, GPT-6 Astra System Cardchecked 2026-10-25 | Model as printed: gpt-6 Astra with Daybreak Blue |
General Knowledge and Reasoning
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| Humanity's Last Exam (no tools) | 53.6% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: none; Fallback: with fallback |
| Humanity's Last Exam (with tools) | 57.2% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 3 more pages | Tools: enabled; exact tools in source |
| SimpleQA Verified | 75.6% | Epoch AI, Epoch AI Benchmarking Hubchecked 2026-10-30 | Effort: max |
| ARC-AGI-2 | 95% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| ARC-AGI-1 | 98.5% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| Humanity's Last Exam Diamond (no tools) | 65.8% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: none; Fallback: with fallback |
| ARC-AGI-3 | 99.9% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| Kimi Hallu Bench Hard w/ tools (lower is better)lower is better | 5.5% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: with tools; Fallback: with fallback |
| Kimi Hallu Bench w/o tools (lower is better)lower is better | 4.2% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Tools: none; Fallback: with fallback |
| Internal hallucination benchmarklower is better | 4.2% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
Tool Use and Workflows
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| Agents' Last Exam (pass rate) | 34.2% | Google, Gemini 4 Argonchecked 2026-10-29; printed on 1 more page | - |
| Agents' Last Exam (score) | 59.3% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| Agents' Last Exam (CLI) | 38.8% | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Dataset revision: v1.1; Fallback: with fallback |
| Agents' Last Exam v1.1 (score) | 73.1% | Google, Gemini 4 Carbonchecked 2026-10-29 | - |
Web Search and Research
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| BrowseComp | 91.5% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 1 more page | - |
Healthcare
Biosecurity
Safety and Alignment
Engineering and Design
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| BenchCAD | 95.9% | OpenAI, GPT-6 Astrachecked 2026-10-14; printed on 1 more page | - |
| Blueprint-Bench 2 (normalized score) | 49.7% | Google, Gemini 4 Carbonchecked 2026-10-29 | - |
| EEBench reward | 69.3% | xAI, Grok 4.7 model cardchecked 2026-10-14 | Effort: max |
| Design Bench (Internal, Elo) | 1076 | Moonshot AI, Introducing Kimi K3.1 Prochecked 2026-10-24 | Effort: max; Fallback: with fallback |
| Internal Design Tasks | 50% | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
Preference and Composite Rankings
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| Artificial Analysis Intelligence Index v4.1.1 | 61.2 | OpenAI, GPT-6 Astrachecked 2026-10-14 | - |
| Artificial Analysis Intelligence Index v5.0 | 57 | Google, Gemini 4 Carbonchecked 2026-10-29 | - |
| Artificial Analysis Intelligence Index v4.3 | 53 | Meta, Meta Muse Wonder model cardchecked 2026-10-19 | Effort: max |
| Personal Superintelligence Index (Meta internal) | 54.5 | Meta, Meta Muse Wonder model cardchecked 2026-10-19 | Effort: max |
Price
API list prices in US dollars per one million tokens, as printed on the developer's pricing page on the day checked. Cost per task is measured by Vals AI.
| Price | Value | Printed by | Setup as printed |
|---|---|---|---|
| Input | $10 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $20 per 1M tokens above 272,000 tokens |
| Cached input | $1 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $2 per 1M tokens above 272,000 tokens |
| Cache write | $12.5 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $25 per 1M tokens above 272,000 tokens |
| Output | $50 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $75 per 1M tokens above 272,000 tokens |
| Input, batch | $5 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $10 per 1M tokens above 272,000 tokens |
| Cached input, batch | $0.5 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $1 per 1M tokens above 272,000 tokens |
| Cache write, batch | $6.25 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $12.5 per 1M tokens above 272,000 tokens |
| Output, batch | $25 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $37.5 per 1M tokens above 272,000 tokens |
| Input, flex | $5 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $10 per 1M tokens above 272,000 tokens |
| Cached input, flex | $0.5 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $1 per 1M tokens above 272,000 tokens |
| Cache write, flex | $6.25 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $12.5 per 1M tokens above 272,000 tokens |
| Output, flex | $25 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $37.5 per 1M tokens above 272,000 tokens |
| Input, fast | $20 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $40 per 1M tokens above 272,000 tokens |
| Cached input, fast | $2 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $4 per 1M tokens above 272,000 tokens |
| Cache write, fast | $25 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $50 per 1M tokens above 272,000 tokens |
| Output, fast | $100 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $150 per 1M tokens above 272,000 tokens |
| Input, ultrafast | $60 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $120 per 1M tokens above 272,000 tokens |
| Cached input, ultrafast | $6 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $12 per 1M tokens above 272,000 tokens |
| Cache write, ultrafast | $75 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $150 per 1M tokens above 272,000 tokens |
| Output, ultrafast | $300 | OpenAI, OpenAI API pricingchecked 2026-10-15 | Context: up to 272,000 tokens; Long context: $450 per 1M tokens above 272,000 tokens |
| Cost per taskVals AI, USD, lower is better | $38.51 | Vals AI, Vibe Code Bench v1.1 leaderboardchecked 2026-10-27 | Harness: OpenHands |
Side by side
More from OpenAI
- GPT-6.1 Terra
- GPT-6.1 Sol
- GPT-6 Luna
- GPT-6 Sol
- GPT-5.6 Luna
- GPT-5.6 Sol
- GPT-5.6 Terra
- GPT-5.5
- GPT-5.5 Pro
- GPT-5.4 mini
- GPT-5.4 nano
- GPT-5.4
- GPT-5.4 Pro
- GPT-5.3-Codex
- GPT-5.2-Codex
- GPT-5.2
- GPT-5.2 Instant
- GPT-5.2 Pro
- GPT-5.1-Codex-Max
- GPT-5.1
- GPT-5.1 Instant
- GPT-5-Codex
- GPT-5
- GPT-5 mini
- GPT-5 Pro
- gpt-oss-120b
- gpt-oss-20b
- ChatGPT agent
- codex-1
- o3
- OpenAI o4-mini
- GPT-4.1
- GPT-4.5
- Deep research
- o3-mini
- o1
- o1-mini
- GPT-4o
- GPT-5.5 Instant
- gpt-5.6-cyber
Every number here is read from the same catalog file as the benchmark table, data version 2026103001. Where one page printed the model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.