BlogLLM Benchmark Scores as Published, Side by Side
AIME: Results as Published
Every result the catalog holds for AIME: 100 models from 14 developers, read from 34 pages. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area Science and MathMaintainer Mathematical Association of America (MAA)Data version 2026103001, as of 2026-10-30
The AIME (American Invitational Mathematics Examination) is the second round of the MAA competition series for high-school students in the United States: 15 problems in three hours, each with an integer answer from 0 to 999. As a benchmark, a given year's two exams (30 problems) are posed to a model, and the score is the share answered correctly; publishers state whether tools such as a Python interpreter were allowed.
AIME 2025
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.2OpenAI, released 2025-12-11 | 100% | OpenAI, Introducing GPT-5.2checked 2026-10-15; printed on 2 more pages | Effort: xhigh; Tools: none |
| GPT-5.2 ProOpenAI, released 2025-12-11 | 100% | OpenAI, Introducing GPT-5.2checked 2026-10-15 | Effort: xhigh; Tools: none |
| GPT-5.1OpenAI, released 2025-11-12 | 94% | OpenAI, Introducing GPT-5.1 for developerschecked 2026-10-15; printed on 2 more pages | Effort: high; Tools: none |
| GPT-5OpenAI, released 2025-08-07 | 61.9% | OpenAI, Introducing GPT-5checked 2026-10-15; printed on 6 more pages | Effort: none; Tools: no tools |
| 94.6% | OpenAI, Introducing GPT-5checked 2026-10-15 | Effort: high; Tools: no tools | |
| 71% | OpenAI, Introducing GPT-5checked 2026-10-15 | Effort: none; Tools: python | |
| 99.6% | OpenAI, Introducing GPT-5checked 2026-10-15 | Effort: high; Tools: python | |
| GPT-5 ProOpenAI, released 2025-08-07 | 96.7% | OpenAI, Introducing GPT-5checked 2026-10-15; printed on 1 more page | Effort: high; Tools: no tools |
| 100% | OpenAI, Introducing GPT-5checked 2026-10-15 | Effort: high; Tools: python | |
| gpt-oss-120bOpenAI, released 2025-08-05 | 45% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: instruct; Harness: Score vs. Output Length chart (Mistral comparison run) |
| 89% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: reasoning; Harness: Score vs. Output Length chart (Mistral comparison run) | |
| gpt-oss-20bOpenAI, released 2025-08-05 | 91.7% | NVIDIA, NVIDIA Nemotron 3 Nano 30B-A3B model cardchecked 2026-10-15 | Tools: none |
| 98.7% | NVIDIA, NVIDIA Nemotron 3 Nano 30B-A3B model cardchecked 2026-10-15 | Tools: with tools | |
| o3OpenAI, released 2025-04-16 | 98.4% | OpenAI, Introducing GPT-5checked 2026-10-15; printed on 2 more pages | Effort: high; Tools: python |
| 88.9% | OpenAI, Introducing GPT-5checked 2026-10-15 | Effort: high; Tools: no tools | |
| GPT-4.1OpenAI, released 2025-04-14 | 37% | Moonshot AI, Kimi K2 release pagechecked 2026-10-15 | - |
| o3-miniOpenAI, released 2025-01-31 | 76.7% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | Effort: medium |
| GPT-4oOpenAI, released 2024-05-13 | 42.1% | OpenAI, Introducing GPT-5checked 2026-10-15 | Tools: python |
| Claude Sonnet 4.6Anthropic, released 2026-02-17 | 95.6% | Microsoft, Microsoft AI: Introducing MAI-Thinking-1checked 2026-10-19; printed on 1 more page | - |
| Claude Opus 4.6Anthropic, released 2026-02-05 | 99.8% | Microsoft, Microsoft AI: Introducing MAI-Thinking-1checked 2026-10-19; printed on 1 more page | - |
| Claude Opus 4.5Anthropic, released 2025-11-24 | 91% | MiniMax, MiniMax-M2.5 model cardchecked 2026-10-15; printed on 1 more page | - |
| Claude Haiku 4.5Anthropic, released 2025-10-15 | 96.3% | Anthropic, Claude Haiku 4.5checked 2026-10-15; printed on 1 more page | Effort: thinking budget 128K; Tools: Python |
| 80.7% | Anthropic, Claude Haiku 4.5checked 2026-10-15 | Effort: thinking budget 128K; Tools: none | |
| Claude Sonnet 4.5Anthropic, released 2025-09-29 | 87% | Anthropic, Claude Sonnet 4.5checked 2026-10-15; printed on 6 more pages | Tools: none |
| 100% | Anthropic, Claude Sonnet 4.5checked 2026-10-15 | Effort: 64K reasoning tokens; Tools: Python | |
| Claude Opus 4.1Anthropic, released 2025-08-05 | 78% | Anthropic, Claude Opus 4.1checked 2026-10-15; printed on 1 more page | Effort: extended thinking, up to 64K |
| Claude Opus 4Anthropic, released 2025-05-22 | 90% | Anthropic, Claude Opus 4 and Claude Sonnet 4checked 2026-10-15; printed on 2 more pages | Effort: extended thinking, up to 64K; Attempt budget: parallel test-time compute: multiple samples, the best selected by an internal scoring model |
| 75.5% | Anthropic, Claude Opus 4 and Claude Sonnet 4checked 2026-10-15 | Effort: extended thinking, up to 64K | |
| Claude Sonnet 4Anthropic, released 2025-05-22 | 70.5% | Anthropic, Claude Sonnet 4.5checked 2026-10-15; printed on 4 more pages | Tools: none |
| Claude 3.7 SonnetAnthropic, released 2025-02-24 | 54.8% | Anthropic, Claude Opus 4 and Claude Sonnet 4checked 2026-10-15 | - |
| Gemini 3 ProGoogle, released 2025-11-18 | 95% | DeepSeek, DeepSeek-V3.2 model cardchecked 2026-10-15; printed on 3 more pages | - |
| Gemini 2.5 FlashGoogle, released 2025-05-20 | 72% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15; printed on 1 more page | Effort: thinking |
| Gemini 2.5 ProGoogle, released 2025-03-25 | 83% | Anthropic, Claude Opus 4 and Claude Sonnet 4checked 2026-10-15; printed on 3 more pages | - |
| Grok 4xAI, released 2025-07-09 | 98.8% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Tools: enabled; python |
| 91.7% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Tools: none | |
| 100% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Harness: heavy mode: 8 parallel trajectories, aggregated; Tools: enabled; python | |
| DeepSeek-V3.2DeepSeek, released 2025-12-01 | 93.1% | DeepSeek, DeepSeek-V3.2 model cardchecked 2026-10-15; printed on 3 more pages | Effort: thinking |
| DeepSeek-V3.2-SpecialeDeepSeek, released 2025-12-01 | 96% | DeepSeek, DeepSeek-V3.2 model cardchecked 2026-10-15 | - |
| DeepSeek-V3.2-ExpDeepSeek, released 2025-09-29 | 89.3% | DeepSeek, DeepSeek-V3.2-Exp model cardchecked 2026-10-15; printed on 1 more page | Tools: none |
| DeepSeek-V3.1-TerminusDeepSeek, released 2025-09-22 | 88.4% | DeepSeek, DeepSeek-V3.2-Exp model cardchecked 2026-10-15 | Tools: none |
| DeepSeek-R1-0528DeepSeek, released 2025-05-28 | 87.5% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-0528-Qwen3-8BDeepSeek, released 2025-05-28 | 76.3% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
| DeepSeek-V3-0324DeepSeek, released 2025-03-24 | 46.7% | Moonshot AI, Kimi K2 release pagechecked 2026-10-15 | - |
| DeepSeek-R1DeepSeek, released 2025-01-20 | 70% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15; printed on 1 more page | - |
| DeepSeek-V3DeepSeek, released 2024-12-26 | 28.8% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: pass@1 |
| Qwen3.5-122B-A10BAlibaba, released 2026-02-24 | 93% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: reasoning; Harness: Score vs. Output Length chart (Mistral comparison run) |
| 80% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: instruct; Harness: Score vs. Output Length chart (Mistral comparison run) | |
| Qwen3.5-397B-A17BAlibaba, released 2026-02-16 | 83.1% | Mistral AI, Mistral Medium 3.5 model cardchecked 2026-10-15 | Effort: max |
| Qwen3-Next-80B-A3BAlibaba, released 2025-09-11 | 65% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: instruct; Harness: Score vs. Output Length chart (Mistral comparison run) |
| 88% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: reasoning; Harness: Score vs. Output Length chart (Mistral comparison run) | |
| Qwen3-30B-A3B-Thinking-2507Alibaba, released 2025-07-30 | 85% | NVIDIA, NVIDIA Nemotron 3 Nano 30B-A3B model cardchecked 2026-10-15 | Effort: thinking; Tools: none |
| Qwen3-235B-A22BAlibaba, released 2025-04-29 | 81.5% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3-32BAlibaba, released 2025-04-29 | 72.9% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
| Qwen3-8BAlibaba, released 2025-04-29 | 67.3% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
| Kimi K2.5Moonshot AI, released 2026-01-27 | 96.1% | Moonshot AI, Kimi K2.5 model cardchecked 2026-10-15; printed on 1 more page | - |
| Kimi K2 ThinkingMoonshot AI, released 2025-11-06 | 94.5% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15; printed on 1 more page | Effort: thinking; Tools: none |
| 100% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Effort: thinking; Harness: heavy mode: 8 parallel trajectories, aggregated; Tools: enabled; python | |
| 99.1% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Effort: thinking; Tools: enabled; python | |
| Kimi K2 Instruct 0905Moonshot AI, released 2025-09-05 | 51% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Tools: none |
| 75.2% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Tools: enabled; python | |
| Kimi K2 InstructMoonshot AI, released 2025-07-11 | 49.5% | Moonshot AI, Kimi K2 release pagechecked 2026-10-15 | - |
| GLM-5Z.ai, released 2026-02-12 | 87.1% | Mistral AI, Mistral Medium 3.5 model cardchecked 2026-10-15 | Effort: max |
| GLM-4.7Z.ai, released 2025-12-22 | 95.7% | Z.ai, GLM-4.7 model cardchecked 2026-10-15 | - |
| GLM-4.6Z.ai, released 2025-09-30 | 93.9% | Z.ai, GLM-4.7 model cardchecked 2026-10-15 | - |
| Mistral Medium 3.5Mistral AI, released 2026-05-22 | 86.3% | Mistral AI, Mistral Medium 3.5 model cardchecked 2026-10-15 | Effort: max |
| Mistral Small 4Mistral AI, released 2026-03-16 | 84% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: high; Harness: Score vs. Output Length chart (Mistral comparison run) |
| 83.8% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: high | |
| 36% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | Effort: none; Harness: Score vs. Output Length chart (Mistral comparison run) | |
| Magistral Medium 1.2Mistral AI, released 2025-09-17 | 84.4% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | - |
| Magistral Small 1.2Mistral AI, released 2025-09-17 | 80.2% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | - |
| Magistral MediumMistral AI, released 2025-06-10 | 64.9% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: pass@1 |
| Mistral Medium 3Mistral AI, released 2025-05-07 | 21.2% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: pass@1 |
| MiniMax M2.5MiniMax, released 2026-02-12 | 86.3% | MiniMax, MiniMax-M2.5 model cardchecked 2026-10-15 | - |
| MiniMax M2.1MiniMax, released 2025-12-20 | 83% | MiniMax, MiniMax-M2.5 model cardchecked 2026-10-15 | - |
| Nemotron 3 Nano 30B-A3BNVIDIA, released 2025-12-15 | 99.2% | NVIDIA, NVIDIA Nemotron 3 Nano 30B-A3B model cardchecked 2026-10-15 | Tools: with tools |
| 89.1% | NVIDIA, NVIDIA Nemotron 3 Nano 30B-A3B model cardchecked 2026-10-15 | Tools: none | |
| MAI-Thinking-1Microsoft, released 2026-08-12 | 97% | Microsoft, Microsoft AI: Introducing MAI-Thinking-1checked 2026-10-19; printed on 1 more page | - |
| Phi-4-reasoning-plusMicrosoft, released 2025-04-30 | 78% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
AIME 2026
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.6 SolOpenAI, released 2026-07-09 | 99.9% | Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15 | Effort: xhigh |
| GPT-5.5OpenAI, released 2026-04-23 | 98.3% | Z.ai, GLM-5.2 model cardchecked 2026-10-15 | - |
| GPT-5.4OpenAI, released 2026-03-05 | 99.2% | Moonshot AI, Kimi K2.6 model cardchecked 2026-10-15; printed on 1 more page | Effort: xhigh |
| GPT-5.2OpenAI, released 2025-12-11 | 96.7% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15 | - |
| Claude Opus 4.8Anthropic, released 2026-05-28 | 95.7% | Z.ai, GLM-5.2 model cardchecked 2026-10-15 | - |
| Claude Opus 4.6Anthropic, released 2026-02-05 | 96.7% | Moonshot AI, Kimi K2.6 model cardchecked 2026-10-15; printed on 1 more page | Effort: max |
| Claude Opus 4.5Anthropic, released 2025-11-24 | 93.3% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15; printed on 2 more pages | - |
| Gemma 4 12BGoogle, released 2026-04-02 | 77.5% | Google, Gemma 4 model cardchecked 2026-10-15 | Tools: none |
| Gemma 4 26B A4BGoogle, released 2026-04-02 | 88.3% | Google, Gemma 4 model cardchecked 2026-10-15; printed on 1 more page | Tools: none |
| Gemma 4 31BGoogle, released 2026-04-02 | 89.2% | Google, Gemma 4 model cardchecked 2026-10-15; printed on 2 more pages | Tools: none |
| Gemma 4 E2BGoogle, released 2026-04-02 | 37.5% | Google, Gemma 4 model cardchecked 2026-10-15 | Tools: none |
| Gemma 4 E4BGoogle, released 2026-04-02 | 42.5% | Google, Gemma 4 model cardchecked 2026-10-15 | Tools: none |
| Gemini 3.1 Pro PreviewGoogle, released 2026-02-19 | 98.3% | Moonshot AI, Kimi K2.6 model cardchecked 2026-10-15; printed on 3 more pages | Effort: high |
| Gemini 3 ProGoogle, released 2025-11-18 | 90.6% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15; printed on 1 more page | - |
| Gemma 3 27BGoogle, released 2025-03-12 | 20.8% | Google, Gemma 4 model cardchecked 2026-10-15 | Effort: no think; Tools: none |
| DeepSeek-V4-Pro (Preview)DeepSeek, released 2026-04-24 | 96.7% | Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15; printed on 1 more page | - |
| DeepSeek-V3.2DeepSeek, released 2025-12-01 | 92.7% | Z.ai, GLM-5 model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.6-27BAlibaba, released 2026-04-22 | 94.1% | Alibaba, Qwen3.6-27B model cardchecked 2026-10-15 | - |
| Qwen3.6-35B-A3BAlibaba, released 2026-04-16 | 92.7% | Alibaba, Qwen3.6-35B-A3B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.6-PlusAlibaba, released 2026-04-02 | 95.1% | Z.ai, GLM-5.1 model cardchecked 2026-10-15 | - |
| Qwen3.5-27BAlibaba, released 2026-02-24 | 92.6% | Alibaba, Qwen3.6-27B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.5-35B-A3BAlibaba, released 2026-02-24 | 91% | Alibaba, Qwen3.6-35B-A3B model cardchecked 2026-10-15 | - |
| Qwen3.5-397B-A17BAlibaba, released 2026-02-16 | 91.3% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3-Max-ThinkingAlibaba | 93.3% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15 | - |
| Qwen3.7-MaxAlibaba | 97% | Z.ai, GLM-5.2 model cardchecked 2026-10-15 | - |
| Kimi K2.6Moonshot AI, released 2026-04-20 | 96.4% | Moonshot AI, Kimi K2.6 model cardchecked 2026-10-15; printed on 2 more pages | - |
| Kimi K2.5Moonshot AI, released 2026-01-27 | 95.8% | Moonshot AI, Kimi K2.6 model cardchecked 2026-10-15; printed on 4 more pages | - |
| GLM-5.2Z.ai, released 2026-06-16 | 99.2% | Z.ai, GLM-5.2 model cardchecked 2026-10-15; printed on 1 more page | - |
| GLM-5.1Z.ai, released 2026-04-07 | 95.3% | Z.ai, GLM-5.1 model cardchecked 2026-10-15; printed on 2 more pages | - |
| GLM-5Z.ai, released 2026-02-12 | 92.7% | Z.ai, GLM-5 model cardchecked 2026-10-15; printed on 1 more page | - |
| GLM-4.7Z.ai, released 2025-12-22 | 92.9% | Z.ai, GLM-5 model cardchecked 2026-10-15 | - |
| MiniMax M2.7MiniMax, released 2026-03-18 | 89.8% | Z.ai, GLM-5.1 model cardchecked 2026-10-15 | - |
| Nemotron 3 UltraNVIDIA, released 2026-06-04 | 94.2% | Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15 | - |
| MAI-Thinking-1Microsoft, released 2026-08-12 | 94.5% | Microsoft, Microsoft AI: Introducing MAI-Thinking-1checked 2026-10-19; printed on 1 more page | - |
| InklingThinking Machines, released 2026-07-14 | 97.1% | Thinking Machines, Thinking Machines Inkling model cardchecked 2026-10-15 | Effort: 0.99 |
AIME 2024
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-4.1OpenAI, released 2025-04-14 | 46.5% | Moonshot AI, Kimi K2 release pagechecked 2026-10-15 | - |
| GPT-4.5OpenAI, released 2025-02-27 | 36.7% | DeepSeek, DeepSeek-V3-0324 model cardchecked 2026-10-15 | - |
| o3-miniOpenAI, released 2025-01-31 | 87.3% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15; printed on 1 more page | Effort: high |
| o1OpenAI, released 2024-12-17 | 83.3% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15; printed on 1 more page | Attempt budget: majority voting, N=64 samples (note 3) |
| 79.2% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15 | - | |
| o1-miniOpenAI, released 2024-09-12 | 63.6% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | Model as printed: o1-mini |
| 63.6% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | Model as printed: OpenAI o1-mini | |
| GPT-4oOpenAI, released 2024-05-13 | 9.3% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | Model as printed: GPT-4o-0513 |
| 9.3% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | Model as printed: GPT-4o 0513 | |
| Claude Opus 4Anthropic, released 2025-05-22 | 48.2% | Moonshot AI, Kimi K2 release pagechecked 2026-10-15 | Effort: no extended thinking |
| Claude Sonnet 4Anthropic, released 2025-05-22 | 43.4% | Moonshot AI, Kimi K2 release pagechecked 2026-10-15 | Effort: no extended thinking |
| Claude 3.7 SonnetAnthropic, released 2025-02-24 | 80% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15; printed on 1 more page | Effort: extended thinking, 64K; Attempt budget: parallel test-time compute: multiple samples, the best selected by an internal scoring model |
| 61.3% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15 | Effort: extended thinking, 64K | |
| 23.3% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15 | Effort: none | |
| Claude 3.5 SonnetAnthropic, released 2024-10-22 | 16% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15; printed on 1 more page | - |
| Gemini 2.5 FlashGoogle, released 2025-05-20 | 82.3% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15; printed on 1 more page | Effort: thinking |
| Grok 3xAI, released 2025-02-17 | 83.9% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15 | Effort: extended thinking |
| 93.3% | Anthropic, Claude 3.7 Sonnetchecked 2026-10-15 | Effort: extended thinking; Attempt budget: majority voting, N=64 samples (note 3) | |
| DeepSeek-R1-0528DeepSeek, released 2025-05-28 | 91.4% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-0528-Qwen3-8BDeepSeek, released 2025-05-28 | 86% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
| DeepSeek-V3-0324DeepSeek, released 2025-03-24 | 59.4% | DeepSeek, DeepSeek-V3-0324 model cardchecked 2026-10-15; printed on 1 more page | - |
| DeepSeek-R1DeepSeek, released 2025-01-20 | 79.8% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15; printed on 3 more pages | - |
| DeepSeek-R1-Distill-Llama-70BDeepSeek, released 2025-01-20 | 70% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Llama-8BDeepSeek, released 2025-01-20 | 50.4% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Qwen-1.5BDeepSeek, released 2025-01-20 | 28.9% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Qwen-14BDeepSeek, released 2025-01-20 | 69.7% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Qwen-32BDeepSeek, released 2025-01-20 | 72.6% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Qwen-7BDeepSeek, released 2025-01-20 | 55.5% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-V3DeepSeek, released 2024-12-26 | 39.6% | DeepSeek, DeepSeek-V3-0324 model cardchecked 2026-10-15; printed on 2 more pages | - |
| Qwen3-235B-A22BAlibaba, released 2025-04-29 | 85.7% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3-32BAlibaba, released 2025-04-29 | 81.4% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
| Qwen3-8BAlibaba, released 2025-04-29 | 76% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
| QwQ-32B-PreviewAlibaba, released 2024-11-27 | 44% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| Qwen-MaxAlibaba | 26.7% | DeepSeek, DeepSeek-V3-0324 model cardchecked 2026-10-15 | - |
| Kimi K2 InstructMoonshot AI, released 2025-07-11 | 69.6% | Moonshot AI, Kimi K2 release pagechecked 2026-10-15 | - |
| Magistral MediumMistral AI, released 2025-06-10 | 73.6% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: pass@1 |
| Magistral SmallMistral AI, released 2025-06-10 | 70.7% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: pass@1 |
| Mistral Medium 3Mistral AI, released 2025-05-07 | 26.8% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: pass@1 |
| Phi-4-reasoning-plusMicrosoft, released 2025-04-30 | 81.3% | DeepSeek, DeepSeek-R1-0528 model cardchecked 2026-10-15 | - |
AIME 2024 cons@64 (majority vote over 64 samples)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| o1-miniOpenAI, released 2024-09-12 | 80% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| GPT-4oOpenAI, released 2024-05-13 | 13.4% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| Claude 3.5 SonnetAnthropic, released 2024-10-22 | 26.7% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Llama-70BDeepSeek, released 2025-01-20 | 86.7% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Llama-8BDeepSeek, released 2025-01-20 | 80% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Qwen-1.5BDeepSeek, released 2025-01-20 | 52.7% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Qwen-14BDeepSeek, released 2025-01-20 | 80% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Qwen-32BDeepSeek, released 2025-01-20 | 83.3% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| DeepSeek-R1-Distill-Qwen-7BDeepSeek, released 2025-01-20 | 83.3% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| QwQ-32B-PreviewAlibaba, released 2024-11-27 | 60% | DeepSeek, DeepSeek-R1 model cardchecked 2026-10-15 | - |
| Magistral MediumMistral AI, released 2025-06-10 | 90% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@64 |
| Magistral SmallMistral AI, released 2025-06-10 | 83.3% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@64 |
| Mistral Medium 3Mistral AI, released 2025-05-07 | 43.3% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@64 |
AIME 2024 and 2025
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| Kimi K2 BaseMoonshot AI, released 2025-07-11 | 31.6% | Xiaomi, Xiaomi MiMo-V2.5-Pro model cardchecked 2026-10-15 | Harness: 2-shot |
| MiMo-V2.5 BaseXiaomi, released 2026-04-27 | 36.9% | Xiaomi, Xiaomi MiMo-V2.5-Pro model cardchecked 2026-10-15 | Harness: 2-shot |
| MiMo-V2.5-Pro BaseXiaomi, released 2026-04-27 | 37.3% | Xiaomi, Xiaomi MiMo-V2.5-Pro model cardchecked 2026-10-15 | Harness: 2-shot |
AIME 2024 maj@4 (majority vote over 4 samples)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| Magistral MediumMistral AI, released 2025-06-10 | 81.3% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@4 |
| Mistral Medium 3Mistral AI, released 2025-05-07 | 34.4% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@4 |
AIME 2025 maj@4 (majority vote over 4 samples)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| Magistral MediumMistral AI, released 2025-06-10 | 72.1% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@4 |
| Mistral Medium 3Mistral AI, released 2025-05-07 | 25.8% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@4 |
AIME 2025 maj@64 (majority vote over 64 samples)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| Magistral MediumMistral AI, released 2025-06-10 | 83.3% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@64 |
| Mistral Medium 3Mistral AI, released 2025-05-07 | 30% | Mistral AI, Announcing Magistralchecked 2026-10-15 | Attempt budget: maj@64 |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.