BlogLLM Benchmark Scores as Published, Side by Side
PaperBench: Results as Published
Every result the catalog holds for PaperBench: 14 models from 7 developers, read from 3 pages. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area ML EngineeringMaintainer OpenAI (Giulio Starace and 12 co-authors)Introduced 2025Data version 2026103001, as of 2026-10-30
PaperBench tests whether an AI agent can reproduce the results of recent machine learning research from scratch. The agent gets 20 papers from ICML 2024 and must write the code and run the experiments without the original code. Grading uses 8,316 rubric items written with the paper authors, and an automated AI judge scores each attempt.
PaperBench
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.6 SolOpenAI, released 2026-07-09 | 90.5 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | Effort: max |
| GPT-5.5OpenAI, released 2026-04-23 | 57.5 | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| GPT-5.2OpenAI, released 2025-12-11 | 63.7 | Moonshot AI, Kimi K2.5 model cardchecked 2026-10-15 | Effort: xhigh |
| Claude Fable 5Anthropic, released 2026-06-09 | 88.8 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| Claude Opus 4.8Anthropic, released 2026-05-28 | 80.3 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| Claude Opus 4.7Anthropic, released 2026-04-16 | 58.5 | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| Claude Opus 4.5Anthropic, released 2025-11-24 | 72.9 | Moonshot AI, Kimi K2.5 model cardchecked 2026-10-15 | - |
| Gemini 3.1 Pro PreviewGoogle, released 2026-02-19 | 46.7 | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| DeepSeek-V3.2DeepSeek, released 2025-12-01 | 47.1 | Moonshot AI, Kimi K2.5 model cardchecked 2026-10-15 | - |
| Qwen3.8-MaxAlibaba, released 2026-08-03 | 93 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| Qwen3.7-MaxAlibaba | 64.8 | Alibaba, Qwen3.8-Max comparisonchecked 2026-10-14 | - |
| Kimi K2.5Moonshot AI, released 2026-01-27 | 63.5 | Moonshot AI, Kimi K2.5 model cardchecked 2026-10-15 | - |
| MiniMax M3MiniMax, released 2026-06-01 | 52.6 | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| MiniMax M2.7MiniMax, released 2026-03-18 | 30.6 | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.