BlogLLM Benchmark Scores as Published, Side by Side
Mistral Medium 3.1: Benchmark Scores as Published
Every result the catalog holds for Mistral Medium 3.1: 10 tests in 7 areas, from 2 pages. All were printed by Mistral AI itself. Each number is as printed; nothing is ranked or rescored.
Developer Mistral AIReleased 2025-08-12Data version 2026103001, as of 2026-10-30
Science and Math
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPQA Diamond | 65.7% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | - |
Multimodal Understanding
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| MMMU-Pro | 43.8% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | - |
General Knowledge and Reasoning
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| MMLU-Pro | 76.8% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | - |
Tool Use and Workflows
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| τ³-bench Banking | 5.7% | Mistral AI, Mistral Medium 3.5 model cardchecked 2026-10-15 | Effort: max; Harness: gpt-5.2 user simulator (low), 4 trials; terminal- or embedding-based agentic search retrieval, highest score |
| τ³-bench Airline | 41.5% | Mistral AI, Mistral Medium 3.5 model cardchecked 2026-10-15 | Effort: max; Harness: gpt-5.2 user simulator (low), 4 trials; Sierra-reported for Claude Sonnet 4.5 and Qwen3.5 |
| τ³-bench Retail | 64.3% | Mistral AI, Mistral Medium 3.5 model cardchecked 2026-10-15 | Effort: max; Harness: gpt-5.2 user simulator (low), 4 trials; Sierra-reported for Claude Sonnet 4.5 and Qwen3.5 |
| τ³-bench Telecom | 46.9% | Mistral AI, Mistral Medium 3.5 model cardchecked 2026-10-15 | Effort: max; Harness: gpt-5.2 user simulator (low), 4 trials; Sierra-reported for Claude Sonnet 4.5 and Qwen3.5 |
Web Search and Research
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| BrowseComp | 7.8% | Mistral AI, Mistral Medium 3.5 model cardchecked 2026-10-15 | Effort: max; Harness: context management, discard-all at 100k tokens (Mistral) |
Instruction Following
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| IFBench | 40.8 | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | - |
Preference and Composite Rankings
| Test | Value | Printed by | Setup as printed |
|---|---|---|---|
| Arena-Hard win rate (GPT-4-1106 judge) | 67.3% | Mistral AI, Mistral Small 4 model cardchecked 2026-10-15 | - |
More from Mistral AI
Every number here is read from the same catalog file as the benchmark table, data version 2026103001. Where one page printed the model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.