BlogLLM Benchmark Scores as Published, Side by Side
ChartQA: Results as Published
Every result the catalog holds for ChartQA: 16 models from 8 developers, read from 3 pages. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area Multimodal UnderstandingMaintainer Ahmed Masry and co-authorsIntroduced 2022Data version 2026103001, as of 2026-10-30
ChartQA asks questions about real chart images that need visual reading plus arithmetic or logical steps. It has 9.6K questions written by people and 23.1K questions generated from human-written chart summaries. A numeric answer counts as correct when it is within 5 percent of the gold value. A non-numeric answer must match exactly.
ChartQA
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-4oOpenAI, released 2024-05-13 | 86% | Mistral AI, Medium is the new largechecked 2026-10-15 | - |
| Claude 3.7 SonnetAnthropic, released 2025-02-24 | 76.3% | Mistral AI, Medium is the new largechecked 2026-10-15 | - |
| Gemma 3 12BGoogle, released 2025-03-12 | 39% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Harness: VLMEvalKit |
| Llama 4 MaverickMeta, released 2025-04-05 | 90% | Meta, Meta Llama 4 model cardchecked 2026-10-15; printed on 1 more page | - |
| Llama 4 Maverick (pre-trained)Meta, released 2025-04-05 | 85.3% | Meta, Meta Llama 4 model cardchecked 2026-10-15 | - |
| Llama 4 ScoutMeta, released 2025-04-05 | 88.8% | Meta, Meta Llama 4 model cardchecked 2026-10-15 | - |
| Llama 4 Scout (pre-trained)Meta, released 2025-04-05 | 83.4% | Meta, Meta Llama 4 model cardchecked 2026-10-15 | - |
| Qwen3-VL-32B-InstructAlibaba, released 2025-10-21 | 84.3% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Harness: VLMEvalKit; Context: 4096 |
| 84% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Harness: VLMEvalKit; Context: 32768 | |
| Qwen3-VL-32B-ThinkingAlibaba, released 2025-10-21 | 78.5% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Effort: thinking; Harness: VLMEvalKit; Context: 4096 |
| 79.1% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Effort: thinking; Harness: VLMEvalKit; Context: 40960 | |
| Qwen3-VL-8B-InstructAlibaba, released 2025-10-15 | 83.1% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Harness: VLMEvalKit; Context: 4096 |
| 83.2% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Harness: VLMEvalKit; Context: 32768 | |
| Qwen3-VL-8B-ThinkingAlibaba, released 2025-10-15 | 78% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Effort: thinking; Harness: VLMEvalKit; Context: 4096 |
| 78.6% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Effort: thinking; Harness: VLMEvalKit; Context: 40960 | |
| Kimi-VL-A3B-InstructMoonshot AI, released 2025-04-10 | 87% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Harness: VLMEvalKit |
| Kimi-VL-A3B-ThinkingMoonshot AI, released 2025-04-10 | 73.3% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Effort: thinking; Harness: VLMEvalKit |
| Mistral Medium 3Mistral AI, released 2025-05-07 | 82.6% | Mistral AI, Medium is the new largechecked 2026-10-15 | - |
| Phi-4-reasoning-vision-15BMicrosoft, released 2026-03-04 | 82.9% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Effort: force thinking; Harness: VLMEvalKit |
| 83.3% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Harness: VLMEvalKit | |
| 76.5% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Effort: force nothink; Harness: VLMEvalKit | |
| Phi-4-multimodal-instructMicrosoft, released 2025-02-26 | 23.5% | Microsoft, Microsoft Phi-4-reasoning-vision-15B model cardchecked 2026-10-15 | Harness: VLMEvalKit |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.