BlogLLM Benchmark Scores as Published, Side by Side
SimpleVQA: Results as Published
Every result the catalog holds for SimpleVQA: 19 models from 7 developers, read from 6 pages. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area Multimodal UnderstandingMaintainer Xianfu Cheng and co-authors (Beihang University, with Baidu and M-A-P among the listed affiliations)Introduced 2025Data version 2026103001, as of 2026-10-30
SimpleVQA tests whether a multimodal model can give a short factual answer about an image. The paper collects 2,025 question-answer pairs, placed in 9 tasks and 9 topics, in English and Chinese, with reference answers that are not supposed to change over time. A separate language model judges each reply right or wrong. The reported result is the share of answers judged correct.
SimpleVQA
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.2OpenAI, released 2025-12-11 | 55.8% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15; printed on 1 more page | - |
| GPT-5 miniOpenAI, released 2025-08-07 | 56.8% | Alibaba, Qwen3.5-122B-A10B model cardchecked 2026-10-15 | - |
| Claude Opus 4.5Anthropic, released 2025-11-24 | 65.7% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15; printed on 2 more pages | - |
| Claude Sonnet 4.5Anthropic, released 2025-09-29 | 57.6% | Alibaba, Qwen3.5-122B-A10B model cardchecked 2026-10-15; printed on 1 more page | - |
| Gemma 4 26B A4BGoogle, released 2026-04-02 | 52.2% | Alibaba, Qwen3.6-35B-A3B model cardchecked 2026-10-15 | - |
| Gemma 4 31BGoogle, released 2026-04-02 | 52.9% | Alibaba, Qwen3.6-27B model cardchecked 2026-10-15; printed on 1 more page | - |
| Gemini 3 ProGoogle, released 2025-11-18 | 73.2% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15; printed on 1 more page | - |
| Muse Spark OpenMeta, released 2026-10-15 | 73.6% | Meta, Meta Muse Spark Open 816B-A42B model cardchecked 2026-10-19 | Effort: xhigh; Tools: with tools |
| Qwen3.6-27BAlibaba, released 2026-04-22 | 56.1% | Alibaba, Qwen3.6-27B model cardchecked 2026-10-15 | - |
| Qwen3.6-35B-A3BAlibaba, released 2026-04-16 | 58.9% | Alibaba, Qwen3.6-35B-A3B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.5-122B-A10BAlibaba, released 2026-02-24 | 61.7% | Alibaba, Qwen3.5-122B-A10B model cardchecked 2026-10-15 | - |
| Qwen3.5-27BAlibaba, released 2026-02-24 | 56% | Alibaba, Qwen3.5-122B-A10B model cardchecked 2026-10-15; printed on 2 more pages | - |
| Qwen3.5-35B-A3BAlibaba, released 2026-02-24 | 58.3% | Alibaba, Qwen3.5-122B-A10B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3.5-397B-A17BAlibaba, released 2026-02-16 | 67.1% | Alibaba, Qwen3.5-397B-A17B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3-VL-235B-A22BAlibaba, released 2025-09-23 | 61.3% | Alibaba, Qwen3.5-122B-A10B model cardchecked 2026-10-15; printed on 1 more page | - |
| Qwen3-VL-235B-A22B-ThinkingAlibaba, released 2025-09-23 | 56.8% | Moonshot AI, Kimi K2.5 model cardchecked 2026-10-15 | - |
| Kimi K3Moonshot AI, released 2026-07-16 | 71.9% | Meta, Meta Muse Spark Open 816B-A42B model cardchecked 2026-10-19 | Effort: max; Tools: with tools |
| Kimi K2.5Moonshot AI, released 2026-01-27 | 71.2% | Moonshot AI, Kimi K2.5 model cardchecked 2026-10-15; printed on 1 more page | - |
| InklingThinking Machines, released 2026-07-14 | 52.3% | Meta, Meta Muse Spark Open 816B-A42B model cardchecked 2026-10-19 | Effort: xhigh; Tools: with tools |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.