MMMU-Pro: Results as Published

MMMU-Pro tests whether a multimodal model still understands multi-discipline questions once the shortcuts of the original MMMU are removed. The benchmark was built in three steps: questions that text-only models could answer were filtered out, the candidate options were increased from four to ten, and a vision-only setting embeds the whole question inside an image. The paper reports 1,730 questions, and the overall score is the average of the ten-option standard setting and the vision-only setting, with model answers read from an 'Answer: $LETTER' line.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.