MMBench: Results as Published

MMBench tests broad multimodal skill with single-answer multiple-choice questions in English and Chinese. The paper describes 3,217 questions across 20 ability dimensions, while the current repository lists 1,164 development questions and 1,784 test questions in each language. A free-form reply is mapped to a choice, by a matching rule when that works and otherwise by a language model. Under CircularEval the choices are rotated, the question is asked once for each order, and it counts as correct only when every pass is correct.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.