HallusionBench: Results as Published

HallusionBench tests whether a model reasons from the image or falls back on language priors and visual illusions. Human experts wrote 1,129 yes/no questions on 346 figures, mixing original images, edited images, charts, tables, maps, and illusions, and arranged them as control pairs. Each answer is marked correct or not. The paper then reports accuracy over all questions, accuracy over a whole figure, and question-pair accuracy, which counts a pair only when the model is right on every image in that pair.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.