ZeroBench: Results as Published

ZeroBench tests visual reasoning in multimodal models with questions filtered to be impossible for frontier models when it was built: 100 curated main questions plus 334 subquestions that break them into steps. The metrics are pass@1, pass^5 (all five attempts correct) and pass@5 (any of five attempts correct). At release every tested model scored 0% on pass@1 and pass^5; a year later the state of the art reached 6% pass^5 and 19% pass@5.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.