ARC-Challenge: Results as Published

The AI2 Reasoning Challenge is a set of natural grade-school science questions written for human tests. The questions are multiple choice and do not use diagrams. The full set has 7,787 questions, and the Challenge Set is the 2,590 that both a retrieval method and a word co-occurrence method answered incorrectly; the other 5,197 form an Easy Set. Scores are the percentage of questions answered correctly on the test split.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.