INCLUDE: Results as Published

INCLUDE measures knowledge and reasoning in the languages where the questions were originally written, using local exams rather than English questions translated into those languages. The full collection has 197,243 multiple-choice pairs from 1,926 examinations in 44 written languages, collected from local sources in 52 countries, covering academic exams, professional certifications and licenses, and regional licenses such as driving and marine licenses. The released base set keeps only four-option questions; the dataset card counts 22,637 of them. The paper reports accuracy.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.