IFBench: Results as Published

IFBench tests whether a language model follows written instructions that carry checkable constraints, such as counting, formatting, and word or character rules. It uses 58 new constraints that the models were not trained on, so it measures generalisation rather than memorised rules. Each reply is checked automatically by a rule-based function, and the paper reports that leading models of the time scored below 50 percent on the new constraints.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.