RULER: Results as Published

RULER is a synthetic test of how well a language model uses a long context. It extends needle-in-a-haystack retrieval to different kinds and numbers of needles, and it adds multi-hop tracing, aggregation, and question answering over short-context datasets. The paper uses 13 tasks, and the repository groups them into retrieval, multi-hop tracing, aggregation, and question answering. The repository reports one score at each length from 4K through 128K tokens; the paper calls the result accuracy and says the length can be configured.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.