FrontierCode: Results as Published

FrontierCode tests whether an AI coding agent's change to a real open-source project is good enough for that project's maintainers to merge, not just code that passes tests. It has 150 tasks drawn from 36 repositories, and Artificial Analysis runs the 100-task Main set with three attempts per task. Each attempt must pass deterministic tests and file-scope limits, and an LLM judge checks it against maintainer criteria, where failing any blocking check scores zero.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.