NL2Repo-Bench: Results as Published

NL2Repo-Bench tests whether a coding agent can build a whole software project from a single natural-language requirements document. The agent starts with an empty workspace and must produce an installable Python library, across 104 tasks in nine library categories graded as easy, medium and hard. Scoring is execution-based, measuring the average share of each project's tests that pass.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.