SWE Atlas Test Writing: Results as Published

SWE Atlas Test Writing asks a coding agent to add tests for a behavior that a prompt describes in ordinary language, inside a real repository. The set has 90 tasks from 11 production repositories in Go, Python, C, and TypeScript. A submission is resolved only if a manifest of the new tests is accepted, those tests pass on the original code and fail after the relevant code is removed, and every mandatory rubric item passes. The reported figure is the task resolve rate, the share of tasks that pass all three checks.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.