BlogLLM Benchmark Scores as Published, Side by Side
SWE-fficiency: Results as Published
Every result the catalog holds for SWE-fficiency: 5 models from 4 developers, read from 1 page. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area Agentic CodingMaintainer Jeffrey Jian Ma, Milad Hashemi, Amir Yazdanbakhsh, and co-authorsIntroduced 2025Data version 2026103001, as of 2026-10-30
SWE-fficiency asks an agent to make a real repository faster on a real workload without breaking behavior. The suite has 498 tasks from nine Python libraries, including numpy, pandas, and scipy. Each task provides a codebase, a slow workload, and the repository tests that must still pass. The score is the speedup ratio, the agent's runtime speedup divided by the human patch's speedup, combined across tasks with a harmonic mean, and a patch that fails the tests or does not apply is scored as if no edit was made.
SWE-fficiency
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.5OpenAI, released 2026-04-23 | 46.6% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| Claude Opus 4.7Anthropic, released 2026-04-16 | 42.2% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| Gemini 3.1 Pro PreviewGoogle, released 2026-02-19 | 19.7% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| MiniMax M3MiniMax, released 2026-06-01 | 34.8% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| MiniMax M2.7MiniMax, released 2026-03-18 | 13.98% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.