BlogLLM Benchmark Scores as Published, Side by Side
Multi-SWE-bench: Results as Published
Every result the catalog holds for Multi-SWE-bench: 10 models from 5 developers, read from 2 pages. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area Agentic CodingMaintainer Daoguang Zan and co-authors (ByteDance)Introduced 2025Data version 2026103001, as of 2026-10-30
Multi-SWE-bench tests whether a model can edit a repository so that a reported software issue is fixed. It covers Java, TypeScript, JavaScript, Go, Rust, C, and C++, with 1,632 instances chosen by 68 expert annotators from 2,456 candidates. The task is to produce a patch that addresses the given issue. The primary score is the resolved rate, the percentage of issues resolved.
Multi-SWE-bench
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.4OpenAI, released 2026-03-05 | 49% | MiniMax, MiniMax-M2.7 model cardchecked 2026-10-15 | - |
| GPT-5OpenAI, released 2025-08-07 | 39.3% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Effort: high; Harness: in-house harness derived from SWE-agent; Tools: enabled; bash and edit tools |
| Claude Sonnet 4.6Anthropic, released 2026-02-17 | 51% | MiniMax, MiniMax-M2.7 model cardchecked 2026-10-15 | - |
| Claude Opus 4.6Anthropic, released 2026-02-05 | 50.3% | MiniMax, MiniMax-M2.7 model cardchecked 2026-10-15 | - |
| Claude Sonnet 4.5Anthropic, released 2025-09-29 | 44.3% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Effort: thinking; Harness: in-house harness derived from SWE-agent; Tools: enabled; bash and edit tools |
| DeepSeek-V3.2-ExpDeepSeek, released 2025-09-29 | 30.6% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Harness: in-house harness derived from SWE-agent; Tools: enabled; bash and edit tools |
| Kimi K2 ThinkingMoonshot AI, released 2025-11-06 | 41.9% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Effort: thinking; Harness: in-house harness derived from SWE-agent; Tools: enabled; bash and edit tools |
| Kimi K2 Instruct 0905Moonshot AI, released 2025-09-05 | 33.5% | Moonshot AI, Kimi K2 Thinking release pagechecked 2026-10-15 | Harness: in-house harness derived from SWE-agent; Tools: enabled; bash and edit tools |
| MiniMax M2.7MiniMax, released 2026-03-18 | 52.7% | MiniMax, MiniMax-M2.7 model cardchecked 2026-10-15 | - |
| MiniMax M2.5MiniMax, released 2026-02-12 | 51.3% | MiniMax, MiniMax-M2.7 model cardchecked 2026-10-15 | - |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.