SWE-MM: Results as Published

SWE-bench Multimodal, which the paper abbreviates as SWE-bench M, tests whether an autonomous software engineering system can fix bugs in visual, user-facing JavaScript software. Each task instance includes at least one image in its problem statement or unit tests. The paper reports 617 task instances in its abstract and 619 in its body and conclusion, drawn from 17 JavaScript libraries used for web interface design, diagramming, data visualization, syntax highlighting and interactive mapping. A task instance counts as resolved when all unit tests pass, and the main metric is the percentage of resolved instances.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.