BlogLLM Benchmark Scores as Published, Side by Side
KernelBench-CUDA: Results as Published
Every result the catalog holds for KernelBench-CUDA: 5 models from 4 developers, read from 1 page. Each number is as printed by the company named beside it; nothing is ranked or rescored. Models are listed by developer, newest first.
Area ML EngineeringMaintainer Scaling Intelligence Lab at Stanford UniversityIntroduced 2025Data version 2026103001, as of 2026-10-30
KernelBench asks a language model to replace a PyTorch machine-learning workload with a GPU kernel, in CUDA or another kernel language, that is both correct and fast. The paper uses 250 PyTorch workloads, and the repository splits the core set into 100 single operators, 100 fused patterns, and 50 full architectures. A kernel is correct when its outputs match the PyTorch reference on randomized inputs. The score, fast_p, is the share of kernels that are correct and faster than that PyTorch baseline by more than a chosen threshold p, using wall-clock time.
KernelBench Hard (TFLOPs relative to hardware peak)
| Model | Value | Printed by | Setup as printed |
|---|---|---|---|
| GPT-5.5OpenAI, released 2026-04-23 | 20.9% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| Claude Opus 4.7Anthropic, released 2026-04-16 | 30.7% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| Gemini 3.1 Pro PreviewGoogle, released 2026-02-19 | 18.6% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| MiniMax M3MiniMax, released 2026-06-01 | 28.8% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
| MiniMax M2.7MiniMax, released 2026-03-18 | 10.5% | MiniMax, MiniMax M3checked 2026-10-15 | Harness: Claude Code |
Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.