--- title: "LLM Benchmark Scores, Side by Side" description: "Benchmark results published by AI labs on release pages and model cards, with their API prices, organized in one table by area and test. Each value stays as printed, with its source one click away." seo_title: "LLM Benchmark Scores, Side by Side" seo_description: "Benchmark results from AI labs' release pages and model cards, with API prices, in one table by area and test. Each value as printed, source one click away." slug: llm-bench status: published published_at: 2026-10-15T12:00:00Z hero_image: /blog/images/posts/llm-bench/hero.jpg hero_alt: "A tilted benchmark table with one shaded column, beside the title LLM benchmark scores, side by side." og_image: /blog/images/posts/llm-bench/hero.jpg author: Arman Obosyan author_url: https://sugra.systems/about section: general primary_keyword: AI model benchmark results tags: - ai --- *A launch page is a table with the publisher's model in the first column. I kept the tables and lost the first column.* ## The table behind every model launch Most model releases arrive with a comparison table: benchmarks down the side, models across the top, and the publisher's newest model in the first column. The numbers are useful, but the tables are difficult to compare. Each release chooses a different set of models, tests, tools and evaluation settings, and each new launch makes the previous table look old. I read these tables for work. We run models every day at Sugra Systems, and choosing one for a task often begins with what its developer says it can do. I started recording those numbers together with their sources and stated conditions. That record became the catalog below. ## What this catalog is The catalog contains results published by model developers on their release pages and model cards, and the API prices from their official pricing pages. Sugra does not run these evaluations. Each value is stored as printed, in its original unit, with the source, date and conditions disclosed by the publisher. Results remain attached to their publications. Two values in the same row may come from different harnesses, reasoning settings or tool policies; placing them side by side does not turn them into a single controlled experiment. Different benchmark versions and metrics remain separate. There is no overall score and no ranking. The table is a sourced catalog of published claims: a place to see what was reported, compare the available evidence and open the original source before drawing a conclusion.
Developer
Models
Area
Benchmarks

Loading the results file.

## The areas Benchmarks are grouped by the kind of work they test. The areas below are the catalog's own; their descriptions are copied here from it.
AreaWhat it covers
Knowledge WorkTasks a professional is paid for: legal analysis, finance and tax questions, office documents, spreadsheets, research memos and other desk work. The tests here hand the model a realistic brief and judge the finished product, often with a human expert or an expert-written rubric.
Agentic CodingChanging and repairing real programs. The model works inside a repository or a terminal over many steps, reads the code, runs commands, and is scored on whether the resulting change passes hidden tests or a reviewer. This is the busiest area on every launch page.
ML EngineeringTraining and improving models rather than ordinary software: reproducing a research paper, running an experiment, tuning a training recipe, writing a compute kernel. The score is usually whether the resulting artifact works or reaches a target.
Science and MathMathematics from contest problems to research-level questions, plus scientific reasoning and laboratory knowledge in chemistry, physics and biology. Dangerous biological capabilities are kept out of this area and sit under Biosecurity.
Long ContextFinding, matching and using information spread across very long inputs: hundreds of thousands or a million tokens of documents, transcripts or structured data. The tests plant facts or relationships deep in the input and check whether the model can still use them.
Computer UseOperating a computer the way a person does: a desktop, a browser, a mobile device, through screenshots and clicks rather than an API. Tasks are everyday chores in real applications, scored on whether the end state is correct. Open-ended web research has its own area.
Multimodal UnderstandingImages, charts, video, audio, scanned documents and sheet music, and reasoning that connects them with text. The model has to read what it sees or hears and answer about it.
CybersecurityFinding and fixing vulnerabilities, reproducing known flaws in controlled test environments, analysing binaries and operating production systems under failure. These tests measure capability in a sandbox; the Safety and Alignment area measures whether the model refuses to misuse it.
General Knowledge and ReasoningBroad knowledge, logic and abstract problem solving, factual accuracy and the mixed question sets that try to cover everything at once. The benchmarks here are the ones most often quoted as a single headline number.
Code GenerationWriting code from a specification without a long agentic process: a function, a component, a drawing, an algorithm. Scored by tests, by rendering the output, or by a judge comparing candidates.
Tool Use and WorkflowsCalling functions and APIs, chaining tools, planning multi-step work and carrying a long task to the end. The environments range from office automation to games, and the score is whether the whole workflow completed.
Web Search and ResearchFinding hard-to-locate facts on the open web and synthesizing an answer from many pages. The questions are built so that no single page answers them.
Instruction FollowingObeying constraints: output formats, length limits, data schemas, forbidden words and multi-step user requirements. The score is how many constraints the answer honored.
MultilingualKnowledge and task performance across natural languages other than English: translation, multilingual question sets and regional exams.
Writing and ConversationCreative and editorial writing, style, role play, and the emotional and social quality of a conversation.
HealthcareClinical and medical knowledge, medical documents, and the quality and safety of help given in a medical conversation, usually graded against rubrics written by physicians.
BiosecurityTests of dangerous biological capability and of dual-use risk: virology know-how, wet-lab troubleshooting, and whether the model refuses requests that could help build a weapon. Some sets also cover chemical and cyber risk. Here a high capability score is not good news, and the refusal scores are the ones to read.
Safety and AlignmentRefusals of harmful requests, resistance to jailbreaks, honesty under pressure, sycophancy, and whether an agent discloses its own failures or takes forbidden shortcuts. Most of these are rates where lower is better; the table says so on each row.
Engineering and DesignCAD modeling, electrical engineering, spatial design and the production of engineering artifacts from a specification, scored on whether the artifact meets the spec.
Preference and Composite RankingsHead-to-head preference rankings by people or judge models, and composite indexes that average many unrelated tests into one number. A specialized index stays in its own area; only the catch-all ones live here.
PriceNot a test: what the developer charges through its API, in US dollars per one million tokens, as printed on its own pricing page on the day it was checked. One row per price component (input, cached input, cache write, output) and per schedule: standard, and the developer's others such as batch, flex, fast and priority. Lower is better. A long-context, regional or non-dollar price stays on the pricing page the value links to.
## The benchmarks, one by one Each benchmark in the current view is described below: who maintains it, what it tests and which cataloged publications reported a result for it.
## How the catalog is maintained The catalog is a public repository, [sugra-llm-benchmarks](https://github.com/armanobosyan/sugra-llm-benchmarks), and this page reads a copy of the file it builds, refreshed when the article is updated; the data version and its date are printed under the table. It is a maintained record, not a scraper: new release pages and model cards are read by hand, because the details that matter often live in headings, footnotes and evaluation notes, such as the benchmark version, the reasoning effort or the tool policy behind a number. Values are added as printed and checked against the repository's data contract, and a correction never replaces the past silently: the new record supersedes the earlier one, which stays in the history. The catalog shows what model developers published. It does not say which model is best for your workload, and it does not replace independent evaluation or a test on your own tasks. If a value here differs from the source it cites, the source is right; open the value, take its record ID and report it in the repository so the catalog can be corrected. The models and the leading scores will keep changing, and the rules of the catalog should not. Writing a number down with its source is what lets an old number remain an honest record.