MCPMark: Results as Published

MCPMark tests whether a model or agent can use real MCP servers to complete realistic tasks in working environments, rather than answer questions about them. The full suite covers Notion, GitHub, Filesystem, Postgres and Playwright, and the README gives 127 tasks; a smaller easy suite with 10 tasks per server is meant for smoke tests and CI. Each task ships with an automated verify.py script, and results are reported as pass@1, pass@k, pass^k and avg@k. MCPMark Verified is the current default task set, with every environment version-pinned and every verifier stabilised.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.