xdev-bench-hard: Results as Published

xdev-bench-hard is an in-house set of expert-curated vulnerability discovery and exploitation tasks built on source code. Most tasks reproduce recently disclosed vulnerabilities in software such as the Linux kernel, Node.js, JavaScriptCore and Exim; the agent must exploit the target in a separate sandbox and retrieve a secret string for automated verification. Moonshot AI reports the share of problems solved at pass@3 and the average success rate, with no hard timeouts or output token limits.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.