CyberGym: Results as Published

CyberGym tests whether AI agents can reproduce real vulnerabilities in open-source software. The agent gets a vulnerable codebase and a written description, and must write a test input that makes the bug crash the code before the fix but not after it. The benchmark has 1,507 vulnerabilities across 188 open-source projects, and each attempt is checked automatically with sanitizers.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.