MultiPL-E: Results as Published

MultiPL-E translates unit-test code generation benchmarks from Python into other programming languages. It translates HumanEval and MBPP, including the function signature, the comment, and simple unit tests, into 18 additional languages so the same problems can be scored in each language. A completion is judged correct only if it passes every unit test. Because generation is stochastic, the paper samples several completions per problem and reports an estimated pass rate.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.