AndroidWorld: Results as Published

AndroidWorld tests whether an autonomous agent can operate Android apps on a live emulator to complete natural-language tasks. The benchmark has 116 hand-crafted tasks across 20 apps, and each task is instantiated with randomly generated parameters, which the authors describe as producing millions of task variations. Each task includes setup, success-checking and teardown logic, so runs are reproducible and rewards come from programmatic checks of app state. Publishers report the task success rate.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.