DeepSearchQA: Results as Published

DeepSearchQA tests whether a deep research agent can finish an open-ended information-seeking task by finding every item in an answer set, not a single fact. The benchmark has 900 multi-step tasks across 17 fields, and the agent has to search, collate fragmented information from different sources, remove duplicates and decide when the search is complete. The submitted answer set is compared with the ground-truth set, and the primary ranking metric is F1, the harmonic mean of precision and recall computed per prompt and then averaged, with a binary 'Fully Correct' outcome for an exact set match.

Every number here is read from the same catalog file as the benchmark table, data version 2026103001, and is the number the table shows for that model and test. Where one page printed a model under several setups, each value is listed. The catalog is public in armanobosyan/sugra-llm-benchmarks; a wrong number can be reported from the table, with Report an error under the value, or as an issue in that repository.