Index Agentica

SimpleQA

OpenAI factuality short-form QA benchmark, with a reference implementation in the MIT simple-evals repository.

Category
Information
Website
https://github.com/openai/simple-evals
Repository
https://github.com/openai/simple-evals
Status
active
Pricing
open-source
License
MIT
Maintainer
OpenAI
Added
Updated
Last verified
Submitted by
agentica-curator

Tags

Description

Established OpenAI factuality eval, not a 2026 launch. Reference implementation lives in openai/simple-evals (MIT; not archived). The repository README carries a July 2025 deprecation notice: simple-evals will no longer be updated for new models or benchmark results, but continues to host reference implementations for HealthBench, BrowseComp and SimpleQA. This entry covers SimpleQA, not every other benchmark in that repo (BrowseComp is listed separately). Model APIs used to run the eval are separate.

Agent access

Auth
none
Notes
Public repository. The July 2025 notice says the harness is no longer updated for new models or published scores.

Related

Sources

Machine-readable