# OpenAI Frontier Evals

> OpenAI's open-source code for frontier capability evals: PaperBench, SWE-Lancer and EVMbench.

- id: `openai-frontier-evals`
- category: Evals & Observability (`evals-observability`)
- url: https://github.com/openai/frontier-evals
- repo: https://github.com/openai/frontier-evals
- status: active
- pricing: open-source
- license: MIT
- tags: benchmark, agents, coding, research, security
- related: OpenAI Evals (https://indexagentica.com/entries/openai-evals/); SWE-Bench Pro (https://indexagentica.com/entries/swe-bench-pro/); MLE-bench (https://indexagentica.com/entries/mle-bench/)
- sources: https://github.com/openai/frontier-evals , https://arxiv.org/abs/2504.01848 , https://arxiv.org/abs/2502.12115
- added: 2026-11-01
- updated: 2026-11-01
- last verified: 2026-11-01
- maintainer: OpenAI
- submitted by: agentica-curator
- page: https://indexagentica.com/entries/openai-frontier-evals/
- json: https://indexagentica.com/api/entries/openai-frontier-evals.json

PaperBench measures end-to-end replication of state-of-the-art AI papers, SWE-Lancer uses real freelance software-engineering tasks with end-to-end tests, and EVMbench covers smart-contract security tasks. Each suite has its own README and a uv-managed environment (`uv sync`). MIT. Last commit October 2026.