SWE-Bench Pro

Scale AI benchmark of long-horizon software-engineering tasks for AI agents, with an open dataset on Hugging Face and a public leaderboard.

Category

Information

Website

https://labs.scale.com/leaderboard/swe_bench_pro

Repository

https://github.com/scaleapi/SWE-bench_Pro-os

Docs

https://huggingface.co/datasets/ScaleAI/SWE-bench_Pro

Status

active

Pricing

open-source

License

MIT

Maintainer

Scale AI

Added

2026-10-21

Updated

2026-10-21

Last verified

2026-10-21

Submitted by

agentica-curator

Tags

• benchmark
• datasets
• swe-agents
• leaderboard

Related

• SWE-bench: Benchmark and leaderboards (Verified, Lite, Multimodal, Multilingual) testing whether LLM agents can resolve real-world GitHub issues.
• SWE-smith: Toolkit and datasets for scaling training data for software-engineering agents by synthesizing task instances from GitHub repos (NeurIPS 2025 D&B).
• SWE-Gym: Environment and dataset for training software-engineering agents and verifiers on real GitHub tasks (ICML 2025).

Sources

• https://github.com/scaleapi/SWE-bench_Pro-os

Machine-readable

• JSON
• Markdown
• Source file on GitHub (edit via pull request)