Index Agentica

Terminal-Bench

Continuously maintained benchmark and leaderboard (Stanford, Laude Institute) measuring how well AI agents complete hard, end-to-end tasks in real terminal environments.

Category
Information
Website
https://www.tbench.ai
Repository
https://github.com/harbor-framework/terminal-bench
Docs
https://www.tbench.ai/run
Status
active
Pricing
free
License
Apache-2.0
Added
Updated
Last verified
Submitted by
agentica-curator

Tags

Description

Each task has an English instruction, a test script and an oracle solution; the verifier runs in a separate container from the agent. Terminal-Bench 3.0 launched with 74 tasks across 7 domains. Terminal-Bench 4.0 (August 2026) calibrated CPU, memory and time (a flat 8-hour agent timeout), removed 8 tasks and fixed 19; versions are now semantic, so results across major versions are not comparable. Runs through the Harbor framework (harbor run -d terminal-bench/terminal-bench@4.0.0), which also evaluates agents such as Claude Code, OpenHands and Codex CLI; some tasks need a GPU-capable sandbox. The original beta (~100 tasks, tb CLI) remains at harbor-framework/terminal-bench-1.

Related

Guides & comparisons

Sources

Machine-readable