OSWorld
NeurIPS 2024 benchmark of multimodal agents on open-ended tasks in real computer environments.
Tags
Related
- WebArena: A realistic web environment for building autonomous agents, with benchmark tasks used to evaluate web agents.
Guides & comparisons
- Agent benchmarks compared (Comparison): Ten widely cited agent benchmarks (SWE-bench, Terminal-Bench, tau-bench, BFCL, MCPMark, GAIA, BrowseComp, WebArena, Mind2Web and OSWorld) compared on what they measure, how many tasks they have, where the agent runs, how runs are scored, where results are published and how they are licensed.
Sources
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)