WebChoreArena
Apache-2.0 benchmark of 532 tedious web-chore tasks built on the WebArena sites, for evaluating browsing agents.
Tags
Description
Research benchmark (repo WebChoreArena/WebChoreArena; not archived; created 2025-05-13; last pushed 2026-07-09; not presented as a brand-new 2026 launch). Root LICENSE is Apache-2.0. Project page https://webchorearena.github.io/ and arXiv 2506.01952 (both HTTP 200 on 2026-11-13). README and the homepage describe 532 curated tasks that extend WebArena toward labor-intensive chores, grouped as massive-memory, calculation, and long-term-memory tasks, on the four existing WebArena simulation websites. Running the benchmark requires a separately deployed WebArena environment and the resets described in the README, including before cross-site tasks. README evaluation examples use model API keys (Azure OpenAI, Anthropic, and Google Gemini are named); no keys were supplied and no evaluator was started. Upstream WebArena components keep their own terms. Agent runs can change the simulated sites — treat them as potentially destructive evaluation environments.
Agent access
Related
- WebArena: A realistic web environment for building autonomous agents, with benchmark tasks used to evaluate web agents.
- VisualWebArena: MIT benchmark of 910 multimodal visual web tasks for agents, extending WebArena with visually grounded sites and evaluation scripts.
- BrowserGym: ServiceNow's Gym environment for web agents that bundles MiniWoB, WebArena, VisualWebArena, WorkArena, AssistantBench and more behind one API.
Sources
- https://github.com/WebChoreArena/WebChoreArena
- https://github.com/WebChoreArena/WebChoreArena/blob/main/LICENSE
- https://github.com/WebChoreArena/WebChoreArena/blob/main/README.md
- https://webchorearena.github.io/
- https://arxiv.org/abs/2506.01952
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)