VisualWebArena
MIT benchmark of 910 multimodal visual web tasks for agents, extending WebArena with visually grounded sites and evaluation scripts.
Tags
Description
Established 2024 research release (web-arena-x/visualwebarena; MIT; not archived; last push observed 2024-11), not a new 2026 product. Website https://jykoh.com/vwa and paper arXiv:2401.13649. README documents 910 VWA tasks, local Python 3.10/3.11 setup (not 3.12), Playwright, optional AMI, and a demo for running on arbitrary webpages. Live-site or environment interactions can have real-world side effects; model APIs are separate.
Agent access
Related
- WebArena: A realistic web environment for building autonomous agents, with benchmark tasks used to evaluate web agents.
- WebVoyager: Multimodal end-to-end web agent research code and 643-task dataset across 15 real websites, with GPT-4V auto-eval protocol.
- WebShop: Princeton simulated e-commerce environment and benchmark for grounded language agents shopping from text instructions.
Sources
- https://jykoh.com/vwa
- https://github.com/web-arena-x/visualwebarena
- https://github.com/web-arena-x/visualwebarena/blob/main/LICENSE
- https://arxiv.org/abs/2401.13649
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)