MCP-Bench
Accenture's benchmark for tool-using agents on complex real-world tasks across 28 live MCP servers and 250 tools (finance, travel, science, academic search).
Tags
Description
Tests cross-tool coordination, precise parameter control and planning, end to end over MCP: the repo ships the 28 server implementations, task synthesis and the runner (run_benchmark.py), and a Hugging Face Space hosts the leaderboard. Accepted to the NeurIPS 2025 Workshop on Scaling Environments for Agents (arXiv 2508.20453). The README badge says Apache-2.0, but the repository has no license file.
Related
- Toolathlon (Tool Decathlon): Benchmark of 108 long-horizon agent tasks across 32 real applications and 604 tools (mostly MCP servers), each checked by a deterministic evaluation script.
- MCPMark: Stress-testing benchmark for agentic models using real MCP environments (Notion, GitHub, Filesystem, Postgres, Playwright), with strict automated verifiers and sandboxed tasks.
- MCP-Universe: Salesforce AI Research benchmark and framework that evaluates LLMs and agents on real-world tasks against live MCP servers, with a public leaderboard.
- Berkeley Function Calling Leaderboard (BFCL): UC Berkeley leaderboard evaluating how accurately LLMs call functions and tools; part of the Gorilla project.
Sources
- https://arxiv.org/abs/2508.20453
- https://github.com/Accenture/mcp-bench
- https://huggingface.co/spaces/mcpbench/mcp-bench
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)