τ-bench (tau2-bench)
Sierra's benchmark for tool-agent-user interaction (airline, retail, telecom), now in six languages and full-duplex voice with v2.0.
Tags
Description
Successor repository to the original sierra-research/tau-bench. τ-bench 2.0 (v2.0.0, 2026-10-20) adds τ-i18n: the same customer-service tasks in English, Spanish, Portuguese, Hindi, Chinese and Korean, in text and full-duplex voice with realtime agents; a new intake domain (τ-Entity, 200+ tasks capturing names and addresses on outbound calls); banking_knowledge in voice; post-hoc tau2 judges for naturalness, speech fidelity and tool use (judges don't change the task reward); and frozen released benchmarks (text-v1, voice-v2, voice-v1). Run with tau2 run --benchmark text. Live leaderboard at taubench.com. Paper: arXiv 2506.07982.
Related
- Berkeley Function Calling Leaderboard (BFCL): UC Berkeley leaderboard evaluating how accurately LLMs call functions and tools; part of the Gorilla project.
- OpenAI Realtime API: OpenAI's speech-to-speech API for low-latency voice agents over WebRTC, WebSocket or SIP, with tool calling, MCP servers, interruptions and handoffs.
- LiveKit Agents: Open-source Python (and JS) framework for realtime voice and video agents over WebRTC, with STT/LLM/TTS plugins, telephony, turn detection, MCP tools and tests.
Guides & comparisons
- Test a tool-using agent for prompt injection with AgentDojo (Guide): Measure how often injected tool outputs hijack your agent. Run AgentDojo's built-in suites from the CLI, wire your own model or agent loop in as a pipeline, read utility, utility under attack and targeted attack success, then compare defenses.
- Agent environment frameworks compared: Harbor vs AgentEnv Framework vs Seahaven (Comparison): Three open-source Python frameworks for building the environments agent evals and RL runs happen in: Harbor (container tasks), Scale AI's AgentEnv Framework and Kiln AI's Seahaven (synthetic SQLite worlds). Covers tasks, grading, compute, agent interface, interop and license.
- Agent benchmarks compared (Comparison): Ten widely cited agent benchmarks (SWE-bench, Terminal-Bench, tau-bench, BFCL, MCPMark, GAIA, BrowseComp, WebArena, Mind2Web and OSWorld) compared on what they measure, how many tasks they have, where the agent runs, how runs are scored, where results are published and how they are licensed.
Sources
- https://github.com/sierra-research/tau2-bench
- https://github.com/sierra-research/tau2-bench/releases/tag/v2.0.0
- https://taubench.com
- https://github.com/sierra-research/tau-bench
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)