APIGen-MT-5k
Salesforce dataset of 5,000 verified multi-turn agent trajectories (retail and airline, tau-bench style) for training and evaluating tool-calling agents.
Tags
Description
Generated by the APIGen-MT pipeline (simulated agent-human interplay) and verified through format checks, function execution and domain policy checks, and semantic verification; a 200-sample human evaluation reported a 99% success rate. ShareGPT-like JSON, a subset of the data used to train Salesforce's xLAM-2 models. Paper: arXiv:2504.03601.
Access: gated on Hugging Face (accept the terms and log in); license CC BY-NC 4.0 (non-commercial).
Agent access
Related
- APIGen Function-Calling Dataset (xlam-function-calling-60k): Salesforce's 60,000-example function-calling dataset generated by APIGen, each example checked for format, by real execution and semantically.
- τ-bench (tau2-bench): Sierra's benchmark for tool-agent-user interaction in real-world domains.
- Berkeley Function Calling Leaderboard (BFCL): UC Berkeley leaderboard evaluating how accurately LLMs call functions and tools; part of the Gorilla project.
- Hugging Face Datasets Hub: Hugging Face hub of ready-to-use datasets for AI models, plus the open-source `datasets` library for loading and manipulating them.
Guides & comparisons
- Test a tool-using agent for prompt injection with AgentDojo (Guide): Measure how often injected tool outputs hijack your agent. Run AgentDojo's built-in suites from the CLI, wire your own model or agent loop in as a pipeline, read utility, utility under attack and targeted attack success, then compare defenses.
Sources
- https://huggingface.co/datasets/Salesforce/APIGen-MT-5k
- https://apigen-mt.github.io/
- https://arxiv.org/abs/2504.03601
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)