Inspect
Open-source framework for large language model and agent evaluations from the UK AI Security Institute.
Tags
Agent access
Guides & comparisons
- Evaluate an AI agent with open-source eval and observability tools (Guide): A hands-on loop for agent quality: instrument with OpenTelemetry, collect traces, turn real traces into a dataset, write code, LLM-as-judge and human evaluators, run offline experiments, then monitor production and gate pull requests in CI.
- Test a tool-using agent for prompt injection with AgentDojo (Guide): Measure how often injected tool outputs hijack your agent. Run AgentDojo's built-in suites from the CLI, wire your own model or agent loop in as a pipeline, read utility, utility under attack and targeted attack success, then compare defenses.
- Observability and eval platforms for AI agents (Comparison): Langfuse, LangSmith, Arize Phoenix, Braintrust, Opik and Laminar compared on license, self-hosting, OpenTelemetry ingestion, MCP access to your traces, free tiers, paid entry prices and built-in evaluation.
- Agent benchmarks compared (Comparison): Ten widely cited agent benchmarks (SWE-bench, Terminal-Bench, tau-bench, BFCL, MCPMark, GAIA, BrowseComp, WebArena, Mind2Web and OSWorld) compared on what they measure, how many tasks they have, where the agent runs, how runs are scored, where results are published and how they are licensed.
Sources
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)