Evals & Observability (54)
Evaluation, tracing and observability for agents and LLM applications.
Machine-readable: /api/evals-observability.json.
- AgentEnv Framework
Scale AI's open-source Python SDK and CLI for building, deploying and scoring containerized agent environments and tasks for RL and evals.
- AgentEvals
LangChain's evaluators and utilities for agents, focused on agent trajectories (the intermediate steps an agent takes).
- AgentGym
Agent training and evaluation suite exposing diverse interactive environments through a common HTTP interface and Python clients.
- AgentOps
Python SDK and platform for AI agent monitoring, LLM cost tracking, benchmarking, testing and debugging.
- Arize AX
Arize's commercial platform for agent observability, evaluation, tracing and experimentation, from a free tier to self-hosted.
- Arize Phoenix
Open-source AI observability and evaluation platform from Arize.
- Braintrust
AI observability platform for agents: trace production, run evals and catch regressions before users see them.
- CyberSecEval
Meta's benchmark suite for LLM cybersecurity risk and defensive skill: attack compliance, insecure code, prompt injection and SOC tasks.
- DeepEval
Open-source LLM evaluation framework with 50+ plug-and-play metrics for agents, RAG and chatbots.
- DeepTeam
Open-source framework from Confident AI to red team LLMs and AI agents.
- EvalPlus
Code-generation evaluation framework with expanded HumanEval+/MBPP+ tests and EvalPerf for checking correctness and runtime efficiency.
- Evidently
Open-source Python framework to evaluate, test and monitor ML and LLM systems, with 100+ built-in metrics, LLM judges, test suites for CI and a monitoring UI.
- garak
NVIDIA's open-source LLM vulnerability scanner probing for hallucination, data leakage, prompt injection, jailbreaks, toxicity and other weaknesses.
- Giskard
Open-source Python library for testing agentic systems: scenario-based checks and LLM-as-judge evals (giskard-checks) plus a vulnerability scanner for red teaming (giskard-scan).
- Guardrails AI
Open-source Python framework for adding input/output guardrails (validators) to LLM applications.
- Harbor
Framework from the Terminal-Bench creators for evaluating and optimizing agents and models in sandboxed environments, run in parallel across cloud sandbox providers.
- Helicone
Open-source AI gateway and LLM observability platform for routing, monitoring, evaluating and experimenting.
- Inspect
Open-source framework for large language model and agent evaluations from the UK AI Security Institute.
- Inspect Evals
Community collection of ready-to-run evaluation tasks for the UK AISI Inspect AI framework.
- Invariant Guardrails
Rule-based guardrails that sit between an agent and its MCP servers or LLM provider, checking and blocking tool calls and messages.
- Laminar
Open-source observability platform purpose-built for AI agents: trace, evaluate and debug agent failures.
- Langfuse
Open-source agent evals and observability platform: trace, evaluate and improve LLM applications and agents.
- Langfuse Experiment Action
GitHub Action that runs a Langfuse experiment in your CI pipeline.
- LangSmith
LangChain's agent and LLM observability and evaluation platform: tracing, monitoring, cost and latency tracking.
- LangWatch
Open-core platform for agent observability and evals: tracing, agent simulation tests, evaluations, prompt management, an AI gateway and coding-agent usage tracking.
- Lighteval
Hugging Face toolkit for evaluating LLMs across local and served-model backends, with customizable tasks, metrics and per-sample results.
- Llama Guard
Meta's open-weight safety classifiers that label LLM prompts and responses as safe or unsafe against the MLCommons hazard taxonomy.
- Llama Prompt Guard 2
Meta's small open-weight classifiers (86M and 22M) that flag prompt injection and jailbreak attempts in user input and untrusted content.
- LlamaFirewall
Meta's open-source guardrail framework for agents that chains scanners for prompt injection, goal hijacking and insecure generated code.
- LM Evaluation Harness
EleutherAI's widely used framework for few-shot evaluation of language models across hundreds of benchmarks, with HF, vLLM and API backends.
- MCP Inspector
Official interactive developer tool for testing and debugging MCP servers in the browser, on the command line or in the terminal.
- Meta Agents Research Environments (ARE)
MIT research platform from Meta for simulating dynamic, multi-step agent tasks, including official Gaia2 evaluation.
- MLflow
Open-source AI engineering platform for agents, LLMs and ML models: OpenTelemetry-based tracing, evaluation with 50+ metrics and LLM judges, prompt registry and an AI Gateway.
- NeMo Guardrails
NVIDIA's open-source toolkit for adding programmable guardrails to LLM-based conversational systems.
- OmniMemEval
MemTensor's evaluation framework for memory backends (LoCoMo, LongMemEval, BEAM, PersonaMem v2, HaluMem) and memory-equipped agents on AgentBench domains.
- OpenAI Evals
OpenAI's open-source framework and registry of benchmarks for evaluating LLMs and LLM systems, including custom private evals.
- OpenAI Frontier Evals
OpenAI's open-source code for frontier capability evals: PaperBench, SWE-Lancer and EVMbench.
- OpenEvals
LangChain's package of ready-made evaluators for LLM apps (sibling project agentevals targets agents).
- OpenInference
OpenTelemetry-compatible conventions and auto-instrumentation packages for tracing LLM calls, retrieval, tools and agent frameworks.
- OpenLIT
Open-source, OpenTelemetry-native platform for AI agents: tracing, evals, guardrails, prompt hub, vault, and cost and GPU monitoring; self-host free.
- OpenLLMetry
Traceloop's open-source observability for GenAI/LLM applications, built on OpenTelemetry.
- Opik
Comet's open-source platform to debug, evaluate and monitor LLM apps, RAG systems and agentic workflows.
- Promptfoo
Open-source tool to test and red-team prompts, agents and RAG: automated evals, vulnerability scanning and model comparison.
- Promptfoo GitHub Action
GitHub Action for running Promptfoo prompt, agent and RAG tests and red-team scans in CI.
- Pydantic Logfire
OpenTelemetry-based observability platform from the Pydantic team for tracing LLM apps and agents, with SQL querying and evals.
- PyRIT
Microsoft's open-source Python Risk Identification Tool for proactively red teaming generative AI systems.
- Ragas
Open-source evaluation toolkit for LLM apps and agents: LLM-based and traditional metrics for RAG and agent workflows, plus automatic test-set generation.
- Seahaven
Kiln AI's open-source Python framework for synthetic, resettable 'worlds' (SQLite state plus mock tools) for agent evals and RL, served via OpenEnv or MCP.
- Snyk Agent Scan
Security scanner that discovers agent components on your machine (agents, MCP servers, skills) and scans them for prompt injections and vulnerabilities.
- TextArena
MIT framework of 100+ text-based games for evaluating and training language-model agents, with a PyPI package and public site.
- ThinkingBox
Microsoft's open-source sandbox and benchmark that runs agents against isolated MCP tool servers and grades the backend state and side effects they leave.
- TruLens
Open-source, OpenTelemetry-native tracing and evaluation for LLM apps and agents, scoring each step with LLM judges and comparing app versions.
- verifiers
Prime Intellect’s MIT library for creating LLM reinforcement-learning environments and evals, with a matching PyPI package.
- W&B Weave
Weights & Biases toolkit for tracking, testing and improving LLM-powered applications.