Observability and eval platforms for AI agents
Langfuse, LangSmith, Arize Phoenix, Braintrust, Opik and Laminar compared on license, self-hosting, OpenTelemetry ingestion, MCP access to your traces, free tiers, paid entry prices and built-in evaluation.
Comparison
| Entry | License | Self-host | OpenTelemetry ingest | MCP access to your data | Free cloud tier | Paid entry | Evaluation built in |
|---|---|---|---|---|---|---|---|
| Langfuse | MIT for the core; ee/ folders under a separate commercial license | Free with no usage limits; Docker Compose for testing, Helm or Terraform (AWS, Azure, GCP) for production. Needs Postgres, ClickHouse, Redis or Valkey, and S3-compatible storage | Yes: OTLP at /api/public/otel with Basic auth (cloud regions or your own host) | Native MCP server at /api/public/mcp (Basic auth); read and write tools on by default. Separate public docs MCP | Hobby: 50k units/month, 30 days data access, 2 users, no card | Core $29/mo with 100k units, then $8 per 100k (lower with volume); Pro $199/mo | LLM-as-a-judge, code evaluators, datasets and experiments (including in CI), human annotation queues |
| LangSmith | Closed platform; SDKs are open source (MIT) | Enterprise add-on only, with a license key | Yes: OTLP at https://api.smith.langchain.com/otel (regional hosts for EU, APAC, AWS) | Hosted remote MCP on all cloud regions, OAuth 2.1 or X-Api-Key; also on self-hosted v0.16+. The standalone server repo is archived | Developer: 1 seat, 5k base traces/month (base traces kept 14 days) | Plus $39 per seat/month with 10k base traces, then pay as you go | Online and offline evals, datasets; Tuned Evaluators billed per run |
| Arize Phoenix | Elastic License 2.0 (source-available; Arize calls it open source) | Free with no feature gates; run locally from the CLI, in Docker or on Kubernetes | Built on OpenTelemetry with OpenInference conventions; OTLP to your Phoenix server (localhost:6006 by default) | Remote MCP built into the server at /mcp (beta, Phoenix 19.0.0+); older npm MCP package in maintenance mode | Phoenix itself is self-hosted. Arize's managed product AX Free gives 25k spans/month, 1 GB, 15-day retention | AX Pro $50/mo with 50k spans, 10 GB, 30-day retention | LLM-as-judge evals (pre-built and custom, client or server side), datasets and experiments |
| Braintrust | Closed platform; SDKs Apache-2.0, autoevals MIT | Enterprise: on-prem or hosted deployment; self-hosted data plane with Terraform modules | Yes: OTLP at https://api.braintrust.dev/otel with a Bearer API key and x-bt-parent header | Hosted remote MCP at https://api.braintrust.dev/mcp, OAuth or API key; EU and self-hosted endpoints | Starter: 1 GB processed data, 10k scores, 14-day retention, $10 model credits, unlimited users | Pro $249/mo with 5 GB (+$3/GB), 50k scores (+$1.50 per 1k), 30-day retention | LLM-as-a-judge, autoevals and custom code scorers; experiments, datasets, playgrounds, human review scores |
| Opik | Apache-2.0 | Free; same codebase as the hosted versions. Docker Compose locally, Helm on Kubernetes | Yes: OTLP at /opik/api/v1/private/otel on Comet cloud, or /api/v1/private/otel self-hosted | Hosted MCP at https://www.comet.com/opik/api/v1/mcp with OAuth, or local uvx opik-mcp with an API key | Free Cloud: 25k spans/month, 60-day retention, up to 10 team members | Pro Cloud $19/mo with 100k spans/month, up to 50 members | Test suites and assertions, datasets and experiments, custom metrics, Agent Playground, prompt optimization |
| Laminar | Apache-2.0 | Free OSS images with Docker Compose, or Helm; Signals and Slack alerts need an enterprise license | Yes: OTLP over gRPC, HTTP/protobuf or HTTP/JSON, cloud and self-hosted | Remote MCP at https://api.lmnr.ai/v1/mcp with a project API key; read-only SQL, trace context and an ask-agent tool | Free: 1 GB data with no overage, 7-day retention, 1 project, 1 seat | Starter $30/mo with 3 GB, then $2/GB, 30-day retention; Pro $150/mo | Evaluations with run comparison (also self-hosted); agent debugger with record and replay |
Verdict: To self-host a permissively licensed platform at no cost, pick Langfuse (MIT core) or Opik (Apache-2.0); Opik also has the cheapest paid cloud tier here. Phoenix is free to self-host with no feature gates, but under ELv2. LangSmith suits LangChain and LangGraph teams that are happy with SaaS. Braintrust centers on evals, and its free tier allows unlimited users. Laminar is built around agent traces and gives agents read-only SQL over MCP. All six accept OTLP, so instrument with OpenTelemetry and keep the option to switch.
What this compares
These six platforms record what an agent did, step by step (model calls, tool calls, retrievals), and let you score those runs, both offline against datasets and online on production traffic. The table covers what decides a choice: whether you can run it yourself and under what license, whether it accepts standard OpenTelemetry, whether your coding agent can read your traces over MCP, and what the free and entry-level plans include.
Prices, allowances and retention come from each vendor's pricing page on 2026-10-12. The vendors meter different things. Langfuse counts units (every trace, observation and score). LangSmith counts base traces. Phoenix's managed sibling Arize AX and Opik count spans. Braintrust and Laminar count GB of processed data. One agent run can be a single LangSmith trace but dozens of Langfuse units or spans, so convert your own traffic before comparing.
Pick by situation
- Data must stay on your infrastructure, at no license cost. Langfuse (MIT core, no usage limits self-hosted), Opik (Apache-2.0, same codebase as its cloud) and Laminar (Apache-2.0, minus Signals and Slack alerts) are permissively licensed. Arize Phoenix is also free to self-host with no feature gates, but under the Elastic License 2.0, which restricts offering it as a managed service. LangSmith and Braintrust self-host only on Enterprise plans.
- You want your coding agent to debug from traces. All six now expose an MCP server for your own data. Laminar's is read-only, with SQL scoped to your project. Langfuse's has write tools on by default, so restrict it with a tool allowlist if the agent should only read. LangSmith, Braintrust and Opik offer OAuth for interactive clients. See Connect an agent to a remote MCP server for client setup.
- You expect to switch later. Every platform here accepts OTLP. Instrument with OpenTelemetry and the GenAI semantic conventions, or with OpenLLMetry (Apache-2.0), rather than only a vendor SDK, and switching platforms becomes mostly a change of exporter endpoint and headers.
- A team on a small budget. Braintrust Starter allows unlimited users, and Opik's Free Cloud allows up to 10 members with 60-day retention. LangSmith Developer is one seat, Laminar Free is one seat and one project, and Langfuse Hobby is two users.
Tracing an agent harness
Several of these platforms document tracing coding agents directly: Langfuse, Opik and Braintrust each have integration pages for Claude Code, Codex and others, and Phoenix has one for Cursor. That is useful if you run agents from the coding agent harnesses comparison and want to see what they did.
Eval-only tools
If you only need tests in CI and no hosted trace store, a framework may be enough. promptfoo (MIT) does evals and red-teaming from config files. DeepEval (Apache-2.0) is a pytest-style metrics library. Inspect (MIT, UK AI Security Institute) is built for model and agent evaluations. Most of the platforms above can ingest their results or run similar scorers.
Not in the table
- Helicone was acquired by Mintlify in March 2026. Mintlify's announcement says Helicone "will continue operating in maintenance mode", with security updates, bug fixes and new models still shipping, and that Mintlify will help customers migrate to another platform. It is still an option as an LLM gateway with logging (free Hobby plan with 10,000 requests), but weigh that status before you build on it.
- W&B Weave (Apache-2.0) is the Weights & Biases tracing and evaluation toolkit. We left it out of the table because we couldn't confirm its current plan limits from a primary source on the day.
What we left out
- LangSmith's per-trace overage price: the pricing page says "pay as you go" without listing the rate in its static content.
- Ingestion speed, query latency and "best for agents" claims: none of the vendors publish benchmarks we could reproduce.
- Exact MCP tool lists, which change often. Check each vendor's MCP page, linked in the sources.
Directory entries in this comparison
- Langfuse: Open-source agent evals and observability platform: trace, evaluate and improve LLM applications and agents.
- LangSmith: LangChain's agent and LLM observability and evaluation platform: tracing, monitoring, cost and latency tracking.
- Arize Phoenix: Open-source AI observability and evaluation platform from Arize.
- Braintrust: AI observability platform for agents: trace production, run evals and catch regressions before users see them.
- Opik: Comet's open-source platform to debug, evaluate and monitor LLM apps, RAG systems and agentic workflows.
- Laminar: Open-source observability platform purpose-built for AI agents: trace, evaluate and debug agent failures.
- Helicone: Open-source AI gateway and LLM observability platform for routing, monitoring, evaluating and experimenting.
- OpenTelemetry GenAI Semantic Conventions: OpenTelemetry conventions for spans, metrics and events emitted by GenAI clients, MCP and model providers, for interoperable agent observability.
- OpenLLMetry: Traceloop's open-source observability for GenAI/LLM applications, built on OpenTelemetry.
- Promptfoo: Open-source tool to test and red-team prompts, agents and RAG: automated evals, vulnerability scanning and model comparison.
- DeepEval: Open-source LLM evaluation framework with 50+ plug-and-play metrics for agents, RAG and chatbots.
- Inspect: Open-source framework for large language model and agent evaluations from the UK AI Security Institute.
- W&B Weave: Weights & Biases toolkit for tracking, testing and improving LLM-powered applications.
Related
- Open-source red-teaming tools for LLM apps and agents (Comparison)
- Evaluate an AI agent with open-source eval and observability tools (Guide)
- Terminal coding agents compared (Comparison)
- Research agent stack (Stack)
- Connect an agent to a remote MCP server (Guide)
- Agent benchmarks compared (Comparison)
- DevOps and observability MCP servers compared (Comparison)
Sources
- Langfuse cloud pricing, accessed
- Langfuse self-hosted pricing and license, accessed
- Langfuse self-hosting, accessed
- Langfuse OpenTelemetry integration, accessed
- Langfuse MCP server, accessed
- Langfuse LICENSE, accessed
- LangSmith pricing, accessed
- LangSmith self-hosted, accessed
- Trace with OpenTelemetry (LangSmith), accessed
- LangSmith Remote MCP, accessed
- LangSmith MCP Server (standalone, deprecated), accessed
- langsmith-sdk repository, accessed
- Phoenix license, accessed
- Phoenix LICENSE (Elastic License 2.0), accessed
- Phoenix docs index (llms.txt), accessed
- Phoenix setup using phoenix.otel, accessed
- Phoenix Remote MCP Server, accessed
- Arize pricing, accessed
- Braintrust pricing, accessed
- Send OpenTelemetry traces and logs to Braintrust, accessed
- Braintrust MCP, accessed
- Braintrust docs index (llms.txt), accessed
- Comet and Opik pricing, accessed
- Opik OpenTelemetry integration, accessed
- Opik MCP server advanced setup, accessed
- Opik self-hosting overview, accessed
- comet-ml/opik repository, accessed
- Laminar pricing, accessed
- Laminar OpenTelemetry, accessed
- Laminar MCP server, accessed
- Self-hosting Laminar, accessed
- Mintlify acquires Helicone to redefine AI knowledge infrastructure, accessed
- Helicone pricing, accessed