OpenLIT
Open-source, OpenTelemetry-native platform for AI agents: tracing, evals, guardrails, prompt hub, vault, and cost and GPU monitoring; self-host free.
Tags
Description
Traces every LLM call, tool call, MCP request and retrieval via OpenTelemetry SDKs (pip install openlit / npm install openlit), evaluates on real traces (LLM-as-a-judge, programmatic, human feedback, CI gates), adds runtime guardrails (prompt injection, sensitive or restricted topics) and manages prompts and secrets without redeploying. Tracks cost (incl. custom pricing) and NVIDIA/AMD/Intel GPUs. Self-host with docker compose up -d.
Agent access
Related
- OpenLLMetry: Traceloop's open-source observability for GenAI/LLM applications, built on OpenTelemetry.
- OpenTelemetry GenAI Semantic Conventions: OpenTelemetry conventions for spans, metrics and events emitted by GenAI clients, MCP and model providers, for interoperable agent observability.
- Langfuse: Open-source agent evals and observability platform: trace, evaluate and improve LLM applications and agents.
- Arize Phoenix: Open-source AI observability and evaluation platform from Arize.
- Opik: Comet's open-source platform to debug, evaluate and monitor LLM apps, RAG systems and agentic workflows.
Sources
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)