MLflow
Open-source AI engineering platform for agents, LLMs and ML models: OpenTelemetry-based tracing, evaluation with 50+ metrics and LLM judges, prompt registry and an AI Gateway.
Tags
Description
Tracing works with any LLM provider and agent framework and supports Python, TypeScript/JavaScript and Java. Evaluation and production monitoring track quality metrics over time; the prompt registry versions prompts with lineage and supports automatic prompt optimization; the AI Gateway is an OpenAI-compatible endpoint with credential management, rate limits, fallbacks and traffic splitting. A setup wizard can launch Claude Code, Codex or OpenCode to add tracing to an app. Run your own MLflow server or connect to a Databricks workspace.
Agent access
Related
- Langfuse: Open-source agent evals and observability platform: trace, evaluate and improve LLM applications and agents.
- Arize Phoenix: Open-source AI observability and evaluation platform from Arize.
- Opik: Comet's open-source platform to debug, evaluate and monitor LLM apps, RAG systems and agentic workflows.
- OpenTelemetry GenAI Semantic Conventions: OpenTelemetry conventions for spans, metrics and events emitted by GenAI clients, MCP and model providers, for interoperable agent observability.
Sources
- https://github.com/mlflow/mlflow
- https://mlflow.org/docs/latest/genai/tracing/
- https://mlflow.org/docs/latest/genai/eval-monitor/
- https://mlflow.org/llms.txt
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)