LiteLLM
MIT Python SDK and self-hosted AI gateway that calls 100+ LLM providers in the OpenAI format. Paid Enterprise is separate and quote-based.
Tags
Description
Vendor-affiliated open project (Berri AI / BerriAI/litellm; not archived; not presented as a 2026 launch). GitHub license metadata is NOASSERTION because the root LICENSE is split: content under litellm/enterprise/ is under the BerriAI Enterprise license in litellm/enterprise/LICENSE.md (Copyright (c) 2024 - present Berrie AI Inc., spelling as in that file); everything else is MIT (Copyright (c) 2023 Berri AI). This entry uses license MIT for the open-source SDK/gateway and discloses the enterprise directory split. The enterprise LICENSE.md says that software may only be used in production with a valid BerriAI Enterprise license and agreement; it was not used here. https://litellm.ai 301s to https://www.litellm.ai/ (HTTP 200). Docs https://docs.litellm.ai/docs/ were HTTP 200. PyPI litellm 1.109.1 (not yanked; first file uploaded 2026-11-12T01:59:54Z = 2026-11-12 02:59:54 Europe/Stockholm). GitHub latest release v1.109.1 was published 2026-11-12T02:07:01Z = 2026-11-12 03:07:01 Europe/Stockholm (not draft/prerelease). Pricing page (https://www.litellm.ai/pricing, HTTP 200): Open Source is $0, free forever to self-host, with no license fee for production OSS use. Enterprise is annual, talk-to-sales, sized to annual gateway request capacity, deployment architecture and support needs, and is described as never per token; a 30-day Enterprise trial key is offered on the Enterprise page and was not requested. No numeric Enterprise list prices were published on that page. Official incident posts describe two 2026 supply-chain events. The 24 March 2026 post (last updated 27 March 2026, with later notes through 30 March) says PyPI packages litellm==1.82.7 and litellm==1.82.8 “were live on March 24, 2026 from 10:39 UTC for about 40 minutes before being quarantined by PyPI.” It says those versions included a credential stealer that harvested environment variables, SSH keys, cloud-provider credentials (AWS, GCP, Azure), Kubernetes tokens and database passwords and exfiltrated them via POST to models.litellm.cloud, which the post says is not an official BerriAI / LiteLLM domain; it also says those versions have already been removed from PyPI, and that customers running the official LiteLLM Proxy Docker image were not impacted because that path pins dependencies in requirements.txt. The same post’s “Who is affected” section also lists installs or upgrades via pip on 24 March 2026 between 10:39 UTC and 16:00 UTC, a wider window than the “about 40 minutes” live period; both figures are recorded as written. The 21 October 2026 post (last updated 20:30 UTC that day) says a malicious litellm==1.107.0 “was live for approximately 2 hours (11:39 UTC to 13:23 UTC)” and, more precisely, that all seven wheels were live from 11:39:51 to 13:23 UTC; it has since been pulled. That post describes the package as containing compiled malware that steals credentials and loads remote-code execution tools. It does not give one end time for both container registries: Docker Hub tags v1.107.0, latest, main-latest and main-stable “were compromised from 11:46 UTC to 14:20 UTC” (pointed at compromised image index sha256:1bbcb1443bc962e45f5aad53fe3d2c30600e0379715fb6c664cdd3ac126c339e from 11:46:56 to 14:20 UTC), with the v1.107.0 image removed and the others pointed at secure sources; the same tag names on GHCR “pointed to the compromised image indexes … from 11:46:56 UTC until they were made unavailable by 13:51 UTC.” The 23 October 2026 attack-chain post, which that October 21 post later calls the canonical record of how the attack happened, agrees those publication and removal times (PyPI quarantined by 13:23; GHCR unavailable by 13:51; Docker Hub tags removed or repointed at 14:20). As of 2026-11-15, PyPI no longer lists 1.82.7, 1.82.8 or 1.107.0, and the current PyPI version remains litellm 1.109.1 (not yanked; first file uploaded 2026-11-12T01:59:54Z = 2026-11-12 02:59:54 Europe/Stockholm). The self-hosted proxy stores provider credentials (encrypted with LITELLM_SALT_KEY when that is set; otherwise the master key is also the at-rest encryption key) and uses LITELLM_MASTER_KEY as the admin credential and default Admin UI password. The config settings docs (https://docs.litellm.ai/docs/proxy/config_settings, HTTP 200) say the proxy will not start when the master key (general_settings.master_key, falling back to LITELLM_MASTER_KEY) is not set, is empty, or has a known unsafe value, unless general_settings.dangerously_permit_weak_or_unset_master_key or the LITELLM_DANGEROUSLY_PERMIT_WEAK_OR_UNSET_MASTER_KEY environment variable is true (documented as local development only; default false). The linked master-key page (https://docs.litellm.ai/docs/proxy/master_key_rotations, HTTP 200) says the proxy then exits at boot with a non-zero status and prints how to fix it. The exception name UnsafeMasterKeyError was not found on those two official pages. Official security best practices say applications and users should use scoped virtual keys, not the master key; store provider credentials, the master key and the salt key in a secret manager; and run the gateway on a private network when possible, exposing only the routes clients need. By default the proxy also serves Swagger UI, ReDoc and the OpenAPI schema without authentication unless that is disabled. Related catalog entry litellm-mcp-gateway covers the MCP gateway surface and is not duplicated here. Upstream model-provider fees still apply when you attach keys. No signup, trial key or spend was used.
Agent access
Related
- LiteLLM MCP Gateway: The LiteLLM Proxy's MCP Gateway gives one fixed endpoint for MCP tools, controls access by key and team, and lets any LiteLLM-supported model call them.
- vLLM: Apache-2.0 high-throughput LLM inference and serving engine with an OpenAI-compatible API, installable via uv or pip.
- Ollama: MIT app and API for running open models locally, plus a paid Ollama Cloud with usage credits and per-token prices.
Guides & comparisons
- Run an agent on a local model with an OpenAI-compatible endpoint (Guide): Start Ollama, vLLM, SGLang or LocalAI bound to 127.0.0.1, point the OpenAI Python SDK at the server's /v1 base URL, enable tool calling, and optionally route the same endpoint through a LiteLLM proxy config.
- GPU clouds and serverless inference for agents (Comparison): Compare GPU rental, serverless GPUs and hosted per-token APIs for agents: OpenAI-compatible base URLs, tools and structured output, BYO models, scale-to-zero and billing, APIs/CLIs, and host-trust notes. No cached dollar figures or invented benchmarks.
- Fully local, private agent (Stack): A local OpenAI-compatible model server, an optional LiteLLM router, Pi as the harness, Basic Memory for on-disk Markdown memory, and Anthropic's sandbox runtime — all on your machine, with no hosted model API and no cloud sandbox.
- Probe an OpenAI-compatible endpoint (Skill): A read-only-by-default Python probe that lists models on an OpenAI-compatible /v1 endpoint, with opt-in chat and tool-call checks, loopback default, and keys from env only.
Sources
- https://www.litellm.ai/
- https://www.litellm.ai/pricing
- https://docs.litellm.ai/docs/
- https://github.com/BerriAI/litellm
- https://github.com/BerriAI/litellm/blob/main/LICENSE
- https://github.com/BerriAI/litellm/blob/main/litellm/enterprise/LICENSE.md
- https://github.com/BerriAI/litellm/blob/main/README.md
- https://github.com/BerriAI/litellm/releases/tag/v1.109.1
- https://pypi.org/project/litellm/
- https://pypi.org/project/litellm/1.109.1/
- https://docs.litellm.ai/blog/security-update-march-2026
- https://docs.litellm.ai/blog/security-incident-oct-21-2026
- https://docs.litellm.ai/blog/security-incident-oct-21-2026-attack-chain
- https://docs.litellm.ai/docs/proxy/security_best_practices
- https://docs.litellm.ai/docs/proxy/config_settings
- https://docs.litellm.ai/docs/proxy/master_key_rotations
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)