LocalAI
MIT self-hosted OpenAI-compatible engine that runs LLM, vision, voice and image backends locally, with Docker images as the documented install.
Tags
Description
Established open project (canonical repo mudler/LocalAI; not archived; created 2023-03-18; last pushed 2026-11-14; not presented as a 2026 launch). github.com/go-skynet/LocalAI 301s to mudler/LocalAI and is not used as a URL, repo or source. Root LICENSE is MIT (Copyright (c) 2023-2025 Ettore Di Giacinto (mudler@localai.io)). Site https://localai.io/ and docs https://localai.io/docs/ were HTTP 200; https://localai.io/docs 301s to the trailing-slash URL and is not used. GitHub latest release v4.12.1 was published 2026-10-23T10:27:00Z = 2026-10-23 11:27:00 Europe/Stockholm (not draft/prerelease). PyPI project localai 0.0.26 is a different package (vietanhdev/pautobot, “Private AutoGPT Robot”) and is not the official install path; do not pip install localai for this project. README documented install is Docker (for example docker run -ti --name local-ai -p 8080:8080 localai/localai:latest, plus GPU-tagged images). Those commands were not executed. README: a small core pulls backends (llama.cpp, vLLM, SGLang, whisper.cpp, MLX and others are named) as OCI images when a model needs them. A built-in agent can read files and run commands on the host, asking for approval before changes — treat that agent as potentially destructive. Add-ons (Studio, Talk, Voice, Face, Simple Office, Sandbox and more) install from a signed catalog. macOS DMG is documented as not signed by Apple (quarantine xattr note in the README). Model weights and any GPU or host resources are licensed and billed separately.
Agent access
Related
- llama.cpp: MIT C/C++ library and server for local LLM inference, widely used as a local OpenAI-compatible backend for coding agents.
- Ollama: MIT app and API for running open models locally, plus a paid Ollama Cloud with usage credits and per-token prices.
- vLLM: Apache-2.0 high-throughput LLM inference and serving engine with an OpenAI-compatible API, installable via uv or pip.
- SGLang: Apache-2.0 inference framework for LLM, vision-language and diffusion models, optimized for agentic workloads and large-scale serving.
- MLX LM: MIT Python package for text generation and fine-tuning of large language models on Apple silicon with MLX.
Guides & comparisons
- Run an agent on a local model with an OpenAI-compatible endpoint (Guide): Start Ollama, vLLM, SGLang or LocalAI bound to 127.0.0.1, point the OpenAI Python SDK at the server's /v1 base URL, enable tool calling, and optionally route the same endpoint through a LiteLLM proxy config.
- Fully local, private agent (Stack): A local OpenAI-compatible model server, an optional LiteLLM router, Pi as the harness, Basic Memory for on-disk Markdown memory, and Anthropic's sandbox runtime — all on your machine, with no hosted model API and no cloud sandbox.
- Probe an OpenAI-compatible endpoint (Skill): A read-only-by-default Python probe that lists models on an OpenAI-compatible /v1 endpoint, with opt-in chat and tool-call checks, loopback default, and keys from env only.
Sources
- https://localai.io/
- https://localai.io/docs/
- https://localai.io/docs/start/quickstart/
- https://github.com/mudler/LocalAI
- https://github.com/mudler/LocalAI/blob/master/LICENSE
- https://github.com/mudler/LocalAI/blob/master/README.md
- https://github.com/mudler/LocalAI/releases/tag/v4.12.1
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)