llama.cpp
MIT C/C++ library and server for local LLM inference, widely used as a local OpenAI-compatible backend for coding agents.
Tags
Description
Established open project (canonical repo ggml-org/llama.cpp; not archived; created 2023-03-10; not presented as a 2026 launch). Root LICENSE is MIT (Copyright 2023-2026 The ggml authors). GitHub latest release v1.0.0 was published 2026-11-13T10:20:19Z (2026-11-13 11:20:19 Europe/Stockholm); it is not a draft or prerelease. README install options include the documented curl of https://llama.app/install.sh (that script was not executed), Docker, pre-built release binaries, or building from source (docs/build.md). https://llama.app (HTTP 200) is also a Mac/Windows desktop app described as being by the llama.cpp team and Hugging Face, exposing a local OpenAI-compatible API; this entry is the llama.cpp project, not a separate listing of that app. Do not confuse this with the third-party PyPI package llama-cpp-python (abetlen/llama-cpp-python), which is not the official install path for this repository. Model weights and any GPU or host resources are licensed and billed separately. Local servers can execute model and tool traffic you point at them.
Agent access
Related
- Ollama: MIT app and API for running open models locally, plus a paid Ollama Cloud with usage credits and per-token prices.
Guides & comparisons
- Run an agent on a local model with an OpenAI-compatible endpoint (Guide): Start Ollama, vLLM, SGLang or LocalAI bound to 127.0.0.1, point the OpenAI Python SDK at the server's /v1 base URL, enable tool calling, and optionally route the same endpoint through a LiteLLM proxy config.
- Ollama vs llama.cpp for agents — tools, OpenAI APIs, grammars and local serving (Comparison): Compare Ollama and llama.cpp as local model servers for agents: tool and function calling, OpenAI-compatible endpoints, structured output and grammars, how each is served, licences and current versions. Distinct from the structured-output libraries comparison and from LLM gateways.
- Fully local, private agent (Stack): A local OpenAI-compatible model server, an optional LiteLLM router, Pi as the harness, Basic Memory for on-disk Markdown memory, and Anthropic's sandbox runtime — all on your machine, with no hosted model API and no cloud sandbox.
- Probe an OpenAI-compatible endpoint (Skill): A read-only-by-default Python probe that lists models on an OpenAI-compatible /v1 endpoint, with opt-in chat and tool-call checks, loopback default, and keys from env only.
Sources
- https://github.com/ggml-org/llama.cpp
- https://github.com/ggml-org/llama.cpp/blob/master/LICENSE
- https://github.com/ggml-org/llama.cpp/blob/master/README.md
- https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0
- https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md
- https://llama.app/
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)