{
  "type": "stack",
  "id": "fully-local-private-agent",
  "title": "Fully local, private agent",
  "summary": "A local OpenAI-compatible model server, an optional LiteLLM router, Pi as the harness, Basic Memory for on-disk Markdown memory, and Anthropic's sandbox runtime — all on your machine, with no hosted model API and no cloud sandbox.",
  "author": "Agentica Author",
  "tags": [
    "local-models",
    "privacy",
    "inference",
    "memory",
    "sandboxes",
    "agents"
  ],
  "published": "2026-11-15",
  "last_verified": "2026-11-15",
  "entries": [
    "ollama",
    "litellm",
    "pi-coding-agent",
    "basic-memory",
    "sandbox-runtime",
    "vllm",
    "sglang",
    "localai",
    "llama-cpp",
    "claude-code",
    "e2b"
  ],
  "links": {
    "html": "https://indexagentica.com/stacks/fully-local-private-agent/",
    "markdown": "https://indexagentica.com/stacks/fully-local-private-agent.md",
    "json": "https://indexagentica.com/api/longform/stacks/fully-local-private-agent.json",
    "source": "https://github.com/Drudley/indexagentica/blob/main/content-long/stacks/fully-local-private-agent.md"
  },
  "status": "published",
  "stack": {
    "use_case": "Run a coding agent on your machine with a loopback model server, optional local router, local-first Markdown memory and an OS-level sandbox, without a hosted model API or a cloud sandbox.",
    "components": [
      {
        "role": "Local model server",
        "entry": "ollama",
        "name": "Ollama",
        "url": "https://indexagentica.com/entries/ollama/",
        "why": "MIT local app and server that binds 127.0.0.1:11434 by default and exposes http://localhost:11434/v1. Swap in vLLM, SGLang, LocalAI or llama.cpp; start any of them with the companion guide."
      },
      {
        "role": "Optional router",
        "entry": "litellm",
        "name": "LiteLLM",
        "url": "https://indexagentica.com/entries/litellm/",
        "why": "MIT OpenAI-compatible proxy you can bind to 127.0.0.1:4000 and point at the local server. One client URL if you change backends. The proxy will not start in current releases without LITELLM_MASTER_KEY."
      },
      {
        "role": "Harness",
        "entry": "pi-coding-agent",
        "name": "Pi",
        "url": "https://indexagentica.com/entries/pi-coding-agent/",
        "why": "MIT terminal agent that documents models.json endpoints for Ollama, vLLM, SGLang and other OpenAI-compatible servers, plus a first-party llama.cpp router path."
      },
      {
        "role": "Memory",
        "entry": "basic-memory",
        "name": "Basic Memory",
        "url": "https://indexagentica.com/entries/basic-memory/",
        "why": "AGPL-3.0 local-first MCP memory stored as Markdown you keep on disk. Install with the documented local command; this stack does not use Basic Memory's hosted cloud path."
      },
      {
        "role": "Sandbox",
        "entry": "sandbox-runtime",
        "name": "Anthropic Sandbox Runtime (srt)",
        "url": "https://indexagentica.com/entries/sandbox-runtime/",
        "why": "Apache-2.0 OS-level sandbox (sandbox-exec, bubblewrap, or Windows srt-sandbox). Defaults deny network and writes until allowed; reads are allowed everywhere unless denied. Officially a beta research preview."
      }
    ]
  },
  "entries_detail": [
    {
      "id": "ollama",
      "name": "Ollama",
      "summary": "MIT app and API for running open models locally, plus a paid Ollama Cloud with usage credits and per-token prices.",
      "url": "https://indexagentica.com/entries/ollama/",
      "json": "https://indexagentica.com/api/entries/ollama.json"
    },
    {
      "id": "litellm",
      "name": "LiteLLM",
      "summary": "MIT Python SDK and self-hosted AI gateway that calls 100+ LLM providers in the OpenAI format. Paid Enterprise is separate and quote-based.",
      "url": "https://indexagentica.com/entries/litellm/",
      "json": "https://indexagentica.com/api/entries/litellm.json"
    },
    {
      "id": "pi-coding-agent",
      "name": "Pi",
      "summary": "Minimal, extensible coding-agent harness and TypeScript toolkit (unified LLM API, agent core, TUI) from Earendil; customize with extensions, skills and prompt templates.",
      "url": "https://indexagentica.com/entries/pi-coding-agent/",
      "json": "https://indexagentica.com/api/entries/pi-coding-agent.json"
    },
    {
      "id": "basic-memory",
      "name": "Basic Memory",
      "summary": "Local-first MCP server that gives AI assistants persistent memory stored as Markdown files in a knowledge base you own.",
      "url": "https://indexagentica.com/entries/basic-memory/",
      "json": "https://indexagentica.com/api/entries/basic-memory.json"
    },
    {
      "id": "sandbox-runtime",
      "name": "Anthropic Sandbox Runtime (srt)",
      "summary": "Lightweight OS-level sandbox enforcing filesystem and network restrictions on agents, MCP servers and arbitrary processes, without a container.",
      "url": "https://indexagentica.com/entries/sandbox-runtime/",
      "json": "https://indexagentica.com/api/entries/sandbox-runtime.json"
    },
    {
      "id": "vllm",
      "name": "vLLM",
      "summary": "Apache-2.0 high-throughput LLM inference and serving engine with an OpenAI-compatible API, installable via uv or pip.",
      "url": "https://indexagentica.com/entries/vllm/",
      "json": "https://indexagentica.com/api/entries/vllm.json"
    },
    {
      "id": "sglang",
      "name": "SGLang",
      "summary": "Apache-2.0 inference framework for LLM, vision-language and diffusion models, optimized for agentic workloads and large-scale serving.",
      "url": "https://indexagentica.com/entries/sglang/",
      "json": "https://indexagentica.com/api/entries/sglang.json"
    },
    {
      "id": "localai",
      "name": "LocalAI",
      "summary": "MIT self-hosted OpenAI-compatible engine that runs LLM, vision, voice and image backends locally, with Docker images as the documented install.",
      "url": "https://indexagentica.com/entries/localai/",
      "json": "https://indexagentica.com/api/entries/localai.json"
    },
    {
      "id": "llama-cpp",
      "name": "llama.cpp",
      "summary": "MIT C/C++ library and server for local LLM inference, widely used as a local OpenAI-compatible backend for coding agents.",
      "url": "https://indexagentica.com/entries/llama-cpp/",
      "json": "https://indexagentica.com/api/entries/llama-cpp.json"
    },
    {
      "id": "claude-code",
      "name": "Claude Code",
      "summary": "Anthropic's agentic coding tool that reads your codebase, edits files, runs commands and integrates with dev tools; available in terminal, IDE, desktop and browser.",
      "url": "https://indexagentica.com/entries/claude-code/",
      "json": "https://indexagentica.com/api/entries/claude-code.json"
    },
    {
      "id": "e2b",
      "name": "E2B",
      "summary": "Open-source, secure cloud sandboxes for AI agents: an isolated machine per agent to run code, browse and use tools.",
      "url": "https://indexagentica.com/entries/e2b/",
      "json": "https://indexagentica.com/api/entries/e2b.json"
    }
  ],
  "related": [
    {
      "type": "guide",
      "id": "run-an-agent-on-a-local-model",
      "title": "Run an agent on a local model with an OpenAI-compatible endpoint",
      "url": "https://indexagentica.com/guides/run-an-agent-on-a-local-model/",
      "json": "https://indexagentica.com/api/longform/guides/run-an-agent-on-a-local-model.json"
    },
    {
      "type": "comparison",
      "id": "local-llm-servers-for-agents",
      "title": "Ollama vs llama.cpp for agents — tools, OpenAI APIs, grammars and local serving",
      "url": "https://indexagentica.com/compare/local-llm-servers-for-agents/",
      "json": "https://indexagentica.com/api/longform/compare/local-llm-servers-for-agents.json"
    },
    {
      "type": "comparison",
      "id": "llm-gateways",
      "title": "LLM gateways compared: OpenRouter vs Vercel AI Gateway vs Cloudflare AI Gateway vs Portkey vs LiteLLM",
      "url": "https://indexagentica.com/compare/llm-gateways/",
      "json": "https://indexagentica.com/api/longform/compare/llm-gateways.json"
    },
    {
      "type": "guide",
      "id": "give-an-agent-long-term-memory",
      "title": "Give an agent long-term memory",
      "url": "https://indexagentica.com/guides/give-an-agent-long-term-memory/",
      "json": "https://indexagentica.com/api/longform/guides/give-an-agent-long-term-memory.json"
    },
    {
      "type": "guide",
      "id": "run-untrusted-code-in-a-sandbox",
      "title": "Run agent-generated code safely in a sandbox",
      "url": "https://indexagentica.com/guides/run-untrusted-code-in-a-sandbox/",
      "json": "https://indexagentica.com/api/longform/guides/run-untrusted-code-in-a-sandbox.json"
    },
    {
      "type": "comparison",
      "id": "self-hosted-agent-sandboxes",
      "title": "Self-hosted sandboxes for agents compared",
      "url": "https://indexagentica.com/compare/self-hosted-agent-sandboxes/",
      "json": "https://indexagentica.com/api/longform/compare/self-hosted-agent-sandboxes.json"
    },
    {
      "type": "comparison",
      "id": "agent-memory-layers",
      "title": "Memory layers for AI agents",
      "url": "https://indexagentica.com/compare/agent-memory-layers/",
      "json": "https://indexagentica.com/api/longform/compare/agent-memory-layers.json"
    },
    {
      "type": "comparison",
      "id": "coding-agent-memory-servers",
      "title": "Memory servers for coding agents compared: m3 Memory vs agentmemory vs Basic Memory vs MCP Memory Service",
      "url": "https://indexagentica.com/compare/coding-agent-memory-servers/",
      "json": "https://indexagentica.com/api/longform/compare/coding-agent-memory-servers.json"
    },
    {
      "type": "stack",
      "id": "coding-agent-starter",
      "title": "Coding agent starter stack",
      "url": "https://indexagentica.com/stacks/coding-agent-starter/",
      "json": "https://indexagentica.com/api/longform/stacks/coding-agent-starter.json"
    },
    {
      "type": "comparison",
      "id": "personal-agent-runtimes",
      "title": "OpenClaw vs Hermes Agent vs Pi: open-source agents you run yourself",
      "url": "https://indexagentica.com/compare/personal-agent-runtimes/",
      "json": "https://indexagentica.com/api/longform/compare/personal-agent-runtimes.json"
    },
    {
      "type": "comparison",
      "id": "coding-agent-harnesses",
      "title": "Terminal coding agents compared",
      "url": "https://indexagentica.com/compare/coding-agent-harnesses/",
      "json": "https://indexagentica.com/api/longform/compare/coding-agent-harnesses.json"
    }
  ],
  "sources": [
    {
      "title": "Ollama, OpenAI compatibility",
      "url": "https://docs.ollama.com/api/openai-compatibility",
      "accessed": "2026-11-15"
    },
    {
      "title": "Ollama FAQ",
      "url": "https://docs.ollama.com/faq",
      "accessed": "2026-11-15"
    },
    {
      "title": "Pi docs, Models",
      "url": "https://docs.pi.dev/latest/models",
      "accessed": "2026-11-15"
    },
    {
      "title": "Pi docs, Local models with llama.cpp",
      "url": "https://docs.pi.dev/latest/llama-cpp",
      "accessed": "2026-11-15"
    },
    {
      "title": "Pi docs, latest landing page",
      "url": "https://docs.pi.dev/latest",
      "accessed": "2026-11-15"
    },
    {
      "title": "LiteLLM docs, OpenAI-compatible endpoints",
      "url": "https://docs.litellm.ai/docs/providers/openai_compatible",
      "accessed": "2026-11-15"
    },
    {
      "title": "LiteLLM docs, Config settings",
      "url": "https://docs.litellm.ai/docs/proxy/config_settings",
      "accessed": "2026-11-15"
    },
    {
      "title": "LiteLLM docs, Docker quick start",
      "url": "https://docs.litellm.ai/docs/proxy/docker_quick_start",
      "accessed": "2026-11-15"
    },
    {
      "title": "Basic Memory README",
      "url": "https://raw.githubusercontent.com/basicmachines-co/basic-memory/main/README.md",
      "accessed": "2026-11-15"
    },
    {
      "title": "Anthropic sandbox-runtime README",
      "url": "https://raw.githubusercontent.com/anthropics/sandbox-runtime/main/README.md",
      "accessed": "2026-11-15"
    },
    {
      "title": "Anthropic sandbox-runtime LICENSE",
      "url": "https://raw.githubusercontent.com/anthropics/sandbox-runtime/main/LICENSE",
      "accessed": "2026-11-15"
    }
  ],
  "front_matter": {
    "id": "fully-local-private-agent",
    "type": "stack",
    "title": "Fully local, private agent",
    "summary": "A local OpenAI-compatible model server, an optional LiteLLM router, Pi as the harness, Basic Memory for on-disk Markdown memory, and Anthropic's sandbox runtime — all on your machine, with no hosted model API and no cloud sandbox.",
    "description": "Assemble a private coding agent from catalog entries that already run locally: Ollama (or another loopback OpenAI-compatible server), optional LiteLLM, Pi, Basic Memory's local install, and sandbox-runtime. Distinct from the coding-agent-starter stack, which uses Claude Code, hosted MCP and E2B. Start the model server with the companion guide; this page only wires the pieces and records licence and default-deny caveats.",
    "author": "Agentica Author",
    "use_case": "Run a coding agent on your machine with a loopback model server, optional local router, local-first Markdown memory and an OS-level sandbox, without a hosted model API or a cloud sandbox.",
    "components": [
      {
        "role": "Local model server",
        "entry": "ollama",
        "why": "MIT local app and server that binds 127.0.0.1:11434 by default and exposes http://localhost:11434/v1. Swap in vLLM, SGLang, LocalAI or llama.cpp; start any of them with the companion guide."
      },
      {
        "role": "Optional router",
        "entry": "litellm",
        "why": "MIT OpenAI-compatible proxy you can bind to 127.0.0.1:4000 and point at the local server. One client URL if you change backends. The proxy will not start in current releases without LITELLM_MASTER_KEY."
      },
      {
        "role": "Harness",
        "entry": "pi-coding-agent",
        "why": "MIT terminal agent that documents models.json endpoints for Ollama, vLLM, SGLang and other OpenAI-compatible servers, plus a first-party llama.cpp router path."
      },
      {
        "role": "Memory",
        "entry": "basic-memory",
        "why": "AGPL-3.0 local-first MCP memory stored as Markdown you keep on disk. Install with the documented local command; this stack does not use Basic Memory's hosted cloud path."
      },
      {
        "role": "Sandbox",
        "entry": "sandbox-runtime",
        "why": "Apache-2.0 OS-level sandbox (sandbox-exec, bubblewrap, or Windows srt-sandbox). Defaults deny network and writes until allowed; reads are allowed everywhere unless denied. Officially a beta research preview."
      }
    ],
    "tags": [
      "local-models",
      "privacy",
      "inference",
      "memory",
      "sandboxes",
      "agents"
    ],
    "entries": [
      "ollama",
      "litellm",
      "pi-coding-agent",
      "basic-memory",
      "sandbox-runtime",
      "vllm",
      "sglang",
      "localai",
      "llama-cpp"
    ],
    "related": [
      "run-an-agent-on-a-local-model",
      "local-llm-servers-for-agents",
      "llm-gateways",
      "give-an-agent-long-term-memory",
      "run-untrusted-code-in-a-sandbox",
      "self-hosted-agent-sandboxes",
      "agent-memory-layers",
      "coding-agent-memory-servers",
      "coding-agent-starter",
      "personal-agent-runtimes",
      "coding-agent-harnesses"
    ],
    "sources": [
      {
        "title": "Ollama, OpenAI compatibility",
        "url": "https://docs.ollama.com/api/openai-compatibility",
        "accessed": "2026-11-15"
      },
      {
        "title": "Ollama FAQ",
        "url": "https://docs.ollama.com/faq",
        "accessed": "2026-11-15"
      },
      {
        "title": "Pi docs, Models",
        "url": "https://docs.pi.dev/latest/models",
        "accessed": "2026-11-15"
      },
      {
        "title": "Pi docs, Local models with llama.cpp",
        "url": "https://docs.pi.dev/latest/llama-cpp",
        "accessed": "2026-11-15"
      },
      {
        "title": "Pi docs, latest landing page",
        "url": "https://docs.pi.dev/latest",
        "accessed": "2026-11-15"
      },
      {
        "title": "LiteLLM docs, OpenAI-compatible endpoints",
        "url": "https://docs.litellm.ai/docs/providers/openai_compatible",
        "accessed": "2026-11-15"
      },
      {
        "title": "LiteLLM docs, Config settings",
        "url": "https://docs.litellm.ai/docs/proxy/config_settings",
        "accessed": "2026-11-15"
      },
      {
        "title": "LiteLLM docs, Docker quick start",
        "url": "https://docs.litellm.ai/docs/proxy/docker_quick_start",
        "accessed": "2026-11-15"
      },
      {
        "title": "Basic Memory README",
        "url": "https://raw.githubusercontent.com/basicmachines-co/basic-memory/main/README.md",
        "accessed": "2026-11-15"
      },
      {
        "title": "Anthropic sandbox-runtime README",
        "url": "https://raw.githubusercontent.com/anthropics/sandbox-runtime/main/README.md",
        "accessed": "2026-11-15"
      },
      {
        "title": "Anthropic sandbox-runtime LICENSE",
        "url": "https://raw.githubusercontent.com/anthropics/sandbox-runtime/main/LICENSE",
        "accessed": "2026-11-15"
      }
    ],
    "last_verified": "2026-11-15",
    "published": "2026-11-15"
  },
  "markdown": "\n## When to use this stack\n\nUse this stack when the agent should stay on your machine: the model is served on loopback, memory is files you hold, and tool processes are sandboxed locally. It is the opposite of [Coding agent starter](https://indexagentica.com/stacks/coding-agent-starter/), which uses [Claude Code](https://indexagentica.com/entries/claude-code/) as the harness, hosted MCP servers and [E2B](https://indexagentica.com/entries/e2b/) as a cloud microVM.\n\nThis page does not teach you how to start vLLM, SGLang, Ollama or LocalAI, or how to place LiteLLM in front of them. That is [Run an agent on a local model with an OpenAI-compatible endpoint](https://indexagentica.com/guides/run-an-agent-on-a-local-model/). Start there, then come back to wire the harness, memory and sandbox.\n\nIt is also not a substitute for [Ollama vs llama.cpp for agents](https://indexagentica.com/compare/local-llm-servers-for-agents/), [LLM gateways compared](https://indexagentica.com/compare/llm-gateways/), [Memory layers for AI agents](https://indexagentica.com/compare/agent-memory-layers/), [Self-hosted sandboxes for agents](https://indexagentica.com/compare/self-hosted-agent-sandboxes/) or [Give an agent long-term memory](https://indexagentica.com/guides/give-an-agent-long-term-memory/). Those pages compare alternatives; this stack picks one local set and shows how they fit.\n\n## The pieces\n\n**Model server.** Start with [Ollama](https://indexagentica.com/entries/ollama/) if you want the default loopback bind (`127.0.0.1:11434`, per the [FAQ](https://docs.ollama.com/faq)) and an OpenAI-compatible URL at `http://localhost:11434/v1`. If you already run [vLLM](https://indexagentica.com/entries/vllm/), [SGLang](https://indexagentica.com/entries/sglang/), [LocalAI](https://indexagentica.com/entries/localai/) or [llama.cpp](https://indexagentica.com/entries/llama-cpp/), keep that server; the companion guide shows the loopback flags. llama.cpp is a first-class option in [Pi](https://indexagentica.com/entries/pi-coding-agent/) (see below) and is compared with Ollama on the serving-layer page, not repeated here.\n\n**Optional router.** Put [LiteLLM](https://indexagentica.com/entries/litellm/) on `127.0.0.1:4000` when you want one client base URL in front of whichever local server you started. Current LiteLLM releases will not start the proxy without `LITELLM_MASTER_KEY` (or `general_settings.master_key`). Official Docker examples already use `-p 127.0.0.1:4000:4000`. Skip the router if the harness can talk to the model server directly.\n\n**Harness.** [Pi](https://indexagentica.com/entries/pi-coding-agent/) is a MIT terminal agent whose [models](https://docs.pi.dev/latest/models) page covers local GGUF files through a llama.cpp router (`/llama`) and OpenAI-, Anthropic- or Google-compatible endpoints in `models.json`. It names Ollama, LM Studio, vLLM, SGLang and other compatible servers as the `models.json` path. Pi is also one of the subjects in [OpenClaw vs Hermes Agent vs Pi](https://indexagentica.com/compare/personal-agent-runtimes/). You can swap in another local-capable harness from [Terminal coding agents compared](https://indexagentica.com/compare/coding-agent-harnesses/) if you prefer; the rest of the stack does not depend on Pi-only APIs.\n\n**Memory.** [Basic Memory](https://indexagentica.com/entries/basic-memory/) is an AGPL-3.0 local-first MCP server that stores memory as Markdown in a knowledge base you own. The documented local install is:\n\n```bash\nuv tool install basic-memory --prerelease=allow\n```\n\nThe project README also offers a hosted cloud path (\"Skip the install — try Basic Memory in the cloud\"). That path is out of scope for this stack. Stay on the local install, and see [Give an agent long-term memory](https://indexagentica.com/guides/give-an-agent-long-term-memory/) and [Memory servers for coding agents](https://indexagentica.com/compare/coding-agent-memory-servers/) if you need to compare other memory layers.\n\n**Sandbox.** [sandbox-runtime](https://indexagentica.com/entries/sandbox-runtime/) (`srt`) is Apache-2.0 and officially a beta research preview: APIs and configuration may change. Install with the documented:\n\n```bash\nnpm install -g @anthropic-ai/sandbox-runtime\n```\n\nIt uses `sandbox-exec` on macOS, `bubblewrap` on Linux, and a Windows `srt-sandbox` user. The README's defaults: network is denied until you allow domains; writes are denied until you allow paths; reads are allowed everywhere unless you add a `denyRead` rule. For how this compares with other self-hosted sandboxes, and for the broader \"treat output as untrusted\" recipe, see [Self-hosted sandboxes for agents](https://indexagentica.com/compare/self-hosted-agent-sandboxes/) and [Run agent-generated code safely in a sandbox](https://indexagentica.com/guides/run-untrusted-code-in-a-sandbox/).\n\n## Wire it\n\n1. Start the model server on loopback, following [the companion guide](https://indexagentica.com/guides/run-an-agent-on-a-local-model/). Confirm you can complete a chat request with the OpenAI Python SDK against that `/v1` URL.\n2. Optionally start LiteLLM on `127.0.0.1:4000` with `LITELLM_MASTER_KEY` set, pointing `api_base` at that same server. Clients then use `http://127.0.0.1:4000/v1` and the master key.\n3. Point Pi at the endpoint. The official `models.json` example for Ollama (from the [models](https://docs.pi.dev/latest/models) page) is:\n\n```json\n{\n  \"$schema\": \"https://pi.dev/schemas/models.schema.json\",\n  \"providers\": {\n    \"ollama\": {\n      \"baseUrl\": \"http://localhost:11434/v1\",\n      \"api\": \"openai-completions\",\n      \"apiKey\": \"ollama\",\n      \"models\": [\n        { \"id\": \"qwen2.5-coder:7b\" }\n      ]\n    }\n  }\n}\n```\n\nThe same page says the dummy key makes the model available to Pi and that Ollama ignores it. To go through LiteLLM instead, use the proxy's loopback `/v1` URL as `baseUrl` and the master key as `apiKey` (via `$LITELLM_MASTER_KEY` or `${LITELLM_MASTER_KEY}`, which Pi documents for environment interpolation). Swap the model `id` for one you have actually pulled; the `qwen2.5-coder:7b` value is Pi's example, not a model downloaded for this page. For GGUF files on disk, follow Pi's [llama.cpp](https://docs.pi.dev/latest/llama-cpp) path rather than this JSON snippet.\n\n4. Add Basic Memory as a local MCP server in the harness, using the local install, not the cloud product.\n5. Wrap the agent's command and tool processes with `srt` and a settings file that keeps network and writes on a default-deny list, then allow only the paths and hosts the agent should touch (for example the project directory, and `127.0.0.1` if the sandbox must reach the local model server).\n\n## Licences and limits\n\n- Ollama, LiteLLM's open-source SDK and proxy, Pi and llama.cpp are MIT. vLLM and SGLang are Apache-2.0. LocalAI is MIT. LiteLLM Enterprise is a separate, quote-based product and is not part of this stack.\n- Basic Memory is AGPL-3.0. If you distribute a modified server, that licence's conditions apply. Its hosted cloud offering is a different deployment, not this stack.\n- sandbox-runtime is Apache-2.0 and a beta research preview. Do not assume a stable config schema.\n- A \"fully local\" setup is only as private as its bind addresses, logs and tool egress. Leave every HTTP server on `127.0.0.1`, and do not open sandbox network to the public internet \"just in case\".\n",
  "raw": "---\nid: fully-local-private-agent\ntype: stack\ntitle: \"Fully local, private agent\"\nsummary: \"A local OpenAI-compatible model server, an optional LiteLLM router, Pi as the harness, Basic Memory for on-disk Markdown memory, and Anthropic's sandbox runtime — all on your machine, with no hosted model API and no cloud sandbox.\"\ndescription: \"Assemble a private coding agent from catalog entries that already run locally: Ollama (or another loopback OpenAI-compatible server), optional LiteLLM, Pi, Basic Memory's local install, and sandbox-runtime. Distinct from the coding-agent-starter stack, which uses Claude Code, hosted MCP and E2B. Start the model server with the companion guide; this page only wires the pieces and records licence and default-deny caveats.\"\nauthor: Agentica Author\nuse_case: \"Run a coding agent on your machine with a loopback model server, optional local router, local-first Markdown memory and an OS-level sandbox, without a hosted model API or a cloud sandbox.\"\ncomponents:\n  - role: Local model server\n    entry: ollama\n    why: \"MIT local app and server that binds 127.0.0.1:11434 by default and exposes http://localhost:11434/v1. Swap in vLLM, SGLang, LocalAI or llama.cpp; start any of them with the companion guide.\"\n  - role: Optional router\n    entry: litellm\n    why: \"MIT OpenAI-compatible proxy you can bind to 127.0.0.1:4000 and point at the local server. One client URL if you change backends. The proxy will not start in current releases without LITELLM_MASTER_KEY.\"\n  - role: Harness\n    entry: pi-coding-agent\n    why: \"MIT terminal agent that documents models.json endpoints for Ollama, vLLM, SGLang and other OpenAI-compatible servers, plus a first-party llama.cpp router path.\"\n  - role: Memory\n    entry: basic-memory\n    why: \"AGPL-3.0 local-first MCP memory stored as Markdown you keep on disk. Install with the documented local command; this stack does not use Basic Memory's hosted cloud path.\"\n  - role: Sandbox\n    entry: sandbox-runtime\n    why: \"Apache-2.0 OS-level sandbox (sandbox-exec, bubblewrap, or Windows srt-sandbox). Defaults deny network and writes until allowed; reads are allowed everywhere unless denied. Officially a beta research preview.\"\ntags: [local-models, privacy, inference, memory, sandboxes, agents]\nentries: [ollama, litellm, pi-coding-agent, basic-memory, sandbox-runtime, vllm, sglang, localai, llama-cpp]\nrelated: [run-an-agent-on-a-local-model, local-llm-servers-for-agents, llm-gateways, give-an-agent-long-term-memory, run-untrusted-code-in-a-sandbox, self-hosted-agent-sandboxes, agent-memory-layers, coding-agent-memory-servers, coding-agent-starter, personal-agent-runtimes, coding-agent-harnesses]\nsources:\n  - title: \"Ollama, OpenAI compatibility\"\n    url: https://docs.ollama.com/api/openai-compatibility\n    accessed: 2026-11-15\n  - title: \"Ollama FAQ\"\n    url: https://docs.ollama.com/faq\n    accessed: 2026-11-15\n  - title: \"Pi docs, Models\"\n    url: https://docs.pi.dev/latest/models\n    accessed: 2026-11-15\n  - title: \"Pi docs, Local models with llama.cpp\"\n    url: https://docs.pi.dev/latest/llama-cpp\n    accessed: 2026-11-15\n  - title: \"Pi docs, latest landing page\"\n    url: https://docs.pi.dev/latest\n    accessed: 2026-11-15\n  - title: \"LiteLLM docs, OpenAI-compatible endpoints\"\n    url: https://docs.litellm.ai/docs/providers/openai_compatible\n    accessed: 2026-11-15\n  - title: \"LiteLLM docs, Config settings\"\n    url: https://docs.litellm.ai/docs/proxy/config_settings\n    accessed: 2026-11-15\n  - title: \"LiteLLM docs, Docker quick start\"\n    url: https://docs.litellm.ai/docs/proxy/docker_quick_start\n    accessed: 2026-11-15\n  - title: \"Basic Memory README\"\n    url: https://raw.githubusercontent.com/basicmachines-co/basic-memory/main/README.md\n    accessed: 2026-11-15\n  - title: \"Anthropic sandbox-runtime README\"\n    url: https://raw.githubusercontent.com/anthropics/sandbox-runtime/main/README.md\n    accessed: 2026-11-15\n  - title: \"Anthropic sandbox-runtime LICENSE\"\n    url: https://raw.githubusercontent.com/anthropics/sandbox-runtime/main/LICENSE\n    accessed: 2026-11-15\nlast_verified: 2026-11-15\npublished: 2026-11-15\n---\n\n## When to use this stack\n\nUse this stack when the agent should stay on your machine: the model is served on loopback, memory is files you hold, and tool processes are sandboxed locally. It is the opposite of [Coding agent starter](https://indexagentica.com/stacks/coding-agent-starter/), which uses [Claude Code](https://indexagentica.com/entries/claude-code/) as the harness, hosted MCP servers and [E2B](https://indexagentica.com/entries/e2b/) as a cloud microVM.\n\nThis page does not teach you how to start vLLM, SGLang, Ollama or LocalAI, or how to place LiteLLM in front of them. That is [Run an agent on a local model with an OpenAI-compatible endpoint](https://indexagentica.com/guides/run-an-agent-on-a-local-model/). Start there, then come back to wire the harness, memory and sandbox.\n\nIt is also not a substitute for [Ollama vs llama.cpp for agents](https://indexagentica.com/compare/local-llm-servers-for-agents/), [LLM gateways compared](https://indexagentica.com/compare/llm-gateways/), [Memory layers for AI agents](https://indexagentica.com/compare/agent-memory-layers/), [Self-hosted sandboxes for agents](https://indexagentica.com/compare/self-hosted-agent-sandboxes/) or [Give an agent long-term memory](https://indexagentica.com/guides/give-an-agent-long-term-memory/). Those pages compare alternatives; this stack picks one local set and shows how they fit.\n\n## The pieces\n\n**Model server.** Start with [Ollama](https://indexagentica.com/entries/ollama/) if you want the default loopback bind (`127.0.0.1:11434`, per the [FAQ](https://docs.ollama.com/faq)) and an OpenAI-compatible URL at `http://localhost:11434/v1`. If you already run [vLLM](https://indexagentica.com/entries/vllm/), [SGLang](https://indexagentica.com/entries/sglang/), [LocalAI](https://indexagentica.com/entries/localai/) or [llama.cpp](https://indexagentica.com/entries/llama-cpp/), keep that server; the companion guide shows the loopback flags. llama.cpp is a first-class option in [Pi](https://indexagentica.com/entries/pi-coding-agent/) (see below) and is compared with Ollama on the serving-layer page, not repeated here.\n\n**Optional router.** Put [LiteLLM](https://indexagentica.com/entries/litellm/) on `127.0.0.1:4000` when you want one client base URL in front of whichever local server you started. Current LiteLLM releases will not start the proxy without `LITELLM_MASTER_KEY` (or `general_settings.master_key`). Official Docker examples already use `-p 127.0.0.1:4000:4000`. Skip the router if the harness can talk to the model server directly.\n\n**Harness.** [Pi](https://indexagentica.com/entries/pi-coding-agent/) is a MIT terminal agent whose [models](https://docs.pi.dev/latest/models) page covers local GGUF files through a llama.cpp router (`/llama`) and OpenAI-, Anthropic- or Google-compatible endpoints in `models.json`. It names Ollama, LM Studio, vLLM, SGLang and other compatible servers as the `models.json` path. Pi is also one of the subjects in [OpenClaw vs Hermes Agent vs Pi](https://indexagentica.com/compare/personal-agent-runtimes/). You can swap in another local-capable harness from [Terminal coding agents compared](https://indexagentica.com/compare/coding-agent-harnesses/) if you prefer; the rest of the stack does not depend on Pi-only APIs.\n\n**Memory.** [Basic Memory](https://indexagentica.com/entries/basic-memory/) is an AGPL-3.0 local-first MCP server that stores memory as Markdown in a knowledge base you own. The documented local install is:\n\n```bash\nuv tool install basic-memory --prerelease=allow\n```\n\nThe project README also offers a hosted cloud path (\"Skip the install — try Basic Memory in the cloud\"). That path is out of scope for this stack. Stay on the local install, and see [Give an agent long-term memory](https://indexagentica.com/guides/give-an-agent-long-term-memory/) and [Memory servers for coding agents](https://indexagentica.com/compare/coding-agent-memory-servers/) if you need to compare other memory layers.\n\n**Sandbox.** [sandbox-runtime](https://indexagentica.com/entries/sandbox-runtime/) (`srt`) is Apache-2.0 and officially a beta research preview: APIs and configuration may change. Install with the documented:\n\n```bash\nnpm install -g @anthropic-ai/sandbox-runtime\n```\n\nIt uses `sandbox-exec` on macOS, `bubblewrap` on Linux, and a Windows `srt-sandbox` user. The README's defaults: network is denied until you allow domains; writes are denied until you allow paths; reads are allowed everywhere unless you add a `denyRead` rule. For how this compares with other self-hosted sandboxes, and for the broader \"treat output as untrusted\" recipe, see [Self-hosted sandboxes for agents](https://indexagentica.com/compare/self-hosted-agent-sandboxes/) and [Run agent-generated code safely in a sandbox](https://indexagentica.com/guides/run-untrusted-code-in-a-sandbox/).\n\n## Wire it\n\n1. Start the model server on loopback, following [the companion guide](https://indexagentica.com/guides/run-an-agent-on-a-local-model/). Confirm you can complete a chat request with the OpenAI Python SDK against that `/v1` URL.\n2. Optionally start LiteLLM on `127.0.0.1:4000` with `LITELLM_MASTER_KEY` set, pointing `api_base` at that same server. Clients then use `http://127.0.0.1:4000/v1` and the master key.\n3. Point Pi at the endpoint. The official `models.json` example for Ollama (from the [models](https://docs.pi.dev/latest/models) page) is:\n\n```json\n{\n  \"$schema\": \"https://pi.dev/schemas/models.schema.json\",\n  \"providers\": {\n    \"ollama\": {\n      \"baseUrl\": \"http://localhost:11434/v1\",\n      \"api\": \"openai-completions\",\n      \"apiKey\": \"ollama\",\n      \"models\": [\n        { \"id\": \"qwen2.5-coder:7b\" }\n      ]\n    }\n  }\n}\n```\n\nThe same page says the dummy key makes the model available to Pi and that Ollama ignores it. To go through LiteLLM instead, use the proxy's loopback `/v1` URL as `baseUrl` and the master key as `apiKey` (via `$LITELLM_MASTER_KEY` or `${LITELLM_MASTER_KEY}`, which Pi documents for environment interpolation). Swap the model `id` for one you have actually pulled; the `qwen2.5-coder:7b` value is Pi's example, not a model downloaded for this page. For GGUF files on disk, follow Pi's [llama.cpp](https://docs.pi.dev/latest/llama-cpp) path rather than this JSON snippet.\n\n4. Add Basic Memory as a local MCP server in the harness, using the local install, not the cloud product.\n5. Wrap the agent's command and tool processes with `srt` and a settings file that keeps network and writes on a default-deny list, then allow only the paths and hosts the agent should touch (for example the project directory, and `127.0.0.1` if the sandbox must reach the local model server).\n\n## Licences and limits\n\n- Ollama, LiteLLM's open-source SDK and proxy, Pi and llama.cpp are MIT. vLLM and SGLang are Apache-2.0. LocalAI is MIT. LiteLLM Enterprise is a separate, quote-based product and is not part of this stack.\n- Basic Memory is AGPL-3.0. If you distribute a modified server, that licence's conditions apply. Its hosted cloud offering is a different deployment, not this stack.\n- sandbox-runtime is Apache-2.0 and a beta research preview. Do not assume a stable config schema.\n- A \"fully local\" setup is only as private as its bind addresses, logs and tool egress. Leave every HTTP server on `127.0.0.1`, and do not open sandbox network to the public internet \"just in case\".\n",
  "html": "<h2>When to use this stack</h2>\n<p>Use this stack when the agent should stay on your machine: the model is served on loopback, memory is files you hold, and tool processes are sandboxed locally. It is the opposite of <a href=\"/stacks/coding-agent-starter/\">Coding agent starter</a>, which uses <a href=\"/entries/claude-code/\">Claude Code</a> as the harness, hosted MCP servers and <a href=\"/entries/e2b/\">E2B</a> as a cloud microVM.</p>\n<p>This page does not teach you how to start vLLM, SGLang, Ollama or LocalAI, or how to place LiteLLM in front of them. That is <a href=\"/guides/run-an-agent-on-a-local-model/\">Run an agent on a local model with an OpenAI-compatible endpoint</a>. Start there, then come back to wire the harness, memory and sandbox.</p>\n<p>It is also not a substitute for <a href=\"/compare/local-llm-servers-for-agents/\">Ollama vs llama.cpp for agents</a>, <a href=\"/compare/llm-gateways/\">LLM gateways compared</a>, <a href=\"/compare/agent-memory-layers/\">Memory layers for AI agents</a>, <a href=\"/compare/self-hosted-agent-sandboxes/\">Self-hosted sandboxes for agents</a> or <a href=\"/guides/give-an-agent-long-term-memory/\">Give an agent long-term memory</a>. Those pages compare alternatives; this stack picks one local set and shows how they fit.</p>\n<h2>The pieces</h2>\n<p><strong>Model server.</strong> Start with <a href=\"/entries/ollama/\">Ollama</a> if you want the default loopback bind (<code>127.0.0.1:11434</code>, per the <a href=\"https://docs.ollama.com/faq\">FAQ</a>) and an OpenAI-compatible URL at <code>http://localhost:11434/v1</code>. If you already run <a href=\"/entries/vllm/\">vLLM</a>, <a href=\"/entries/sglang/\">SGLang</a>, <a href=\"/entries/localai/\">LocalAI</a> or <a href=\"/entries/llama-cpp/\">llama.cpp</a>, keep that server; the companion guide shows the loopback flags. llama.cpp is a first-class option in <a href=\"/entries/pi-coding-agent/\">Pi</a> (see below) and is compared with Ollama on the serving-layer page, not repeated here.</p>\n<p><strong>Optional router.</strong> Put <a href=\"/entries/litellm/\">LiteLLM</a> on <code>127.0.0.1:4000</code> when you want one client base URL in front of whichever local server you started. Current LiteLLM releases will not start the proxy without <code>LITELLM_MASTER_KEY</code> (or <code>general_settings.master_key</code>). Official Docker examples already use <code>-p 127.0.0.1:4000:4000</code>. Skip the router if the harness can talk to the model server directly.</p>\n<p><strong>Harness.</strong> <a href=\"/entries/pi-coding-agent/\">Pi</a> is a MIT terminal agent whose <a href=\"https://docs.pi.dev/latest/models\">models</a> page covers local GGUF files through a llama.cpp router (<code>/llama</code>) and OpenAI-, Anthropic- or Google-compatible endpoints in <code>models.json</code>. It names Ollama, LM Studio, vLLM, SGLang and other compatible servers as the <code>models.json</code> path. Pi is also one of the subjects in <a href=\"/compare/personal-agent-runtimes/\">OpenClaw vs Hermes Agent vs Pi</a>. You can swap in another local-capable harness from <a href=\"/compare/coding-agent-harnesses/\">Terminal coding agents compared</a> if you prefer; the rest of the stack does not depend on Pi-only APIs.</p>\n<p><strong>Memory.</strong> <a href=\"/entries/basic-memory/\">Basic Memory</a> is an AGPL-3.0 local-first MCP server that stores memory as Markdown in a knowledge base you own. The documented local install is:</p>\n<pre><code class=\"language-bash\">uv tool install basic-memory --prerelease=allow</code></pre>\n<p>The project README also offers a hosted cloud path (&quot;Skip the install — try Basic Memory in the cloud&quot;). That path is out of scope for this stack. Stay on the local install, and see <a href=\"/guides/give-an-agent-long-term-memory/\">Give an agent long-term memory</a> and <a href=\"/compare/coding-agent-memory-servers/\">Memory servers for coding agents</a> if you need to compare other memory layers.</p>\n<p><strong>Sandbox.</strong> <a href=\"/entries/sandbox-runtime/\">sandbox-runtime</a> (<code>srt</code>) is Apache-2.0 and officially a beta research preview: APIs and configuration may change. Install with the documented:</p>\n<pre><code class=\"language-bash\">npm install -g @anthropic-ai/sandbox-runtime</code></pre>\n<p>It uses <code>sandbox-exec</code> on macOS, <code>bubblewrap</code> on Linux, and a Windows <code>srt-sandbox</code> user. The README&#39;s defaults: network is denied until you allow domains; writes are denied until you allow paths; reads are allowed everywhere unless you add a <code>denyRead</code> rule. For how this compares with other self-hosted sandboxes, and for the broader &quot;treat output as untrusted&quot; recipe, see <a href=\"/compare/self-hosted-agent-sandboxes/\">Self-hosted sandboxes for agents</a> and <a href=\"/guides/run-untrusted-code-in-a-sandbox/\">Run agent-generated code safely in a sandbox</a>.</p>\n<h2>Wire it</h2>\n<ol><li>Start the model server on loopback, following <a href=\"/guides/run-an-agent-on-a-local-model/\">the companion guide</a>. Confirm you can complete a chat request with the OpenAI Python SDK against that <code>/v1</code> URL.</li><li>Optionally start LiteLLM on <code>127.0.0.1:4000</code> with <code>LITELLM_MASTER_KEY</code> set, pointing <code>api_base</code> at that same server. Clients then use <code>http://127.0.0.1:4000/v1</code> and the master key.</li><li>Point Pi at the endpoint. The official <code>models.json</code> example for Ollama (from the <a href=\"https://docs.pi.dev/latest/models\">models</a> page) is:</li></ol>\n<pre><code class=\"language-json\">{\n  &quot;$schema&quot;: &quot;https://pi.dev/schemas/models.schema.json&quot;,\n  &quot;providers&quot;: {\n    &quot;ollama&quot;: {\n      &quot;baseUrl&quot;: &quot;http://localhost:11434/v1&quot;,\n      &quot;api&quot;: &quot;openai-completions&quot;,\n      &quot;apiKey&quot;: &quot;ollama&quot;,\n      &quot;models&quot;: [\n        { &quot;id&quot;: &quot;qwen2.5-coder:7b&quot; }\n      ]\n    }\n  }\n}</code></pre>\n<p>The same page says the dummy key makes the model available to Pi and that Ollama ignores it. To go through LiteLLM instead, use the proxy&#39;s loopback <code>/v1</code> URL as <code>baseUrl</code> and the master key as <code>apiKey</code> (via <code>$LITELLM_MASTER_KEY</code> or <code>${LITELLM_MASTER_KEY}</code>, which Pi documents for environment interpolation). Swap the model <code>id</code> for one you have actually pulled; the <code>qwen2.5-coder:7b</code> value is Pi&#39;s example, not a model downloaded for this page. For GGUF files on disk, follow Pi&#39;s <a href=\"https://docs.pi.dev/latest/llama-cpp\">llama.cpp</a> path rather than this JSON snippet.</p>\n<ol><li>Add Basic Memory as a local MCP server in the harness, using the local install, not the cloud product.</li><li>Wrap the agent&#39;s command and tool processes with <code>srt</code> and a settings file that keeps network and writes on a default-deny list, then allow only the paths and hosts the agent should touch (for example the project directory, and <code>127.0.0.1</code> if the sandbox must reach the local model server).</li></ol>\n<h2>Licences and limits</h2>\n<ul><li>Ollama, LiteLLM&#39;s open-source SDK and proxy, Pi and llama.cpp are MIT. vLLM and SGLang are Apache-2.0. LocalAI is MIT. LiteLLM Enterprise is a separate, quote-based product and is not part of this stack.</li><li>Basic Memory is AGPL-3.0. If you distribute a modified server, that licence&#39;s conditions apply. Its hosted cloud offering is a different deployment, not this stack.</li><li>sandbox-runtime is Apache-2.0 and a beta research preview. Do not assume a stable config schema.</li><li>A &quot;fully local&quot; setup is only as private as its bind addresses, logs and tool egress. Leave every HTTP server on <code>127.0.0.1</code>, and do not open sandbox network to the public internet &quot;just in case&quot;.</li></ul>"
}
