Index Agentica

Fully local, private agent

A local OpenAI-compatible model server, an optional LiteLLM router, Pi as the harness, Basic Memory for on-disk Markdown memory, and Anthropic's sandbox runtime — all on your machine, with no hosted model API and no cloud sandbox.

Type
Stack
Author
Agentica Author
Published
Last verified

Use case

Run a coding agent on your machine with a loopback model server, optional local router, local-first Markdown memory and an OS-level sandbox, without a hosted model API or a cloud sandbox.

Components

RoleComponentWhy
Local model serverOllamaMIT local app and server that binds 127.0.0.1:11434 by default and exposes http://localhost:11434/v1. Swap in vLLM, SGLang, LocalAI or llama.cpp; start any of them with the companion guide.
Optional routerLiteLLMMIT OpenAI-compatible proxy you can bind to 127.0.0.1:4000 and point at the local server. One client URL if you change backends. The proxy will not start in current releases without LITELLM_MASTER_KEY.
HarnessPiMIT terminal agent that documents models.json endpoints for Ollama, vLLM, SGLang and other OpenAI-compatible servers, plus a first-party llama.cpp router path.
MemoryBasic MemoryAGPL-3.0 local-first MCP memory stored as Markdown you keep on disk. Install with the documented local command; this stack does not use Basic Memory's hosted cloud path.
SandboxAnthropic Sandbox Runtime (srt)Apache-2.0 OS-level sandbox (sandbox-exec, bubblewrap, or Windows srt-sandbox). Defaults deny network and writes until allowed; reads are allowed everywhere unless denied. Officially a beta research preview.

When to use this stack

Use this stack when the agent should stay on your machine: the model is served on loopback, memory is files you hold, and tool processes are sandboxed locally. It is the opposite of Coding agent starter, which uses Claude Code as the harness, hosted MCP servers and E2B as a cloud microVM.

This page does not teach you how to start vLLM, SGLang, Ollama or LocalAI, or how to place LiteLLM in front of them. That is Run an agent on a local model with an OpenAI-compatible endpoint. Start there, then come back to wire the harness, memory and sandbox.

It is also not a substitute for Ollama vs llama.cpp for agents, LLM gateways compared, Memory layers for AI agents, Self-hosted sandboxes for agents or Give an agent long-term memory. Those pages compare alternatives; this stack picks one local set and shows how they fit.

The pieces

Model server. Start with Ollama if you want the default loopback bind (127.0.0.1:11434, per the FAQ) and an OpenAI-compatible URL at http://localhost:11434/v1. If you already run vLLM, SGLang, LocalAI or llama.cpp, keep that server; the companion guide shows the loopback flags. llama.cpp is a first-class option in Pi (see below) and is compared with Ollama on the serving-layer page, not repeated here.

Optional router. Put LiteLLM on 127.0.0.1:4000 when you want one client base URL in front of whichever local server you started. Current LiteLLM releases will not start the proxy without LITELLM_MASTER_KEY (or general_settings.master_key). Official Docker examples already use -p 127.0.0.1:4000:4000. Skip the router if the harness can talk to the model server directly.

Harness. Pi is a MIT terminal agent whose models page covers local GGUF files through a llama.cpp router (/llama) and OpenAI-, Anthropic- or Google-compatible endpoints in models.json. It names Ollama, LM Studio, vLLM, SGLang and other compatible servers as the models.json path. Pi is also one of the subjects in OpenClaw vs Hermes Agent vs Pi. You can swap in another local-capable harness from Terminal coding agents compared if you prefer; the rest of the stack does not depend on Pi-only APIs.

Memory. Basic Memory is an AGPL-3.0 local-first MCP server that stores memory as Markdown in a knowledge base you own. The documented local install is:

uv tool install basic-memory --prerelease=allow

The project README also offers a hosted cloud path ("Skip the install — try Basic Memory in the cloud"). That path is out of scope for this stack. Stay on the local install, and see Give an agent long-term memory and Memory servers for coding agents if you need to compare other memory layers.

Sandbox. sandbox-runtime (srt) is Apache-2.0 and officially a beta research preview: APIs and configuration may change. Install with the documented:

npm install -g @anthropic-ai/sandbox-runtime

It uses sandbox-exec on macOS, bubblewrap on Linux, and a Windows srt-sandbox user. The README's defaults: network is denied until you allow domains; writes are denied until you allow paths; reads are allowed everywhere unless you add a denyRead rule. For how this compares with other self-hosted sandboxes, and for the broader "treat output as untrusted" recipe, see Self-hosted sandboxes for agents and Run agent-generated code safely in a sandbox.

Wire it

  1. Start the model server on loopback, following the companion guide. Confirm you can complete a chat request with the OpenAI Python SDK against that /v1 URL.
  2. Optionally start LiteLLM on 127.0.0.1:4000 with LITELLM_MASTER_KEY set, pointing api_base at that same server. Clients then use http://127.0.0.1:4000/v1 and the master key.
  3. Point Pi at the endpoint. The official models.json example for Ollama (from the models page) is:
{
  "$schema": "https://pi.dev/schemas/models.schema.json",
  "providers": {
    "ollama": {
      "baseUrl": "http://localhost:11434/v1",
      "api": "openai-completions",
      "apiKey": "ollama",
      "models": [
        { "id": "qwen2.5-coder:7b" }
      ]
    }
  }
}

The same page says the dummy key makes the model available to Pi and that Ollama ignores it. To go through LiteLLM instead, use the proxy's loopback /v1 URL as baseUrl and the master key as apiKey (via $LITELLM_MASTER_KEY or ${LITELLM_MASTER_KEY}, which Pi documents for environment interpolation). Swap the model id for one you have actually pulled; the qwen2.5-coder:7b value is Pi's example, not a model downloaded for this page. For GGUF files on disk, follow Pi's llama.cpp path rather than this JSON snippet.

  1. Add Basic Memory as a local MCP server in the harness, using the local install, not the cloud product.
  2. Wrap the agent's command and tool processes with srt and a settings file that keeps network and writes on a default-deny list, then allow only the paths and hosts the agent should touch (for example the project directory, and 127.0.0.1 if the sandbox must reach the local model server).

Licences and limits

Directory entries in this stack

Related

Sources

Machine-readable