Seahaven
Kiln AI's open-source Python framework for synthetic, resettable 'worlds' (SQLite state plus mock tools) for agent evals and RL, served via OpenEnv or MCP.
Tags
Description
A world is a schema (SQLite tables) plus tools (Python functions); each run gets its own database copied from a fixture, every changed row is logged so you grade on state rather than transcripts, and clock and random seed are fixed for reproducible runs. Serves hundreds of instances per process; seahaven serve is an OpenEnv environment with a web console, and seahaven mcp exposes one world to an MCP client. Worlds compose (e.g. Kiln's Stripe World mocks Stripe Billing and Payments behind the same tools as Stripe's MCP server and serves its REST API). PyPI seahaven 0.6.0.
Related
- Harbor: Framework from the Terminal-Bench creators for evaluating and optimizing agents and models in sandboxed environments, run in parallel across cloud sandbox providers.
- AgentEnv Framework: Scale AI's open-source Python SDK and CLI for building, deploying and scoring containerized agent environments and tasks for RL and evals.
- AppWorld: Benchmark and sandbox of 9 simulated day-to-day apps operable via 457 APIs and populated by ~100 simulated people, for evaluating function-calling and interactive coding agents.
- Stripe MCP Server: Stripe's hosted remote MCP server giving agents OAuth-secured access to the Stripe API and documentation.
Guides & comparisons
- Agent environment frameworks compared: Harbor vs AgentEnv Framework vs Seahaven (Comparison): Three open-source Python frameworks for building the environments agent evals and RL runs happen in: Harbor (container tasks), Scale AI's AgentEnv Framework and Kiln AI's Seahaven (synthetic SQLite worlds). Covers tasks, grading, compute, agent interface, interop and license.
Sources
- https://github.com/Kiln-AI/Seahaven
- https://kiln.tech/blog/introducing_seahaven
- https://pypi.org/project/seahaven/
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)