AgentEnv Framework

Scale AI's open-source Python SDK and CLI for building, deploying and scoring containerized agent environments and tasks for RL and evals.

Category

Evals & Observability

Website

https://www.agentenvframework.com

Repository

https://github.com/scaleapi/agentenv-framework

Docs

https://www.agentenvframework.com/docs

Status

active

Pricing

open-source

License

Apache-2.0

Maintainer

Scale AI

Added

2026-10-27

Updated

2026-10-27

Last verified

2026-10-27

Submitted by

agentica-curator

Tags

• evals
• rl-environments
• sandbox
• benchmarks
• python

Description

Environments are containerized servers speaking the open agentenv-framework-protocol; the framework builds them into versioned images, deploys them behind a gateway, points an agent at them over MCP, REST or a generated CLI, and grades what the agent did. Agents are described by A2A Agent Cards, so the same task runs against Claude Code, OpenAI agents or custom agents. Sandboxes: local Docker, Modal containers, and Modal and E2B VMs, plus custom providers via plugins; optional AWS/GCP store backends and an OpenEnv extra. Try it with uvx --from agentenv-framework agent-env run hello. Requires Python 3.11.4+; PyPI agentenv-framework 0.9.1364. Scale says every RL environment it builds runs on AgentEnv.

Related

• Harbor: Framework from the Terminal-Bench creators for evaluating and optimizing agents and models in sandboxed environments, run in parallel across cloud sandbox providers.
• Modal: Cloud platform for production AI: inference, training, sandboxes and serverless functions on one platform.
• E2B: Open-source, secure cloud sandboxes for AI agents: an isolated machine per agent to run code, browse and use tools.
• Agent2Agent Protocol (A2A): Open standard, hosted by the Linux Foundation, that lets independent AI agents discover each other, delegate work and exchange results across vendors.

Guides & comparisons

• Agent environment frameworks compared: Harbor vs AgentEnv Framework vs Seahaven (Comparison): Three open-source Python frameworks for building the environments agent evals and RL runs happen in: Harbor (container tasks), Scale AI's AgentEnv Framework and Kiln AI's Seahaven (synthetic SQLite worlds). Covers tasks, grading, compute, agent interface, interop and license.

Sources

• https://github.com/scaleapi/agentenv-framework
• https://labs.scale.com/blog/introducing-agentenv
• https://pypi.org/project/agentenv-framework/

Machine-readable

• JSON
• Markdown
• Source file on GitHub (edit via pull request)