RL environment libraries compared — NeMo Gym, OpenEnv and Verifiers
Compare NeMo Gym, OpenEnv and Verifiers as libraries for writing and running RL environments: authoring model, isolation, documented trainers, install paths, licences and current versions. Distinct from Harbor/AgentEnv/Seahaven and from the agent RL stacks comparison.
Comparison
| Entry | Authoring model | Runtime and isolation | Training integrations | Official install and versions | License |
|---|---|---|---|---|---|
| NeMo Gym | An environment is a dataset of tasks, an agent harness, a verifier, and per-task state. Tutorials cover single-step and multi-step tool calling, session state, and multi-reward verification. The documented CLI includes gym env start and gym eval run. | Local coordinating servers for resources, agent and model. Release notes document multiple sandbox providers. A GPU is not required for the library itself; particular resource servers or inference may need one, and the official quick start uses a model API key. | Official training tutorials cover NeMo RL, Unsloth and VeRL. The ecosystem docs also describe combining Harbor, OpenEnv, Reasoning Gym and Verifiers environments with NeMo Gym's own. | Official quick start is git clone, then uv venv --python 3.13.14 && uv sync (Python >=3.13.14), not a pip one-liner. PyPI nemo-gym 0.7.0 matches GitHub v0.7.0 (12 October 2026). | Apache License 2.0 (repository LICENSE) |
| OpenEnv | Subclass Environment and implement reset(), step() and state, plus Action/Observation models. Scaffold with openenv init. Rewards are computed in the environment (observation fields or a rubric). openenv import can wrap a Verifiers or AgentEnv source. | Each environment is documented as a Docker image that can run locally, on a cloud sandbox, or as a Hugging Face Space, over a Gymnasium-style reset/step/state API on WebSocket/HTTP. openenv serve runs on the host for local development; use Docker or Spaces for isolated training. | The README and training guide list TRL (environment_factory on GRPOTrainer, and Harbor traces with AsyncGRPOTrainer), Unsloth, SkyRL, ART, Oumi, torchforge, Miles and others. Training stays in the framework; the environment supplies tools and the reward. | pip install openenv (Python >=3.10). PyPI and GitHub latest are 0.13.0 (12 November 2026). The project notes that it is in early development and APIs may change. | BSD 3-Clause (repository LICENSE; copyright 2026, Hugging Face, Inc.) |
| verifiers | A Taskset.load() builds Task objects from TaskData (prompt, files, scoring). Most packages need only a Taskset and Task; a custom Env is for bespoke multi-agent control flow. Score with @vf.reward. Isolated grading uses ScoringConfig(mode="isolated"). | An evaluator process plus optional agent and grading sandboxes (Docker and other runtimes are documented). vf-validate can check task setup or a gold solution without calling a model. vf-run runs a few in-process episodes. | Documented as tightly integrated with prime-rl (the orchestrator drives verifiers environments) and Prime Intellect's Environments Hub and Hosted Training (account and compute cost possible). OpenEnv can import a Verifiers source; NeMo Gym documents a Verifiers agent integration. | Getting started is git clone and uv sync (example vf-validate reverse-text -n 1). PyPI verifiers 0.4.1 matches GitHub v0.4.1 (3 November 2026), Python >=3.11,<3.15. The README also recommends the Prime CLI (uv tool install prime). | MIT (repository LICENSE; copyright 2026 Prime Intellect) |
Verdict: Need a Gymnasium-style reset/step/state service you can package as Docker or a Hugging Face Space and call from several trainers? Start with OpenEnv (and openenv import if you already have a Verifiers or AgentEnv source). Need NVIDIA's environment servers (dataset, harness and verifier) and the documented path into NeMo RL, or its VeRL and Unsloth tutorials? Use NeMo Gym. Need Prime Intellect's taskset/harness/trace model and prime-rl? Use Verifiers. This is a different layer from Harbor, AgentEnv Framework and Seahaven, and from the training engines on the agent RL stacks page.
Gym libraries, not training stacks or eval-task frameworks
Agent RL stacks compared covers training engines and harness gateways — verl, SkyRL, Agent Lightning and OpenPipe ART. Post-training a tool-use model is a recommended SFT/LoRA/DPO/GRPO path on TRL and PEFT. Agent environment frameworks compared is one layer over, covering Harbor, AgentEnv Framework and Seahaven as task- and world-authoring frameworks; that page already notes that AgentEnv and Seahaven can serve environments through OpenEnv.
This page compares the three RL environment libraries now in the directory: NeMo Gym, OpenEnv and verifiers. They are what you use to define tasks, rewards and (often) isolated runtimes that a trainer can call. prime-rl and NeMo RL appear here only as documented consumers of those environments, not as subjects.
The comparison table is generated from the front matter. Project READMEs include scale or catalog wording (for example OpenEnv's "40+ ready-to-use environments", NeMo Gym's "thousands of concurrent environments", or prime-rl's large-cluster claims). Those figures are the projects' own and are not independently counted or measured here.
How each library models an environment
NeMo Gym defines an environment as the complete system an agent interacts with: a dataset of tasks, an agent harness, a verifier, and per-task state. You start coordinating servers with gym env start and score rollouts with gym eval run. Environment tutorials walk through single-step and multi-step tool calling, session state, and multi-reward verification. The official docs are at docs.nvidia.com/nemo/gym/main/about/.
OpenEnv is a client-server layer inspired by Gymnasium. You subclass Environment and implement reset(), step() and state; step() returns an observation that already carries done and reward, not a Gymnasium-style tuple. Your First Environment scaffolds this with openenv init. The README documents a catalog of ready-to-use environments (Echo, coding, TextArena, OpenSpiel, BrowserGym, Harbor and others) at the environment catalog. Official docs: huggingface.co/docs/openenv/index. The documentation root without /index redirects, so do not use that shorter path in published links. The canonical repository is openenv/OpenEnv; older GitHub paths under huggingface/ and meta-pytorch/ redirect to it.
verifiers is organised around a taskset that loads Task objects from serialisable TaskData, a harness (for example Claude Code, Codex or mini-swe-agent), an agent (harness × model × runtime policy) that produces a Trace, and an environment that defines control flow across an episode. Most packages only need a Taskset and Task; you write a custom Env when you want bespoke multi-agent flow. The building environments guide shows verifiers.v1 hooks such as setup and @vf.reward, including isolated grading with ScoringConfig(mode="isolated"). In-tree docs live under docs/; there is no separate docs hostname.
Runtime and isolation
NeMo Gym's quick start runs local servers on a development machine (Linux, macOS, or Windows via WSL2). A GPU is not required for the library itself. The official example configures a model endpoint (OpenAI is the documented quick-start case; other providers, including vLLM, are listed in the docs). Release notes describe several sandbox providers; treat any environment that executes agent or task code as a potentially destructive sandbox.
OpenEnv packages each environment as a Docker image that the docs say can run locally, on a cloud sandbox, or as a Hugging Face Space, speaking reset / step / state over WebSocket or HTTP, with optional MCP tool serving in production mode. For local authoring, openenv serve runs on the host after you install the environment's dependencies; the README says to use Docker or Spaces for isolated training and deployment. openenv push deploys to Spaces — we did not run that.
verifiers splits work across an evaluator process and optional agent and grading sandboxes. Isolated scoring copies only declared artifacts into a fresh grading container. Getting started documents vf-validate as a local check that does not call a model, and vf-run for a few in-process episodes. Harbor-based tasksets need Python 3.12+ and the harbor extra.
Which trainers they document
These libraries are not substitutes for a training engine. Use them to produce graded rollouts; pick the trainer separately.
- OpenEnv keeps training in the caller's framework. The training guide and README name TRL (OpenEnv guide, including
GRPOTrainerwithenvironment_factoryand Harbor traces viaAsyncGRPOTrainer), Unsloth, SkyRL, ART, Oumi, torchforge, Miles and others.openenv importcan wrap an existing Verifiers environment or an AgentEnv bundle. Harbor integration is documented for running real coding-agent harnesses and capturing tokens for RL. - NeMo Gym publishes training tutorials, including a NeMo RL GRPO path, plus Unsloth and VeRL. The ecosystem page describes combining environments from Harbor, OpenEnv, Reasoning Gym and Verifiers. NeMo RL itself is an Apache-2.0 trainer: GitHub latest is v0.8.0 (6 November 2026). Its PyPI package
nemo-rlis still 0.0.0 (uploaded 3 June 2025) and is not the current install path — official docs use the NGC container or a git checkout. - verifiers is documented as tightly integrated with prime-rl. prime-rl's overview describes an orchestrator that owns the data plane across verifiers training and eval environments. GitHub latest is v0.10.0 (13 October 2026). There is still no PyPI package named
prime-rlorprime_rl(HTTP 404 on 13 November 2026). The README documents a git/uv install and a curl ofscripts/install.sh(not executed here) and states that you currently need at least one NVIDIA GPU. The Environments Hub and Hosted Training are Prime Intellect products that may require an account and can incur compute cost; they were not used here.
Official install, licences and versions (13 November 2026)
| Package | Licence file | Official install documented as | PyPI / GitHub latest | Requires Python |
|---|---|---|---|---|
| nemo-gym | Apache License 2.0 | git clone and uv venv --python 3.13.14 && uv sync | 0.7.0 / v0.7.0 | >=3.13.14 |
| openenv | BSD 3-Clause | pip install openenv | 0.13.0 / v0.13.0 | >=3.10 |
| verifiers | MIT | git clone and uv sync; README also recommends uv tool install prime | 0.4.1 / v0.4.1 | >=3.11,<3.15 |
| prime-rl (trainer, not a subject) | Apache License 2.0 | git/uv (or the project's install script) | no PyPI package / v0.10.0 | (see prime-rl docs; GPU required) |
| nemo-rl (trainer, not a subject) | Apache License 2.0 | NGC container or git clone, not the 0.0.0 PyPI wheel | PyPI 0.0.0; GitHub v0.8.0 | (see NeMo RL install docs) |
When linking repositories, use the canonical paths above. Do not use redirecting GitHub URLs such as huggingface/openenv, meta-pytorch/OpenEnv, or willccbb/verifiers.
A minimal OpenEnv check we ran
The following commands are the official scaffold and static contract check from the OpenEnv README and CLI reference. We ran them on the box with PyPI openenv 0.13.0 in a Python 3.13.5 virtualenv, without Docker and without calling a hosted Space or a model:
pip install openenv==0.13.0
openenv init my_env
openenv validate my_env --level static --skip-build
openenv init created 12 files (including server/my_env_environment.py, models.py, openenv.yaml and a server/Dockerfile). openenv validate reported policy v1, level static, PASS static.manifest, and verdict PASS. We did not run openenv serve, openenv build, openenv push, or a runtime/semantic validate.
What we did not run
We did not install NeMo Gym: the official pin is Python >=3.13.14, and this box has Python 3.13.5. We did not clone verifiers or run vf-validate reverse-text, and we did not install or run prime-rl or NeMo RL (both are GPU training stacks; prime-rl's docs say you currently need at least one NVIDIA GPU; NeMo RL's official path uses an NGC container or a full checkout). No environment server was left listening, and no API keys, Spaces deploys or paid compute were used.
Directory entries in this comparison
- NeMo Gym: NVIDIA Apache-2.0 library for building, evaluating and scaling agent environments, verifiers and harnesses, including sandboxed code-execution tasks.
- OpenEnv: BSD-3-Clause framework for building and serving isolated agentic RL environments over a Gymnasium-style reset/step/state API.
- verifiers: Prime Intellect’s MIT library for creating LLM reinforcement-learning environments and evals, with a matching PyPI package.
- prime-rl: Prime Intellect’s Apache-2.0 asynchronous RL training framework, installed from git/uv (no verified PyPI package) and integrated with verifiers environments.
- NeMo RL: NVIDIA Apache-2.0 post-training / reinforcement-learning library for large-scale model training, with official docs and NGC containers.
- Harbor: Framework from the Terminal-Bench creators for evaluating and optimizing agents and models in sandboxed environments, run in parallel across cloud sandbox providers.
- TRL: Hugging Face Transformer Reinforcement Learning library for post-training methods such as SFT, DPO, PPO and GRPO on top of transformers.
- SkyRL: Modular reinforcement-learning stack with a training engine, long-horizon agent layer and Gymnasium-style tool-use environments for LLMs.
- Unsloth: Open-source LLM fine-tuning library and desktop app for running and training models locally, with a dual Apache-2.0 / AGPL-3.0 license split.
- ART (Agent Reinforcement Trainer): Open-source Python framework for training multi-step agents with GRPO, with an OpenAI-compatible client and a separate GPU training server.
- verl: Distributed reinforcement-learning training library for LLMs, with GRPO/PPO dataflows and pluggable training and rollout engines for agentic post-training.
- Agent Lightning: Microsoft's lightweight agentic RL framework that trains models through real agent harnesses unchanged, using a proxy gateway, verl/vLLM trainer and Kubernetes rollouts.
- AgentEnv Framework: Scale AI's open-source Python SDK and CLI for building, deploying and scoring containerized agent environments and tasks for RL and evals.
- Seahaven: Kiln AI's open-source Python framework for synthetic, resettable 'worlds' (SQLite state plus mock tools) for agent evals and RL, served via OpenEnv or MCP.
Related
- Agent environment frameworks compared: Harbor vs AgentEnv Framework vs Seahaven (Comparison)
- Agent RL stacks compared — verl, SkyRL, Agent Lightning and ART (Comparison)
- Post-training a tool-use model — TRL, PEFT, and scale-out RL (Stack)
Sources
- NVIDIA-NeMo/Gym README, accessed
- NVIDIA-NeMo/Gym LICENSE, accessed
- nemo-gym PyPI metadata, accessed
- NVIDIA-NeMo/Gym latest GitHub release, accessed
- NeMo Gym documentation, About, accessed
- NeMo Gym environment tutorials, accessed
- NeMo Gym training tutorials, accessed
- NeMo Gym tutorial, NeMo RL GRPO, accessed
- NeMo Gym ecosystem, accessed
- openenv/OpenEnv README, accessed
- openenv/OpenEnv LICENSE, accessed
- openenv PyPI metadata, accessed
- openenv/OpenEnv latest GitHub release, accessed
- OpenEnv documentation index, accessed
- OpenEnv, Your First Environment, accessed
- OpenEnv, Core Concepts, accessed
- OpenEnv, Training with OpenEnv, accessed
- OpenEnv CLI reference, accessed
- OpenEnv environment catalog, accessed
- TRL documentation, OpenEnv, accessed
- PrimeIntellect-ai/verifiers README, accessed
- PrimeIntellect-ai/verifiers LICENSE, accessed
- verifiers PyPI metadata, accessed
- PrimeIntellect-ai/verifiers latest GitHub release, accessed
- verifiers docs, Overview, accessed
- verifiers docs, Getting started, accessed
- verifiers docs, Building environments, accessed
- PrimeIntellect-ai/prime-rl README, accessed
- PrimeIntellect-ai/prime-rl LICENSE, accessed
- prime-rl docs, Overview, accessed
- PrimeIntellect-ai/prime-rl latest GitHub release, accessed
- NVIDIA-NeMo/RL README, accessed
- NVIDIA-NeMo/RL LICENSE, accessed
- NVIDIA-NeMo/RL latest GitHub release, accessed
- NeMo RL documentation index, accessed
- nemo-rl PyPI metadata, accessed