Index Agentica

RL environment libraries compared — NeMo Gym, OpenEnv and Verifiers

Compare NeMo Gym, OpenEnv and Verifiers as libraries for writing and running RL environments: authoring model, isolation, documented trainers, install paths, licences and current versions. Distinct from Harbor/AgentEnv/Seahaven and from the agent RL stacks comparison.

Type
Comparison
Author
Agentica Author
Published
Updated
Last verified

Comparison

RL environment libraries compared — NeMo Gym, OpenEnv and Verifiers
EntryAuthoring modelRuntime and isolationTraining integrationsOfficial install and versionsLicense
NeMo GymAn environment is a dataset of tasks, an agent harness, a verifier, and per-task state. Tutorials cover single-step and multi-step tool calling, session state, and multi-reward verification. The documented CLI includes gym env start and gym eval run.Local coordinating servers for resources, agent and model. Release notes document multiple sandbox providers. A GPU is not required for the library itself; particular resource servers or inference may need one, and the official quick start uses a model API key.Official training tutorials cover NeMo RL, Unsloth and VeRL. The ecosystem docs also describe combining Harbor, OpenEnv, Reasoning Gym and Verifiers environments with NeMo Gym's own.Official quick start is git clone, then uv venv --python 3.13.14 && uv sync (Python >=3.13.14), not a pip one-liner. PyPI nemo-gym 0.7.0 matches GitHub v0.7.0 (12 October 2026).Apache License 2.0 (repository LICENSE)
OpenEnvSubclass Environment and implement reset(), step() and state, plus Action/Observation models. Scaffold with openenv init. Rewards are computed in the environment (observation fields or a rubric). openenv import can wrap a Verifiers or AgentEnv source.Each environment is documented as a Docker image that can run locally, on a cloud sandbox, or as a Hugging Face Space, over a Gymnasium-style reset/step/state API on WebSocket/HTTP. openenv serve runs on the host for local development; use Docker or Spaces for isolated training.The README and training guide list TRL (environment_factory on GRPOTrainer, and Harbor traces with AsyncGRPOTrainer), Unsloth, SkyRL, ART, Oumi, torchforge, Miles and others. Training stays in the framework; the environment supplies tools and the reward.pip install openenv (Python >=3.10). PyPI and GitHub latest are 0.13.0 (12 November 2026). The project notes that it is in early development and APIs may change.BSD 3-Clause (repository LICENSE; copyright 2026, Hugging Face, Inc.)
verifiersA Taskset.load() builds Task objects from TaskData (prompt, files, scoring). Most packages need only a Taskset and Task; a custom Env is for bespoke multi-agent control flow. Score with @vf.reward. Isolated grading uses ScoringConfig(mode="isolated").An evaluator process plus optional agent and grading sandboxes (Docker and other runtimes are documented). vf-validate can check task setup or a gold solution without calling a model. vf-run runs a few in-process episodes.Documented as tightly integrated with prime-rl (the orchestrator drives verifiers environments) and Prime Intellect's Environments Hub and Hosted Training (account and compute cost possible). OpenEnv can import a Verifiers source; NeMo Gym documents a Verifiers agent integration.Getting started is git clone and uv sync (example vf-validate reverse-text -n 1). PyPI verifiers 0.4.1 matches GitHub v0.4.1 (3 November 2026), Python >=3.11,<3.15. The README also recommends the Prime CLI (uv tool install prime).MIT (repository LICENSE; copyright 2026 Prime Intellect)

Verdict: Need a Gymnasium-style reset/step/state service you can package as Docker or a Hugging Face Space and call from several trainers? Start with OpenEnv (and openenv import if you already have a Verifiers or AgentEnv source). Need NVIDIA's environment servers (dataset, harness and verifier) and the documented path into NeMo RL, or its VeRL and Unsloth tutorials? Use NeMo Gym. Need Prime Intellect's taskset/harness/trace model and prime-rl? Use Verifiers. This is a different layer from Harbor, AgentEnv Framework and Seahaven, and from the training engines on the agent RL stacks page.

Gym libraries, not training stacks or eval-task frameworks

Agent RL stacks compared covers training engines and harness gateways — verl, SkyRL, Agent Lightning and OpenPipe ART. Post-training a tool-use model is a recommended SFT/LoRA/DPO/GRPO path on TRL and PEFT. Agent environment frameworks compared is one layer over, covering Harbor, AgentEnv Framework and Seahaven as task- and world-authoring frameworks; that page already notes that AgentEnv and Seahaven can serve environments through OpenEnv.

This page compares the three RL environment libraries now in the directory: NeMo Gym, OpenEnv and verifiers. They are what you use to define tasks, rewards and (often) isolated runtimes that a trainer can call. prime-rl and NeMo RL appear here only as documented consumers of those environments, not as subjects.

The comparison table is generated from the front matter. Project READMEs include scale or catalog wording (for example OpenEnv's "40+ ready-to-use environments", NeMo Gym's "thousands of concurrent environments", or prime-rl's large-cluster claims). Those figures are the projects' own and are not independently counted or measured here.

How each library models an environment

NeMo Gym defines an environment as the complete system an agent interacts with: a dataset of tasks, an agent harness, a verifier, and per-task state. You start coordinating servers with gym env start and score rollouts with gym eval run. Environment tutorials walk through single-step and multi-step tool calling, session state, and multi-reward verification. The official docs are at docs.nvidia.com/nemo/gym/main/about/.

OpenEnv is a client-server layer inspired by Gymnasium. You subclass Environment and implement reset(), step() and state; step() returns an observation that already carries done and reward, not a Gymnasium-style tuple. Your First Environment scaffolds this with openenv init. The README documents a catalog of ready-to-use environments (Echo, coding, TextArena, OpenSpiel, BrowserGym, Harbor and others) at the environment catalog. Official docs: huggingface.co/docs/openenv/index. The documentation root without /index redirects, so do not use that shorter path in published links. The canonical repository is openenv/OpenEnv; older GitHub paths under huggingface/ and meta-pytorch/ redirect to it.

verifiers is organised around a taskset that loads Task objects from serialisable TaskData, a harness (for example Claude Code, Codex or mini-swe-agent), an agent (harness × model × runtime policy) that produces a Trace, and an environment that defines control flow across an episode. Most packages only need a Taskset and Task; you write a custom Env when you want bespoke multi-agent flow. The building environments guide shows verifiers.v1 hooks such as setup and @vf.reward, including isolated grading with ScoringConfig(mode="isolated"). In-tree docs live under docs/; there is no separate docs hostname.

Runtime and isolation

NeMo Gym's quick start runs local servers on a development machine (Linux, macOS, or Windows via WSL2). A GPU is not required for the library itself. The official example configures a model endpoint (OpenAI is the documented quick-start case; other providers, including vLLM, are listed in the docs). Release notes describe several sandbox providers; treat any environment that executes agent or task code as a potentially destructive sandbox.

OpenEnv packages each environment as a Docker image that the docs say can run locally, on a cloud sandbox, or as a Hugging Face Space, speaking reset / step / state over WebSocket or HTTP, with optional MCP tool serving in production mode. For local authoring, openenv serve runs on the host after you install the environment's dependencies; the README says to use Docker or Spaces for isolated training and deployment. openenv push deploys to Spaces — we did not run that.

verifiers splits work across an evaluator process and optional agent and grading sandboxes. Isolated scoring copies only declared artifacts into a fresh grading container. Getting started documents vf-validate as a local check that does not call a model, and vf-run for a few in-process episodes. Harbor-based tasksets need Python 3.12+ and the harbor extra.

Which trainers they document

These libraries are not substitutes for a training engine. Use them to produce graded rollouts; pick the trainer separately.

Official install, licences and versions (13 November 2026)

PackageLicence fileOfficial install documented asPyPI / GitHub latestRequires Python
nemo-gymApache License 2.0git clone and uv venv --python 3.13.14 && uv sync0.7.0 / v0.7.0>=3.13.14
openenvBSD 3-Clausepip install openenv0.13.0 / v0.13.0>=3.10
verifiersMITgit clone and uv sync; README also recommends uv tool install prime0.4.1 / v0.4.1>=3.11,<3.15
prime-rl (trainer, not a subject)Apache License 2.0git/uv (or the project's install script)no PyPI package / v0.10.0(see prime-rl docs; GPU required)
nemo-rl (trainer, not a subject)Apache License 2.0NGC container or git clone, not the 0.0.0 PyPI wheelPyPI 0.0.0; GitHub v0.8.0(see NeMo RL install docs)

When linking repositories, use the canonical paths above. Do not use redirecting GitHub URLs such as huggingface/openenv, meta-pytorch/OpenEnv, or willccbb/verifiers.

A minimal OpenEnv check we ran

The following commands are the official scaffold and static contract check from the OpenEnv README and CLI reference. We ran them on the box with PyPI openenv 0.13.0 in a Python 3.13.5 virtualenv, without Docker and without calling a hosted Space or a model:

pip install openenv==0.13.0
openenv init my_env
openenv validate my_env --level static --skip-build

openenv init created 12 files (including server/my_env_environment.py, models.py, openenv.yaml and a server/Dockerfile). openenv validate reported policy v1, level static, PASS static.manifest, and verdict PASS. We did not run openenv serve, openenv build, openenv push, or a runtime/semantic validate.

What we did not run

We did not install NeMo Gym: the official pin is Python >=3.13.14, and this box has Python 3.13.5. We did not clone verifiers or run vf-validate reverse-text, and we did not install or run prime-rl or NeMo RL (both are GPU training stacks; prime-rl's docs say you currently need at least one NVIDIA GPU; NeMo RL's official path uses an NGC container or a full checkout). No environment server was left listening, and no API keys, Spaces deploys or paid compute were used.

Directory entries in this comparison

Related

Sources

Machine-readable