verl
Distributed reinforcement-learning training library for LLMs, with GRPO/PPO dataflows and pluggable training and rollout engines for agentic post-training.
Tags
Description
Open-source implementation of HybridFlow, initiated by ByteDance Seed and maintained by the verl community. PyPI verl 0.9.1 supports Python 3.10–3.12; follow the repository installation guide for compatible GPU and engine dependencies. Integrates FSDP, Megatron-LM, vLLM and SGLang, including multi-turn agentic rollouts. The official README at b17218b documents optional workflows under in-tree recipe/, with per-recipe REQUIRED_VERL.txt specifying a tag/fixed SHA or an explicit VERL_COMMIT and related revision pins. Match the recipe and engine stack to that revision rather than assuming the latest PyPI release works. Agent Lightning's installation guide separately supports pinned verl 0.7.1/0.8.0 stacks (CUDA 12.9/13.0); standalone PyPI verl 0.9.1 is not a claimed drop-in upgrade for that integration. Apache-2.0 software; GPU resources are not included.
Agent access
Guides & comparisons
- RL environment libraries compared — NeMo Gym, OpenEnv and Verifiers (Comparison): Compare NeMo Gym, OpenEnv and Verifiers as libraries for writing and running RL environments: authoring model, isolation, documented trainers, install paths, licences and current versions. Distinct from Harbor/AgentEnv/Seahaven and from the agent RL stacks comparison.
- Fine-tuning frameworks compared — Axolotl, LLaMA-Factory and Unsloth (Comparison): Compare Axolotl, LLaMA-Factory and Unsloth as SFT, LoRA/QLoRA and preference-tuning frameworks for tool-use models: documented methods, config style, chat/tool datasets, hardware backends, licences and current versions. Cross-links the agent RL stacks comparison.
- Agent RL stacks compared — verl, SkyRL, Agent Lightning and ART (Comparison): Choose an agent reinforcement-learning stack by the layer you need: distributed training, long-horizon environments, an existing-harness gateway, or a trajectory-and-reward client. Backend ownership matters more than a headline benchmark.
- Post-training a tool-use model — TRL, PEFT, and scale-out RL (Stack): A recommended post-training stack for chat and tool-use models: TRL and PEFT for SFT, LoRA/QLoRA, DPO and GRPO, with OpenRLHF and AReaL as the distributed and asynchronous RL options. Cross-links the fine-tuning frameworks and agent RL stacks comparisons.
Sources
- https://github.com/verl-project/verl
- https://github.com/verl-project/verl/blob/main/LICENSE
- https://pypi.org/project/verl/
- https://github.com/verl-project/verl/blob/b17218ba58a6fbcdfb0de32772e3c2a79f88529d/README.md
- https://microsoft.github.io/agent-lightning/stable/00-installation/
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)