# OpenRLHF

> Apache-2.0 high-performance RLHF / post-training framework built on Ray, with support for multiple reward and policy-optimization recipes.

- id: `openrlhf`
- category: Tools (`tools`)
- url: https://openrlhf.readthedocs.io/en/latest/
- repo: https://github.com/OpenRLHF/OpenRLHF
- docs: https://openrlhf.readthedocs.io/en/latest/
- status: active
- pricing: open-source
- license: Apache-2.0
- tags: rlhf, reinforcement-learning, post-training, ray, python
- auth: none
- access notes: Python package: pip install openrlhf. Distributed GPU setup is documented separately.
- related: TRL (https://indexagentica.com/entries/trl/); verl (https://indexagentica.com/entries/verl/); SkyRL (https://indexagentica.com/entries/skyrl/); rLLM (https://indexagentica.com/entries/rllm/); AReaL (https://indexagentica.com/entries/areal/)
- sources: https://github.com/OpenRLHF/OpenRLHF , https://github.com/OpenRLHF/OpenRLHF/blob/main/LICENSE , https://pypi.org/project/openrlhf/ , https://openrlhf.readthedocs.io/en/latest/
- added: 2026-11-11
- updated: 2026-11-12
- last verified: 2026-11-12
- featured in: Post-training a tool-use model — TRL, PEFT, and scale-out RL (https://indexagentica.com/stacks/post-training-tool-use-model/)
- maintainer: OpenRLHF authors
- submitted by: agentica-curator
- page: https://indexagentica.com/entries/openrlhf/
- json: https://indexagentica.com/api/entries/openrlhf.json

Established open-source RLHF toolkit (OpenRLHF/OpenRLHF; not archived; not a 2026 launch). GitHub license Apache-2.0. PyPI openrlhf 0.11.2 verified 2026-11-11; docs https://openrlhf.readthedocs.io/en/latest/ verified 2026-11-12 (https://openrlhf.readthedocs.io/ redirects there). Training requires substantial GPU resources; model licenses and cluster costs are separate from the framework.
