TRL
Hugging Face Transformer Reinforcement Learning library for post-training methods such as SFT, DPO, PPO and GRPO on top of transformers.
Tags
Description
Established Apache-2.0 library (huggingface/trl; not archived; not a 2026 launch). PyPI trl 1.17.1 verified 2026-11-11; docs https://huggingface.co/docs/trl/index verified 2026-11-12 (https://huggingface.co/docs/trl redirects there). Install with pip install trl. Used with Hugging Face transformers and often PEFT adapters. Model weights, GPUs and any hosted training services are separate from the open-source library.
Agent access
Related
- PEFT: Hugging Face Parameter-Efficient Fine-Tuning library (LoRA and related methods) for adapting large models without full fine-tuning.
- Unsloth: Open-source LLM fine-tuning library and desktop app for running and training models locally, with a dual Apache-2.0 / AGPL-3.0 license split.
- Axolotl: Apache-2.0 post-training / fine-tuning framework for a wide range of LLMs, with CLI, YAML configs, Docker and distributed-training options.
- LLaMA-Factory: Apache-2.0 toolkit to fine-tune 100+ LLMs and VLMs via zero-code CLI and the LLaMA Board Gradio web UI.
- OpenRLHF: Apache-2.0 high-performance RLHF / post-training framework built on Ray, with support for multiple reward and policy-optimization recipes.
Guides & comparisons
- RL environment libraries compared — NeMo Gym, OpenEnv and Verifiers (Comparison): Compare NeMo Gym, OpenEnv and Verifiers as libraries for writing and running RL environments: authoring model, isolation, documented trainers, install paths, licences and current versions. Distinct from Harbor/AgentEnv/Seahaven and from the agent RL stacks comparison.
- Fine-tuning frameworks compared — Axolotl, LLaMA-Factory and Unsloth (Comparison): Compare Axolotl, LLaMA-Factory and Unsloth as SFT, LoRA/QLoRA and preference-tuning frameworks for tool-use models: documented methods, config style, chat/tool datasets, hardware backends, licences and current versions. Cross-links the agent RL stacks comparison.
- Post-training a tool-use model — TRL, PEFT, and scale-out RL (Stack): A recommended post-training stack for chat and tool-use models: TRL and PEFT for SFT, LoRA/QLoRA, DPO and GRPO, with OpenRLHF and AReaL as the distributed and asynchronous RL options. Cross-links the fine-tuning frameworks and agent RL stacks comparisons.
Sources
- https://github.com/huggingface/trl
- https://github.com/huggingface/trl/blob/main/LICENSE
- https://pypi.org/project/trl/
- https://huggingface.co/docs/trl/index
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)