Index Agentica

Fine-tuning frameworks compared — Axolotl, LLaMA-Factory and Unsloth

Compare Axolotl, LLaMA-Factory and Unsloth as SFT, LoRA/QLoRA and preference-tuning frameworks for tool-use models: documented methods, config style, chat/tool datasets, hardware backends, licences and current versions. Cross-links the agent RL stacks comparison.

Type
Comparison
Author
Agentica Author
Published
Updated
Last verified

Comparison

Fine-tuning frameworks compared — Axolotl, LLaMA-Factory and Unsloth
EntryConfig styleDocumented methodsTool-calling and chat dataHardware and backendsLicense and versions
AxolotlYAML configs reused across preprocess, train, eval, quantization and inference, driven by the axolotl CLI (train, fetch, agent-docs, config-schema). Docker images and a PyPI package are documented.Full fine-tuning, LoRA, QLoRA, GPTQ, QAT (int8/int4/FP8/NVFP4/MXFP4), preference tuning (DPO, IPO, KTO, ORPO), RL (GRPO, GDPO), and reward / process reward modelling. The agent-docs CLI also lists SimPO under preference tuning.Conversation datasets via chat_template: ShareGPT or OpenAI-style messages, tokenizer or custom Jinja templates, multi-turn versus single-turn instruct. Tool use is documented with a tools column and assistant tool_calls / tool roles (JSON Schema tools).README: NVIDIA GPU (Ampere or newer for bf16 and Flash Attention) or AMD GPU; Python >=3.12; PyTorch >=2.13.0. Multi-GPU via FSDP2 and DeepSpeed; multi-node via Torchrun or Ray. Docker is documented.Apache-2.0 (LICENSE); PyPI axolotl 0.21.1 and GitHub release v0.21.1; Python >=3.12 (checked 2026-11-11)
LLaMA-FactoryZero-code CLI (llamafactory-cli train, chat, export) with YAML examples, plus the LLaMA Board Gradio web UI (llamafactory-cli webui). Custom datasets are registered in data/dataset_info.json.README table: pre-training, SFT, reward modeling, PPO, DPO, KTO, ORPO and SimPO, each with full, freeze, LoRA, QLoRA, OFT and QOFT. GRPO is not in that table; the changelog points to a separate EasyR1 project for GRPO.Alpaca and ShareGPT formats (OpenAI messages are documented as a ShareGPT special case). ShareGPT documents human/gpt/observation/function_call roles and an optional tools column; Glaive Function Calling and AgentInstruct are listed SFT datasets.Python >=3.11. Optional CUDA (min 11.6, recommend 12.2), DeepSpeed, bitsandbytes, vLLM and FlashAttention. Docker paths for CUDA, AMD ROCm and Ascend NPU. The README includes an estimated VRAM table, marked as estimated.Apache-2.0 (LICENSE); PyPI llamafactory 0.9.5 and GitHub release v0.9.5; Python >=3.11 (checked 2026-11-11). Model-weight licences are separate, as the README states.
UnslothThree documented surfaces: the Unsloth Desktop app, Unsloth Studio web UI (unsloth studio), and Unsloth Core (Python: uv pip install unsloth --torch-backend=auto; FastLanguageModel).README and docs list LoRA, QLoRA, full fine-tuning, pretraining, FP8, and RL including GRPO (GSPO via GRPOConfig). A preference page documents DPO, ORPO and KTO, and also mentions PPO and a SimPO notebook.The datasets guide covers raw corpus (CPT), instruct (SFT) and multi-turn conversation (SFT) in ShareGPT or ChatML/OpenAI messages, with standardize_sharegpt. Tool calling is documented mainly as a runtime/API loop, not as a dedicated training schema.README: Windows, Linux, WSL and macOS; NVIDIA, AMD and Intel GPUs, CPUs and Vulkan. Multi-GPU is described via Accelerate/DeepSpeed (FSDP, DDP), with a note that official multi-GPU support is still being announced. AMD image: unsloth/unsloth-rocm.Dual licence: core Apache-2.0 (LICENSE; PyPI license_expression Apache-2.0); optional Studio UI AGPL-3.0 (studio/LICENSE.AGPL-3.0). PyPI unsloth 2026.11.3; Python >=3.9,<3.15 (checked 2026-11-11).

Verdict: Choose by workflow, not by unverified speed claims. Use Axolotl when you want a YAML/CLI pipeline that documents SFT, (Q)LoRA, preference tuning and GRPO in one project. Use LLaMA-Factory when you want a zero-code CLI or Gradio UI and an explicit ShareGPT tool-call dataset schema; treat EasyR1 as a separate GRPO project. Use Unsloth for a Python, Desktop or Studio workflow with documented LoRA/QLoRA and GRPO/DPO—and read the Apache-2.0/AGPL-3.0 split if you ship Studio UI. For multi-turn agent RL infrastructure, see the agent RL stacks comparison; these three are post-training frameworks.

SFT and LoRA trainers, not agent RL stacks

Agent RL stacks compared covers distributed training engines and harness gateways — verl, SkyRL, Agent Lightning and OpenPipe ART — whose job is multi-turn agent reinforcement learning. This page is one layer down the training stack: post-training frameworks you use to run supervised fine-tuning, LoRA/QLoRA and preference optimization on chat and tool-use models.

The three directory entries are Axolotl, LLaMA-Factory and Unsloth. They overlap on (Q)LoRA and chat data, and some also document GRPO or PPO, but they are not drop-in substitutes for those RL stacks. For a library-level recommended path that starts with TRL and PEFT and then scales out with OpenRLHF or AReaL, see Post-training a tool-use model. Other trainers, including ms-SWIFT and the deprecated torchtune entry, are listed under Other options below. LLaMA-Factory’s README also lists Unsloth among optional training “tricks,” so the libraries can compose.

The comparison table is generated from the front matter. README and docs include vendor speed or VRAM figures (for example Unsloth’s “2× faster / 70% less VRAM” wording, or LLaMA-Factory’s starred estimated memory table). Those numbers are not independently verified here and are not used as selection criteria.

How you configure them

Axolotl is YAML-first. The README says a single configuration file can be reused across dataset preprocessing, training, evaluation, quantization and inference. The CLI entry points include axolotl train (for example axolotl train examples/llama-3/lora-1b.yml), axolotl fetch for example configs, axolotl agent-docs for bundled method references, and axolotl config-schema. Installation is documented via uv pip install axolotl[deepspeed] after a pinned PyTorch, or with the axolotlai/axolotl Docker image.

LLaMA-Factory advertises “zero-code” CLI and a Gradio LLaMA Board UI. The documented commands are llamafactory-cli train, chat and export on YAML files under examples/, plus llamafactory-cli webui. Custom datasets are registered in data/dataset_info.json; the data README is the format specification.

Unsloth documents three surfaces: a native Desktop app (platform installers), Studio (unsloth studio, also install.sh / Homebrew / Docker unsloth/unsloth), and Core, the Python package installed with uv pip install unsloth --torch-backend=auto and used through FastLanguageModel. Training examples in the docs are Python (and Colab notebooks), not a project-wide YAML schema like Axolotl.

Documented methods — only what each project lists

Do not assume every popular acronym is available in all three.

Tool-calling and multi-turn chat data

If you are fine-tuning a tool-use model, the dataset schema matters as much as the trainer.

Hardware and backends

Licences and versions (11 November 2026)

Checked on PyPI, GitHub latest releases, and each repository’s licence file(s):

PackageLicence filePyPI / GitHubRequires Python
axolotlApache License 2.0 (LICENSE)0.21.1 / v0.21.1>=3.12
llamafactoryApache License 2.0 (LICENSE)0.9.5 / v0.9.5>=3.11
unslothApache License 2.0 (LICENSE) for the core package; AGPL-3.0 (studio/LICENSE.AGPL-3.0) for optional Studio UIPyPI 2026.11.3 (GitHub latest release tag v0.1.916-beta, same calendar day)>=3.9,<3.15

Axolotl’s PyPI metadata does not set a licence classifier; the Apache-2.0 text is from the repo LICENSE. Unsloth’s PyPI license_expression is Apache-2.0, which matches the core package, not the Studio UI. The Unsloth README states this dual-licensing model explicitly. Model weights, GPUs and any hosted extras remain separate from these software licences.

When linking the LLaMA-Factory repository, use hiyouga/LlamaFactory, the canonical path. The hyphenated hiyouga/LLaMA-Factory path redirects to it, so don't use that in published links.

Other options

The table above stays a three-way comparison of Axolotl, LLaMA-Factory and Unsloth. These related directory entries are not added as subjects, so the criteria rows stay focused on those YAML/UI-oriented frameworks.

What we ran, and what we did not

We did not install these packages or run CLI help against a live install. A useful smoke test (axolotl config-schema, llamafactory-cli help, or importing FastLanguageModel) would pull PyTorch and related CUDA/ROCm wheels, which is a large download and is not GPU-free in practice. No fine-tune was run, and no speed, memory or accuracy figure from project blogs or notebooks was reproduced.

A safe selection sequence

  1. Name the layer. If you need a multi-turn agent RL stack (rollouts, rewards, a training backend), start from Agent RL stacks compared. If you want the Hugging Face trainer layer (TRL and PEFT) and a documented scale-out path, use Post-training a tool-use model. If you need a YAML/UI-oriented SFT, (Q)LoRA or preference-tuning framework, stay on this page.
  2. List the methods you actually need. If you need GRPO in the same project as SFT, Axolotl and Unsloth document it; LLaMA-Factory’s official approaches table does not.
  3. Check the dataset schema. Tool-call SFT is explicitly specified in Axolotl’s conversation docs and LLaMA-Factory’s ShareGPT format. Unsloth’s first-class documented path is multi-turn ShareGPT/ChatML plus a separate runtime tool-calling guide.
  4. Match hardware and licence. Note Unsloth’s AGPL-3.0 Studio UI if you will ship that component; Axolotl and LLaMA-Factory are Apache-2.0. Confirm CUDA versus ROCm versus NPU install docs before you pin a stack.
  5. Pin versions. Record the PyPI versions above (or newer pins), the YAML or Python entry point, and any engine flags (DeepSpeed, FSDP, loss_type for GRPO).

Directory entries in this comparison

Related

Sources

Machine-readable