Index Agentica

Structured-output libraries compared — Outlines, XGrammar, Guidance and llguidance

Compare Outlines, XGrammar, Guidance and llguidance by constraint form (regex, JSON Schema, CFG), documented engine integrations, licences, current PyPI versions and official JSON Schema coverage limits. The libraries compose: Outlines and Guidance can sit on these backends.

Type
Comparison
Author
Agentica Author
Published
Updated
Last verified

Comparison

Structured-output libraries compared — Outlines, XGrammar, Guidance and llguidance
EntryRoleConstraint formsJSON Schema coverageDocumented integrationsLicense and PyPI
OutlinesUser-facing structured-generation library. For local steerable models you can select outlines_core, llguidance or xgrammar as the constraint backend.JSON Schema (Pydantic, dataclasses, TypedDict, GenSON, callables, JsonSchema wrappers), regex, and Lark CFGs. Defaults: outlines_core for JSON/regex; llguidance for CFGs. outlines_core does not support grammars.Docs describe how to pass schemas and whitespace_pattern (outlines_core only; llguidance/xgrammar backends error if it is set). No official keyword-level coverage table in the output-type documentation.Features matrix: local LlamaCpp, MLXLM, Transformers, Transformers MultiModal, VLLMOffline; server Anthropic, Dottxt, Gemini, LM Studio, Mistral, Ollama, OpenAI, SGLang, TGI, VLLM. Type support varies (Anthropic none listed; Gemini JSON Schema partial; SGLang grammars expect EBNF).Apache-2.0; PyPI outlines 1.3.3; Python >=3.10,<3.14 (checked 2026-11-10)
XGrammarPortable constrained-decoding engine: compile a grammar, then fill token bitmasks during sampling.JSON Schema (builtin JSON grammar, Pydantic, or a schema string), EBNF (documented as compatible with llama.cpp GBNF), Lark, and Structural Tag for mixed reasoning and tool-calling output.Docs cover compile_json_schema and compile_builtin_json_grammar. No official keyword-by-keyword coverage table comparable to llguidance's docs/json_schema.md was found in the project documentation.README lists official integration into Mirai, OpenVINO GenAI, Modular MAX, TensorRT-LLM, vLLM, SGLang and MLC-LLM (self-described as default for several—project wording). Quick Start shows a transformers logits processor. Outlines: Transformers and MLXLM, not LlamaCpp.Apache-2.0; PyPI xgrammar 0.2.9; Python >=3.8,<4 (checked 2026-11-10)
GuidancePython interface for steering models: interleaves prompts, control flow and constrained generation. Uses llguidance as the grammar engine since Guidance v0.2.0 (2025-01-07, per the llguidance README).Regex, context-free grammars, and JSON Schema (treated as a CFG; guidance.json / gen_json with Pydantic). The README documents fast-forwarding of tokens that the grammar has already forced.The README shows JSON generation from a Pydantic schema. Keyword-level support is documented in llguidance's JSON Schema converter, not in the Guidance README itself.README names Transformers, llama.cpp, OpenAI, and "etc." CFG quality depends on whether the backend LLM has full Guidance support.MIT (LICENSE.md); PyPI guidance 0.3.1; Python >=3.10 (checked 2026-11-10)
llguidanceLow-level token-mask library used by Guidance and several inference engines.A documented large subset of JSON Schema, plus CFGs in a Lark variation that can embed JSON schemas and regular expressions.Draft 2020-12 converter. Departures: fixed property order; format enforced by default; unique keys not enforced for additional/pattern properties. Remote $ref unsupported. MaskBench rates: oneOf 68%, pattern 99% (no lookarounds), format 74%. Unsupported keywords error unless x-guidance.lenient.README: Guidance; OpenAI Structured Output (JSON Schema only); llama.cpp PR 10224 (b4613, -DLLAMA_LLGUIDANCE=ON); Chromium window.ai; SGLang --grammar-backend llguidance; vLLM PRs 14589/14779 (v0.8.2); LLGTRT (TensorRT-LLM). Outlines can use it on Transformers, LlamaCpp and MLXLM.MIT (Microsoft copyright in LICENSE; microsoft/llguidance redirects to guidance-ai/llguidance); PyPI llguidance 1.9.1; Python >=3.10 (checked 2026-11-10)

Verdict: Pick by layer, not by a generic quality ranking. Use Outlines when you want one Python API across many model clients and can live with each client's documented type limits. Use XGrammar when an engine you already run documents an XGrammar integration, or you need EBNF, Lark or Structural Tag at the engine layer. Use Guidance when you want a programming model that interleaves control flow with constraints (it sits on llguidance). Use llguidance directly when you need its documented JSON Schema subset and the engine hooks listed in its README. These pieces compose rather than replace each other.

Two layers, four packages

Agent guardrail frameworks mentions validators and structured-output checks; it does not compare the libraries that constrain decoding. This page covers four directory entries that do: Outlines, XGrammar, Guidance and llguidance.

They are not four interchangeable drop-in libraries. Two are user-facing APIs; two are token-mask engines that those APIs (and inference servers) can sit on:

The comparison table is generated from the front matter. The notes below add only documented details, plus a small CPU-only compile test. We did not benchmark mask latency or claim a winner on speed.

How each constrains decoding

The shared mechanism is constrained decoding: at each step, disallowed tokens are masked out of the distribution so the sampler cannot emit them. XGrammar’s constrained-decoding guide describes a binary mask over the vocabulary, applied by setting disallowed logits to \(-\infty\). llguidance’s README describes the same idea as computing a token mask from a context-free grammar and a tokenizer.

What you can express differs:

JSON Schema coverage — only what official docs state

Only llguidance publishes a keyword-level coverage page (docs/json_schema.md). It aims at JSON Schema Draft 2020-12 semantics or an error, with documented departures:

Those percentages are the project’s documented feature-support rates on MaskBench schemas, not model quality scores. We did not rerun MaskBench.

Outlines documents how to pass a schema and the JsonSchema options whitespace_pattern and ensure_ascii. whitespace_pattern “is applied by the outlines_core backend; the llguidance and xgrammar backends do not support it and raise an error if one is set.” The output-type docs do not include a keyword-by-keyword coverage table; they defer to each model client’s documented type support (for example Gemini accepts only Pydantic classes, dataclasses and TypedDicts as JSON Schema input).

XGrammar documents compiling a builtin JSON grammar or a JSON Schema (string or Pydantic). We did not find an official keyword-level coverage table in the project docs comparable to llguidance’s page, so this comparison does not invent one.

Guidance documents JSON generation from a schema/Pydantic model and points at the CFG mechanism; detailed keyword support is the llguidance converter’s.

Engine integrations — as official docs state them

Do not treat any project’s “we are the default backend” wording as an independently verified fact about another engine’s current defaults. The lists below are what each library’s own docs say.

Licences and versions (10 November 2026)

Checked on PyPI and in each repository’s licence file:

PackageLicence filePyPI versionRequires Python
outlinesApache License 2.01.3.3>=3.10,<3.14
xgrammarApache License 2.00.2.9>=3.8,<4
guidanceMIT (LICENSE.md)0.3.1>=3.10
llguidanceMIT (Microsoft copyright in LICENSE)1.9.1>=3.10

guidance and llguidance do not set a PyPI licence classifier; the MIT text is from the repo files. The llguidance wheel metadata reports License-Expression: MIT. The GitHub URL github.com/microsoft/llguidance redirects to guidance-ai/llguidance. The Index Agentica xgrammar entry still records PyPI 0.2.8 (verified 2026-11-09); PyPI served 0.2.9 on 2026-11-10.

What we ran, and what we did not

In a venv outside the site checkout, on CPU, we installed llguidance 1.9.1 and outlines-core 0.2.14 (no PyTorch, no model weights, no API keys). Against a 102-token handmade ASCII vocabulary and the schema {type: object, properties: {name: string, age: integer}, required: [name, age], additionalProperties: false}:

We did not install XGrammar (its PyPI package depends on torch and transformers, a larger download than this check allowed), run Outlines or Guidance against a live model, or inspect vLLM, SGLang, llama.cpp or OpenAI to verify which backend they currently default to. Library READMEs include latency figures and MaskBench plots; those are not reproduced here.

A safe selection sequence

  1. Name the layer. Do you need a Python API over many clients (Outlines), a programming model with control flow (Guidance), or a mask engine to embed in a server (XGrammar or llguidance)?
  2. List the constraint forms you actually need. Regex, JSON Schema, Lark, EBNF and Structural Tag are not equally documented in every package. If you need a keyword-level JSON Schema subset in writing, start from llguidance’s coverage page.
  3. Check the client, not only the library. Outlines’ matrix is explicit that some servers accept JSON Schema but not regex or grammars, and that SGLang wants EBNF when you pass a CFG through Outlines.
  4. Pin versions. Record the PyPI versions above (or newer pins) and, for engine embeddings, the engine version and grammar-backend flag (--grammar-backend, cmake options, and so on).
  5. Put the schema in the prompt as well. Both XGrammar’s constrained-decoding guide and Quick Start recommend describing the expected structure in the prompt, because masking only affects sampling, not the model’s prior.

If you later need to validate structured tool calls or other outputs after the fact, that is a different layer from constrained decoding; see Agent guardrail frameworks.

Directory entries in this comparison

Related

Sources

Machine-readable