Promptfoo
Open-source tool to test and red-team prompts, agents and RAG: automated evals, vulnerability scanning and model comparison.
Tags
Description
README: 'Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed.' npm promptfoo 0.124.1 requires Node >=22.22.0.
Agent access
Related
- Promptfoo GitHub Action: GitHub Action for running Promptfoo prompt, agent and RAG tests and red-team scans in CI.
- DeepTeam: Open-source framework from Confident AI to red team LLMs and AI agents.
- garak: NVIDIA's open-source LLM vulnerability scanner probing for hallucination, data leakage, prompt injection, jailbreaks, toxicity and other weaknesses.
- PyRIT: Microsoft's open-source Python Risk Identification Tool for proactively red teaming generative AI systems.
Guides & comparisons
- Evaluate an AI agent with open-source eval and observability tools (Guide): A hands-on loop for agent quality: instrument with OpenTelemetry, collect traces, turn real traces into a dataset, write code, LLM-as-judge and human evaluators, run offline experiments, then monitor production and gate pull requests in CI.
- Test a tool-using agent for prompt injection with AgentDojo (Guide): Measure how often injected tool outputs hijack your agent. Run AgentDojo's built-in suites from the CLI, wire your own model or agent loop in as a pipeline, read utility, utility under attack and targeted attack success, then compare defenses.
- Observability and eval platforms for AI agents (Comparison): Langfuse, LangSmith, Arize Phoenix, Braintrust, Opik and Laminar compared on license, self-hosting, OpenTelemetry ingestion, MCP access to your traces, free tiers, paid entry prices and built-in evaluation.
- Runtime guardrails for agents: Guardrails AI, NeMo Guardrails, Invariant and HOL Guard (Comparison): Four open-source guardrail layers compared: where each one sits, what a rule looks like, whether it can check tool calls, which checks need a model, what leaves your machine and how actively each is maintained. All four were installed and run on 2026-11-01.
- Open-source red-teaming tools for LLM apps and agents (Comparison): Promptfoo, DeepTeam and Giskard compared as red-teaming tools for LLM apps and agents. Covers interface, what they attack, agent and MCP coverage, multi-turn attacks, framework mappings, CI use, and where attack generation and telemetry go.
- Agent benchmarks compared (Comparison): Ten widely cited agent benchmarks (SWE-bench, Terminal-Bench, tau-bench, BFCL, MCPMark, GAIA, BrowseComp, WebArena, Mind2Web and OSWorld) compared on what they measure, how many tasks they have, where the agent runs, how runs are scored, where results are published and how they are licensed.
Sources
- https://github.com/promptfoo/promptfoo
- https://www.promptfoo.dev/docs/intro/
- https://www.promptfoo.dev/blog/promptfoo-joining-openai/
- https://www.npmjs.com/package/promptfoo
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)