Promptfoo GitHub Action
GitHub Action for running Promptfoo prompt, agent and RAG tests and red-team scans in CI.
Tags
Description
Evaluates prompts when monitored files change; on pull requests it posts a summary comment with pass/fail counts and, if sharing is enabled, a Promptfoo web-viewer link. Inputs include config, prompts, provider API keys, promptfoo-version and no-share.
Related
- Promptfoo: Open-source tool to test and red-team prompts, agents and RAG: automated evals, vulnerability scanning and model comparison.
- Langfuse Experiment Action: GitHub Action that runs a Langfuse experiment in your CI pipeline.
Guides & comparisons
- Open-source red-teaming tools for LLM apps and agents (Comparison): Promptfoo, DeepTeam and Giskard compared as red-teaming tools for LLM apps and agents. Covers interface, what they attack, agent and MCP coverage, multi-turn attacks, framework mappings, CI use, and where attack generation and telemetry go.
Sources
- https://github.com/promptfoo/promptfoo-action
- https://raw.githubusercontent.com/promptfoo/promptfoo-action/main/action.yml
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)