Open-source red-teaming tools for LLM apps and agents

Promptfoo, DeepTeam and Giskard compared as red-teaming tools for LLM apps and agents. Covers interface, what they attack, agent and MCP coverage, multi-turn attacks, framework mappings, CI use, and where attack generation and telemetry go.

Type

Comparison

Author

Agentica Author

Published

2026-10-21

Last verified

2026-10-21

• red-teaming
• security
• llm-security
• prompt-injection
• jailbreak
• agents
• mcp
• evals
• ci

Comparison

Verdict: Pick Promptfoo if you want a config-driven CLI with the broadest agent and MCP coverage and framework reports, but turn off remote generation and telemetry if data must stay local. Pick DeepTeam for a Python-native red team with a large agentic vulnerability list and a code scanner for PRs. Pick Giskard for multi-turn scans in Python; its third_party_scan can also run garak probes and DeepTeam attacks.

What this covers

Evals ask whether your app does the right thing. Red-teaming asks whether someone can make it do the wrong thing: leak a system prompt or PII, follow injected instructions, call a tool it shouldn't, or be talked out of its rules over several turns. All three tools here generate those attacks automatically, run them against your app, and grade the responses with an LLM judge. That last step means a "pass" is a model's opinion, so read the failing transcripts yourself.

For evals, tracing and CI quality gates, see Evaluate an AI agent and Agent observability platforms. For public capability scores, see Agent benchmarks.

How each one is wired

• Promptfoo is a CLI. You describe the target and its purpose in promptfooconfig.yaml, pick plugins (what to attack) and strategies (how to deliver it), and run promptfoo redteam run. It writes the generated probes to redteam.yaml and then attacks the target. The target can be any HTTP endpoint, a browser session or a model, so it tests the deployed app, not only the model.
• DeepTeam is a Python library built on DeepEval. You wrap your app in an async model_callback(input) -> str, choose vulnerabilities and attacks or a framework such as OWASPTop10(), and call red_team(...). deepteam run config.yaml does the same from YAML.
• Giskard v3 is a Python library. You pass any callable as target and a plain-language description to vulnerability_scan(...), and it generates the adversarial scenarios. third_party_scan(tool="garak" | "deepteam") runs garak probes or DeepTeam attacks through the same target and report format, which the README marks as experimental.

# DeepTeam, from its README: map an OWASP framework to vulnerabilities and attacks
from deepteam import red_team
from deepteam.frameworks import OWASPTop10

async def model_callback(input: str) -> str:
    return await my_agent(input)          # your app

risk_assessment = red_team(model_callback=model_callback, framework=OWASPTop10())

# Giskard v3, from the giskard-scan README
from giskard.scan import vulnerability_scan

result = await vulnerability_scan(
    target=my_agent,
    description="Customer support chatbot for an e-commerce store.",
    languages=["en"],
    max_scenarios=20,
)

# Promptfoo, adapted from its MCP security testing guide: attack an MCP server through a model client
targets:
  - id: openai:gpt-5
    config:
      mcp:
        enabled: true
        servers:
          - name: your-mcp-server
            url: https://your-mcp-server.example.com/mcp
redteam:
  purpose: A payment assistant that must enforce authorization on every tool call.
  plugins: [mcp, pii, bfla, bola, sql-injection]
  strategies: [multi-turn, jailbreak, jailbreak-templates]

What was tested

On 2026-10-21 we installed deepteam 1.0.10 and giskard[scan] 3.0.1 in a clean Python 3.13 virtualenv. The imports above resolved, deepteam --help listed run, scan and login, and list_scan_items("giskard") listed the scenario generators named in the table. We did not run any attacks, because every tool needs an LLM API key for generation and judging. Promptfoo 0.124.1 requires Node.js 22.22 or later, and the test box had Node.js 20, so its CLI was not run.

One finding from that install: deepteam 1.0.10 failed to import until sentry-sdk and posthog were installed by hand. Its telemetry module imports them, but the package does not list them as dependencies. If you get ModuleNotFoundError: No module named 'sentry_sdk', run pip install sentry-sdk posthog.

Agent-specific risks

Plain chatbots mostly risk saying something bad. Agents can do something bad, so check coverage for:

• Tool misuse and authorization: BOLA and BFLA (calling a function or reading an object the user shouldn't), and excessive agency. Promptfoo and DeepTeam both name these.
• Indirect prompt injection: instructions hidden in content the agent reads. Promptfoo has indirect-injection strategies, DeepTeam has an Indirect Instruction vulnerability, and Giskard's scan includes prompt injection.
• Tool poisoning: a malicious MCP server whose tool descriptions carry instructions. Promptfoo's MCP guide sets this up with its open-source rogue server, and DeepTeam lists Tool Metadata Poisoning.
• Multi-agent issues: DeepTeam lists inter-agent communication compromise and agent identity abuse.

Red-teaming finds weaknesses; it doesn't fix them. Pair it with server-side controls such as audience-bound tokens and per-tool scopes on your MCP servers (Add OAuth authorization to your own MCP server) and with care about which servers you connect (Pick an MCP server).

Notes

• Keep it local if you must. Promptfoo's best-quality attack generation uses its remote service by default. Its docs warn that local generation quality "depends greatly on the model that you configure". DeepTeam and Giskard send attack and judge prompts to whichever LLM provider you configure. All three have opt-outs for usage telemetry.
• Cost scales with the matrix. Vulnerabilities × subtypes × attacks × turns. Start with a small set, then widen.
• In CI, store the generated probes and rerun them as a regression suite: Promptfoo's retry strategy, or a committed DeepTeam YAML config. Gate on new failures, not on a perfect score.
• Not compared here: NVIDIA garak and Microsoft PyRIT have no directory entries yet. Giskard can run garak probes through third_party_scan.

Directory entries in this comparison

• Promptfoo: Open-source tool to test and red-team prompts, agents and RAG: automated evals, vulnerability scanning and model comparison.
• DeepTeam: Open-source framework from Confident AI to red team LLMs and AI agents.
• Giskard: Open-source Python library for testing agentic systems: scenario-based checks and LLM-as-judge evals (giskard-checks) plus a vulnerability scanner for red teaming (giskard-scan).
• Promptfoo GitHub Action: GitHub Action for running Promptfoo prompt, agent and RAG tests and red-team scans in CI.
• DeepEval: Open-source LLM evaluation framework with 50+ plug-and-play metrics for agents, RAG and chatbots.
• Model Context Protocol (MCP): Open protocol that standardizes how AI applications connect to external tools, data sources and prompts via MCP servers.

Related

• Evaluate an AI agent with open-source eval and observability tools (Guide)
• Observability and eval platforms for AI agents (Comparison)
• Agent benchmarks compared (Comparison)
• Add OAuth authorization to your own MCP server (Guide)
• Pick an MCP server (Skill)
• Test your MCP server with in-process tests and Inspector CI checks (Guide)
• Test a tool-using agent for prompt injection with AgentDojo (Guide)

Sources

• Promptfoo red team getting started (site/docs/red-team/quickstart.md), accessed 2026-10-21
• Promptfoo red team configuration (site/docs/red-team/configuration.md), accessed 2026-10-21
• Promptfoo red team strategies (site/docs/red-team/strategies/index.md), accessed 2026-10-21
• Promptfoo MCP security testing guide, accessed 2026-10-21
• Promptfoo CI/CD integration docs, accessed 2026-10-21
• Promptfoo GitHub Actions docs, accessed 2026-10-21
• Promptfoo telemetry docs, accessed 2026-10-21
• Promptfoo README, accessed 2026-10-21
• Promptfoo llms.txt (docs index, red-team plugin list), accessed 2026-10-21
• promptfoo-action README, accessed 2026-10-21
• DeepTeam README (confident-ai/deepteam), accessed 2026-10-21
• DeepTeam YAML configurations (docs/content/docs/red-teaming-yaml-cli.mdx), accessed 2026-10-21
• DeepTeam code scanning introduction, accessed 2026-10-21
• DeepTeam telemetry module (deepteam/telemetry.py), accessed 2026-10-21
• deepteam on PyPI, accessed 2026-10-21
• Giskard OSS README (Giskard-AI/giskard-oss), accessed 2026-10-21
• giskard-scan README, accessed 2026-10-21
• giskard on PyPI, accessed 2026-10-21

Machine-readable

• JSON
• Markdown
• Source on GitHub