Giskard
Open-source Python library for testing agentic systems: scenario-based checks and LLM-as-judge evals (giskard-checks) plus a vulnerability scanner for red teaming (giskard-scan).
Tags
Description
Giskard v3 is a modular rewrite for dynamic, multi-turn testing of AI agents: pip install giskard for checks, giskard[scan] for the scanner (prompt injection, jailbreaks, harmful content, RAG and quality evaluation), plus provider extras for LLM judges. Requires Python 3.12+. Optional aggregated telemetry sends no prompts or outputs and is turned off with DO_NOT_TRACK=1 or GISKARD_TELEMETRY_DISABLED=1. Giskard v2 remains available but is no longer actively maintained.
Agent access
Related
- DeepEval: Open-source LLM evaluation framework with 50+ plug-and-play metrics for agents, RAG and chatbots.
- DeepTeam: Open-source framework from Confident AI to red team LLMs and AI agents.
- Promptfoo: Open-source tool to test and red-team prompts, agents and RAG: automated evals, vulnerability scanning and model comparison.
- Ragas: Open-source evaluation toolkit for LLM apps and agents: LLM-based and traditional metrics for RAG and agent workflows, plus automatic test-set generation.
- Inspect: Open-source framework for large language model and agent evaluations from the UK AI Security Institute.
Guides & comparisons
- Open-source red-teaming tools for LLM apps and agents (Comparison): Promptfoo, DeepTeam and Giskard compared as red-teaming tools for LLM apps and agents. Covers interface, what they attack, agent and MCP coverage, multi-turn attacks, framework mappings, CI use, and where attack generation and telemetry go.
Sources
- https://github.com/Giskard-AI/giskard-oss
- https://docs.giskard.ai/oss
- https://pypi.org/project/giskard/
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)