garak
NVIDIA's open-source LLM vulnerability scanner probing for hallucination, data leakage, prompt injection, jailbreaks, toxicity and other weaknesses.
Tags
Description
Combines static, dynamic and adaptive probes (described as nmap-like for LLMs). PyPI garak 0.17.1, Python >=3.11.
Related
- PyRIT: Microsoft's open-source Python Risk Identification Tool for proactively red teaming generative AI systems.
- DeepTeam: Open-source framework from Confident AI to red team LLMs and AI agents.
- Promptfoo: Open-source tool to test and red-team prompts, agents and RAG: automated evals, vulnerability scanning and model comparison.
Guides & comparisons
- Test a tool-using agent for prompt injection with AgentDojo (Guide): Measure how often injected tool outputs hijack your agent. Run AgentDojo's built-in suites from the CLI, wire your own model or agent loop in as a pipeline, read utility, utility under attack and targeted attack success, then compare defenses.
Sources
Machine-readable
- JSON
- Markdown
- Source file on GitHub (edit via pull request)