AI red teaming & LLM evaluation
PyRIT is an open-source generative AI red teaming framework from Microsoft. It is released under the MIT license and is offered as: Library, CLI, Self-hosted.
Best for
Security engineers who want a free Python framework for building multi-turn red teaming runs against models and web apps
Consider instead if
PyRIT, the Python Risk Identification Tool, is an open-source framework from Microsoft for red teaming generative AI systems. Its docs describe it as a framework for automated and human-led red teaming. You write Python, or use the pyrit_scan command-line scanner, to run single-turn and multi-turn attacks against targets such as OpenAI, Azure, Anthropic, Google and Hugging Face models, custom HTTP endpoints and web apps. Scorers judge the responses, and results are stored in SQLite or Azure SQL. A local web app called CoPyRIT is also available. The code is MIT licensed and free to use. PyRIT is a framework you build tests with, not an inline filter for live traffic.
This summary is our own wording of the sourced facts on this page.
Each value links to the page that states it, with the date we checked it.
As documented by the vendor or project. The tag is our label from a controlled vocabulary.
Our editorial model, not a standard. The primary layer follows the category, except for tools whose main documented job is detection and response; other layers rest on a capability the sources document. How the model works.
| Layer | Role | How it applies | Based on |
|---|---|---|---|
| L4 Testing Testing and red teaming | Primary | Attack and evaluate models, apps and agents on purpose, before release and after every change, to find what the other layers miss. | Follows its category, AI red teaming & LLM evaluation |
The risk list is OWASP's (Top 10 for LLM Applications, 2025). The mapping is our analysis, and each row rests on a capability the sources document.
| Risk | How it applies | Based on |
|---|---|---|
| LLM01 Prompt Injection | Ships a Jailbreak scenario that tests a model against jailbreak templates. | Includes a Jailbreak scenario that tests how vulnerable a model is to jailbreak templates |
| LLM02 Sensitive Information Disclosure | Ships scenarios that cover data leakage against a target system. | Includes standardized scenarios that cover data leakage, content harms and psychosocial risks |
More in AI red teaming & LLM evaluation, ranked within the category by Coverage Score.
NVIDIA
Probes language models and LLM apps for prompt injection, jailbreaks, data leakage and more.
Best for: Teams that want a free command-line scanner they can point at any model or LLM endpoint before release
Reviewed
Giskard AI
Generates adversarial test suites for LLM agents and RAG apps, runs them and reports failures by threat type.
Best for: Python teams that want a free library to scan an LLM agent or RAG app for prompt injection and harmful output, and to write pass or fail checks for it
Reviewed
OpenAI (Promptfoo)
Runs evals and red team scans against LLM apps from a CLI or CI pipeline and writes a vulnerability report.
Best for: Developer teams that want one open-source CLI for both prompt evals and red teaming, run locally and in CI
Reviewed
Yes. PyRIT is released under the MIT License, and the code is on GitHub at microsoft/PyRIT.
The docs list OpenAI, Azure, Anthropic, Google and Hugging Face targets, custom HTTP endpoints and WebSockets, and web apps through Playwright.
The docs list SQLite and Azure SQL as memory backends, and scorers that use LLMs, Azure AI Content Safety or custom logic.
PyRIT is released under the MIT license.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
PyRIT is offered as: Library, CLI, Self-hosted.
Source Source, checked October 21, 2026: microsoft.github.io (opens in a new tab) Checked
Pricing model for PyRIT: Free.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
Latest release of PyRIT: September 4, 2026.
Source Source, checked October 21, 2026: api.github.com (opens in a new tab) Checked
We have no commercial relationship with Microsoft.