AI red teaming & LLM evaluation
AI Red Teaming Agent is an automated red teaming tool in Microsoft Foundry from Microsoft. It is a commercial product offered as: Cloud-native, Library. Pricing model: usage-based.
Best for
Teams already building on Microsoft Foundry that want automated safety and agent red teaming scans before release
Consider instead if
The AI Red Teaming Agent is a Microsoft Foundry feature that scans model and application endpoints by simulating adversarial probing. It builds on Microsoft's open-source PyRIT framework and Foundry's risk and safety evaluations, and it reports an Attack Success Rate by risk category and attack technique. Text-based scans cover hateful, sexual, violent and self-harm content, protected material, code vulnerabilities and ungrounded attributes. Agent-specific tests for prohibited actions, sensitive data leakage and task adherence run only in cloud red teaming, which Microsoft lists for five regions. You can also run scans locally with the Azure AI Evaluation SDK, which Microsoft's how-to page labels preview. Microsoft's Foundry pricing page says red teaming evaluations are billed on its AI evaluations meter, and we could not read the rates.
This summary is our own wording of the sourced facts on this page.
Each value links to the page that states it, with the date we checked it.
As documented by the vendor or project. The tag is our label from a controlled vocabulary.
Our editorial model, not a standard. The primary layer follows the category, except for tools whose main documented job is detection and response; other layers rest on a capability the sources document. How the model works.
| Layer | Role | How it applies | Based on |
|---|---|---|---|
| L4 Testing Testing and red teaming | Primary | Attack and evaluate models, apps and agents on purpose, before release and after every change, to find what the other layers miss. | Follows its category, AI red teaming & LLM evaluation |
The risk list is OWASP's (Top 10 for LLM Applications, 2025). The mapping is our analysis, and each row rests on a capability the sources document.
| Risk | How it applies | Based on |
|---|---|---|
| LLM01 Prompt Injection | Runs jailbreak, indirect prompt injection and multi-turn attack strategies against a target. | Attack strategies include a Jailbreak strategy for user-injected prompt attacks, plus encoding and obfuscation strategies from PyRIT; Indirect prompt injection (XPIA) tests whether an agent follows malicious instructions hidden in external data such as emails or documents; Multi-turn and Crescendo strategies run attacks across several conversation turns |
| LLM02 Sensitive Information Disclosure | Agent red teaming probes for leakage of financial, medical and personal data in cloud runs. | Agent red teaming for sensitive data leakage from knowledge bases and tool calls, using synthetic data and mock tools |
| LLM06 Excessive Agency | Tests whether an agent performs prohibited, high-risk or irreversible actions. | Agent red teaming for prohibited, high-risk and irreversible actions based on user-provided policies |
More in AI red teaming & LLM evaluation, ranked within the category by Coverage Score.
NVIDIA
Probes language models and LLM apps for prompt injection, jailbreaks, data leakage and more.
Best for: Teams that want a free command-line scanner they can point at any model or LLM endpoint before release
Reviewed
Giskard AI
Generates adversarial test suites for LLM agents and RAG apps, runs them and reports failures by threat type.
Best for: Python teams that want a free library to scan an LLM agent or RAG app for prompt injection and harmful output, and to write pass or fail checks for it
Reviewed
OpenAI (Promptfoo)
Runs evals and red team scans against LLM apps from a CLI or CI pipeline and writes a vulnerability report.
Best for: Developer teams that want one open-source CLI for both prompt evals and red teaming, run locally and in CI
Reviewed
The concept page says it uses Microsoft's open-source PyRIT framework and Foundry's Risk and Safety Evaluations to scan, evaluate and report on attack and response pairs.
Microsoft's local-scan how-to labels the agent preview, with no service-level agreement. Check Microsoft's current documentation for its status.
Agent-specific categories (prohibited actions, sensitive data leakage and task adherence) are cloud only. Microsoft lists East US 2, France Central, Sweden Central, Switzerland West and US North Central as the cloud regions.
AI Red Teaming Agent is closed source (proprietary).
Source Source, checked October 21, 2026: learn.microsoft.com (opens in a new tab) Checked
AI Red Teaming Agent is offered as: Cloud-native, Library.
Source Source, checked October 21, 2026: learn.microsoft.com (opens in a new tab) Checked
Pricing model for AI Red Teaming Agent: Usage-based: Microsoft's Foundry observability pricing page says red teaming evaluations are billed on the AI evaluations meter. Its rate table is script-rendered, so no price could be read..
Source Source, checked October 21, 2026: azure.microsoft.com (opens in a new tab) Checked
Status of AI Red Teaming Agent: Active.
Source Source, checked October 21, 2026: learn.microsoft.com (opens in a new tab) Checked
We have no commercial relationship with Microsoft.