AI red teaming & LLM evaluation
ART is an open-source adversarial machine learning library from LF AI & Data Foundation. It is released under the MIT license and is offered as: Library, Self-hosted.
Best for
Ownership
The project homepage states that IBM moved ART to LF AI in July 2020 and describes it as an LF AI & Data Foundation project. Source, checked October 21, 2026: adversarial-robustness-toolbox.org (opens in a new tab)
Consider instead if
ART is a Python library for attacking and hardening classic machine learning models, hosted by the LF AI & Data Foundation since IBM moved it there in July 2020. The homepage counts 39 attack modules and 29 defense modules, plus robustness, certification and verification metrics, across images, tables, audio and video. Models from TensorFlow, Keras, PyTorch, scikit-learn, XGBoost and other libraries plug in through wrapper classes. Use the inference attacks (membership inference, model inversion, reconstruction) when the question is whether a model leaks its training data. Neither the homepage nor the docs mention large language models, so it does not test prompts or chat apps, and for text classifiers TextAttack has the more specific recipes. It is MIT licensed; the latest release dates from July 2025.
This summary is our own wording of the sourced facts on this page.
Each value links to the page that states it, with the date we checked it.
As documented by the vendor or project. The tag is our label from a controlled vocabulary.
Our editorial model, not a standard. The primary layer follows the category, except for tools whose main documented job is detection and response; other layers rest on a capability the sources document. How the model works.
| Layer | Role | How it applies | Based on |
|---|---|---|---|
| L4 Testing Testing and red teaming | Primary | Attack and evaluate models, apps and agents on purpose, before release and after every change, to find what the other layers miss. | Follows its category, AI red teaming & LLM evaluation |
The risk list is OWASP's (Top 10 for LLM Applications, 2025). The mapping is our analysis, and each row rests on a capability the sources document.
| Risk | How it applies | Based on |
|---|---|---|
| LLM04 Data and Model Poisoning | Includes poisoning attack modules for ML models; the docs do not describe LLM-specific use. | Provides evasion attacks against machine learning models; Provides poisoning attacks against machine learning models; Provides extraction and inference attacks against machine learning models; Counts 39 attack modules and 29 defense modules, with 3 robustness metrics, 1 certification metric and 1 verification metric |
| LLM02 Sensitive Information Disclosure | Model inversion and membership inference attacks test the training data leakage OWASP LLM02 describes, on classic ML models rather than LLMs. | Inference attacks test whether a model leaks training data: membership inference, attribute inference, model inversion and reconstruction |
| LLM10 Unbounded Consumption | Extraction attacks copy a model through its predictions, the model theft path in LLM10; ART runs them against classic ML models. | Provides evasion attacks against machine learning models; Provides poisoning attacks against machine learning models; Provides extraction and inference attacks against machine learning models; Counts 39 attack modules and 29 defense modules, with 3 robustness metrics, 1 certification metric and 1 verification metric |
More in AI red teaming & LLM evaluation, ranked within the category by Coverage Score.
NVIDIA
Probes language models and LLM apps for prompt injection, jailbreaks, data leakage and more.
Best for: Teams that want a free command-line scanner they can point at any model or LLM endpoint before release
Reviewed
Giskard AI
Generates adversarial test suites for LLM agents and RAG apps, runs them and reports failures by threat type.
Best for: Python teams that want a free library to scan an LLM agent or RAG app for prompt injection and harmful output, and to write pass or fail checks for it
Reviewed
OpenAI (Promptfoo)
Runs evals and red team scans against LLM apps from a CLI or CI pipeline and writes a vulnerability report.
Best for: Developer teams that want one open-source CLI for both prompt evals and red teaming, run locally and in CI
Reviewed
Yes. ART is released under the MIT license, and the code is on GitHub at Trusted-AI/adversarial-robustness-toolbox.
The project homepage says ART is an LF AI & Data Foundation project and that IBM moved it to LF AI in July 2020.
The ART docs and README describe attacks on machine learning models in general and do not mention large language models. For LLM prompt injection or jailbreak testing, look at the LLM-focused tools in this category.
Yes, for the model types it supports. The docs list membership inference, attribute inference, model inversion and reconstruction attacks, and the homepage counts 39 attack modules and 29 defense modules in total.
ART is released under the MIT license.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
ART is offered as: Library, Self-hosted.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
Pricing model for ART: Free.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
Latest release of ART: July 7, 2025.
Source Source, checked October 21, 2026: api.github.com (opens in a new tab) Checked
We have no commercial relationship with LF AI & Data Foundation.