AI red teaming & LLM evaluation
b3 benchmark is an agent backbone LLM security benchmark from Check Point (Lakera) and UK AI Security Institute. It is open source under the MIT license and is offered as: CLI, Library.
Best for
Teams choosing a backbone model for an AI agent who want to compare how candidate models hold up against crowdsourced attacks in agent settings
Ownership
Check Point completed its acquisition of Lakera in the fourth quarter of 2025. Source, checked October 21, 2026: checkpoint.com (opens in a new tab)
The Backbone Breaker Benchmark, b3, measures how well the language model inside an AI agent resists attack. It rests on threat snapshots, which isolate the points in an agent's run where a model call can be exploited, and on attacks crowdsourced through Lakera's Gandalf Agent Breaker game. The attack types include direct and indirect instruction override and direct and indirect tool invocation, plus denial of AI service. It is part of the UK AI Security Institute's inspect_evals collection, so you install it with pip and run it against a model with the inspect command. A lower-quality public dataset is on Hugging Face. It compares models; it does not scan your own application.
This summary is our own wording of the sourced facts on this page.
Each value links to the page that states it, with the date we checked it.
As documented by the vendor or project. The tag is our label from a controlled vocabulary.
Our editorial model, not a standard. The primary layer follows the category, except for tools whose main documented job is detection and response; other layers rest on a capability the sources document. How the model works.
| Layer | Role | How it applies | Based on |
|---|---|---|---|
| L4 Testing Testing and red teaming | Primary | Attack and evaluate models, apps and agents on purpose, before release and after every change, to find what the other layers miss. | Follows its category, AI red teaming & LLM evaluation |
The risk list is OWASP's (Top 10 for LLM Applications, 2025). The mapping is our analysis, and each row rests on a capability the sources document.
| Risk | How it applies | Based on |
|---|---|---|
| LLM01 Prompt Injection | Measures how models resist direct and indirect instruction override attacks. | Attack types include direct and indirect instruction override |
| LLM02 Sensitive Information Disclosure | Data exfiltration is one of the attack objectives it covers. | Covers data exfiltration as one of its attack objectives |
| LLM06 Excessive Agency | Includes direct and indirect tool invocation attacks against the agent's model. | Attack types include direct and indirect tool invocation |
More in AI red teaming & LLM evaluation, ranked within the category by Coverage Score.
NVIDIA
Probes language models and LLM apps for prompt injection, jailbreaks, data leakage and more.
Best for: Teams that want a free command-line scanner they can point at any model or LLM endpoint before release
Reviewed
Giskard AI
Generates adversarial test suites for LLM agents and RAG apps, runs them and reports failures by threat type.
Best for: Python teams that want a free library to scan an LLM agent or RAG app for prompt injection and harmful output, and to write pass or fail checks for it
Reviewed
OpenAI (Promptfoo)
Runs evals and red team scans against LLM apps from a CLI or CI pipeline and writes a vulnerability report.
Best for: Developer teams that want one open-source CLI for both prompt evals and red teaming, run locally and in CI
Reviewed
The README says to install it with pip install inspect-evals[b3] and run uv run inspect eval inspect_evals/b3 with a model name. You can also call it from Python.
How the choice of backbone model affects the security of an AI agent, using threat snapshots and crowdsourced attacks. It compares models and does not scan your own application.
The paper lists authors from Lakera AI, ETH Zurich, the UK AI Security Institute and the University of Oxford. Check Point completed its acquisition of Lakera in the fourth quarter of 2025.
b3 benchmark is released under the MIT license.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
b3 benchmark is offered as: CLI, Library.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
Pricing model for b3 benchmark: Free.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
Status of b3 benchmark: Active.
Source Source, checked October 21, 2026: github.com (opens in a new tab) Checked
We have no commercial relationship with Check Point (Lakera) and UK AI Security Institute.