AI Defense Stack

Comparison: Red teaming

DeepTeam vs promptfoo

DeepTeam and Promptfoo both attack LLM apps with jailbreaks and prompt injection; the decision is whether your red teaming lives in Python code or in a config file and CLI that also runs evals on pull requests. DeepTeam, from Confident AI, is a Python framework you call from code or a YAML config. Promptfoo is a Node.js CLI and library driven by promptfooconfig.yaml, with a documented GitHub Action.

  • Last reviewed

Executive summary

What it is
DeepTeam and Promptfoo are open-source red teaming tools that generate attacks against an LLM app and evaluate the responses. DeepTeam fits Python teams that call it from code; Promptfoo fits teams that run tests from a config file and CI.
The risk
An LLM app released without adversarial testing can follow injected instructions, give way to multi-turn jailbreaks or leak data, and a prompt or model change can bring back a weakness that an earlier run caught.
What to buy or do
Choose DeepTeam to script attacks in Python, including named multi-turn jailbreaks and custom vulnerabilities. Choose Promptfoo to test HTTP endpoints, code or browser flows from one config and run on pull requests. OpenAI's docs say Promptfoo is part of OpenAI.

Which should you choose?

Our analysis, built from the facts in the table below.

Choose DeepTeam if

Our analysis
  • You want to call red teaming from Python, passing a model callback, a list of vulnerabilities and a list of attacks.
  • You want named multi-turn jailbreaks such as Linear, Tree and Crescendo, and a README that lists more than 50 vulnerabilities and more than 20 attacks.
  • You want to define your own vulnerabilities by name and criteria in a YAML config and run them with deepteam run, in the CLI or in CI.

See DeepTeam.

Choose Promptfoo if

Our analysis
  • You want a CLI-first workflow driven by a promptfooconfig.yaml file, with a report command for failure cases.
  • You want to test an HTTP endpoint, custom code or a browser target, as the red team quickstart lists.
  • You want evals and red teaming together, with a documented GitHub Action for pull requests.

See Promptfoo.

Side by side

Every cell links to the page that states it, with the date we checked it. We compile from public pages and repositories and do not run either product. Coverage Score is computed within one category from sourced facts (how it works).

DeepTeam compared with Promptfoo: each row shows the fact for both tools with its source
Attribute DeepTeam Promptfoo
Vendor Confident AI Source, checked October 21, 2026: confident-ai.com (opens in a new tab) OpenAI (Promptfoo) Source, checked October 21, 2026: developers.openai.com (opens in a new tab)
Status Active Source, checked October 21, 2026: api.github.com (opens in a new tab) Active Source, checked October 21, 2026: api.github.com (opens in a new tab)
Ownership No ownership change documented in the sources we checked Promptfoo announced on March 9, 2026 that it had agreed to be acquired by OpenAI. OpenAI's developer documentation now says Promptfoo is part of OpenAI; we found no published closing date. Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
Open source and license Open source, Apache-2.0 Source, checked October 21, 2026: github.com (opens in a new tab) Open source, MIT Source, checked October 21, 2026: github.com (opens in a new tab)
GitHub stars 3,035 GitHub stars as of Oct 21, 2026; last push Oct 20, 2026 Source, checked October 21, 2026: api.github.com (opens in a new tab) 26,250 GitHub stars as of Oct 21, 2026; last push Oct 22, 2026 Source, checked October 21, 2026: api.github.com (opens in a new tab)
Latest release November 12, 2025 Source, checked October 21, 2026: api.github.com (opens in a new tab) October 8, 2026 Source, checked October 21, 2026: api.github.com (opens in a new tab)
Deployment Library, CLI, Self-hosted Source, checked October 21, 2026: github.com (opens in a new tab) CLI, Library, Self-hosted, SaaS, On-prem Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
Pricing model Free: The framework is free. On the vendor's pricing page, the Enterprise plan (custom pricing) lists an AI red teaming module. (as listed on October 21, 2026; check the vendor for current pricing) Source, checked October 21, 2026: confident-ai.com (opens in a new tab) Freemium: Community: Free Forever, with red teaming at 10k probes per month. Enterprise: Custom, with managed cloud deployment. On-Premise: Custom, deployed on your own infrastructure. (as listed on October 21, 2026; check the vendor for current pricing) Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
OWASP LLM risks Our analysis
  • LLM01 Prompt Injection
  • LLM02 Sensitive Information Disclosure
  • LLM09 Misinformation
  • LLM01 Prompt Injection
  • LLM02 Sensitive Information Disclosure
  • LLM09 Misinformation
Documented capabilities
  • Runs single-turn prompt injection attacks
  • Runs jailbreaking attacks, including linear, tree and Crescendo jailbreaking
  • Runs multi-turn attacks such as Sequential Jailbreak and Bad Likert Judge
  • Tests for PII leakage among its data privacy vulnerabilities
  • Tests for misinformation among its vulnerability types
  • Tests for toxicity among its vulnerability types
  • Uses LLMs to generate adversarial attacks and to evaluate the target's outputs
  • Can red team against OWASP Top 10 for LLMs 2025, NIST AI RMF and MITRE ATLAS
  • Accepts custom vulnerabilities defined by a name and criteria in a YAML config
  • Runs from a YAML config with deepteam run, in the CLI or in CI
Source, checked October 21, 2026: github.com (opens in a new tab) Source, checked October 21, 2026: trydeepteam.com (opens in a new tab) Source, checked October 21, 2026: trydeepteam.com (opens in a new tab)
  • Scans for jailbreaks among more than 50 vulnerability types
  • Scans for injections among more than 50 vulnerability types
  • Scans a target automatically across many vulnerability types from one command
  • Generates adversarial inputs with plugins and wraps them in attack strategies
  • Supports custom policies that enforce an organization's own guidelines
  • Offers red team presets for the OWASP LLM Top 10, NIST AI RMF and MITRE ATLAS, and lists EU AI Act and ISO 42001 as framework filters
  • Provides plugins that test for PII leakage: direct, across sessions, through social engineering and through API or database access
  • Provides hallucination and overreliance plugins that test for misinformation
  • Produces a report with test types, severity levels and suggested mitigations
  • Runs evals on pull requests through a documented GitHub Action
Source, checked October 21, 2026: promptfoo.dev (opens in a new tab) Source, checked October 21, 2026: promptfoo.dev (opens in a new tab) Source, checked October 21, 2026: promptfoo.dev (opens in a new tab) Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
Integrations 1 listed
  • OpenAI models as the simulator and evaluation LLMs
Source, checked October 21, 2026: trydeepteam.com (opens in a new tab) 		5 listed
  • OpenAI models
  • Anthropic models
  • AWS Bedrock models
  • Hugging Face models
  • GitHub Actions
Source, checked October 21, 2026: github.com (opens in a new tab) Source, checked October 21, 2026: promptfoo.dev (opens in a new tab) Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
Documentation
  • Documentation site
  • Quickstart
Source, checked October 21, 2026: trydeepteam.com (opens in a new tab) Source, checked October 21, 2026: trydeepteam.com (opens in a new tab)
  • Documentation site
  • Quickstart
Source, checked October 21, 2026: promptfoo.dev (opens in a new tab) Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
Language and install Python: pip install -U deepteam Source, checked October 21, 2026: trydeepteam.com (opens in a new tab) Node.js 22.22.0 or newer, then npx promptfoo@latest redteam setup Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
How you run it red_team() with a model callback, vulnerabilities and attacks, or deepteam run config.yaml Source, checked October 21, 2026: trydeepteam.com (opens in a new tab) promptfoo redteam run with a promptfooconfig.yaml, then promptfoo redteam report Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
What you point it at A Python model callback that wraps your LLM system, or a model provider such as OpenAI, Azure, Anthropic or Bedrock in YAML Source, checked October 21, 2026: trydeepteam.com (opens in a new tab) HTTP endpoints, direct models, custom Python or JavaScript code and a browser provider Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
What you need to start OPENAI_API_KEY set as an environment variable, because it uses LLMs to generate attacks and evaluate outputs Source, checked October 21, 2026: trydeepteam.com (opens in a new tab) An OpenAI API key is listed as optional Source, checked October 21, 2026: promptfoo.dev (opens in a new tab)
Coverage Score 71 out of 100 Computed from sourced facts. What this measures 80 out of 100 Computed from sourced facts. What this measures
Where each is stronger Our analysis
  • Documents a Python API for red teaming and a YAML CLI that accepts custom vulnerabilities defined by a name and criteria.
  • Lists named multi-turn attacks (Linear, Tree and Crescendo Jailbreaking, Sequential Jailbreak, Bad Likert Judge).
  • Documents more kinds of target: HTTP endpoints, direct models, custom code and a browser provider.
  • Documents a GitHub Action that runs on pull requests, and a report command that shows failure cases and raw inputs and outputs.
  • Publishes a SECURITY.md that promises acknowledgement within one business day; we did not find a security policy in the DeepTeam sources we checked.

Verdict

Our analysis

Our analysis

Pick by workflow. DeepTeam fits Python teams that want to call red teaming from code, define custom vulnerabilities in YAML and use named multi-turn jailbreaks such as Crescendo. Promptfoo fits teams that prefer a config file and a CLI, need to test an HTTP endpoint or browser flow, or want evals and red teaming on pull requests through its GitHub Action. Both are open source, both cover jailbreaks and prompt injection, and both document OWASP, NIST AI RMF and MITRE ATLAS coverage. OpenAI's documentation says Promptfoo is part of OpenAI, so check the repository for how the project continues. We compare documented features only and did not measure results.

DeepTeam vs Promptfoo: FAQ

Are DeepTeam and promptfoo open source?

Yes. DeepTeam is released under the Apache-2.0 license and promptfoo under the MIT License, and the code is on GitHub at confident-ai/deepteam and promptfoo/promptfoo.

What is the main difference between DeepTeam and promptfoo?

DeepTeam is a Python framework you call from code or run from a YAML config. Promptfoo is a CLI and library configured with a YAML file, with evals, a report view and a GitHub Action. The table above shows the source for each fact.

Which tool lists multi-turn attacks?

DeepTeam's README lists Linear, Tree and Crescendo Jailbreaking, Sequential Jailbreak and Bad Likert Judge as multi-turn attacks. The Promptfoo quickstart we read describes strategies that wrap generated inputs in attack patterns, but it does not name multi-turn attacks.

Do both tools need an OpenAI key?

DeepTeam's quickstart says to set OPENAI_API_KEY because it uses LLMs to generate attacks and evaluate outputs, and it points to docs for a custom LLM. Promptfoo's red team quickstart lists the key as optional.

Key facts you can cite

Keep exploring

Sources & last reviewed

Last reviewed . We compile from the vendors' own public pages and repositories and we do not run the products. Source links open in a new tab. Found a mistake? Report a correction.

Drafted with AI assistance; facts checked against primary sources on October 21, 2026.

DeepTeam (8 sources)
Promptfoo (14 sources)
Comparison rows (8 sources)