Methodology: how we source facts and score tools

Sourcing standard

We compile from public primary sources and we do not run or benchmark the tools ourselves. For each listing we open the vendor's own pages, the project's repository and license file, and official standards pages, and we record what they say.

• Primary sources only. A source is the tool's own site or documentation, its code repository, or an official standards or cloud-provider page. Each listing names the hosts it is allowed to cite.
• The page that states the fact. We cite a specific docs or pricing page, not just a homepage, whenever one exists.
• No aggregators as sources. These hosts are rejected by the build: g2.com, capterra.com, crunchbase.com, wikipedia.org, producthunt.com, medium.com, linkedin.com, theresanaiforthat.com.
• Unverified means absent. If we cannot verify a fact we leave it out. The page then says "Not documented in the sources we checked", never "No".
• A minimum to be listed. A tool needs a verified vendor, status, deployment, open-source status, pricing model, website and at least three capabilities. Otherwise it is not listed.
• Prices only from the vendor. We show a price only if it is stated on the vendor's own pricing page, with the date, and a reminder to check the vendor for current pricing.
• GitHub numbers are dated. Stars, archived status and the last push come from the GitHub API on the day we checked. Stars show attention, not quality (how we use them).

What a fact is

Every factual field is stored as a fact with four parts: the value, the source URL (https only), the date we opened the page, and an optional short locator that tells a checker where on the page to look. The build fails if a source or date is missing, if the date is in the future, or if a field has an unknown name. On a page, each fact shows an evidence chip with the source host and the check date, linked to the source.

What is our analysis

Some text is ours, not a fact: taglines, descriptions, "best for" lines, "consider instead if" lines, and the mapping to the OWASP and MITRE ATLAS frameworks. Our words must not contain a factual claim that is missing from the sourced facts, and the mapping is always marked Our analysis. Where we compare two tools, both sides must have verified facts and we state where each is stronger.

Coverage Score

The Coverage Score runs from 0 to 100. It measures how much a tool documents in public: risk coverage, capabilities, deployment options, openness, documentation, integrations, maintenance and transparency. It is computed in code from the sourced facts in each listing. There is no score field in the data, and the build rejects one.

Scores compare tools within one category. Category pages sort by score, and the home page never shows a global ranking. This is rubric version 1.0, dated October 20, 2026. If a weight changes, the version and this date change with it.

The rubric (weights add up to 100)

Coverage Score criteria, points and rules

Criterion	Points	Computed from	Rule
Risk coverage	25	owasp_llm	25 x (OWASP LLM risks mapped within the category list, each backed by a sourced capability) / (size of the category list)
Capability breadth	20	capabilities[].tag	20 x min(1, distinct capability tags from the category vocabulary / the category target)
Deployment flexibility	10	deployment	1 mode = 4, 2 modes = 7, 3 or more = 10
Openness	10	open_source	OSI-approved license = 10; source-available or a custom license = 6; proprietary with a stated free tier = 3; proprietary = 0
Documentation	10	docs	public docs site 4 + API reference 3 + quickstart 3
Integrations	10	integrations[].tag	1 point per distinct integration tag, up to 10
Maintenance	5	last_release or github.pushed_at	newest activity within 180 days of the review date = 5; within 365 days = 3; older = 1; archived or deprecated = 0
Pricing transparency	5	pricing	free or open source, or public prices on the vendor page = 5; a free tier or inclusion stated without prices = 3; contact sales or not published = 0
Trust transparency	5	trust	security or trust page (or SECURITY.md) = 2; stated attestations or a documented disclosure process = 3
Total	100

Missing data scores 0 for that criterion and appears as "Not documented in the sources we checked". A tool page shows the total, the nine sub-scores as bars, and the list of facts that fed each bar.

Category settings (our analysis)

Two numbers depend on the category: which OWASP LLM risks count as relevant, and how many distinct capabilities make a tool "broad" in that category. We chose them from what each kind of control is meant to do.

Relevant OWASP LLM risks and capability target per category

Category	Relevant OWASP LLM risks	Capability target
LLM firewalls & guardrails	LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM05 Improper Output Handling, LLM07 System Prompt Leakage, LLM09 Misinformation, LLM10 Unbounded Consumption	8
AI red teaming & LLM evaluation	LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM09 Misinformation	7
Model & ML supply-chain security	LLM03 Supply Chain, LLM04 Data and Model Poisoning	5
AI security posture management & discovery	LLM02 Sensitive Information Disclosure, LLM03 Supply Chain, LLM06 Excessive Agency, LLM08 Vector and Embedding Weaknesses	6
Agent & MCP security	LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage	6
GenAI data-leak prevention & privacy	LLM02 Sensitive Information Disclosure, LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses	5
AI governance, risk & compliance	LLM02 Sensitive Information Disclosure, LLM04 Data and Model Poisoning, LLM06 Excessive Agency, LLM09 Misinformation	6

What the score does not measure

• We do not run the tools, so there is no measurement of detection accuracy, false-positive rate, latency or throughput.
• It says nothing about support quality, ease of use or price-performance.
• A public benchmark may appear in a separate "Public benchmarks" list on a tool page, with its source and date. It never feeds the score.
• A documentation-based score favors vendors with public docs and open-source tools. A low score can mean "little published", not "poor".

Independence guarantees

• The score function receives the listing with the sponsorship, affiliate and ownership fields removed, so it cannot read them.
• A build check confirms that the function's source does not mention those fields and that switching sponsorship on for a sample listing leaves its score unchanged.
• Sponsored placements are labeled "Sponsored", sit outside ranked lists, and have no effect on scores, ordering, picks or comparison verdicts. See the disclosure page.

GitHub stars

Tools whose listing records their own GitHub repository carry a stars badge on the tools list, the layer pages and comparison pages. It shows the star count, the date we read it and the date of the latest push to the repository. A tool with no recorded repository shows no badge: that covers every commercial tool without public code, and an open-source project that lives inside a larger shared repository.

Stars are a popularity signal, not a measure of security. They count GitHub accounts that starred the repository, so a large number says nothing about detection accuracy, how the code is reviewed or how the maintainers handle a vulnerability report. A general framework can collect stars for features that have little to do with security, while a narrow scanner with a few hundred stars may be the one that fits your job. Sorting the tools list by stars is a way to browse it, not a ranking.

Stars do not feed the Coverage Score. The score reads two other values from the same GitHub record, both in the Maintenance criterion (5 of 100 points): the latest push date, which counts as activity next to the latest release date, and the archived flag. The rule is: newest activity within 180 days of the review date = 5; within 365 days = 3; older = 1; archived or deprecated = 0. Activity is counted against the listing's review date. A refresh that brings a newer push date or a new archived flag can therefore change that criterion, while a change in stars changes no score.

The as-of date is the day, in Pacific time, that we last read the repository from the GitHub API. The star count, the archived flag and the last push in one badge come from that single response. The last push is the UTC date GitHub reports, so a push on a Pacific evening can carry the next day's date. Before each deploy a script reads every listed repository again, one request each, without signing in. GitHub allows 60 such requests per hour, so when the allowance runs out the tools not yet reached keep their earlier values and their earlier date, and two badges on one page can show different dates. A refresh does not move a listing's "Last reviewed" date, because nobody re-checked the rest of the listing.

Two text labels mark repositories that may have stopped moving. "archived" appears when GitHub marks the repository archived. "inactive" appears when the last push is more than 365 days (12 months) before the as-of date; an archived repository shows only "archived". We picked 12 months because past that point the Maintenance criterion drops to its lowest score above zero. Treat "inactive" as a reason to read the project's issues and releases before you adopt it: a finished library can go a year without a push and still work.

OWASP and MITRE ATLAS mapping

The list of risks is OWASP's: the OWASP Top 10 for LLM Applications, 2025 edition, published at genai.owasp.org. The technique list is MITRE's, from atlas.mitre.org. The mapping from a tool to a risk or a technique is ours. Each mapped entry must point to at least one capability the tool documents with a source, and every place it appears is labeled Our analysis. A mapping says a documented capability addresses part of a risk, not that the tool removes the risk.

How we place tools in layers

A tool's layer tells a buyer which budget line it can fill, so placement follows written rules. AI Security Layers is the directory's own model for organizing AI security controls; it is not a standard, and OWASP, MITRE and NIST did not create or endorse it. Every placement is our analysis, marked Our analysis, and rests on a capability the tool's own sources document:

1. The category comes first. A tool sits in the layer of the category that matches its main documented job, so its primary layer is never a judgment call.
2. A tool gets at most two more layers, and only when its own sources document a capability that does that layer's job. A matching tag in our vocabulary is not enough: the capability has to describe the layer's work.
3. We never reword a capability, change a tag or move a category to reach a layer, and sponsorship or an affiliate link never places a tool.
4. When more than two layers qualify, we keep the two with the most qualifying capabilities. On a tie, a layer the vendor sells as a named product or module wins, then the lower layer number.
5. Archived and deprecated tools keep their layers. Layer pages list them last, with their badge.
6. A tool whose main job fits no layer, for example deepfake or voice fraud detection, is not forced into one. We log it, and three or more such tools would trigger a proposal to change the model. Detection and response for a security operations center has been layer 8 since tools documenting it as their main job were added.
7. Monitoring and response has no category, so it is a tool's primary layer only when the tool documents at least two capabilities that match the layer and its description names detection and response as its main job. Otherwise it is a secondary layer, like any other.
8. Layers never change the Coverage Score. The score measures what a vendor documents inside its own category.

Vocabulary

Capabilities and integrations are tagged from a controlled vocabulary so that tools can be counted and compared. Each tag has a one-line definition.

Freshness

We re-verify each listing at least every 90 days. The "Last reviewed" date on a page changes only when we make a real edit, and the visible date, the date in the page's structured data and the date in the sitemap are the same value. Every fact carries its own check date, so you can see which parts are older.

Limits

The market moves quickly: products are renamed, acquired and retired. A fact is true as of the date shown and may have changed since. Treat a listing as a starting point and confirm details with the vendor. This is information, not legal, compliance or purchasing advice. If you find an error, tell us on the corrections page.