Quick answer: A broad AI assisted scan tests every page and every subdomain with many specialist agents, so it can produce tens of thousands of raw findings. The large majority of that number is informational observation, not a real vulnerability. The raw count is never a measure of the attack surface. The correct reading is to sort findings into confidence tiers: confirmed, probable, hypothesis and informational. An operator first looks at the small set of confirmed critical and high severity findings, then evaluates the rest in context. The value of a good scanner is not in how much it finds, but in how well it filters the noise.

When you run a comprehensive AI scan against a modern web application, the first number you see is often startling. Tens of thousands of findings. The instinct is to assume the target is riddled with holes. The truth is far from that. This article explains why that big number appears, why it is misleading, and how to get from thousands of lines down to the few dozen that actually matter.

Why over production happens

A broad scan over produces by design. There are several structural reasons.

The first is fan out. The tool does not test only the home page. It tests every subdomain and every page it discovers, separately. Forty live subdomains and hundreds of pages mean a separate test set for each. This coverage depth is desirable, because the goal is to leave nothing unexamined, but it directly inflates the finding volume.

The second is the number of specialists. Dozens of distinct specialist agents run in a single scan. Each tries its own vulnerability class and produces its own observations. When injection, identity, header, configuration and information disclosure categories are multiplied out, the count climbs quickly.

The third is informational observations. A missing security header, an exposed version string, a default error page are not vulnerabilities on their own. They are recorded for context, but they land in the raw list as individual lines. In a typical scan these informational records make up the bulk of the total.

Our article on AI driven attack surface discovery covers the technical side of this fan out.

The raw count is not the attack surface

This is the most critical point. The total finding count printed at the top of a report is not the measure of real risk on that target. The real measure is how many were confirmed and how many of those are high impact.

Here is a concrete example. A typical broad scan can show a distribution like this.

Tier Typical share Meaning
Informational ninety percent and above Observation, not a vulnerability by itself
Low severity a few percent Minor hardening opportunity
Medium severity one or two percent Worth review, needs context
High severity tens of records Serious, awaiting verification or confirmed
Critical single digits Highest priority, mostly confirmed

So in an output of tens of thousands of lines, the part that genuinely needs action is often just a few dozen high and critical records. The number of confirmed results is smaller still. This scarcity is not a defect, it is the natural outcome of an honest tool. A tool that declares every observation critical is an untrustworthy tool.

Our article on verified vulnerabilities and false positive free testing goes deeper on the measurement side.

Confidence tiers and the triage funnel

The way to filter noise is to place findings into clear confidence tiers. A tier tells you how much you can trust a finding.

A confirmed finding carries independent proof. A measured delay in a time based injection, raw output in a reflected script, expected content in a file read. This tier is ready for action.

A probable finding carries a strong signal but no definitive proof yet. It enters the verification queue.

A hypothesis is a theoretical risk suggested by a pattern or a version. Until verified, it is not a claim.

Informational is an observation kept only for the record.

The triage funnel turns these tiers into a flow.

Stage Input Output
Raw collection all agent observations tens of thousands of records
Deduplication copies of the same finding merged unique records
Severity filter informational separated high and critical promoted
Verification gate probable and critical candidates confirmed or rejected
Operator review confirmed handful report and priority

Thanks to this funnel, the operator deals not with tens of thousands of lines but with the few dozen that passed the verification gate. We covered the verification logic in detail in our AI pentest reporting article.

What the operator should look at

When an AI scan report opens, the first place to look is not the total count. The order is this.

First, confirmed critical and high findings. These carry proof and are handled immediately. Then probable findings awaiting verification. These are quickly confirmed by hand, because sources of false positives exist such as network jitter in a timing measurement or context in a reflection. Last, the informational pile. These are evaluated together as a hardening checklist, not treated one by one as critical.

You can find the broader logic of AI assisted scanning in our article on how it differs from traditional scanners.

Signal and noise balance

The goal of a good tool is not to increase the finding count but to separate signal from noise. A security team has limited time. A tool that flags every observation as high severity tires the team with thousands of false alarms, and the real critical finding is lost in that crowd. This is the classic trap known as alert fatigue.

Confidence tiers and the verification gate exist precisely to prevent this fatigue. The maturity of a tool is measured not by how much it finds, but by how honestly it classifies what it finds. Our article on how a security AI is measured addresses this measurement side. We discussed the tendency of AI agents to drift toward exaggeration and hallucination in our article on AI forensic agents and hallucination.

The KAOS and DSET approach

KAOS, the AI scanning tool we develop at DSET, applies this article's core principle at the code level. However broad the scan, every finding is bound to a confidence tier, and the report summary leads with the confirmed result rather than the raw number. If an observation carries no proof it is not declared critical, it is marked as a hypothesis. This keeps the operator from drowning in thousands of lines and lets them focus on what truly matters. For more about KAOS, see our article on our AI cybersecurity scanning tool.

In a comprehensive security test, separating real risk from noise is not merely a report feature but a work of expertise. Our team at Ankara Hacettepe Teknokent turns automated scan output into a verified and prioritized report. Contact DSET.

Frequently asked questions

Why does the scan produce so many findings, is the target really that exposed? No. The large part of a high finding count is informational observation, not a real vulnerability. Testing every page and every subdomain separately and running dozens of specialists in parallel inflates the number. The real risk lies in the small set of confirmed critical and high severity findings.

Is it correct to compare two tools by raw finding count? It is not. A tool that produces more findings is not better, it is often noisier. The correct comparison is based on the number of confirmed findings, the false positive rate and the quality of proof.

Should I ignore informational findings completely? No, but you should not treat them one by one as critical. Evaluating them as a collective hardening checklist, and fixing missing headers and disclosures in bulk, is the most efficient path.

Sources

  • NIST SP 800-115, Technical Guide to Information Security Testing and Assessment, principles of finding evaluation and reporting.
  • OWASP Web Security Testing Guide, verification of test output and severity classification.
  • FIRST CVSS, the severity scoring framework.
  • DSET blog: verified vulnerabilities and false positive free testing.