Research · October 24, 2026

Why scanners miss the bugs auditors care about (research note)

Research note: a perspective from KN Cyber's research practice. It describes a problem we're studying and a research direction. It isn't a product announcement or a claim of results.

Most product teams run some kind of automated application security scanning: static analysis (SAST) in the pipeline, a dynamic scanner (DAST) against staging, dependency checks on every build. That's good practice. The problem is the confidence it creates. A clean scan report feels like evidence that the application is secure. Often it's evidence that the application doesn't contain the kinds of bugs the scanner knows how to find.

Those aren't always the bugs that matter to an auditor, a customer's security reviewer, or an attacker.

What auditors and reviewers actually care about

A SOC 2 auditor doesn't test your code for SQL injection. They test whether your controls work: whether access is limited to the right people, whether changes are reviewed, whether vulnerabilities are found and fixed on time. Customer and partner reviewers ask similar questions from the outside: show us penetration testing evidence, show us MFA and access reviews, show us that only authorized users can reach our data.

At the application layer, that comes down to one question more than any other: can a user reach data or actions they shouldn't?

The industry's own rankings say the same thing. In the OWASP Top 10:2025, Broken Access Control is the number one category. In the OWASP API Security Top 10 (2023), the first entry is Broken Object Level Authorization: an API that returns another user's record when you change an ID. Broken Function Level Authorization and Unrestricted Access to Sensitive Business Flows are on the list too.

In practice, that's the account record returned to the wrong customer, the payment endpoint that skips the merchant ownership check, the admin endpoint a regular user can call, or the data export with no tenant check.

Why scanners struggle with exactly those bugs

They don't know your rules

A scanner can recognize a dangerous pattern, like unsanitized input reaching a database query. It can't know that a support agent at Customer A should never see Customer B's accounts, or that a discount should apply only once. Those rules live in your requirements, your data model, and your team's heads, not in the code patterns a tool matches.

They struggle with state, roles, and multi-step flows

Many serious flaws only appear after logging in as a specific role, creating an object, and then acting on it as a different user. Automated crawlers have historically had trouble getting that deep.

A frequently cited study, Doupé, Cova, and Vigna's "Why Johnny Can't Pentest" (DIMVA 2010), tested eleven black-box web scanners against a purpose-built vulnerable application. Several planted vulnerabilities were missed by every scanner, including:

  • A parameter manipulation flaw that let anyone view other users' profile pages by changing an ID in the URL
  • A forceful-browsing flaw that let someone reach paid content without an account, which the authors called application-specific
  • A business logic flaw in coupon handling that needed domain knowledge to spot
  • Stored injection and multi-step cross-site scripting bugs that needed the scanner to create accounts or complete steps it didn't manage

A group of students with average security skills, analyzing the same application, found all the vulnerabilities except the forceful-browsing one. The study is more than fifteen years old and scanners have improved. But the categories it highlighted, authorization, logic, and stateful flows, are the same ones at the top of OWASP's lists today.

Static analysis misses much of what's real

Static tools have a related problem. A 2023 study at ACM ESEC/FSE, "Comparison and Evaluation on Static Application Security Testing (SAST) Tools for Java," built a benchmark of 165 real open-source Java programs, each with a known CVE. Tools that performed well on synthetic benchmarks detected only 12.7% of the real-world vulnerabilities. Even combining all the tools studied, 70.9% went undetected.

The lesson isn't that SAST is useless. It's that synthetic benchmark scores don't predict real-world coverage, and a tool's silence tells you less than it seems.

Clean reports read as assurance

The quieter risk is organizational. When the pipeline is green, it's easy to tell a buyer "we scan every build," and easy for that to be heard as "we've tested for the issues you care about." Auditors and plan reviewers increasingly ask for penetration test evidence specifically because they know automated scans aren't the same thing.

What closes the gap today

None of this needs new technology to start fixing.

  1. Write down your authorization model. Roles, tenants, and which objects each role can read or change. You can't test rules nobody has written down.
  2. Test authorization directly. Add automated tests that try to access another tenant's or another user's objects, and call privileged functions as an unprivileged role. Run them in CI alongside your scanners.
  3. Scan authenticated, with multiple roles. Give your DAST tool credentials for each role, not just one, and confirm it reaches the authenticated parts of the app.
  4. Threat-model the sensitive business flows. Exports, bulk actions, invitations, billing, anything involving PHI. Ask how each could be abused.
  5. Get a human-led penetration test at least annually and after major changes, scoped to cover web apps, APIs, and mobile apps, with retesting after fixes. Many customer security terms already expect it.
  6. Keep the evidence trail. Findings, fixes, and retest results, mapped to your vulnerability management control. That's what an auditor samples.

What we're researching

KN Cyber's research practice is interested in a narrower question: how can automated scanners find more of the vulnerabilities that matter, and how can teams tell how much a scan actually covered?

Directions we're exploring include:

  • Giving tools application context. Can a scanner use a declared authorization model (roles, ownership, tenancy) to generate meaningful access-control tests instead of guessing?
  • Measuring what was missed, not just what was found. Reports that show coverage across roles, endpoints, and flows, so a clean result is interpretable.
  • Linking findings to assurance evidence. Connecting test results to the controls auditors and buyers evaluate, so evidence stays current between point-in-time assessments. This is related to our APSOS™ research on continuous product security assurance.

These are open research questions. We'll publish what we learn, including what doesn't work.

How KN Cyber helps now

While the research continues, the practical work is available today: penetration testing for web apps, APIs, mobile apps, cloud, and networks; application security program reviews; and SOC 2 readiness that connects your testing to the controls your auditor will examine. See Product security and penetration testing and SOC 2 readiness. Pricing is scoped to your organization's size and needs.

Want to know what your scanners are missing? Book a consultation.

Sources

Accessed October 22, 2026.

← All insights