The cases
| Finding | Age | Class | Impact if exploited | Discovery / status |
|---|---|---|---|---|
| OpenBSD | 27 years | (details limited publicly) | — | <1,000 autonomous runs, <$20K compute |
| FFmpeg | 16 years | Memory safety in a decoder path | Media-file-triggered code execution across the FFmpeg-dependent ecosystem | Survived >5M fuzz iterations; patched |
| wolfSSL — CVE-2026-5194 | — | Certificate-validation logic | Certificate forgery; impersonation of legitimate TLS endpoints incl. banking sites | Patched |
| FreeBSD NFS client — CVE-2026-4747 | 17 years | Memory corruption | Unauthenticated RCE from a malicious NFS server on a reachable segment; no MitM | Patched; notably absent from Anthropic’s initial ledger (VulnCheck) |
| Firefox (Mozilla) | Various | Various | — | 271 found and fixed during testing |
A note on CVE-2026-4747: VulnCheck flagged that the single CVE Anthropic itself highlighted was, at least initially, missing from Anthropic’s own disclosure ledger — a small but telling inconsistency for a project whose credibility rests on the ledger being the record.
Where the model beats fuzzing — and where it doesn’t
The FFmpeg result is the most instructive. Coverage-guided fuzzing is excellent at shallow crashes and bad at bugs that require satisfying several semantic preconditions in sequence, or understanding what the code is for. A 16-year-old bug that survived five million iterations is almost by definition one of those.
Bug classes where model-based discovery appears to outperform fuzzing, from the public Glasswing and Black Hat material:
- Logic errors in validation (the wolfSSL class): the code runs correctly and returns the wrong answer. Fuzzers have no oracle for “this certificate should have been rejected.”
- State-dependent memory corruption (the FreeBSD NFS class): reaching the vulnerable path requires a protocol conversation, which random mutation rarely produces.
- Long-lived bugs in heavily fuzzed code: survivorship bias means what remains after years of fuzzing is exactly what fuzzing cannot reach.
- Cross-function reasoning: bugs where a precondition is established in one function and violated in another, which coverage metrics do not capture.
Where fuzzing still wins: throughput-per-dollar on shallow input-parsing bugs, and reproducibility. Shoshitaishvili’s Black Hat keynote (Week 6) framed the human skill as having shifted toward building search strategies, validation pipelines and exploitability checks around the model rather than finding bugs by hand.
The exploitation gap — hold both facts at once
The Register’s July reporting is a necessary counterweight: as of late July only 126 Glasswing findings had published CVEs and just one was confirmed exploited in the wild. VulnCheck’s blunt summary: “AI-assisted vulnerability discovery and frontier capabilities have been overhyped relative to the evidence available today… the impact has been real but modest.”
Microsoft’s data cuts the other way. An internal system converted 182 of 200 Linux kernel vulnerabilities into crash-level proofs of concept, many working root exploits, at an average of $3.61 and 21 minutes each (Week 6). Both facts are true simultaneously:
- Exploitation of Glasswing findings by attackers is, so far, rare.
- The cost of exploitation has collapsed.
The gap between those two is the window defenders currently have. It is not guaranteed to stay open.
The UCL rediscovery benchmark
University College London researchers proposed a benchmark testing whether other models can independently rediscover Glasswing findings from the unpatched code. It matters for two reasons:
- Embargo risk becomes an engineering estimate. If a second-tier model rediscovers a finding in N runs, the embargo on that finding is worth roughly N runs of compute to an attacker. This is the quantitative version of VulnCheck’s “findings age into liabilities” concern (Week 4).
- Cleaner capability measurement. Rediscovery rate against a known ground truth beats raw finding counts, which are inflated by duplicates and withdrawals — recall the curl 5→1 case.
Early results were not published at the time of writing; watch arXiv under the authors’ names.
Institutional evaluations
- UK AI Security Institute: Mythos Preview was the first model to complete both of AISI’s cyber ranges — multi-host, multi-stage intrusion scenarios requiring reconnaissance, exploitation, lateral movement and objective completion, not single-bug discovery. This is the capability that connects Week 5 to Weeks 1–3: a model that can complete a cyber range is doing, in a sanctioned environment, what the Hugging Face agents did in an unsanctioned one.
- Mozilla: 271 Firefox vulnerabilities found and fixed during the testing period — the clearest example of the intended Glasswing model, a well-resourced maintainer able to absorb findings at volume. It is also the implicit benchmark against which the two-person-project problem (Week 4) should be read: Mozilla is what capacity looks like, and most maintainers are not Mozilla.
Sources
- Anthropic, Claude Mythos Preview system card + Project Glasswing announcement, Apr 2026; Glasswing update, May 2026 — anthropic.com
- UK AI Security Institute, Mythos Preview pre-deployment evaluation — aisi.gov.uk
- Mozilla security advisories, Glasswing testing period — mozilla.org/security
- VulnCheck, Observations on Anthropic’s Vulnerability Disclosure Ledger — https://www.vulncheck.com/blog/anthropic-ledger (CVE-2026-4747 omission)
- The Register, AI-found bugs aren’t proving any easier to exploit despite the hype, 28 Jul 2026 — https://www.theregister.com/security/2026/07/28/ai-found-bugs-arent-proving-any-easier-to-exploit-despite-the-hype/5279637
- Security Point Break, Black Hat 2026: Fighting an Adversarial Tsunami, 6 Aug 2026 — https://securitypointbreak.com/2026/08/06/black-hat-2026-fighting-an-adversarial-tsunami/ (Microsoft PoC-generation figures)
- CSO Online, Microsoft wants you to rethink your approach to cyber defense, Aug 2026 — csoonline.com
- UCL rediscovery benchmark — preprint pending; check arXiv
Editorial note: OpenBSD (27y, <1,000 runs, <$20K), FFmpeg (16y, >5M fuzz iterations), wolfSSL CVE-2026-5194, AISI “both ranges,” Mozilla 271 — taken from the commissioning brief, not independently re-verified. Check against the Mythos system card and vendor advisories; verify CVE numbers at cve.org.
Back to the light read (2 min).