Skip to content
Oday Bakkour
Back to Knowledge Hub

Cloudflare's Security Audit Skill: I Ran It on My Own API

Oday Bakkour profile photo
Oday Bakkour
5 min read
Share
Cloudflare's Security Audit Skill: I Ran It on My Own API

Executive Summary

  • What it is: security-audit is an open-source (MIT) skill from Cloudflare that turns your coding agent into a security auditor. It is the skill that seeded Cloudflare's own vulnerability discovery harness.
  • Why now: it is #1 on GitHub trending this week with about 15k new stars (47.9k in total).
  • How it works: six phases, each with its own agents. Every lead goes to a fresh verifier that tries to disprove it, and only records that survive end up in the report.
  • My run: on Helpdock's public widget API, scoped and capped: 12 agent runs, 24 coverage units, 14 of them clean, 4 leads. Result: 0 confirmed, 3 need validation, 1 rejected.
  • Why 0 confirmed: I had no OS-enforced sandbox, so the skill ran no code at all. It refuses to confirm what it did not observe, and keeps those leads as "needs validation" with an exact next step.

I made a short video of this on X. The numbers in it come from the run below.

Cloudflare's security audit skill is #1 on GitHub this week

Cloudflare's security audit skill is #1 on GitHub this week

It was already in my repo

I added this skill to Helpdock, my open-source help desk, on September 16, together with my own review skills. The copy under .agents/skills/security-audit is byte-identical to Cloudflare's repo. The rule in my AGENTS.md:

AGENTS.md
## Review skills

The repository ships review skills under `.agents/skills/`: `clean-code-guard`, `test-guard`, `docs-guard` and `security-audit`. Run the matching one before opening a PR that touches production code, tests or docs.

And the AI-actions milestone is only done when "security-audit findings are fixed or accepted in writing". The skill is part of the definition of done, not a one-off scan.

Helpdock's AGENTS.md: run the matching review skill before opening a PR

Helpdock's AGENTS.md: run the matching review skill before opening a PR

Install it

install.sh
npx skills add https://github.com/cloudflare/security-audit-skill \
  --skill security-audit
# add --global for a user-level install

Then start your agent in the repo and ask for it in plain words:

prompt
security audit this codebase

A direct audit request runs the full workflow and writes its files to ~/security-audit-skill/<repo-name>/run-<N> unless you name another directory. A security question uses guidance mode and writes nothing (README).

The six phases

  1. Reconnaissance: map the architecture, trust boundaries and input surfaces into architecture.md, and seed a deterministic coverage-ledger.json.
  2. Coverage-led hunting: isolated hunters each take ledger units. A coverage critic looks for what they missed.
  3. Candidate validation: every unique lead goes to a fresh verifier that tries to disprove it.
  4. Structured output: confirmed, needs_validation and rejected records in findings.json, validated against report-schema.json.
  5. Independent record verification: fresh agents re-check the final source claims.
  6. Reporting: REPORT.md, FINDINGS-DETAIL.md and NEEDS-VALIDATION.md, derived only from verified records.

The design rule I like most: "The agent that checks a finding is never the agent that found it." Another one keeps the report honest: a missing extra layer is a hardening note, not a vulnerability, when the first layer already stops the attack.

My run: scoped, capped, source only

A full audit of a big repo spends a lot of agent calls, so the skill has profiles (quick, standard, deep), scoped runs and a budget counted in agent invocations. I ran it on a fresh export of Helpdock with Claude Code, limited to the public chat-widget API:

prompt
Run a full security audit of this codebase with the security-audit skill,
quick profile, scoped to apps/api/src/widget only. Budget: at most 14 agent
invocations. No OS-enforced sandbox here, so source analysis only.

What came back after about 40 minutes:

  • Agent runs: 12 of 14: 4 recon, 3 hunters, 1 coverage critic, 4 verifiers
  • Coverage units: 24: 14 covered clean, 5 linked to leads, 1 deferred, 4 out of scope
  • Leads: 4 unique
  • Verdicts: 0 confirmed, 3 need validation, 1 rejected
  • Validators: validate-findings.cjs and validate-coverage-ledger.cjs both pass
Six phases, each with its own agents: 12 agent runs on my widget API

Six phases, each with its own agents: 12 agent runs on my widget API

The rejected lead is a good example of the verifier doing its job: the hunter flagged a behavior, and the verifier showed it was the documented contract, covered by an existing integration test.

The three that need validation each come with a repository trace, the exact fact that blocked confirmation, a small local test to settle it, and a read-only check for production. I'm working through them in Helpdock now, so I'm not publishing the details until they're settled.

Every lead faces a fresh verifier: 0 confirmed, 3 need validation, 1 rejected

Every lead faces a fresh verifier: 0 confirmed, 3 need validation, 1 rejected

Why 0 confirmed is the right answer

The skill only runs target code inside an OS-enforced sandbox: no external network, an allowlisted environment, write access only to scratch directories, and CPU, memory and time limits. I didn't have that set up, so it ran nothing. Without a local observation, a lead cannot be confirmed, so it stays needs_validation with no severity attached.

That is what I want from a security tool. A scary-looking list of "critical" items that nobody observed is worse than three precise next steps.

The report also listed what the code does well: one gate, one access function, digest-only storage for visitor secrets, and row-level security for brand isolation. That part is useful to keep when the code changes.

Tips from the run

  • Scope it. Start with your most exposed surface (for me, the anonymous widget API) and a budget.
  • Run it again. The README says one run found roughly half of what repeated runs found in total. Runs are additive: the next one reads the previous ledger and targets the gaps.
  • Give it a sandbox if you want confirmed findings with observed results.
  • Put it in your definition of done, the way my AGENTS.md does, so findings are fixed or accepted in writing before a release.
Run it on your repo: install, ask, scope it, run it again

Run it on your repo: install, ask, scope it, run it again

Sources

Add Oday Bakkour as a preferred source on Google

Comments

Share your thoughts and join the conversation

Leave a Comment

Loading comments...
RELATED