ONLINEAGENT_OPS 2026.Q3 HOME ARTICLES CRAFT RECORD BLOG MAP HUBS FAQ SEARCH
HOMETHE CRAFTA Security Checklist for AI-Built Software
THE CRAFT · CHECKLIST

A Security Checklist for AI-Built Software

Security checks for software you did not fully write, in three tiers by what a mistake costs, plus three AI-specific checks standard practice misses.

READ3 min
WORDS795
SECTIONS6
SOURCES2
TYPEGUIDE
CHECKED25 AUG 26
TL;DR — THE SHORT VERSION

Check software you did not fully write in order of what a mistake would cost, so stopping partway still covers the expensive part.

  • Tier 1: before it goes public. Search for secrets, including in version history, check that each dependency is real, turn debug off and tighten CORS.
  • Tier 2: before other people’s data. Validate on the server, parameterise queries, check authorisation as well as login, and rate-limit anything that costs.
  • Tier 3: before money or credentials. Keep secrets out of logs and the repository, put a human in front of anything irreversible, test the restore, and have someone who reads code review it.
  • Three AI-specific checks. Treat fetched content as data, read each skill or tool before installing it, and handle model output like input from a stranger.
  • Scanners help but do not finish the job. Automated tools catch common flaws faster; they do not understand your business logic.

A checklist for software you did not fully write. Ordered by what it costs to get wrong, so you can stop partway and still have covered the expensive part.

◈ HOW TO USE THIS

Tier 1 before anything touches the internet. Tier 2 before it touches anyone else's data. Tier 3 before it touches money or credentials. Stopping early is fine. Skipping to tier 3 is not.

THE THREE TIERS · IN ORDER
Ordered by what it costs to get wrong. Stopping early is fine; skipping ahead is not.
Tier 1 · before it goes anywhere publicSecrets, version history, dependencies, debug mode, CORS.
Tier 2 · before it touches anyone else’s dataServer-side validation, parameterised queries, escaping, authentication and authorisation, rate limits.
Tier 3 · before money, credentials or personal recordsNothing sensitive logged, secrets from the environment, a human before anything irreversible, a tested restore, a person who reads code.
Reasoning — summarises this page’s three tiers and its box on how to use them. Page checked 25 Aug 2026.

Tier 1 · Before it goes anywhere public

  • Search the whole repository for secrets. API keys, tokens, passwords, connection strings. Generated code puts them inline because examples do.
  • Check what is in version control. A key removed from the file is still in the history.
  • Read every dependency in the manifest. Does each one exist, is it maintained, is the name what you expected? Models invent package names, and people register the ones they invent.
  • Confirm debug mode is off and stack traces are not returned to users.
  • Check CORS is not wide open. A generated example almost always uses a permissive setting.

Tier 2 · Before it touches anyone else's data

  • Every input is validated server-side. Client-side validation is a convenience, not a control.
  • Every database query is parameterised. Never build SQL by joining strings.
  • Output is escaped where it is rendered. Not where it is stored.
  • Authentication is checked on every protected route — not on the page that links to it.
  • Authorisation is separate from authentication. Logged in is not the same as allowed.
  • Rate limits exist on anything that sends, writes or costs money.
  • Error messages say less than you think they need to. "Invalid credentials", not "no user with that email".

Tier 3 · Before money, credentials or personal records

  • Nothing sensitive is logged. Verbose logging is a data exposure that outlives the system.
  • Secrets come from the environment, and the environment is not in the repository.
  • Anything irreversible has a human in front of it — see permission tiers.
  • Test the restore, not the backup. An untested backup is a belief.
  • Have someone who reads code read it. At this tier that is not optional, and no amount of prompting substitutes.

An untested backup is a belief, not a backup.

The AI-specific ones

Fetched content is data, never instruction

If your system reads pages, files, emails or comments and passes them to a model, that text can contain instructions and the model cannot reliably tell the difference. A related, measured risk is poisoned memory rather than a single fetched page: one 2026 study found poisoning an agent's persistent state raised attack success from 24.6% to between 64% and 74%.Wang et al., "Your Agent, Their Asset", arXiv 2604.04759, read at source 9 Sep 2026: “poisoning any single CIK dimension increases the average attack success rate from 24.6% to 64-74%”

Read every skill and tool you install

An audit of 3,984 agent skills found 13.4% carried a critical-severity issue and 76 carried confirmed malicious payloads.Snyk, "ToxicSkills", 5 Feb 2026, read at source 11 Sep 2026 and quotes re-read 17 Sep 2026: “13.4% of all skills, or 534 in total, all contain at least one critical-level security issue” across 3,984 skills scanned from ClawHub and skills.sh; “over a third of the ecosystem is affected: 36.82% (1,467 skills) have at least one security flaw”, and “our human-in-the-loop process confirming that 76 of Agent Skills contained malicious payloads”. Detail on skills are a supply chain.

TOXICSKILLS · AGENT SKILLS SCANNED
Snyk’s audit of skills from ClawHub and skills.sh, published 5 Feb 2026.
Skills scanned3,984
At least one security flaw1,467
At least one critical-level issue534
Confirmed malicious payloads76
Snyk, "ToxicSkills", 5 Feb 2026, read at source 11 Sep 2026 and quotes re-read 17 Sep 2026: “13.4% of all skills, or 534 in total, all contain at least one critical-level security issue” across 3,984 skills scanned from ClawHub and skills.sh; “over a third of the ecosystem is affected: 36.82% (1,467 skills) have at least one security flaw”, and “our human-in-the-loop process confirming that 76 of Agent Skills contained malicious payloads”.

Assume the model output is hostile input

If a model's output reaches a shell, a query, a file path or a browser, treat it exactly as you would treat text from a stranger — because functionally it is.

Tools that automate part of this

Some of the checklist can be run rather than read. None of it replaces the tier-three requirement that someone who reads code reads it.

Strix — open-source AI pentesting

Describes itself as autonomous AI agents that find and fix application vulnerabilities. Apache 2.0 — verified against the repository and the project's own site, 22 Aug 2026.github.com/usestrix/strix · strix.ai/open-source-pentesting, checked 22 Aug 2026

The star count is a moving target and worth showing rather than picking one. Reported at 28.9k on 1 July, 32.8k on 3 July, 36k on 5 July, 43k in early August, and above 50k by late August — an average of roughly 2,500 a week over those two months.Range across AppSecSanta, CoddyKit, Rohit Raj, Trendshift and the project site, Jun–Aug 2026; no links to those write-ups were kept. On 23 Sep 2026 GitHub’s API for usestrix/strix returned “stargazers_count: 64221”. Any single figure here would be wrong within a fortnight.

Worth knowing before you run it: one published account pointed it at a live site, found nothing, spent roughly $17 in tokens in ten minutes, and had their API key auto-disabled for the usage spike.Not read at source on re-check: Protego, “Strix AI Pentester: An Honest Review”, Jul 2026. The site served a bot check on 23 Sep 2026, so the account is reported as first read, not re-verified. That is a token economics problem as much as a security one.

What it is good for: catching the common, well-documented classes — the tier-one and tier-two items on this page — faster and more consistently than a manual pass.

What it does not do: understand your business logic. An authorisation flaw where the wrong user can see the right-shaped data is invisible to a scanner and obvious to a person who knows the domain.

And the recursive caution: a security tool driven by an agent is still an agent. It reads your codebase, which is exactly the read-scope question raised in permission tiers. Run it on a copy first.

What this is not

This is not a penetration test and not legal advice. It catches the common, high-impact issues behind most real incidents in software assembled quickly.

If you are handling payments, health data or anything regulated, this checklist is where you start, not where you stop.

◈ IF YOU ARE CITING THIS

Cite the original source, not this page. Every figure here names the organisation that issued it and the date it was published.

Or check it yourself. How to check the figures here names the feed or document behind each recurring source.

ABOUTMETHODVERIFYPRIVACYCONTACTINDEXAI PROMPT GENEER · EVERY ARTICLE CARRIES ITS OWN CHECKED DATE