Check software you did not fully write in order of what a mistake would cost, so stopping partway still covers the expensive part.
- Tier 1: before it goes public. Search for secrets, including in version history, check that each dependency is real, turn debug off and tighten CORS.
- Tier 2: before other people’s data. Validate on the server, parameterise queries, check authorisation as well as login, and rate-limit anything that costs.
- Tier 3: before money or credentials. Keep secrets out of logs and the repository, put a human in front of anything irreversible, test the restore, and have someone who reads code review it.
- Three AI-specific checks. Treat fetched content as data, read each skill or tool before installing it, and handle model output like input from a stranger.
- Scanners help but do not finish the job. Automated tools catch common flaws faster; they do not understand your business logic.
A checklist for software you did not fully write. Ordered by what it costs to get wrong, so you can stop partway and still have covered the expensive part.
Tier 1 before anything touches the internet. Tier 2 before it touches anyone else's data. Tier 3 before it touches money or credentials. Stopping early is fine. Skipping to tier 3 is not.
Tier 1 · Before it goes anywhere public
- Search the whole repository for secrets. API keys, tokens, passwords, connection strings. Generated code puts them inline because examples do.
- Check what is in version control. A key removed from the file is still in the history.
- Read every dependency in the manifest. Does each one exist, is it maintained, is the name what you expected? Models invent package names, and people register the ones they invent.
- Confirm debug mode is off and stack traces are not returned to users.
- Check CORS is not wide open. A generated example almost always uses a permissive setting.
Tier 2 · Before it touches anyone else's data
- Every input is validated server-side. Client-side validation is a convenience, not a control.
- Every database query is parameterised. Never build SQL by joining strings.
- Output is escaped where it is rendered. Not where it is stored.
- Authentication is checked on every protected route — not on the page that links to it.
- Authorisation is separate from authentication. Logged in is not the same as allowed.
- Rate limits exist on anything that sends, writes or costs money.
- Error messages say less than you think they need to. "Invalid credentials", not "no user with that email".
Tier 3 · Before money, credentials or personal records
- Nothing sensitive is logged. Verbose logging is a data exposure that outlives the system.
- Secrets come from the environment, and the environment is not in the repository.
- Anything irreversible has a human in front of it — see permission tiers.
- Test the restore, not the backup. An untested backup is a belief.
- Have someone who reads code read it. At this tier that is not optional, and no amount of prompting substitutes.
An untested backup is a belief, not a backup.
The AI-specific ones
Fetched content is data, never instruction
If your system reads pages, files, emails or comments and passes them to a model, that text can contain instructions and the model cannot reliably tell the difference. A related, measured risk is poisoned memory rather than a single fetched page: one 2026 study found poisoning an agent's persistent state raised attack success from 24.6% to between 64% and 74%.Wang et al., "Your Agent, Their Asset", arXiv 2604.04759, read at source 9 Sep 2026: “poisoning any single CIK dimension increases the average attack success rate from 24.6% to 64-74%”
Read every skill and tool you install
An audit of 3,984 agent skills found 13.4% carried a critical-severity issue and 76 carried confirmed malicious payloads.Snyk, "ToxicSkills", 5 Feb 2026, read at source 11 Sep 2026 and quotes re-read 17 Sep 2026: “13.4% of all skills, or 534 in total, all contain at least one critical-level security issue” across 3,984 skills scanned from ClawHub and skills.sh; “over a third of the ecosystem is affected: 36.82% (1,467 skills) have at least one security flaw”, and “our human-in-the-loop process confirming that 76 of Agent Skills contained malicious payloads”. Detail on skills are a supply chain.
Assume the model output is hostile input
If a model's output reaches a shell, a query, a file path or a browser, treat it exactly as you would treat text from a stranger — because functionally it is.
Tools that automate part of this
Some of the checklist can be run rather than read. None of it replaces the tier-three requirement that someone who reads code reads it.
Strix — open-source AI pentesting
Describes itself as autonomous AI agents that find and fix application vulnerabilities. Apache 2.0 — verified against the repository and the project's own site, 22 Aug 2026.github.com/usestrix/strix · strix.ai/open-source-pentesting, checked 22 Aug 2026
The star count is a moving target and worth showing rather than picking one. Reported at 28.9k on 1 July, 32.8k on 3 July, 36k on 5 July, 43k in early August, and above 50k by late August — an average of roughly 2,500 a week over those two months.Range across AppSecSanta, CoddyKit, Rohit Raj, Trendshift and the project site, Jun–Aug 2026; no links to those write-ups were kept. On 23 Sep 2026 GitHub’s API for usestrix/strix returned “stargazers_count: 64221”. Any single figure here would be wrong within a fortnight.
Worth knowing before you run it: one published account pointed it at a live site, found nothing, spent roughly $17 in tokens in ten minutes, and had their API key auto-disabled for the usage spike.Not read at source on re-check: Protego, “Strix AI Pentester: An Honest Review”, Jul 2026. The site served a bot check on 23 Sep 2026, so the account is reported as first read, not re-verified. That is a token economics problem as much as a security one.
What it is good for: catching the common, well-documented classes — the tier-one and tier-two items on this page — faster and more consistently than a manual pass.
What it does not do: understand your business logic. An authorisation flaw where the wrong user can see the right-shaped data is invisible to a scanner and obvious to a person who knows the domain.
And the recursive caution: a security tool driven by an agent is still an agent. It reads your codebase, which is exactly the read-scope question raised in permission tiers. Run it on a copy first.
What this is not
This is not a penetration test and not legal advice. It catches the common, high-impact issues behind most real incidents in software assembled quickly.
If you are handling payments, health data or anything regulated, this checklist is where you start, not where you stop.