Checking AI output is necessary, but checking all of it at the same depth can cost more than it saves: check fully where mistakes are expensive and sample the rest.
- Compare checking time with doing time. If proper verification takes longer than doing the task yourself, automation has moved the time rather than saved it, and added a mistake you might miss.
- Constant checkpoints stop being read. An agent that asks approval for every step trains people to approve without looking; one checkpoint you read beats five you do not.
- Uniform checking is unsorted, not thorough. It gives routine formatting the same attention as an irreversible send.
- Check the high-risk parts fully. Anything irreversible or public, specific names, dates, figures and citations, and anything executable, which you check by running it.
- Over-checking can end with a working tool dropped. People decide the tool does not save time when the cost was the verification design, not the model.
Checking AI output is necessary. Checking all of it, all the time, at the same depth, is not.
It costs more than the work saved, it exhausts the attention you need for the parts that matter, and it makes people rubber-stamp.
The arithmetic people skip
Take an illustrative task. It takes you 30 minutes. The model does it in 2. Verifying properly takes 25.Reasoning, September 2026 — the minutes are an illustration, not a measurement: 30 − (2 + 25) = 3.
You saved three minutes and took on the risk of a mistake you might miss. That is not a win, and it is easy to miss in the tasks people are proudest of automating.
If verification costs more than doing it, you have not saved time. You have relocated it and added a failure mode.
The honest test, before automating anything: how long does checking take, and is that number smaller than doing it? If you cannot answer, that is the finding.
Review theatre
The specific failure of over-auditing is a checkpoint that fires so often it stops being read.
An agent that asks permission for every step soon trains you to click approve without looking; Anthropic reports the pattern in its own coding tool. The checkpoint is still in the process. It is no longer in your attention, which is the only place it was ever doing anything.Anthropic, How we built Claude Code auto mode: a safer way to skip permissions, 25 Mar 2026, read at source 23 Sep 2026: “Claude Code users approve 93% of permission prompts.” and “Over time that leads to approval fatigue, where people stop paying close attention to what they're approving.” One vendor describing its own product; the approval share is Anthropic’s figure, not an independent measurement.
Which is why automations puts one checkpoint at the last reversible moment rather than five throughout: one checkpoint you read beats five you do not.
The cost of checking everything equally
Attention is finite. Spending it uniformly means the tier-4 formatting gets the same scrutiny as the tier-1 irreversible send.
Uniform checking is not thorough. It is unsorted — and unsorted attention tends to under-serve the parts that matter, because those parts are rarer. That is reasoning, not a measured effect.
The tiers are on auditing AI-generated work. The point of them is not to check more. It is to check less, in the right places.
The failure that gets blamed on the tool
Over-auditing has an outcome that is easy to blame on something else: people can conclude the tool does not work and stop using it.Reasoning, September 2026 — this site’s argument about how abandonment happens; no study measuring how often it happens is cited here.
They were not wrong that it was not saving time. They were wrong about why. The verification design was the cost, not the model — and because the abandonment looks like a sensible judgement, it is easy never to revisit it.
Where to actually spend the attention
- Everything irreversible or public. Fully, every time, no sampling.
- Every specific claim — names, dates, figures, citations. Specific details are where models tend to invent — see why AI makes things up.Kalai, Nachum, Vempala & Zhang, Why Language Models Hallucinate, 4 Sep 2025, §1.1, read at source 23 Sep 2026: the hallucination rate “after pretraining, should be at least the fraction of training facts that appear once.” A formal lower bound, not a measurement.
- Anything executable — but by running it, not by reading it.
- Nothing else, at that depth. Sample the rest and accept that some formatting errors will ship.
And one habit that replaces a great deal of checking: require evidence rather than assertion. A diff, a list, a count. An artefact you can scan in five seconds does more than a review you will not do.
Under-checking ships errors you did not catch. Over-checking ships errors you stopped being able to catch, because the attention was already spent.
Both look like diligence from the inside. Only one of them is.
Quick answers
Can you check AI output too much?
Yes. If verification costs more than doing the task yourself, you have not saved time: you have relocated it and added a failure mode. Checking AI output is necessary; checking all of it, all the time, at the same depth, is not.
What is review theatre?
A checkpoint that fires so often it stops being read. An agent that asks permission for every step soon trains you to click approve without looking; Anthropic reports the pattern in its own coding tool. The checkpoint is still in the process, but no longer in your attention.
How do I know if I am over-auditing?
Estimate how long checking properly takes and compare it with doing the task yourself. If checking is not clearly less, the automation is not saving time, and if you cannot answer, that is the finding. If you are clicking approve without looking, the checkpoint has already stopped doing anything.
What should always be checked in full?
Everything irreversible or public, fully, every time. Every specific claim (names, dates, figures, citations), because specific details are where models tend to invent. Anything executable, by running it rather than reading it. Sample the rest.