ONLINEAGENT_OPS 2026.Q3 HOME HUBSMODELSBUILDINGCOMPANIESFILMMAKINGSECURITYLIVING WITH ITSKILLSALL HUBS, ONE TERMINAL → ARTICLES CRAFT RECORD BLOG TOOLS SKILLS SEARCH /
HOME›ARTICLES›DO AI DETECTORS WORK? WHAT THE EVIDENCE SAYS
ARTICLES · EXPLAINERBEGINNER

Do AI Detectors Work? What the Evidence Says

Do AI detectors work? Not well enough to prove anything. What they measure, who they wrongly flag, what Turnitin and OpenAI say, and what to do if accused.

READ9 min
WORDS1,810
SECTIONS7
TYPEEXPLAINER
CHECKED5 OCT 26

AI detectors do not work well enough to prove that someone used AI. They give a guess based on how predictable the writing is, they wrongly flag real human writing, and simple edits fool them. The companies that make them say their scores should not be the only evidence. If you are holding a detector score, about yourself or someone else, treat it as a reason to ask questions, not as an answer.

TL;DR — THE SHORT VERSION
  • They measure predictability, not authorship. Plain, regular writing looks “AI-like” to them, whoever wrote it.
  • They wrongly flag people. In a peer-reviewed study, seven detectors marked more than half of essays by non-native English writers as AI-written.
  • The makers say so. Turnitin says its score “should not be used as the sole basis for adverse actions against a student”. OpenAI withdrew its own detector in 2023 for low accuracy.
  • Image and video detectors struggle too, especially on real-world fakes. Labels added when content is made, like Content Credentials, are a different and more useful signal.
  • If you are accused: ask for the evidence and the policy, and show your process: drafts, version history, notes and sources.

What does an AI detector actually measure?

The plain point: mostly how predictable the words are. Text that an AI model finds easy to predict gets marked as more likely AI-written.

Researchers at Stanford describe the main measure, called perplexity, as “how ‘surprised’ or ‘confused’ a generative language model is when trying to guess the next word in a sentence”. Low surprise means low perplexity, and they note that “Most GPT detectors use text perplexity to detect AI-generated text”.1

Some tools add a second measure, burstiness: how much that predictability varies across a document. The detector GPTZero defines it as “a measure of how much writing patterns and text perplexities vary over the entire document”, and says it has since moved to a deep-learning system in which these are no longer the whole test.2

◈ IN PLAIN TERMS

Imagine guessing the next word of a sentence. “I drank a cup of…” is easy: tea or coffee. A detector asks an AI to play that game on your text. If the AI guesses your words easily all the way through, the detector leans towards “AI-written”. The trouble is that careful, simple, rule-following human writing is easy to guess too.

TAKEAWAY

A detector score is a judgement about style, not a record of who typed the words.

How accurate are AI detectors?

The plain point: far less accurate than their marketing, and easy to fool.

A large public test, a benchmark called RAID, ran 12 detectors against more than 6 million pieces of text. Its authors note that many detectors “claim to detect machine-generated text with extremely high accuracy (99% or more)”, but found “current detectors are easily fooled by adversarial attacks, variations in sampling strategies, repetition penalties, and unseen generative models”.3 In plain words: a different AI model, a different setting, or a quick rewording can break them.

OpenAI tried and gave up. In January 2023 OpenAI released its own classifier. On its own test it caught 26% of AI-written text and wrongly labelled human writing as AI 9% of the time. In July 2023 it switched it off: “As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy.”4

TAKEAWAY

If the company behind ChatGPT could not build a detector it trusted, treat any detector’s confidence with care.

Who gets wrongly flagged?

The plain point: people whose writing is plain and regular, which often means people writing in a second language.

The Stanford team ran seven detectors on 91 essays written by non-native English speakers for the TOEFL English test. The detectors “incorrectly labeled more than half of the TOEFL essays as ‘AI-generated’ (average false-positive rate: 61.3%)”. All seven agreed on 19.8% of them, and at least one detector flagged 97.8%. The same detectors did well on essays by US eighth-graders.5 A false positive is a wrong alarm: human writing marked as AI.

The authors’ conclusion is direct: “we strongly caution against the use of GPT detectors in evaluative or educational settings, particularly when assessing the work of non-native English speakers.”6

OpenAI found the same problem with its own attempt. It says its detector “labeled human-written text like Shakespeare and the Declaration of Independence as AI-generated”, and that it could affect students learning English and students whose writing was “particularly formulaic or concise”.7

TAKEAWAY

A high score on careful, simple writing is exactly the case where detectors are known to be wrong.

What do the detector makers themselves say?

The plain point: do not punish anyone on the score alone.

Turnitin, a detector built into many school systems, says: “Our AI writing detection model may not always be accurate (it may misidentify human-written, AI-generated, and AI-paraphrased text), so it should not be used as the sole basis for adverse actions against a student.” It hides scores between 1% and 19%, because its testing “found that there is a higher incidence of false positives” in that range, and it needs at least 300 words of prose in a supported language to give a score at all.8

OpenAI, asked “Do AI detectors work?”, answers: “In short, not in our experience.”9

At least one university switched detection off. Vanderbilt disabled Turnitin’s AI detector in August 2023, working out that at a claimed 1% false-positive rate, “around 750 student papers could have been incorrectly labeled” from its 75,000 papers in 2022.10 At the scale of a whole university, even a small error rate means hundreds of wrong labels.

What you seeWhat it meansWhat it does not mean
A percentageMeans: the share of the text the tool judged likely AI-written (Turnitin’s meaning).Not: how sure the tool is, or proof of anything.
A low score with an asteriskMeans: Turnitin judged the result too unreliable to show.Not: a small amount of cheating.
“Highly likely AI”Means: the text is very predictable to the tool.Not: that a person did not write it.

Turnitin Guides, Using the AI Writing Report, read at source 5 Oct 2026 in a browser: the overall percentage “indicates the qualifying text (prose sentences contained in a long-form writing format) within the submitted document that Turnitin’s AI writing detection model determines could be generated by AI”. The third row is this site’s reading of the perplexity research cited above.

What about AI image and video detectors?

The plain point: they have the same weakness. They do well on the fakes they were trained on and much worse on new ones found in the wild.

A 2025 study collected deepfakes that actually circulated online in 2024 and tested open-source detectors on them. Their scores on a standard accuracy measure fell sharply compared with older tests: “AUC decreasing by 50% for video, 48% for audio, and 45% for image models compared to previous benchmarks”. Commercial tools did better but “do not yet reach the accuracy of deepfake forensic analysts”.11 AUC is a common score for how well a detector separates real from fake. A deepfake is AI-made media of a real person.

Labels added at creation are a different idea. Instead of guessing afterwards, the maker marks the file. Content Credentials attach a signed record of where a file came from, but that record “can be separated” from the file.12 A watermark hides a signal in the content itself, yet the US standards body NIST found “most current watermarking schemes have been consistently found vulnerable to removal”.13 So a label present is useful evidence about where a file came from; a label missing proves nothing.

How to spot fake images, video and voices gives the checks that work without software, starting with where the file first appeared.

TAKEAWAY

For images and video, trust provenance over detectors: find the earliest copy, and look for a label added when it was made.

What to do if you are accused of using AI

The plain point: stay calm, ask what the evidence is, and show how you made the work.

  1. Ask for the evidence and the rule. Which tool, which score, and which policy says what? A score alone is what Turnitin itself says should not decide the matter.
  2. Show your process. Drafts, notes, sources you read, and the file’s edit history. In Google Docs, the version history shows earlier versions and who changed what: Google’s help page says “You can find who updated the file and their changes.”14 Word and other editors have similar features.
  3. Offer to talk it through. Someone who wrote a piece can explain why they made each choice. OpenAI’s guidance for teachers points to the same idea: asking students to share their AI conversations and show their work.15
  4. Bring the evidence about detectors. The Turnitin, OpenAI and Stanford quotes on this page are short enough to print and take with you.
  5. If you did use AI, say how, and check what your school’s or employer’s rules allow. Rules differ from place to place.

Reasoning, 5 Oct 2026 — the order of these steps is this site’s advice, built from the sources cited in them; it is not legal advice.

If you are the one holding the score: do the same in reverse. Ask about process, not the number. How to spot AI writing lists the signals worth a closer look, and why none of them is proof. Students can see AI for students for using AI within the rules.

TAKEAWAY

Process is the fairest evidence on both sides. Keep your drafts; ask for theirs.

Questions people ask

Can an AI detector prove I used AI?

No. Turnitin says its model “may not always be accurate” and should not be the sole basis for action, and OpenAI withdrew its own classifier for low accuracy.16

Why did a detector flag my own writing?

Probably because it was predictable to the tool: clear, simple and regular. The Stanford study found detectors flag that kind of writing, common among non-native English writers, far more often.5

What does a Turnitin AI percentage mean?

It is the share of qualifying sentences the model judged could be AI-generated, not how confident it is. Scores from 1% to 19% are hidden behind an asterisk because they are less reliable.17

Can teachers tell if you used ChatGPT?

Not reliably from the text alone, and asking ChatGPT does not help: OpenAI says its answers to “did you write this?” are “random and have no basis in fact”.18 Teachers who know a student’s work, and ask about drafts and choices, have better evidence than any score.

Do watermarks solve the problem?

Not yet. They only exist where the AI maker adds them, and NIST reports most can be removed.19 A watermark found is a strong clue; none found tells you nothing. See watermarking.

SOURCES
  1. 1Liang, Yuksekgonul, Mao, Wu & Zou, GPT detectors are biased against non-native English writers, Patterns 4(7), July 2023, read at source 5 Oct 2026“Most GPT detectors use text perplexity to detect AI-generated text, which might inadvertently penalize non-native writers who use a more limited range of linguistic expressions.” ↑
  2. 2GPTZero, What is perplexity & burstiness for AI detection?, 1 Mar 2023 with a later update, read at source 5 Oct 2026“As of autumn 2023, GPTZero no longer uses perplexity and burstiness for its AI detection because we migrated to a deep-learning based architecture.” ↑
  3. 3Dugan et al., RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors, ACL 2024, read at source 5 Oct 2026“current detectors are easily fooled by adversarial attacks, variations in sampling strategies, repetition penalties, and unseen generative models”. ↑
  4. 4OpenAI, New AI classifier for indicating AI-written text, 31 Jan 2023, with update of 20 Jul 2023, read at source 5 Oct 2026“our classifier correctly identifies 26% of AI-written text (true positives) as ‘likely AI-written,’ while incorrectly labeling human-written text as AI-written 9% of the time (false positives).” ↑
  5. 5Liang et al., Patterns 4(7), July 2023, read at source 5 Oct 2026“All detectors unanimously identified 19.8% of the human-written TOEFL essays as AI authored, and at least one detector flagged 97.8% of TOEFL essays as AI generated.” ↑a“which might inadvertently penalize non-native writers who use a more limited range of linguistic expressions”. ↑b
  6. 6Liang et al., Patterns 4(7), July 2023, read at source 5 Oct 2026, recommendations section.↑
  7. 7OpenAI Help Center, How can educators respond to students presenting AI-generated content as their own?, “Updated: last month”, re-read directly at source 13 Oct 2026“When we at OpenAI tried to train an AI-generated content detector, we found that it labeled human-written text like Shakespeare and the Declaration of Independence as AI-generated.” ↑
  8. 8Turnitin Guides, Using the AI Writing Report, re-read directly at source 13 Oct 2026“Our testing has found that there is a higher incidence of false positives when the percentage is between 0 and 19.” ↑
  9. 9OpenAI Help Center, How can educators respond to students presenting AI-generated content as their own?, read at source 5 Oct 2026 in a browser“Our research into detectors didn’t show them to be reliable enough given that educators could be making judgments about students with potentially lasting consequences.” ↑
  10. 10Vanderbilt University, Guidance on AI Detection and Why We’re Disabling Turnitin’s AI Detector, 16 Aug 2023, read at source 5 Oct 2026“Vanderbilt has decided to disable Turnitin’s AI detection tool for the foreseeable future”. ↑
  11. 11Chandra et al., Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024, arXiv, 4 Mar 2025, read at source 5 Oct 2026“AUC decreasing by 50% for video, 48% for audio, and 45% for image models compared to previous benchmarks.” ↑
  12. 12C2PA, FAQs, read at source 1 Oct 2026 (as cited on this site’s Content Credentials page): “While C2PA Manifests are typically embedded in the asset, they can be separated.”↑
  13. 13NIST, NIST AI 100-4: Reducing Risks Posed by Synthetic Content, November 2024, read at source 1 Oct 2026 (as cited on this site’s watermarking page).↑
  14. 14Google Docs Editors Help, Find what’s changed in a file, read at source 5 Oct 2026“Choose the latest version. You can find who updated the file and their changes.” ↑
  15. 15OpenAI Help Center, How can educators respond…, read at source 5 Oct 2026 in a browser“One technique some teachers have found useful is encouraging students to share specific conversations from ChatGPT”. ↑
  16. 16Turnitin Guides, Using the AI Writing Report, and OpenAI, New AI classifier, both read at source 5 Oct 2026“the AI classifier is no longer available due to its low rate of accuracy”. ↑
  17. 17Turnitin Guides, Using the AI Writing Report, read at source 5 Oct 2026 in a browser“the AI indicator will display an asterisk (*) for percentages between 0 and 20 to call attention to the fact that the score is less reliable.” ↑
  18. 18OpenAI Help Center, How can educators respond…, read at source 5 Oct 2026 in a browser.↑
  19. 19NIST, NIST AI 100-4, November 2024, read at source 1 Oct 2026 (as cited on this site’s watermarking page): “most current watermarking schemes have been consistently found vulnerable to removal”.↑

Liang et al., GPT detectors are biased against non-native English writers, Patterns, July 2023 · Dugan et al., RAID, ACL 2024 · GPTZero, perplexity and burstiness · OpenAI, New AI classifier, 31 Jan 2023 (withdrawn 20 Jul 2023) · OpenAI Help Center, How can educators respond… · Turnitin Guides, Using the AI Writing Report · Vanderbilt University, Guidance on AI Detection, 16 Aug 2023 · Chandra et al., Deepfake-Eval-2024, 2025 · Google, Find what’s changed in a file. All read at source 5 Oct 2026.

C2PA, FAQs, and NIST, NIST AI 100-4, both read 1 Oct 2026, as recorded on this site’s Content Credentials and watermarking pages.

The comparison on this page (guessing the next word of a sentence) is this site’s own way of explaining, not a claim from these sources.

Free printable AI cheat sheets, and AI news every week.

Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.

SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK
aipromptgeneer://$ about this siteAI PROMPT GENEERAI explained in plain English. Every claim links to its source, with the date it was checked.NEWSLETTERWhat changed in AI each week, in plain English.SUBSCRIBE FREE ↗AI PROMPT GENEER · MACHINE ASSISTANCE DISCLOSEDEVERY ARTICLE CARRIES ITS OWN CHECKED DATE