ONLINEAGENT_OPS 2026.Q3 HOME HUBSMODELSBUILDINGCOMPANIESFILMMAKINGSECURITYLIVING WITH ITSKILLSALL HUBS, ONE TERMINAL → ARTICLES CRAFT RECORD BLOG TOOLS SKILLS SEARCH /
HOME›RECORD›HOW TO CHECK A CITATION AN AI GAVE YOU
RECORD · GUIDEBEGINNER

How to Check a Citation an AI Gave You

AI tools can invent sources that look real. The one-minute check, what a court did when lawyers skipped it, and what a study of 636 citations found.

READ8 min
WORDS1,580
SECTIONS7
TYPEGUIDE
CHECKED17 SEP 26

An AI can produce a citation that looks perfect and points to nothing. Author, title, journal, year, page numbers — all in the right format, for a paper nobody wrote. It is also one of the AI failures you can check for yourself in about a minute. Here is the check, and what happened to people who skipped it.

TL;DR — THE SHORT VERSION
  • A well-formatted citation is not evidence the source exists. Format is the easy part for a language model.
  • Open every source before you rely on it. Not a sample. Every one.
  • If it exists, check it says what you were told it says. Real sources get misdescribed too.
  • If your search comes back empty, believe the search. That is the lesson of a well-known court case.
  • The tool is not the problem. The court said so. Skipping the check is.

The court case worth knowing

In 2023 a federal judge in New York sanctioned two lawyers and their firm. The order says they “submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question.” The court imposed “a penalty of $5,000” and required letters to the judges whose names had been attached to the invented opinions.1

The detail worth remembering is not the invented cases. It is that a check was tried. At the hearing, one lawyer said he had entered the citation for one of the cases into a lookup site. The judge asked what he found. The answer, from the transcript quoted in the order: “I couldn’t find it.” The case went into the brief anyway.2

The same order is clear that using the tool was not the offence: “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.”

TAKEAWAY

The failure was not using an AI. It was not believing the search that came back empty.

It is not a one-off

A researcher keeps a public database of court decisions where generative AI produced false material — “typically fake citations”, in the database’s own words. At its update of 14 September 2026 it listed “2041 cases identified so far”. It is careful about its own limits: it “does not track the (necessarily wider) universe of all fake citations or use of AI in court filings”, and it also covers some decisions “where AI use was alleged but not confirmed.” The count rises with each update; re-check it rather than repeating this one.3

Outside courts, a 2023 study in Scientific Reports had ChatGPT write short literature reviews on 42 topics, then checked all 636 citations against databases and websites. Its headline: “55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated.” Book chapters were worst: “Even with GPT-4, however, 70% of the cited book chapters are fabricated.”4

CITATIONS THAT DID NOT EXIST · 2023 STUDY
Share of 636 citations in ChatGPT-written literature reviews that were fabricated, by model. Both models are older than today’s tools.
GPT-3.555%
GPT-418%
Walters & Wilder, Fabrication and errors in the bibliographic citations generated by ChatGPT, Scientific Reports, 2023, read at source 16 Sep 2026: “55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated.”

GPT-3.5 and GPT-4 are older models, and today’s tools may do better or worse; this site has not found a comparable study of current models. The study is here for the shape of the problem, not today’s rate. And it found a second problem that a quick existence check misses: “43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors.”5

TAKEAWAY

Two separate failures: sources that do not exist, and real sources described wrongly. The check has to catch both.

The one-minute check

  1. Search the exact title in quotation marks. A real paper, book or ruling usually leaves a trace: the publisher, a library catalogue, a court database, another paper citing it.
  2. If there is a DOI, resolve it. A DOI is an identifier that looks like 10.1000/182. The DOI Foundation explains that one “is resolvable using our proxy server by displaying it as a link: https://doi.org/10.1000/182.” Put https://doi.org/ in front and open it. No result, or a different paper, is a red flag. A generated citation can carry a real identifier that belongs to some other work, so check that the title matches too.6
  3. Check the author. Does this person exist, and do they work in this field?
  4. If it exists, open it and find the claim. Search inside the document for the figure or phrase you were given. A real source cited for something it never says is the harder failure to spot, because the first three checks all pass.
  5. If you cannot find it, it does not go in. Not “probably fine”, not “I’ll check later”.

Why it happens

Stated as reasoning: a language model produces text that fits the pattern of what usually comes next. The pattern of a citation — surname, initial, year, title, journal, volume, pages — is extremely regular and easy to reproduce. Whether that particular paper exists is a separate fact the pattern does not contain. So a model can be fluent at the form and wrong about the fact at the same time. The longer version is on why AI makes things up.

Tools that search the web as they answer can link to real pages, which helps with the first failure. They can still describe those pages wrongly, which is the second. Step 4 exists for that.

TWO FAILURES · WHICH CHECK CATCHES WHICH
THE SOURCE DOES NOT EXIST
Yes: A title search catches it
Yes: Resolving the DOI catches it
Yes: Checking the author often catches it
IT EXISTS, BUT SAYS SOMETHING ELSE
No: A title search misses it
No: Resolving the DOI misses it
Yes: Opening it and finding the claim catches it
Reasoning, September 2026 — summarises the five checks on this page.

For figures inside a source, see how to read a number before you repeat it. For a claim that reached you as an image, see how to check a viral screenshot. On who carries responsibility for AI-assisted work, see AI at work. The wider habit is on critical thinking.

What this page could not verify

  • How often current AI tools invent citations. The study quoted here tested older models; this site has not found a comparable study of today’s.
  • The full case database. Its count was read from the database’s own page; the individual cases were not checked here.
  • Two other copies of the court order refused automated access; the order was read on FindLaw.

Questions people ask

Does ChatGPT make up fake citations?

It can. A 2023 study in Scientific Reports found that 55% of GPT-3.5 citations and 18% of GPT-4 citations were fabricated.4 Those are older models, so the study shows the shape of the problem rather than today’s rate.

How do I check if a source from AI is real?

Search the exact title in quotation marks, resolve the DOI if there is one, and check the author exists. If the source exists, open it and find the claim you were given. The one-minute check sets out each step.

What happened to the lawyers who used ChatGPT citations?

In 2023 a federal judge in New York sanctioned two lawyers and their firm for submitting non-existent opinions created by ChatGPT. The court imposed a penalty and required letters to the judges whose names were attached to the invented opinions.1

How do I look up a DOI?

Put https://doi.org/ in front of it and open the link.6 No result, or a different paper, is a red flag, and check that the title matches too.

QUICK CHECK · 3 QUESTIONS

Pick an answer to see if you got it. Nothing is recorded.

1. What does the page say a neatly formatted citation tells you?

  • That the source exists, because every detail is filled in
  • Nothing about whether the source exists, because format is the easy part for an AI
  • That the source has been checked by experts
Show the answer

Answer: Nothing about whether the source exists, because format is the easy part for an AI. The page explains that a model can be fluent at the pattern of a citation and wrong about whether the paper exists.

2. What lesson does the page draw from the court case it describes?

  • Lawyers should not use AI tools at all
  • Check a sample of the citations and trust the rest
  • If your search for a source comes back empty, believe it
Show the answer

Answer: If your search for a source comes back empty, believe it. The page says a lawyer searched, could not find the case, and used it anyway; the court said using the tool itself was not the offence.

3. Why does the page say to open a source and find the claim, even once you know it exists?

  • Real sources can be described wrongly, and title or DOI checks miss that
  • To see how many times it has been cited
  • Because a DOI only works after the document is opened
Show the answer

Answer: Real sources can be described wrongly, and title or DOI checks miss that. The page names two failures, sources that do not exist and real sources described wrongly, and only reading the source catches the second.

Related on this site: Asking AI About Your Health · How to Cite ChatGPT and AI (APA, MLA, Chicago) · What Is Deep Research in AI?

SOURCES
  1. 1Mata v. Avianca, Inc., No. 22-cv-1461, S.D.N.Y., Opinion and Order on Sanctions, 22 June 2023, read at source on FindLaw 16 Sep 2026“A penalty of $5,000 is jointly and severally imposed on Respondents”. ↑a“A penalty of $5,000 is jointly and severally imposed on Respondents”. ↑b
  2. 2Mata v. Avianca, Inc., Opinion and Order on Sanctions, 22 June 2023, read at source 16 Sep 2026.↑
  3. 3Damien Charlotin, AI Hallucination Cases database, last updated 14 September 2026, read at source 16 Sep 2026.↑
  4. 4Walters & Wilder, Fabrication and errors in the bibliographic citations generated by ChatGPT, Scientific Reports, 2023, read at source 16 Sep 2026“55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated”. ↑a“55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated”. ↑b
  5. 5Walters & Wilder, Scientific Reports, 2023, read at source 16 Sep 2026“43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors”. ↑
  6. 6DOI Foundation, What is a DOI?, read at source 16 Sep 2026.↑a↑b

Mata v. Avianca, Inc., Opinion and Order on Sanctions (S.D.N.Y., 22 June 2023) · Damien Charlotin, AI Hallucination Cases database · Walters & Wilder, Fabrication and errors in the bibliographic citations generated by ChatGPT, Scientific Reports (2023) · DOI Foundation, What is a DOI?. All read at source on 16 September 2026.

Free printable AI cheat sheets, and AI news every week.

Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.

SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK
aipromptgeneer://$ about this siteAI PROMPT GENEERAI explained in plain English. Every claim links to its source, with the date it was checked.NEWSLETTERWhat changed in AI each week, in plain English.SUBSCRIBE FREE ↗AI PROMPT GENEER · MACHINE ASSISTANCE DISCLOSEDEVERY ARTICLE CARRIES ITS OWN CHECKED DATE