An AI can produce a citation that looks perfect and points to nothing. Author, title, journal, year, page numbers — all in the right format, for a paper nobody wrote. It is also one of the AI failures you can check for yourself in about a minute. Here is the check, and what happened to people who skipped it.
- A well-formatted citation is not evidence the source exists. Format is the easy part for a language model.
- Open every source before you rely on it. Not a sample. Every one.
- If it exists, check it says what you were told it says. Real sources get misdescribed too.
- If your search comes back empty, believe the search. That is the lesson of a well-known court case.
- The tool is not the problem. The court said so. Skipping the check is.
The court case worth knowing
In 2023 a federal judge in New York sanctioned two lawyers and their firm. The order says they “submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question.” The court imposed “a penalty of $5,000” and required letters to the judges whose names had been attached to the invented opinions.Mata v. Avianca, Inc., No. 22-cv-1461, S.D.N.Y., Opinion and Order on Sanctions, 22 June 2023, read at source on FindLaw 16 Sep 2026: “A penalty of $5,000 is jointly and severally imposed on Respondents”.
The detail worth remembering is not the invented cases. It is that a check was tried. At the hearing, one lawyer said he had entered the citation for one of the cases into a lookup site. The judge asked what he found. The answer, from the transcript quoted in the order: “I couldn’t find it.” The case went into the brief anyway.Mata v. Avianca, Inc., Opinion and Order on Sanctions, 22 June 2023, read at source 16 Sep 2026.
The same order is clear that using the tool was not the offence: “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.”
The failure was not using an AI. It was not believing the search that came back empty.
It is not a one-off
A researcher keeps a public database of court decisions where generative AI produced false material — “typically fake citations”, in the database’s own words. At its update of 14 September 2026 it listed “2041 cases identified so far”. It is careful about its own limits: it “does not track the (necessarily wider) universe of all fake citations or use of AI in court filings”, and it also covers some decisions “where AI use was alleged but not confirmed.” The count rises with each update; re-check it rather than repeating this one.Damien Charlotin, AI Hallucination Cases database, last updated 14 September 2026, read at source 16 Sep 2026.
Outside courts, a 2023 study in Scientific Reports had ChatGPT write short literature reviews on 42 topics, then checked all 636 citations against databases and websites. Its headline: “55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated.” Book chapters were worst: “Even with GPT-4, however, 70% of the cited book chapters are fabricated.”Walters & Wilder, Fabrication and errors in the bibliographic citations generated by ChatGPT, Scientific Reports, 2023, read at source 16 Sep 2026: “55% of the GPT-3.5 citations but just 18% of the GPT-4 citations are fabricated”.
GPT-3.5 and GPT-4 are older models, and today’s tools may do better or worse; this site has not found a comparable study of current models. The study is here for the shape of the problem, not today’s rate. And it found a second problem that a quick existence check misses: “43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors.”Walters & Wilder, Scientific Reports, 2023, read at source 16 Sep 2026: “43% of the real (non-fabricated) GPT-3.5 citations but just 24% of the real GPT-4 citations include substantive citation errors”.
Two separate failures: sources that do not exist, and real sources described wrongly. The check has to catch both.
The one-minute check
- Search the exact title in quotation marks. A real paper, book or ruling usually leaves a trace: the publisher, a library catalogue, a court database, another paper citing it.
- If there is a DOI, resolve it. A DOI is an identifier that looks like
10.1000/182. The DOI Foundation explains that one “is resolvable using our proxy server by displaying it as a link: https://doi.org/10.1000/182.” Puthttps://doi.org/in front and open it. No result, or a different paper, is a red flag. A generated citation can carry a real identifier that belongs to some other work, so check that the title matches too.DOI Foundation, What is a DOI?, read at source 16 Sep 2026. - Check the author. Does this person exist, and do they work in this field?
- If it exists, open it and find the claim. Search inside the document for the figure or phrase you were given. A real source cited for something it never says is the harder failure to spot, because the first three checks all pass.
- If you cannot find it, it does not go in. Not “probably fine”, not “I’ll check later”.
Why it happens
Stated as reasoning: a language model produces text that fits the pattern of what usually comes next. The pattern of a citation — surname, initial, year, title, journal, volume, pages — is extremely regular and easy to reproduce. Whether that particular paper exists is a separate fact the pattern does not contain. So a model can be fluent at the form and wrong about the fact at the same time. The longer version is on why AI makes things up.
Tools that search the web as they answer can link to real pages, which helps with the first failure. They can still describe those pages wrongly, which is the second. Step 4 exists for that.
Related
For figures inside a source, see how to read a number before you repeat it. For a claim that reached you as an image, see how to check a viral screenshot. On who carries responsibility for AI-assisted work, see AI at work. The wider habit is on critical thinking.
What this page could not verify
- How often current AI tools invent citations. The study quoted here tested older models; this site has not found a comparable study of today’s.
- The full case database. Its count was read from the database’s own page; the individual cases were not checked here.
- Two other copies of the court order refused automated access; the order was read on FindLaw.
Mata v. Avianca, Inc., Opinion and Order on Sanctions (S.D.N.Y., 22 June 2023) · Damien Charlotin, AI Hallucination Cases database · Walters & Wilder, Fabrication and errors in the bibliographic citations generated by ChatGPT, Scientific Reports (2023) · DOI Foundation, What is a DOI?. All read at source on 16 September 2026.