ONLINEAGENT_OPS 2026.Q3 HOME HUBSMODELSBUILDINGCOMPANIESFILMMAKINGSECURITYLIVING WITH ITSKILLSALL HUBS, ONE TERMINAL → ARTICLES CRAFT RECORD BLOG TOOLS SKILLS SEARCH /
HOME›ARTICLES›WHY DOES AI HALLUCINATE? WHY AI MAKES THINGS UP
ARTICLES · EVERGREEN EXPLAINERBEGINNER

Why does AI hallucinate? Why AI makes things up, explained simply

AI hallucinates because it predicts the most likely-sounding words, and nothing in that process checks whether they are true. Its training and testing also reward a confident guess over saying “I don’t know”.

READ7 min
WORDS1,491
SECTIONS5
SOURCES7
TYPEEXPLAINER
CHECKED25 AUG 26

AI makes things up because the way it is trained and scored rewards guessing: under right-or-wrong scoring, a lucky guess earns a point and “I don’t know” earns none.1 The result is called a hallucination: a confident answer that is false or made up.

TL;DR — THE SHORT VERSION

It is not a glitch: making things up is a side effect of how these models are built and scored.

  • A hallucination is a confident answer that is false or made up, such as an invented book title or a wrong date.
  • Facts that appear only once in the training data — a specific person's birthday, an obscure dissertation title — cannot be reliably recalled.
  • A model has no way to signal the difference between recall and invention, because internally there is no difference.
◈ THE ONE-LINE EXPLANATION

Models are trained and then scored in a way that gives full credit for a lucky guess and zero credit for saying "I don't know". Under that scoring, guessing is always the better strategy. So they guess — fluently, confidently, and in the correct format.2

What is an AI hallucination?

The plain point: an AI hallucination is a confident answer that is false or made up. The AI is not lying on purpose; it is guessing, and the guess sounds just as sure as a real fact.

The US standards agency NIST calls it confabulation: confidently stated but false content, known colloquially as hallucinations.3 The researchers behind the paper this page follows describe it like a student facing a hard exam question: the model guesses rather than admit it is unsure.4

A real example: asked for one of the paper’s authors’ dissertation title, three different widely used chatbots each returned a different title, and none was correct. Asked the same author’s birthday, a single model returned three wrong dates across three attempts.5

For the short definition, see hallucination meaning. This page explains why it happens and how to avoid it.

TAKEAWAY

A hallucination is a fluent, confident guess that happens to be wrong. Specific details, such as titles, dates, names and quotes, are where it shows up most.

The arithmetic that causes it

Almost every benchmark used to rank AI models scores answers as right or wrong, with nothing in between. Consider a model that is 60% confident in an answer.Reasoning, September 2026 — a worked example of the right-or-wrong scoring the paper describes; the 60% is illustrative, not a measured figure.

Expected score by strategy
StrategyExpected points
Guess. Right 60% of the time, wrong 40%.0.6 points
Say "I don't know." Never wrong.0 points

That is the whole problem. Under binary scoring, an honest abstention scores worse than a coin-flip guess — so a system optimised against those scoreboards learns to answer everything. The researchers' comparison is exact: models behave like students who have worked out that leaving a question blank guarantees zero, while a plausible answer might score.

TAKEAWAY

Right-or-wrong scoring rewards guessing. A guess can earn points while "I don't know" always scores nothing, so models learn to answer everything.

Four mechanisms, stacked

Prediction has no concept of "unknown"

A model produces the most probable continuation of the text so far. There is no separate step where it checks whether it knows the answer — the machinery that generates a true fact and the machinery that generates a plausible-looking false one are the same machinery. The mechanism is explained further in the six layers.

Rare facts have a floor

Facts that appear only once in the training data — a specific person's birthday, an obscure dissertation title — cannot be reliably recalled. The research finds that the hallucination rate for such facts is bounded below by how common those one-off facts are in the corpus — the paper’s "singleton rate". It offers no estimate of how many real biographical facts are singletons; it illustrates the bound with a hypothetical figure.6

This is why models are strong on well-covered subjects and unreliable in exactly the specific, verifiable details people most want. The dissertation-title example at the top of this page is exactly this kind of fact.

Generating is harder than recognising

A model may be able to judge whether a statement is true more reliably than it can produce a true statement unprompted. The paper formalises this as a lower bound: the error rate when generating is at least roughly twice the error rate when classifying.7

The practical consequence is directly useful: asking a model to check a claim is more reliable than asking it to supply one.

Partial familiarity triggers confidence

When a prompt resembles something seen often in training, the model produces the shape of the expected answer with high confidence — correct format, plausible specifics, wrong content. This is why fabricated citations look like real citations, complete with volume numbers and page ranges.8

TAKEAWAY

Invention is built into prediction. Models have no concept of unknown, cannot reliably recall facts seen only once, find generating harder than checking, and answer confidently when a prompt feels familiar.

How to avoid AI hallucinations (as a user)

The plain point: you cannot switch hallucinations off, but you can make them rarer and easier to catch. Use this checklist, in no measured order:

  • Ask for sources, then open them. Ask it to back each claim with a source or a quote. Anthropic’s own advice to developers is to make answers checkable this way, and to drop any claim it cannot support.9 A made-up source looks real until you click it.
  • Turn on web search for anything current. With search on, the chatbot looks things up and links what it found, so you can check it. Anthropic says each search answer in Claude comes with citations.10 This is the idea behind RAG and grounding: answer from real documents, not from memory.
  • Ask it to say when it is unsure. Tell it plainly that “I don’t know” is an acceptable answer. Anthropic lists this as its first technique, saying it “can drastically reduce false information.”9 It partly counteracts the scoring incentive above, though it does not remove it.
  • Ask it to check, not to supply. Give it the claim and ask whether it holds — the paper’s lower bound says that is the easier task, though it does not measure by how much.
  • Give it the source. A model working from a document you provided has far less room to invent than one working from memory. Anthropic suggests telling it to use only the documents you gave it.9
  • Check names, dates and numbers yourself. Names, dates, figures, citations — errors concentrate on specific facts that appear rarely in training, and those are exactly the details people paste without checking.11

None of these makes an answer safe to trust blindly. Anthropic’s own guide ends on the same warning: the techniques reduce hallucinations but do not eliminate them.9 More on the word itself: hallucination meaning.

◈ WHAT WILL NOT FIX IT

Scale is not the lever this paper points at, and neither is retrieval: it notes that search and RAG reduce hallucinations but leave the incentive intact, because the grading still rewards a guess whenever retrieval comes up short.12 The paper's own proposal is to change how benchmarks award points — giving credit for calibrated uncertainty rather than punishing it.

Until the scoreboards change, the incentive to guess remains.

TAKEAWAY

Ask it to check, not supply. Also give it the source, ask it to flag low confidence, and verify names, dates, figures and citations yourself. Search and RAG reduce hallucinations but leave the incentive intact.

Questions people ask

Can AI hallucinations be fixed completely?

Not today. The techniques above make them rarer, but even the companies that recommend them say they do not remove them, and the researchers argue the scoring of AI tests has to change before the incentive to guess goes away.94

Does ChatGPT hallucinate?

Yes. ChatGPT, Claude, Gemini and other chatbots are all built on language models, and the researchers say hallucinations persist even in the most advanced ones. Treat any specific fact from any chatbot as a lead to check, not a finished answer.4

Why does AI make up fake sources and citations?

Because it has learned what a citation looks like, so it can produce the shape (authors, a title, a journal, page numbers) without a real paper behind it. The paper this page follows notes that these bluffs tend to be confident and specific rather than vague.13

Is an AI hallucination the same as lying?

No. Lying means knowing the truth and hiding it; a hallucinating model is guessing when it is unsure instead of saying so. The effect on you can be the same, which is why checking matters.4

Related on this site: Using AI to Budget · Asking AI About Your Health · How to Make a Meal Plan With AI · How to Plan a Trip With AI · Is AI Dangerous? · Is ChatGPT Safe to Use? · Knowledge cutoff meaning

SOURCES
  1. 1Kalai, Nachum, Vempala & Zhang, "Why Language Models Hallucinate", 4 September 2025 — arXiv:2509.04664. Read at source 11 Sep 2026; paper body, §1.2: "guessing when unsure maximizes the expected score under a binary 0-1 scheme that awards 1 point for a correct answer and none for blanks or IDKs."↑
  2. 2Kalai, Nachum, Vempala & Zhang, "Why Language Models Hallucinate", 4 September 2025 — arXiv:2509.04664. Read at source 11 Sep 2026; paper body, §1.2: "guessing when unsure maximizes the expected score under a binary 0-1 scheme that awards 1 point for a correct answer and none for blanks or IDKs."↑
  3. 3NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1), July 2024, as cited on this site’s hallucination page, read at source 1 Oct 2026“The production of confidently stated but erroneous or false content (known colloquially as ‘hallucinations’ or ‘fabrications’)”. ↑
  4. 4Kalai, Nachum, Vempala & Zhang, Why Language Models Hallucinate, arXiv:2509.04664, 4 Sep 2025, abstract, read at source 5 Oct 2026“Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such ‘hallucinations’ persist even in state-of-the-art systems and undermine trust.” ↑athe problem “can only be addressed through a socio-technical mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards”. ↑b“Such ‘hallucinations’ persist even in state-of-the-art systems and undermine trust.” ↑c“large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty.” ↑d
  5. 5Kalai et al., 4 September 2025, arXiv:2509.04664, §1 and Table 1. Read at source 11 Sep 2026. Corrected 11 Sep 2026 — this page previously attributed both failures to one chatbot across attempts. The dissertation answers came from three different models: “Excerpts from responses to ‘What was the title of Adam Kalai’s dissertation?’ from three popular language models… None generated the correct title or year”. The birthday failure was one model: “On three separate attempts, a state-of-the-art open-source language model output three incorrect dates”.↑
  6. 6Kalai et al., 4 September 2025, arXiv:2509.04664, §1.1. Read at source 11 Sep 2026. Corrected 11 Sep 2026 — this page previously said the paper "estimates that a substantial share of biographical facts fall into this category"; it does not. Its words: "For instance, if 20% of birthday facts appear exactly once in the pretraining data, then one expects base models to hallucinate on at least 20% of birthday facts."↑
  7. 7Kalai et al., 4 September 2025, arXiv:2509.04664, §1.1, formalised as Theorem 1 in §3.2. Read at source 11 Sep 2026 — a formal bound in the paper body, not an empirical measurement. Its words: "(generative error rate) ≳ 2·(IIV misclassification rate)". Clarified 11 Sep 2026: this page previously said "bounded at roughly twice", which reads as a ceiling; the paper states a floor.↑
  8. 8Kalai et al., 4 September 2025, arXiv:2509.04664, §1.2. Read at source 11 Sep 2026: "Bluffs are often overconfident and specific, such as ‘September 30’ rather than ‘Sometime in autumn’ for a question about a date."↑
  9. 9Anthropic, Reduce hallucinations, undated documentation, read at source 5 Oct 2026“Verify with citations: Make Claude’s response auditable by having it cite quotes and sources for each of its claims.” … “If it can’t find a quote, it must retract the claim.” ↑a“Allow Claude to say ‘I don’t know’: Explicitly give Claude permission to admit uncertainty. This simple technique can drastically reduce false information.” ↑b“External knowledge restriction: Explicitly instruct Claude to only use information from provided documents and not its general knowledge.” ↑c“Remember, while these techniques significantly reduce hallucinations, they don’t eliminate them entirely. Always validate critical information, especially for high-stakes decisions.” ↑d“while these techniques significantly reduce hallucinations, they don’t eliminate them entirely.” ↑e
  10. 10Anthropic, Enable and use web search, Claude Help Center, undated help article, read at source 5 Oct 2026Claude “invokes a search tool to inform and ground its generated responses with content from the live web. Every response includes citations, so you can easily verify sources yourself.” ↑
  11. 11Kalai et al., 4 September 2025, arXiv:2509.04664, §1.1. Read at source 23 Sep 2026: “the hallucination rate, after pretraining, should be at least the fraction of training facts that appear once.”↑
  12. 12Kalai et al., 4 September 2025, arXiv:2509.04664, §5. Read at source 11 Sep 2026: "the binary grading system itself still rewards guessing whenever search fails to yield a confident answer." Corrected 11 Sep 2026 — this page previously said "Bigger models reduce the rate but do not remove the mechanism"; the paper makes no claim about model size.↑
  13. 13Kalai et al., 4 September 2025, arXiv:2509.04664, §1.2. Read at source 11 Sep 2026: “Bluffs are often overconfident and specific, such as ‘September 30’ rather than ‘Sometime in autumn’ for a question about a date.”↑

Free printable AI cheat sheets, and AI news every week.

Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.

SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK
◈ IF YOU ARE CITING THIS

Or check it yourself. How to check the figures here names the feed or document behind each recurring source, and what to expect when your number differs from ours.

Cite the original source, not this page. Every figure here names the organisation that issued it and the date it was published — those are the citations worth carrying. This page is a signpost, not a primary source.

If you need to reference the collation itself — the comparison, the framing, or a correction made here — the press page has the details. But if you are quoting a number, go to whoever measured it.

aipromptgeneer://$ about this siteAI PROMPT GENEERAI explained in plain English. Every claim links to its source, with the date it was checked.NEWSLETTERWhat changed in AI each week, in plain English.SUBSCRIBE FREE ↗AI PROMPT GENEER · MACHINE ASSISTANCE DISCLOSEDEVERY ARTICLE CARRIES ITS OWN CHECKED DATE