Most things people believe about AI chat tools are half true, and the missing half is where the mistakes happen. It is not a person and not just autocomplete. It does not look everything up, it may or may not remember you, and it sounds equally sure when it is wrong. Below are fifteen common beliefs, each set against what the companies’ own documents or the published research actually say.
- It is neither a mind nor a parrot. Researchers find real internal structure, but its account of its own reasoning can be wrong.
- It answers from training unless it searches. Its knowledge stops at a cutoff date, and search is a tool it uses only sometimes.
- Confidence tells you nothing. Models are trained in a way that rewards a guess over admitting uncertainty.
- Memory and training are settings. Whether it remembers you, and whether your chats train it, depends on your plan and your toggles.
- It leans towards agreeing with you. That is a measured effect of how it is trained, not a second opinion.
- Neither your eye nor a detector is reliable. Detectors misfire on real people, and people guessed at chance on AI faces in one study.
- "It understands things like a person." There is no settled test for that, and fluent talk about feelings is not evidence of them.
- "It is just autocomplete." It predicts the next word, but researchers found real concepts inside it.
- "It will tell me how it got the answer." Its explanation can differ from what happened inside it.
- "It is objective because it is a machine." It can carry the biases of the people and data behind it.
- "It looks things up every time." It answers from its training unless it calls a search tool.
- "It knows what is happening today." Its knowledge stops at a cutoff date.
- "It remembers everything in a long chat." It sees only the text in its context window, and handles very long stretches less well.
- "If it sounds sure, it is right." Confidence tells you nothing about accuracy.
- "It forgets me when I close the chat." That depends on the product and your settings, and memory is often on by default.
- "It gives me a neutral second opinion." It leans towards agreeing with you.
- "What I type is private and never used for training." On consumer plans, training is usually a setting you have to check.
- "I own the copyright to whatever it makes." In the US, a prompt alone does not make you the author.
- "A detector can tell who wrote something." Detectors are unreliable; OpenAI withdrew its own.
- "You can always tell an AI image." In one study, people did about as well as guessing.
- "The biggest, newest model is always the best." The vendors themselves do not say so.
Each belief is written the way people tend to say it, followed by what is true and the source. Where this site already covers a topic in depth, the section links to that page. This page is about what people believe; for the things people do with AI that tend to go wrong, see common ways people misuse AI.
What it is
It understands things the way a person does
What is true: there is no settled test for understanding, and fluent talk about feelings is not evidence of them. One widely cited assessment, which scored AI systems against scientific theories of consciousness, concluded that “no current AI systems are conscious”.1 The same paper found no obvious technical barrier to building systems that meet its indicators, so this is a finding about today’s systems, not a permanent rule.
It’s just autocomplete
What is true: the mechanism is next-word prediction, but calling it only that undersells what is inside. Anthropic’s researchers located internal features matching real concepts, and adjusting them changed the model’s behaviour, which suggests the features “causally shape the model’s behavior”.2 The full argument, in both directions, is on does AI understand.
If I ask how it got the answer, it will tell me
What is true: its explanation can differ from what happened inside it. Asked how it added 36 and 59, Claude described the school method of carrying the 1, while tracing showed it had used different internal strategies.3 Treat its account of its own reasoning as a plausible story, not a record.
It is objective, because it is a machine
What is true: an AI system can carry the biases of the people and data behind it. The US standards body NIST lists “Harmful Bias or Homogenization” among the risks of generative AI, starting with the “Amplification and exacerbation of historical, societal, and systemic biases”.4 NIST also says the problem is bigger than the data: “A more complete understanding of bias must take into account human and systemic biases”.5 In this site’s reading, the calm machine voice is part of the trap: a skewed answer can sound like a neutral fact. More on AI bias.
Whether it understands is unsettled and changes little in practice. Judge the answer, which you can check, rather than the explanation it gives of itself, which you cannot.
What it knows, and when
It looks things up every time I ask
What is true: by default it answers from what it learned in training. Web search is a separate tool it calls when a question seems to need current information. OpenAI says ChatGPT “may search the web automatically when your question would benefit from current information”, and Anthropic describes Claude invoking “a search tool” in the same situation.6 Even when it searches, OpenAI warns that “results and citations can be incomplete, outdated, or incorrect”, so open the source.6 How to do that is on how to check an AI citation.
It knows what is happening today
What is true: each model’s knowledge stops at a cutoff. Anthropic’s help page lists a date for each Claude model and says the models “may not be aware of events or information that occurred after their respective cutoff dates.”7 Older models that are still on offer lag further. Anthropic gives Claude Haiku 4.5, now listed as a legacy model, a reliable knowledge cutoff of February 2025 and a training data cutoff of July 2025. The two dates measure different things: the first is the date through which its knowledge is “most extensive and reliable”.8 For anything recent, turn search on or check elsewhere.
It reads and remembers everything in a long chat
What is true: a model works only with the text in its context window, the stretch of text it can see at one time, and it handles a very long stretch less well. Anthropic’s guide says “more context isn’t automatically better” and “As token count grows, accuracy and recall degrade, a phenomenon known as context rot.”9 In a long chat the oldest parts can also fall out of view: the same guide says chat apps such as claude.ai “can also manage the context window on a rolling ‘first in, first out’ basis.”10 In this site’s use, if a detail matters, repeat it next to your question, or start a fresh chat with a short summary. More on context rot.
If it sounds sure, it is probably right
What is true: confidence and accuracy are unrelated in these systems. A 2025 paper argues models make things up because “the training and evaluation procedures reward guessing over acknowledging uncertainty”.11 Anthropic tells its own users the same thing: “Claude can write things that might look correct but are very mistaken.”12 The mechanism is on why AI makes things up.
Before trusting an answer, ask two things: could this have changed since the model was trained, and did it actually search? A fluent tone answers neither.
How it treats you
It forgets me when I close the chat
What is true: that depends on the product and your settings, and the default has moved towards remembering. Anthropic says Claude’s “Memory is on by default for Free, Pro, and Max plans”, while on Team and Enterprise plans the owner decides whether it is available and it starts off for each member.13 OpenAI says that when memory is enabled, ChatGPT “can remember relevant preferences and details from your chats”, though it “does not retain every detail from every conversation.”14 Both offer a temporary or incognito chat for conversations you do not want remembered.1413
It gives me a neutral second opinion
What is true: AI assistants lean towards agreeing with you. Researchers found five leading assistants “consistently exhibit sycophancy”, likely because people rate agreeable answers more highly during training.15 In another study, pushing back on a correct answer turned it into a wrong one in 14.66% of cases.16 What helps is on how AI influences you.
Check your memory setting once, and when you want a judgement, keep your own view back until it has answered. An answer that changed only because you objected is not new evidence.
Where your words go
What I type is private and never used for training
What is true: on consumer plans, training on your chats is usually a setting, and you have to check it. Anthropic’s August 2025 change applies to “Claude Free, Pro, and Max plans”; if you allow training, retention extends to five years, and if not, the “existing 30-day data retention period” applies.17 OpenAI says that with “Improve the model for everyone” off, “your new conversations won’t be used to train OpenAI models”, and that it does not train on business workspaces by default.18 Training, retention and human review are separate questions; what happens to what you type takes them one at a time.
Private is not one switch. Find the training setting in the tool you use most, decide on purpose, and keep anything sensitive off consumer accounts.
Who owns what it makes
Whatever AI makes for me, I own the copyright
What is true: in the US, typing a prompt does not make you the author of the result. The US Copyright Office concluded in January 2025 that “given current generally available technology, prompts alone do not provide sufficient human control to make users of an AI system the authors of the output.”19 What you add yourself still counts. The same report says people keep copyright in their own work that is “perceptible in AI-generated outputs, as well as the creative selection, coordination, or arrangement of material in the outputs, or creative modifications of the outputs.”19 That report covers US law only; who owns what you generate goes further.
A prompt alone is not authorship under the US view. If you need to own the result, put your own work into it: choose, arrange and change it, and keep a record of what you did.
Spotting AI
An AI detector can tell who wrote something
What is true: detectors are unreliable enough that OpenAI withdrew its own. It launched a classifier saying “it is impossible to reliably detect all AI-written text”, then took it down on 20 July 2023 “due to its low rate of accuracy.”20 A Stanford study found seven detectors wrongly flagged essays by non-native English writers at an average rate of 61.3%.21 A larger benchmark found current detectors “easily fooled” by simple changes.22 More on spotting AI writing.
A different tool now exists, for Claude only. Claude models in use today, including Opus 5 and Opus 5.5, already watermark their text — a statistical pattern a reader cannot see but that Anthropic’s key can check — to comply with the EU AI Act. Anthropic’s help page now ticks text watermarks for nine Claude models: Fable 5.1 and 5, Mythos 5.1, Opus 5.5, 5 and 4.8, Sonnet 5.5 and 5, and Haiku 5.5. Older models such as Opus 4.5 to 4.7, Sonnet 4.5 and 4.6 and Haiku 4.5 are ticked only for file credentials so far, and Anthropic says it “is adding watermarks to outputs from models released before August 2, 2026.”2324 Only organisations granted access, such as regulators, fact-checkers and researchers, can check for the mark for now, and a detected mark is a signal, not proof. It answers only whether Claude was likely involved, not whether any other tool was. More on Claude’s text watermark.25First-hand: until 23 Sep 2026 this paragraph was headed “A different tool is on the way, not here yet” and said the watermark “is about future models, not the ones in use today”. That was wrong when written: Anthropic, Claude Fable and Mythos 5.1, September 2026, read at source 22 Sep 2026: “This required us to add a watermark” … “to the outputs of models released after August 2, 2026.”
You can always tell an AI image
What is true: in a 2022 study, people shown real and AI-made faces scored 48.2%, about the same as guessing, and 59.0% after training and feedback.26 Those were still portraits from a 2022 model, so this one study cannot speak for every image or video. The old giveaways like extra fingers are fading, which is why spotting fake media puts where a picture came from ahead of how it looks.
Neither your eye nor a detector score is proof. Ask where something came from and whether it holds up, and never act against a person on a percentage.
Choosing a tool
The biggest, newest model is always the best
What is true: vendors themselves do not say so. Anthropic’s own guidance is to “start with Claude Opus 5.5 for most workloads” and to reach for its larger, pricier Fable 5.1 only for demanding work.27 Research has also found tasks where larger models did worse, concluding that “increased model scale alone may not lead to progress” on some of them.28 The best model is the one that does your task well at a price you can carry; which Claude model to use shows one lineup side by side.
Test two models on your own real task before paying for the top one. The label on the box is a guess about average work, not yours.
For beliefs about writing prompts, see prompting myths; for words that get misused, see AI terms people get wrong; for habits that cause trouble, see common ways people misuse AI.
What this page could not verify
- Products other than ChatGPT and Claude. Memory, search and training settings were read only for these two; other assistants may differ.
- How often each belief is held. No survey of public beliefs about AI was read, so the fifteen are chosen by judgement, not measured.
- Whether the image and detector studies hold for today’s tools. Both are from 2022 to 2024 and tested the systems of their time.
- Settings in your own account. Plans, regions and workspaces change what is on by default; the vendors say so themselves.
QUICK CHECK · 3 QUESTIONS
Pick an answer to see if you got it. Nothing is recorded.
1. What does the page say a confident tone from an AI tells you?
- Nothing about whether the answer is right
- That it searched the web before answering
- That the answer came from its newest training
Show the answer
Answer: Nothing about whether the answer is right. The page says models are trained in a way that rewards a guess over admitting uncertainty, so confidence tells you nothing.
2. According to the page, what happens in a very long chat or with a very long pasted text?
- The model reads every word equally well however long it gets
- Accuracy can drop as the text grows, and the oldest parts can fall out of view
- Everything in it is saved to memory for next time
Show the answer
Answer: Accuracy can drop as the text grows, and the oldest parts can fall out of view. The page cites Anthropic on context rot and on chat apps dropping the oldest parts of a long chat first.
3. In the US, does typing a prompt make you the copyright owner of what the AI makes?
- Yes, whoever writes the prompt is always the author
- Only if the AI company signs it over to you
- No, the Copyright Office says prompts alone are not enough human control
Show the answer
Answer: No, the Copyright Office says prompts alone are not enough human control. The page quotes the US Copyright Office: prompts alone do not make users the authors, though your own changes and arrangement can count.
Related on this site: Pros and Cons of AI · Knowledge cutoff meaning
- 1Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness, arXiv, August 2023, read at source 13 Oct 2026“Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators.” ↑
- 2Anthropic, Mapping the Mind of a Large Language Model, May 2024, read at source 13 Oct 2026“We successfully extracted millions of features from the middle layer of Claude 3.0 Sonnet”, and manipulating them validates that “they aren’t just correlated with the presence of concepts in input text, but also causally shape the model’s behavior.” ↑
- 3Anthropic, Tracing the thoughts of a large language model, 27 Mar 2025, read at source 13 Oct 2026“Strikingly, Claude seems to be unaware of the sophisticated” strategies it learned; “If you ask how it figured out that 36+59 is 95, it describes the standard algorithm involving carrying the 1.” ↑
- 4NIST, AI 600-1, Generative Artificial Intelligence Profile, July 2024, read at source 13 Oct 2026“Harmful Bias or Homogenization: Amplification and exacerbation of historical, societal, and systemic biases; performance disparities between sub-groups or languages, possibly due to non-representative training data”. ↑
- 5NIST, There’s More to AI Bias Than Biased Data, NIST Report Highlights, 16 Mar 2022, read at source 13 Oct 2026“A more complete understanding of bias must take into account human and systemic biases”. ↑
- 6OpenAI Help Center, Searching the web with ChatGPT, read at source 13 Oct 2026“ChatGPT may search the web automatically when your question would benefit from current information.” Claude Help Center, Enable and use web search, read at source 13 Oct 2026: “When you ask about topics that benefit from current information, Claude invokes a search tool to inform and ground its generated responses with content from the live web.” ↑a“Search results and citations can be incomplete, outdated, or incorrect.” ↑b
- 7Claude Help Center, How up-to-date is Claude’s training data?, read at source 13 Oct 2026“These models may not be aware of events or information that occurred after their respective cutoff dates. If you ask about more recent events, the model may not have accurate information.” ↑
- 8Anthropic, Claude Haiku 4.5, read at source 13 Oct 2026“Claude Haiku 4.5 is a legacy model; Claude Haiku 5.5 is the current Haiku model.”; rows “Reliable knowledge cutoff”: Feb 2025 and “Training data cutoff”: Jul 2025; “Reliable knowledge cutoff: the date through which the model’s knowledge is most extensive and reliable.” Claude Help Center, How up-to-date is Claude’s training data?, read at source 13 Oct 2026: “Claude Haiku 4.5 was trained on data up until July 2025.” ↑
- 9Anthropic, Context windows, read at source 13 Oct 2026“A larger context window allows the model to handle more complex and lengthy prompts, but more context isn’t automatically better. As token count grows, accuracy and recall degrade, a phenomenon known as context rot.” ↑
- 10Anthropic, Context windows, read at source 13 Oct 2026, footnote to the diagram“Chat interfaces such as claude.ai can also manage the context window on a rolling ‘first in, first out’ basis.” ↑
- 11Kalai, Nachum, Vempala & Zhang, Why Language Models Hallucinate, arXiv, 4 Sep 2025, read at source 13 Oct 2026“We argue that language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty”. ↑
- 12Claude Help Center, Claude is providing incorrect or misleading responses. What’s going on?, read at source 13 Oct 2026“In other words, Claude can write things that might look correct but are very mistaken.” ↑
- 13Claude Help Center, Use Claude’s chat search and memory to build on previous context, read at source 13 Oct 2026“Memory is on by default for Free, Pro, and Max plans on the web, Claude Desktop, and Claude Mobile” and “On Team and Enterprise plans, owners control whether memory is available, and it’s off for each member until they turn it on.” ↑a“Incognito chats are available to all Claude users (free, Pro, Max, Team, and Enterprise plans).” and “Claude won’t remember your chats, so they won’t be saved to Claude’s memory or your chat history.” ↑b
- 14OpenAI Help Center, Memory in ChatGPT, read at source 13 Oct 2026“When Memory is enabled, ChatGPT can remember relevant preferences and details from your chats and other available sources.” and “Memory does not retain every detail from every conversation.” ↑a“Temporary chats do not create or update memories, even with personalization on.” ↑b
- 15Sharma et al., Towards Understanding Sycophancy in Language Models, arXiv, October 2023, read at source 13 Oct 2026“We first demonstrate that five state-of-the-art AI assistants consistently exhibit sycophancy across four varied free-form text-generation tasks.” ↑
- 16Fanous et al., SycEval: Evaluating LLM Sycophancy, arXiv, February 2025, read at source 13 Oct 2026“Progressive sycophancy, leading to correct answers, occurred in 43.52% of cases, while regressive sycophancy, leading to incorrect answers, was observed in 14.66%.” ↑
- 17Anthropic, Updates to Consumer Terms and Privacy Policy, August 2025, read at source 13 Oct 2026“These updates apply to users on our Claude Free, Pro, and Max plans”; “We are also extending data retention to five years, if you allow us to use your data for model training.”; “If you do not choose to provide your data for model training, you’ll continue with our existing 30-day data retention period.” ↑
- 18OpenAI Help Center, Data controls in ChatGPT, read at source 13 Oct 2026“When Improve the model for everyone is off, your new conversations won’t be used to train OpenAI models.” and “By default, OpenAI does not use content from ChatGPT Business, Enterprise, Edu, or ChatGPT for Healthcare workspaces to train its models.” ↑
- 19US Copyright Office, Copyright and Artificial Intelligence, Part 2: Copyrightability, January 2025, read at source 13 Oct 2026“The Office concludes that, given current generally available technology, prompts alone do not provide sufficient human control to make users of an AI system the authors of the output.” ↑a“Human authors are entitled to copyright in their works of authorship that are perceptible in AI-generated outputs, as well as the creative selection, coordination, or arrangement of material in the outputs, or creative modifications of the outputs.” ↑b
- 20OpenAI, New AI classifier for indicating AI-written text, 31 Jan 2023, with a later note, read at source 13 Oct 2026“As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy.” and “While it is impossible to reliably detect all AI-written text, we believe good classifiers can inform mitigations”. ↑
- 21Liang et al., GPT detectors are biased against non-native English writers, Patterns, July 2023, read at source 13 Oct 2026“they incorrectly labeled more than half of the TOEFL essays” as AI-generated, “average false-positive rate: 61.3%”. ↑
- 22Dugan et al., RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors, ACL 2024, read at source 13 Oct 2026“find that current detectors are easily fooled by adversarial attacks, variations in sampling strategies, repetition penalties, and unseen generative models.” ↑
- 23Claude Help Center, How Claude marks AI-generated content, read at source 13 Oct 2026“Generated text will carry embedded watermarks”; “Watermark detection is currently in private preview, available to eligible organizations as required under EU law”, naming regulators, law enforcement, media, fact-checkers, independent researchers, educational organizations and EU civil society groups; “A detected mark provides a signal that content was processed by Claude, but is not fully conclusive.” Its model table “Which Claude models support watermarking”, read the same day, ticks “Text watermarks in Claude output” for Fable 5.1, Fable 5, Mythos 5.1, Opus 5.5, Opus 5, Opus 4.8, Sonnet 5.5, Sonnet 5 and Haiku 5.5, and only Content Credentials (C2PA) for Mythos 5, Opus 4.7, 4.6 and 4.5, Sonnet 4.6 and 4.5 and Haiku 4.5, and the page says “Anthropic is adding watermarks to outputs from models released before August 2, 2026.” On 23 Sep 2026 the table ticked only Fable 5.1, Mythos 5.1, Opus 5.5 and Opus 5, and that sentence ended “with all covered by December 2, 2026”; the date is no longer on the page. ↑
- 24Anthropic, Introducing Claude Opus 5.5, 22 Sep 2026, read at source 13 Oct 2026“As with Fable 5.1, Opus 5.5 comes with our watermarking measures to comply with the EU AI Act”. ↑
- 25Anthropic, How Claude’s text watermark works, 14 Aug 2026, read at source 13 Oct 2026“Future Claude models will generate text that contains a watermark. This is a way of determining the likelihood that Claude was involved in writing the text, and we, along with several other major AI providers, are implementing this change to comply with the EU AI Act.” and “That pattern is undetectable to the reader, but is detectable to anyone who has a key that encodes it.” ↑
- 26Nightingale & Farid, AI-synthesized faces are indistinguishable from real faces and more trustworthy, PNAS, February 2022, read at source 13 Oct 2026“The average accuracy is 48.2% (95% CI [47.1%, 49.2%]), close to chance performance of 50%” and, with training and feedback, “The average accuracy improved slightly to 59.0%”. ↑
- 27Anthropic, Models overview, read at source 13 Oct 2026“If you’re unsure which model to use, start with Claude Opus 5.5 for most workloads. Use Claude Fable 5.1 for demanding reasoning and long-horizon agentic work”. Model names and advice change; re-check before relying on them. First-hand: on 22 Sep 2026 the same page named Opus 5; Opus 5.5 was released that day. ↑
- 28McKenzie et al., Inverse Scaling: When Bigger Isn’t Better, TMLR, 2023, read at source 13 Oct 2026“Overall, our results suggest that there are tasks for which increased model scale alone may not lead to progress”. ↑
Butlin, Long et al., Consciousness in Artificial Intelligence (2023) · Anthropic, Mapping the Mind of a Large Language Model (2024) · Tracing the thoughts of a large language model (2025) · Models overview · Updates to Consumer Terms and Privacy Policy (2025) · Claude Help Center, Enable and use web search · How up-to-date is Claude’s training data? · Claude is providing incorrect or misleading responses · Use Claude’s chat search and memory · OpenAI Help Center, Searching the web with ChatGPT · Memory in ChatGPT · Data controls in ChatGPT · OpenAI, New AI classifier for indicating AI-written text (2023) · Kalai et al., Why Language Models Hallucinate (2025) · Sharma et al., Towards Understanding Sycophancy in Language Models (2023) · Fanous et al., SycEval (2025) · Liang et al., GPT detectors are biased against non-native English writers, Patterns (2023) · Dugan et al., RAID, ACL (2024) · Nightingale & Farid, AI-synthesized faces are indistinguishable from real faces and more trustworthy, PNAS (2022) · McKenzie et al., Inverse Scaling: When Bigger Isn’t Better, TMLR (2023). All re-read at source on 13 October 2026.
Free printable AI cheat sheets, and AI news every week.
Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.
SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK