ONLINEAGENT_OPS 2026.Q3 HOME HUBSMODELSBUILDINGCOMPANIESFILMMAKINGSECURITYLIVING WITH ITSKILLSALL HUBS, ONE TERMINAL → ARTICLES CRAFT RECORD BLOG TOOLS SKILLS SEARCH /
HOME›ARTICLES›HOW DOES CHATGPT WORK? WHAT HAPPENS WHEN YOU
ARTICLES · EXPLAINERBEGINNER

How Does ChatGPT Work? What Happens When You Press Send

What happens when you press send in ChatGPT, Claude or Gemini: tokens, hidden instructions, the context window and next-word prediction, in plain English.

READ9 min
WORDS1,856
SECTIONS12
TYPEEXPLAINER
CHECKED5 OCT 26

When you press send, ChatGPT does not look up a finished answer. The app bundles your message with hidden instructions and the conversation so far, cuts it into small chunks called tokens, and hands it to a large language model. The model then writes the reply one chunk at a time, each time picking a likely next piece. Claude and Gemini work the same basic way. This page covers those few seconds; how the model was made in the first place is told in how an AI is actually built.

◈ IN PLAIN TERMS

Picture a very well-read improviser with no memory of yesterday. Every time you press send, someone hands them the whole script again: stage directions you never see, every line spoken so far, and your new line. The improviser reads it all and carries on, one word at a time, choosing what sounds most fitting. That is a chatbot answering you.

What happens when you press send, step by step

The plain point: six things happen, mostly in a data centre, before the first word appears.

  1. The app packs a bundle: your message, the chatbot’s standing instructions, the earlier messages and, if memory is on, saved notes about you.
  2. The bundle is cut into tokens, the small chunks of text an AI actually reads.
  3. It must fit the context window, the limit on how much the model can keep in view.
  4. The model predicts the reply: it picks a likely next token, adds it, and repeats.
  5. It may stop to use a tool, such as a web search, then read the results and carry on.
  6. The reply comes back word by word, because it is being written word by word.

Step 1: your message is not sent alone

The plain point: the model sees far more than what you typed.

Hidden instructions come first. Every chatbot starts with standing instructions from the company that built it, called a system prompt. NIST, the US government’s standards agency, describes instructions “often prepended to the model’s other input” (placed in front of it), which “is known as a SYSTEM PROMPT.”1 They also outrank you. OpenAI’s developer guide says “developer messages are instructions provided by the application developer, prioritized ahead of user messages.”2

The whole conversation is sent again, every time. The model keeps nothing between messages. Anthropic, the maker of Claude, says so in its developer documentation: “The Messages API is stateless, which means that you always send the full conversational history to the API.”3 The chat app does the resending for you, which is why the chatbot seems to remember what you said five messages ago. If memory is on, saved notes from earlier chats are added too (see memory below).

Step 2: your words become tokens

The plain point: an AI does not read letters or whole words. It reads tokens, which sit in between.

Google’s documentation explains: “Tokens can be single characters like z or whole words like cat. Long words are broken up into several tokens.”4 As a rough rule, OpenAI says “1 token is approximately three-quarters of a word.”5 A short question is a handful of tokens; a pasted report is thousands. Each company splits text its own way, so counts differ between tools.

Step 3: everything has to fit in the context window

The plain point: the model can only keep so much in view, and the whole bundle counts, including the reply.

That limit is the context window, which Anthropic describes as “all the text a language model can reference when generating a response, including the response itself.” When a long chat outgrows it, Anthropic notes that chat apps can drop the oldest parts first, on a “first in, first out” basis. A fuller window is not a smarter one either: “As token count grows, accuracy and recall degrade, a phenomenon known as context rot.”6 That is why a very long chat can seem to forget its own beginning, and why a fresh chat often helps.

Step 4: the model predicts the reply, one token at a time

The plain point: this is the heart of it. The model does not retrieve an answer; it builds one, piece by piece.

The model is a large language model (LLM). Google’s course for developers says a language model “estimates the probability of a token or sequence of tokens occurring within a longer sequence of tokens.”7 In plain words: given everything so far, which piece of text most likely comes next?

So the model reads the bundle, picks one next token, adds it to the end, and goes again with the slightly longer text. Anthropic puts it simply: “Claude writes text one word at a time.” Nobody typed the answers in; the same article says models like Claude “aren’t programmed directly by humans” but trained on large amounts of data.8 The knowledge lives in learned patterns, not in a database of facts.

How it weighs your words. ChatGPT-style models use a design called a transformer. Its key trick, attention, is described by IBM as “algorithms designed to determine which parts of a data sequence an AI model should ‘pay attention to’ at any particular moment.” That is how a word early in your message can still shape a word late in the reply. The same IBM page spells out the name: “GPT-3 (short for Generative Pre-trained Transformer)”.9 See GPT meaning.

Some models think first. OpenAI says “Reasoning models use internal reasoning tokens before producing a response.”10 It is the same next-token process with a hidden scratchpad. See reasoning model meaning.

TAKEAWAY

A chatbot’s reply is written fresh each time, one small piece after another, from learned patterns. That explains both how fluent it is and how it can be wrong.

Why you get a different answer each time

The plain point: the model does not always take the single most likely next word, so the same question can come back worded differently.

A setting called temperature controls how adventurous those picks are. Anthropic’s glossary calls it “a parameter that controls the randomness of a model’s predictions during text generation,” and warns that even at zero, identical inputs “may produce different outputs”.11 OpenAI agrees that model output “is non-deterministic.”2

Chat apps set the dial for you, and some companies are retiring it. Google deprecated it in its Gemini API in July 2026, and Anthropic now says the sampling settings “are not supported on Claude 4.7 and later models”, advising people to “use prompting to guide the model’s behavior instead.”12 Want a steadier answer? Say exactly what you want and show an example.

Why it can sound sure and still be wrong

The plain point: the same machinery produces true and false sentences, and sounds equally confident either way.

A 2025 research paper puts it this way: “large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty.”13 These are hallucinations, and names, dates, figures and quotes are where they bite most. Why AI makes things up explains the causes.

The plain point: on its own, a model only knows what it learned in training. Tools let it fetch fresh information or take actions.

With tool use, the model writes a neat request, the app carries it out, and the result goes back into the bundle. Anthropic: “Claude determines when to call a tool based on the user’s request and the tool’s description.”14

Web search is the tool most people meet. Anthropic says it lets Claude answer “with up-to-date information beyond its knowledge cutoff” (the date its training data stops; see knowledge cutoff meaning), ending with “a final response with cited sources.”15 OpenAI describes the same for its models: answers “with sourced citations.”16 A cited answer is easier to check, not automatically right: open the link and see that it says what the chatbot claims.

Memory: what it keeps between chats

The plain point: “memory” is an app feature that saves notes about you. It does not change the model.

OpenAI says that “When Memory is enabled, ChatGPT can remember relevant preferences and details from your chats”.17 Anthropic says Claude’s memory “is on by default for Free, Pro, and Max plans”.18 Those notes are simply added to the bundle in step 1. AI memory meaning explains how to view or delete them, and what happens to what you type covers storage and training settings.

Where the work happens

The plain point: for most chatbots, the answer is worked out on the company’s computers, not your phone.

Running a finished model is called inference, which IBM defines as “the act of using a trained AI model to make predictions on new data.”19 Your question does not retrain it. One answer uses little electricity: Google measured its median Gemini text prompt at 0.24 watt-hours.20 Billions of answers add up, which is the story in what AI actually runs on.

ChatGPT, Claude and Gemini: same recipe, different packaging

The plain point: the steps above apply to all the big chatbots. What differs is the model inside and the wrapping around it.

IBM puts the line between old and new chatbots like this: “Traditional chatbots follow predefined rules, decision trees and scripted conversation flows. AI chatbots use technologies such as natural language processing (NLP), generative AI and large language models (LLMs).”21 The system prompt, the window size, which tools are on and how memory works are each company’s choices, which is why two chatbots can feel very different. ChatGPT vs Claude vs Gemini compares them.

TAKEAWAY

Knowing the steps tells you how to get better answers: put the context it needs in the chat, start fresh when a chat gets long, ask it to search for current facts, and check anything specific before you rely on it.

Questions people ask

Does ChatGPT look things up on the internet?

Only when a search tool is on and the chatbot decides to use it; otherwise it answers from what it learned in training. Anthropic describes search as reaching information “beyond its knowledge cutoff”.15

Does ChatGPT learn from my conversations?

Not while you chat: the model is fixed, and memory is separate note-keeping. Some companies can use chats to train future models when a setting is on; Anthropic, for example, says training applies to “Free, Pro, and Max accounts when this setting is on”.22 Is ChatGPT safe? shows where ChatGPT’s own setting is.

Does ChatGPT understand what I say?

It predicts text from patterns; whether that counts as understanding is a live debate. Does AI understand anything? sets out both sides.

Why does ChatGPT forget things in long chats?

Everything must fit in the context window, and apps can drop the oldest messages first. Anthropic adds that accuracy and recall “degrade” as the window fills.6

Is ChatGPT the same as GPT?

No. GPT is the family of models; ChatGPT is the app around them. IBM notes that GPT-3 “powered the launch of OpenAI’s ChatGPT”.23

Related on this site: How to Use ChatGPT · Context Window vs AI Memory · What Is Attention in AI?

SOURCES
  1. 1NIST, AI 100-2e2025, Adversarial Machine Learning, March 2025, read at source 1 Oct 2026instructions “often prepended to the model’s other input” … “is known as a SYSTEM PROMPT.” ↑
  2. 2OpenAI, Text generation, read at source 5 Oct 2026“developer messages are instructions provided by the application developer, prioritized ahead of user messages.” ↑a“Because the content generated from a model is non-deterministic, prompting to get your desired output is a mix of art and science.” ↑b
  3. 3Anthropic, Using the Messages API, read at source 5 Oct 2026“The Messages API is stateless, which means that you always send the full conversational history to the API.” ↑
  4. 4Google, Understand and count tokens, updated 23 Sep 2026, read at source 1 Oct 2026“Tokens can be single characters like z or whole words like cat. Long words are broken up into several tokens.” ↑
  5. 5OpenAI, Understanding and counting tokens, read at source 22 Sep 2026“1 token is approximately three-quarters of a word.” ↑
  6. 6Anthropic, Context windows, read at source 1 Oct 2026“all the text a language model can reference when generating a response, including the response itself.” · “a rolling ‘first in, first out’ basis.” · “As token count grows, accuracy and recall degrade, a phenomenon known as context rot.” ↑a“As token count grows, accuracy and recall degrade”. ↑b
  7. 7Google, Introduction to Large Language Models, updated 9 Jan 2026, read at source 1 Oct 2026“A language model estimates the probability of a token or sequence of tokens occurring within a longer sequence of tokens.” ↑
  8. 8Anthropic, Tracing the thoughts of a large language model, 27 Mar 2025, read at source 1 Oct 2026“Claude writes text one word at a time.” · “Language models like Claude aren’t programmed directly by humans—instead, they’re trained on large amounts of data.” ↑
  9. 9IBM, What is a transformer model?, 28 Mar 2025, read at source 1 Oct 2026attention mechanisms are “algorithms designed to determine which parts of a data sequence an AI model should ‘pay attention to’ at any particular moment.” · “the GPT-3 (short for Generative Pre-trained Transformer) model that powered the launch of OpenAI’s ChatGPT”. ↑
  10. 10OpenAI, Reasoning models, read at source 1 Oct 2026“Reasoning models use internal reasoning tokens before producing a response.” ↑
  11. 11Anthropic, Glossary, read at source 1 Oct 2026“Temperature is a parameter that controls the randomness of a model’s predictions during text generation.” · “Even with temperature set to 0, the results will not be fully deterministic and identical inputs may produce different outputs across API calls.” ↑
  12. 12Google, Gemini API release notes, entry dated 21 July 2026, read in a browser 16 Sep 2026“The sampling parameters temperature, top_p and top_k are now deprecated.” · Anthropic, Using the Messages API, read at source 5 Oct 2026: “The temperature, top_p, and top_k sampling parameters are not supported on Claude 4.7 and later models” … “use prompting to guide the model’s behavior instead.” ↑
  13. 13Kalai et al., Why Language Models Hallucinate, arXiv:2509.04664, 4 Sep 2025, read at source 1 Oct 2026“large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty.” ↑
  14. 14Anthropic, Tool use with Claude, read at source 1 Oct 2026“Claude determines when to call a tool based on the user’s request and the tool’s description.” ↑
  15. 15Anthropic, Web search tool, read at source 5 Oct 2026“allowing it to answer questions with up-to-date information beyond its knowledge cutoff.” · “At the end of its turn, Claude provides a final response with cited sources.” ↑a“up-to-date information beyond its knowledge cutoff.” ↑b
  16. 16OpenAI, Web search, read at source 5 Oct 2026“Web search allows models to access up-to-date information from the internet and provide answers with sourced citations.” ↑
  17. 17OpenAI, Memory FAQ, read at source 22 Sep 2026“When Memory is enabled, ChatGPT can remember relevant preferences and details from your chats and other available sources.” ↑
  18. 18Anthropic, Use Claude’s chat search and memory, read at source 1 Oct 2026“Memory is on by default for Free, Pro, and Max plans on the web, Claude Desktop, and Claude Mobile”. ↑
  19. 19IBM, What is AI inference?, updated 13 Jan 2026, read at source 1 Oct 2026“AI inference is the act of using a trained AI model to make predictions on new data.” ↑
  20. 20Google, Measuring the environmental impact of delivering AI at Google Scale, Aug 2025, read at source 11 Sep 2026“the median Gemini Apps text prompt uses 0.24 watt-hours (Wh) of energy”. ↑
  21. 21IBM, What Is a Chatbot?, updated 16 Jul 2026, read at source 1 Oct 2026“Traditional chatbots follow predefined rules, decision trees and scripted conversation flows. AI chatbots use technologies such as natural language processing (NLP), generative AI and large language models (LLMs).” ↑
  22. 22Anthropic, Updates to Consumer Terms and Privacy Policy, August 2025, read at source 11 Sep 2026training applies to “Free, Pro, and Max accounts when this setting is on”. ↑
  23. 23IBM, What is a transformer model?, read at source 1 Oct 2026“the GPT-3 (short for Generative Pre-trained Transformer) model that powered the launch of OpenAI’s ChatGPT”. ↑

Kalai et al., Why Language Models Hallucinate, 4 Sep 2025, read at source 1 Oct 2026.

Sources read before 5 Oct 2026 were read for this site’s linked detail pages (system prompt, token, context window, LLM, transformer, reasoning model, temperature, hallucination, tool use, AI memory, inference, chatbot, what AI runs on, what happens to what you type) and are re-used here with the date they were read.

The comparisons on this page (the improviser handed the whole script) and the six-step outline are this site’s own ways of explaining, not claims from these sources.

Free printable AI cheat sheets, and AI news every week.

Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.

SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK
aipromptgeneer://$ about this siteAI PROMPT GENEERAI explained in plain English. Every claim links to its source, with the date it was checked.NEWSLETTERWhat changed in AI each week, in plain English.SUBSCRIBE FREE ↗AI PROMPT GENEER · MACHINE ASSISTANCE DISCLOSEDEVERY ARTICLE CARRIES ITS OWN CHECKED DATE