In AI, a transformer means a design for an AI model that reads a whole passage at once and works out which words matter to each other. It is the “T” in GPT, and it is the design best known for powering large language models, the AI inside chatbots.
Think of reading a mystery novel. When you hit the word “he”, your eyes flick back to work out who “he” is. A transformer does that for every word in the passage at the same time: each word looks around at all the others and decides which ones help explain it. That looking-around is called attention.
In a sentence: “In the lecture she explained that GPT is built on a transformer, which is why the letter T is in its name.”
What the word means in AI
The plain point: a transformer is a blueprint, not a product. Many different AI models are built to it.
IBM describes it as a kind of neural network (a large web of adjustable numbers that learns from examples) that is good at handling things in order, like words in a sentence.1
The design was introduced in a 2017 research paper called Attention Is All You Need. Its authors said it ran on attention alone and was quicker to train than the designs before it.2
“Quicker to train” is the key. Older designs read one word after another. A transformer can work on all the words in parallel, so it can be fed far more text in the same time.3
How attention works: a simple example
The plain point: attention lets the model use the rest of the sentence to decide what each word means.
IBM gives an example: “on Friday, the judge issued a sentence”. On its own, “sentence” could mean a line of words or a punishment. Because “judge” and “issued” are nearby, the model can tell it means a punishment.1
The model does not read words as whole words. It reads small chunks called tokens. Attention runs across every token in view, which is one reason the size of the context window matters so much.
Attention is why context changes the answer. Give a model the surrounding detail and it has more to weigh each word against.
Why it matters to you
The plain point: you do not need to understand the maths, but the name explains a lot of product names.
- GPT is named after it. IBM spells the letters out: “Generative Pre-trained Transformer”.1 Generative means it makes new text. Pre-trained means it learned from a huge pile of text before you ever used it.
- It is not only for words. IBM says the same design also does well with pictures, speech and number patterns over time.1
- It still guesses. A transformer predicts likely next tokens. A better design does not stop it from making things up.
Where the word comes from
The design and its name come from a research paper by Ashish Vaswani and colleagues, “Attention Is All You Need”, first posted on 12 June 2017. It opens by proposing “a new simple network architecture, the Transformer, based solely on attention mechanisms”.2
Often confused with
| Word | What it means |
|---|---|
| Transformer (in AI) | Means: the design, or blueprint, that a model is built to. |
| LLM | Means: a large language model, which is a finished model. LLMs are the best-known use of the transformer design (see the IBM quote above). See LLM meaning. |
| Diffusion model | Means: a different method, used for most image and video tools, that turns noise into pictures. See diffusion model meaning. |
| Transformer (everyday) | Means: a box that changes electrical voltage, or the toy robots. Nothing to do with AI. |
Questions people ask
What does GPT stand for?
Generative Pre-trained Transformer. IBM spells it out when describing “the GPT-3 (short for Generative Pre-trained Transformer) model” behind the launch of ChatGPT.1
Are transformers only used for text?
No. IBM notes that transformer models also do well at computer vision, speech recognition and time series forecasting.1
Learn more on this site
- Where it fits in history: The history of AI places the 2017 paper on the timeline.
- How a model is trained: How an AI is actually built covers pretraining, the stage where a transformer learns.
- Who builds them: Who builds AI.
- The basics: the AI glossary and AI explained for complete beginners.
- Related: GPT meaning.
- Related: Machine translation meaning.
- Related: What Is Attention in AI? Attention Meaning in Plain English.
QUICK CHECK · 3 QUESTIONS
Pick an answer to see if you got it. Nothing is recorded.
1. What is a transformer in AI?
- A design that many AI models are built to
- One chatbot product made by one company
- A box that changes electrical voltage
Show the answer
Answer: A design that many AI models are built to. The page says a transformer is a blueprint, not a product, and many models are built to it; the voltage box is the everyday meaning.
2. What does attention let a model do?
- Read one word at a time, in strict order
- Remember every chat you have ever had
- Use the rest of the sentence to work out each word’s meaning
Show the answer
Answer: Use the rest of the sentence to work out each word’s meaning. The page’s judge example shows nearby words like ‘judge’ and ‘issued’ telling the model which meaning of ‘sentence’ fits.
3. Why could transformers be fed far more text than older designs?
- They skip most of the words they read
- They work on all the words in parallel
- They only read very short sentences
Show the answer
Answer: They work on all the words in parallel. The page says older designs read one word after another, while a transformer works on all the words at once, so it trains quicker.
Related terms: Supervised learning · Synthetic data · Training data · Turing test · Vision-language model (VLM) · World model
- 1IBM, What is a transformer model?, published 28 Mar 2025, read at source 1 Oct 2026“The transformer model is a type of neural network architecture that excels at processing sequential data, most prominently associated with large language models (LLMs).” ↑aattention mechanisms are “algorithms designed to determine which parts of a data sequence an AI model should ‘pay attention to’ at any particular moment.” ↑b“the GPT-3 (short for Generative Pre-trained Transformer) model that powered the launch of OpenAI’s ChatGPT”. ↑c“Transformer models have also achieved elite performance in other fields of artificial intelligence (AI), such as computer vision, speech recognition and time series forecasting.” ↑d“the GPT-3 (short for Generative Pre-trained Transformer) model that powered the launch of OpenAI’s ChatGPT”. ↑e“Transformer models have also achieved elite performance in other fields of artificial intelligence (AI), such as computer vision, speech recognition and time series forecasting.” ↑f
- 2Vaswani et al., Attention Is All You Need, arXiv:1706.03762, submitted 12 Jun 2017, read at source 1 Oct 2026“We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.” ↑a“We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.” ↑b
- 3Vaswani et al., Attention Is All You Need, arXiv:1706.03762, read at source 1 Oct 2026“more parallelizable and requiring significantly less time to train.” ↑
Vaswani et al., Attention Is All You Need, arXiv:1706.03762, submitted 12 Jun 2017, read at source 1 Oct 2026.
The comparisons on this page (the mystery novel, the blueprint) are this site’s own ways of explaining, not claims from these sources.
Free printable AI cheat sheets, and AI news every week.
Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.
SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK