In AI, distillation means training a smaller model to copy the answers of a bigger one, so you get much of the big model’s skill in a package that is faster and lighter to run. The big model is called the teacher; the small one is the student.
Think of a senior chef writing a short recipe card for a new cook. The cook never sees the years of training behind it, only the finished results to copy. The cook will not match the chef at everything, but for the dishes on the card they get close, and they work much faster in a small kitchen.
In a sentence: “The small model on my phone is a distillation of the big one, which is why it’s quick and works offline but is a bit less clever.”
What the word means in AI
The plain point: a big model teaches a small model by example.
IBM, which builds and sells AI systems, describes it in teacher-and-student terms: it moves what a large “teacher model” has learned into a smaller “student model.”1
A 2015 paper by Geoffrey Hinton, Oriol Vinyals and Jeff Dean, titled Distilling the Knowledge in a Neural Network, showed that what several big models knew could be squeezed into one model that is easier to put to work.2
In everyday English, to distil is to boil a liquid and keep the concentrated part. The AI meaning borrows that idea of keeping the essence.
Where the word comes from
The name comes from a 2015 paper by Geoffrey Hinton, Oriol Vinyals and Jeff Dean, which described packing a big model’s knowledge into a smaller one “by distilling the knowledge”. The paper credits Rich Caruana and his collaborators with showing earlier that this kind of compression was possible.3 Like distilling a drink, the idea is to keep the strong part and lose the bulk.
A simple example
The plain point: distilled models trade a little skill for a lot of speed.
In 2019, researchers built DistilBERT, a distilled copy of an earlier Google language model called BERT. They reported that the student was much smaller and faster and kept most of the teacher’s ability.4
That is the usual bargain. IBM points out that most big chatbot models are too heavy to run on a phone, and a distilled student can be small enough to.1
Smaller versions of a model usually trade some skill for speed. Distillation is one way such versions are made, so check a small model on your hardest task before relying on it.
Why it matters to you
The plain point: distillation is a normal tool, and also the centre of a fight between AI companies.
- It puts AI on your own devices. Small students are part of how models run on phones and laptops. See Local or cloud.
- It can be done without permission. Anyone who can ask a model millions of questions can collect the answers and train a student on them. IBM notes it has become a way to move skills from large, often proprietary models into smaller, often open ones.1
- It is behind current accusations. In September 2026, Anthropic accused several China-based AI companies of distilling its Claude models at scale. These are allegations; the reporting carried no response from the companies.5 What this does and does not mean for users is covered in Are Chinese AI models safe?
Often confused with
| Word | What it means |
|---|---|
| Distillation | Means: a small model learns by copying a big model’s answers. |
| Fine-tuning | Means: extra training on your own examples to teach a style or task. See fine-tuning meaning. |
| Quantization | Means: shrinking the same model by storing its numbers less precisely. No teacher involved. See quantization meaning. |
| Stealing the weights | Means: taking the model files themselves. Distillation copies behaviour from outside, through the answers. |
Questions people ask
Why do companies use distillation?
To get a model small and fast enough to run cheaply or on a phone. IBM notes that most large commercial language models are too big for that.1
How much does a distilled model lose?
It varies. In one example, DistilBERT was 40% smaller and 60% faster than the model it copied while keeping 97% of its language understanding.4
Is it allowed to distil someone else’s model?
Distillation itself is a standard technique.1 Using a rival’s model as the teacher is disputed: in September 2026 Anthropic accused several China-based AI companies of “distillation attacks”.5 Check the terms of any AI service before using its answers to train a model.
Learn more on this site
- The models at the centre of the dispute: Chinese AI models.
- How the big teacher models are made: How an AI is actually built.
- Models you can download: open weights meaning.
- The basics: the AI glossary and AI explained for complete beginners.
- Related: Small language model (SLM) meaning.
QUICK CHECK · 3 QUESTIONS
Pick an answer to see if you got it. Nothing is recorded.
1. What is distillation in AI?
- Storing a model’s numbers less precisely to make it smaller
- Teaching a model with your own examples so it learns a style
- Training a smaller model to copy the answers of a bigger one
Show the answer
Answer: Training a smaller model to copy the answers of a bigger one. Distillation trains a small student model to copy a big teacher model’s answers, so it keeps much of the skill in a lighter package.
2. What is the usual bargain with a distilled model?
- It is bigger and slower but smarter
- It trades a little skill for a lot of speed
- It is just as good at everything, only smaller
Show the answer
Answer: It trades a little skill for a lot of speed. Distilled models trade a little skill for a lot of speed, and can be small enough to run on a phone.
3. What does the page suggest before you rely on a small model?
- Check it on your hardest task
- Check how many people already use it
- Nothing, because small models are always as good
Show the answer
Answer: Check it on your hardest task. Smaller versions usually trade some skill for speed, so the page says to check a small model on your hardest task first.
Related terms: Clustering · Computer vision · Constitutional AI · Emergent abilities · Fine-tuning · Foundation model
- 1IBM, What is knowledge distillation?, published 16 Apr 2024, read at source 1 Oct 2026“Knowledge distillation is a machine learning technique that aims to transfer the learnings of a large pre-trained model, the ‘teacher model,’ to a smaller ‘student model.’” ↑a“most commercially viable LLMs are too large and computationally demanding to be used locally on mobile phones or other edge devices.” ↑b“knowledge distillation has emerged as an important means of transferring the advanced capabilities of large, often proprietary models to smaller, often open-source models.” ↑c“most commercially viable LLMs are too large and computationally demanding to be used locally on mobile phones or other edge devices.” ↑d“knowledge distillation has emerged as an important means of transferring the advanced capabilities of large, often proprietary models to smaller, often open-source models.” ↑e
- 2Hinton, Vinyals and Dean, Distilling the Knowledge in a Neural Network, arXiv:1503.02531, submitted 9 Mar 2015, read at source 1 Oct 2026“it is possible to compress the knowledge in an ensemble into a single model which is much easier to deploy”. ↑
- 3Hinton, Vinyals and Dean, Distilling the Knowledge in a Neural Network, arXiv:1503.02531, submitted 9 Mar 2015, read at source 12 Oct 2026“Caruana and his collaborators have shown that it is possible to compress the knowledge in an ensemble into a single model”; “by distilling the knowledge” ↑
- 4Sanh et al., DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter, arXiv:1910.01108, submitted 2 Oct 2019, read at source 1 Oct 2026“it is possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster.” ↑a“it is possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster.” ↑b
- 5TechCrunch, Anthropic details distillation campaigns from Alibaba, Moonshot AI, and DeepSeek, 10 Sep 2026, read at source 1 Oct 2026“A new report released Thursday by Anthropic alleged persistent distillation attacks by China-based AI companies”. ↑a“A new report released Thursday by Anthropic alleged persistent distillation attacks by China-based AI companies”. ↑b
Hinton, Vinyals and Dean, Distilling the Knowledge in a Neural Network, arXiv:1503.02531, 9 Mar 2015, read at source 1 Oct 2026.
Sanh et al., DistilBERT, a distilled version of BERT, arXiv:1910.01108, 2 Oct 2019, read at source 1 Oct 2026.
TechCrunch, Anthropic details distillation campaigns from Alibaba, Moonshot AI, and DeepSeek, 10 Sep 2026, read at source 1 Oct 2026. Allegations are reported as allegations.
The comparisons on this page (the chef’s recipe card, distilling a liquid) are this site’s own ways of explaining, not claims from these sources.
Free printable AI cheat sheets, and AI news every week.
Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.
SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK