Mixture of experts, or MoE, means an AI model split into many smaller parts, called experts, where only a few of them do the work for each piece of text. The model can be very large overall but answer faster, because most of it rests at any moment.
Think of a big hospital. It has dozens of specialists, but a receptionist sends each patient to just one or two of them. Nobody needs every doctor for every visit, so patients are seen quickly. But all the doctors still have to be in the building, ready, all day. An MoE model works the same way: the receptionist is called the router.
In a sentence: “The model is huge on paper, but because it is a mixture of experts, only a small part of it works on each word, so replies still come back quickly.”
What the term means
The plain point: the model is divided into specialist parts, and a router picks which parts handle each input.
IBM describes the setup as separate sub-networks, each good at part of the data, working together.1
The router is itself trained. It learns which experts suit which inputs and switches on only those.2 “Expert” is just the name for one of these parts. It is not a separate AI with a job title, and you never choose it yourself.
A 2017 paper by Noam Shazeer and colleagues showed the idea working at large scale for language and translation.3
A simple example
The plain point: for every small piece of text, only a couple of experts switch on.
IBM uses Mistral’s Mixtral 8x7B model as its example. Each layer of the model has eight experts. For each token (a small chunk of text), the router picks two of the eight and combines their results.2
So the other six sit idle for that token. That is where the speed comes from.
Why it matters to you
The plain point: MoE makes big models quicker to answer, but it does not make them small.
- Faster answers. IBM says using only the experts needed lets big models run faster when answering, a stage called inference.2
- The whole model still needs memory. Every expert must be loaded and ready, even the idle ones. This matters if you want to run a model on your own computer; see VRAM meaning.2
- You will see it in model names and cards. A big MoE model can still be far too large for a home machine. Run AI on your own computer gives a real case, and Gemma 4 vs Muse Glimmer covers a model family with both an MoE version and a dense one.
When a model card says MoE, look at two numbers: the total size, which decides the memory you need, and the active size, which decides how quickly it answers.
Often confused with
| Word | What it means |
|---|---|
| Mixture of experts | Means: one model with many parts inside, where a router switches on a few per token. |
| Dense model | Means: the usual design, where the whole model works on every token.2 |
| Several AI models working together | Means: separate models or agents passing work between them. MoE all happens inside one model. |
| Distillation | Means: making a genuinely smaller model by training it to copy a bigger one. See distillation meaning. |
Where the word comes from
The idea is older than today’s chatbots. IBM traces it to a 1991 University of Toronto paper: “the core premise behind MoE models originates from the 1991 paper ‘Adaptive Mixture of Local Experts’”, which proposed training one system made of separate networks.4 In 2017, Noam Shazeer and colleagues scaled the idea up to very large neural networks, with “A trainable gating network” picking a few experts for each example.3
Questions people ask
Does mixture of experts make a model smaller?
No. It makes each answer cheaper to produce, but the whole model still has to fit in memory. IBM notes that “the entirety of the model’s parameters must be loaded into memory” (IBM’s page has a typo, “most”, for “must”).2
Is a mixture of experts several AI models working together?
No, it is one model with separate parts inside. IBM’s definition: MoE “divides an artificial intelligence (AI) model into separate sub-networks (or ‘experts’)”.1
What does the router do in a mixture of experts?
It decides which experts work on each piece of text. IBM describes it as “a gating network (or router)” trained “to activate only the specific expert(s) best suited to a given input”.2
Learn more on this site
- Running models at home: local AI meaning, Local or cloud and Run AI on your own computer.
- The chips and memory underneath: What AI actually runs on.
- The design underneath chatbot models: transformer meaning.
- The basics: the AI glossary and AI explained for complete beginners.
QUICK CHECK · 3 QUESTIONS
Pick an answer to see if you got it. Nothing is recorded.
1. What is a mixture of experts model?
- Several separate AI models passing work between them
- A team of people who check an AI’s answers
- One model split into many parts, where only a few work on each piece of text
Show the answer
Answer: One model split into many parts, where only a few work on each piece of text. A router picks a few of the model’s parts, called experts, for each piece of text, while the rest rest.
2. What does the router do?
- It decides which experts work on each piece of text
- It connects the model to the internet
- It lets you choose your favourite expert
Show the answer
Answer: It decides which experts work on each piece of text. The router is itself trained to switch on only the experts that suit each input; you never choose them yourself.
3. Does a mixture of experts make a model smaller?
- Yes, it only needs memory for two experts
- No, it answers faster, but the whole model still has to fit in memory
- Yes, it shrinks the model each time it answers
Show the answer
Answer: No, it answers faster, but the whole model still has to fit in memory. Every expert must be loaded and ready, even the idle ones, so check the total size for memory and the active size for speed.
Related terms: Frontier model · Inference · Latent space · Model collapse · Open weights · Overfitting
- 1IBM, What is mixture of experts?, undated page, read at source 1 Oct 2026“Mixture of experts (MoE) is a machine learning approach that divides an artificial intelligence (AI) model into separate sub-networks (or ‘experts’), each specializing in a subset of the input data, to jointly perform a task.” ↑a“Mixture of experts (MoE) is a machine learning approach that divides an artificial intelligence (AI) model into separate sub-networks (or ‘experts’), each specializing in a subset of the input data, to jointly perform a task.” ↑b
- 2IBM, What is mixture of experts?, read at source 1 Oct 2026models train “a gating network (or router) to activate only the specific expert(s) best suited to a given input.” ↑a“For every token, at each layer, a router network selects two of those eight experts to process the data.” ↑bit achieves “faster performance during inference time” by “selectively activating only the specific experts needed for a given task”. ↑c“the entirety of the model’s parameters most be loaded into memory, meaning that the computational efficiency enjoyed by SMoEs in most regards does not apply to their RAM/VRAM requirements.” (“most” is IBM’s typo for “must”.) ↑d“In a typical deep learning model—what in this context is referred to as a dense model—the entire network is executed in order to process any and all inputs.” ↑e“the entirety of the model’s parameters most be loaded into memory, meaning that the computational efficiency enjoyed by SMoEs in most regards does not apply to their RAM/VRAM requirements.” ↑fmodels train “a gating network (or router) to activate only the specific expert(s) best suited to a given input.” ↑g
- 3Shazeer et al., Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer, arXiv:1701.06538, submitted 23 Jan 2017, read at source 1 Oct 2026“A trainable gating network determines a sparse combination of these experts to use for each example.” ↑a“A trainable gating network determines a sparse combination of these experts to use for each example.” ↑b
- 4IBM, What is mixture of experts?, undated page, read at source 12 Oct 2026“the core premise behind MoE models originates from the 1991 paper ‘Adaptive Mixture of Local Experts.’”; the paper “proposed training an AI system composed of separate networks” ↑
Shazeer et al., Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer, arXiv:1701.06538, 23 Jan 2017, read at source 1 Oct 2026.
The comparison on this page (the hospital and its receptionist) is this site’s own way of explaining, not a claim from these sources.
Free printable AI cheat sheets, and AI news every week.
Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.
SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK