Scaling laws means the pattern, measured by researchers, that AI language models get steadily better at predicting text as you give them more data, more size and more computing power. Whether that pattern will keep delivering smarter AI is one of the biggest open arguments in the field.
Think of training for a marathon. Your first extra kilometres a week make a big difference to your time. Later, each small gain needs far more training. The improvement is real and fairly predictable, but each step takes more effort, and it is not clear where the limit is.
In a sentence: “The company is betting on scaling laws: train a bigger model on more data and it should get better again.”
What the term means
The plain point: bigger models, trained on more text with more computing, make fewer prediction mistakes, and the improvement follows a smooth curve.
The best-known version is a 2020 paper by Jared Kaplan and colleagues. They found that a model’s error falls in a regular way as three things grow: the model’s size (its number of parameters), the amount of training text, and the computing power used.1
Two words to unpack. Loss is a score for how wrong the model’s guesses about the next word are; lower is better. A power law is a kind of smooth curve where gains keep coming, but each one needs many times more resources than the last.
“Law” here does not mean a rule of nature or a legal rule. It is a pattern seen in experiments, which could stop holding.
A simple example
The plain point: the laws tell labs how to spend a fixed budget of computing.
In 2022, a team led by Jordan Hoffmann tested how to split computing between model size and training text. Their answer was to grow both together, and they found many large models of the day had been given too little text.2 Their model built on that recipe was called Chinchilla.
That is what scaling laws are used for in practice: planning the next, bigger training run before committing months of computing to it.
The debate: will scaling keep working?
The plain point: people agree the pattern has held so far. They disagree on how long it lasts and what it really buys.
| View | The argument |
|---|---|
| It can keep going | Argument: Epoch AI, a research group, looked at power, chips, data and timing, and judged that much larger training runs are likely possible by 2030.3 |
| The text may run out | Argument: the supply of public human writing is finite. One study projected that training sets will match all of it at some point between 2026 and 2032 if trends continue.4 |
| It measures the wrong thing | Argument: Gary Marcus, a long-time critic of this approach, argues that better word prediction is not the same as understanding.5 |
The laws describe lower prediction error, not intelligence. When someone uses them to forecast human-level AI by a date, that is an argument on top of the data, not the data itself.
Why it matters to you
The plain point: much of today’s AI building is a bet that this pattern keeps holding.
- It shapes the building. If bigger keeps meaning better, ever more chips and power make sense; if it stops, that logic weakens. What AI actually runs on shows what that means physically.
- It feeds the predictions. Many forecasts about AI’s future lean on scaling continuing. AGI meaning shows how widely those forecasts differ.
- It links to the data question. If human text runs short, labs may train on AI-made text, which carries its own risk: model collapse.
Where the word comes from
The phrase is the title of the 2020 paper by Jared Kaplan and colleagues, “Scaling Laws for Neural Language Models”, which opens: “We study empirical scaling laws for language model performance on the cross-entropy loss.”6 The pattern had been reported before: in December 2017, Joel Hestness and colleagues published results that “show power-law generalization error scaling across a breadth of factors”.7 “Law” here means a regular pattern found by measuring, which is what “empirical” means, not a rule of nature that must keep holding.
Questions people ask
What is the Chinchilla scaling law?
A 2022 correction to the earlier advice: train on more data, not just a bigger model. The paper found that “current large language models are significantly undertrained” and that “for every doubling of model size the number of training tokens should also be doubled.”2
Will AI run out of training data?
Possibly. One study projects that models “will be trained on datasets roughly equal in size to the available stock of public human text data between 2026 and 2032”.4 See synthetic data meaning for one proposed answer, and model collapse meaning for its risk.
Do scaling laws mean AI will keep getting smarter?
Not necessarily. The laws measure how well a model predicts text. Critics such as Gary Marcus argue that “the measures that have scaled have not captured what we desperately need to improve: genuine comprehension.”5
Learn more on this site
- Where scale enters the build: How an AI is actually built.
- Why only a few labs can afford it: Who builds AI.
- A related debate: emergent abilities meaning.
- Why scores need care: How AI is tested.
- The basics: the AI glossary and AI explained for complete beginners.
QUICK CHECK · 3 QUESTIONS
Pick an answer to see if you got it. Nothing is recorded.
1. What do scaling laws describe?
- A rule of nature that AI must always obey
- A legal limit on how large AI models can be
- A pattern where more size, text and computing cut mistakes
Show the answer
Answer: A pattern where more size, text and computing cut mistakes. The page says scaling laws are a measured pattern: bigger models trained on more text with more computing make fewer prediction mistakes.
2. What did the team behind Chinchilla find?
- Model size and training text should grow together
- Only the model size needs to grow
- Less computing works better than more
Show the answer
Answer: Model size and training text should grow together. The page says the Chinchilla team found both should grow together, and that many large models had been given too little text.
3. Do scaling laws prove that AI will become more intelligent?
- Yes, the curve shows intelligence rising each year
- No, they describe fewer prediction mistakes, not intelligence
- Yes, but only once the text runs out
Show the answer
Answer: No, they describe fewer prediction mistakes, not intelligence. The page says the laws describe lower prediction error, not intelligence, and forecasts of human-level AI are arguments on top of the data.
Related terms: Pretraining · Reinforcement learning · RLHF · Singularity (technological singularity) · Superintelligence (ASI) · Supervised learning
- 1Kaplan et al., Scaling Laws for Neural Language Models, arXiv:2001.08361, submitted 23 Jan 2020, read at source 1 Oct 2026“The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude.” ↑
- 2Hoffmann et al., Training Compute-Optimal Large Language Models, arXiv:2203.15556, submitted 29 Mar 2022, read at source 1 Oct 2026“current large language models are significantly undertrained” and “for every doubling of model size the number of training tokens should also be doubled.” ↑a“current large language models are significantly undertrained” and “for every doubling of model size the number of training tokens should also be doubled.” ↑b
- 3Epoch AI, Can AI scaling continue through 2030?, 20 Aug 2024, read at source 1 Oct 2026“our analysis suggests that training runs of around 2e29 FLOP are likely possible by 2030.” ↑
- 4Villalobos et al., Will we run out of data? Limits of LLM scaling based on human-generated data, arXiv:2211.04325, revised 4 Jun 2024, read at source 1 Oct 2026“models will be trained on datasets roughly equal in size to the available stock of public human text data between 2026 and 2032”. ↑a“models will be trained on datasets roughly equal in size to the available stock of public human text data between 2026 and 2032”. ↑b
- 5Gary Marcus, Deep Learning Is Hitting a Wall, Nautilus, 10 Mar 2022, read at source 1 Oct 2026“the measures that have scaled have not captured what we desperately need to improve: genuine comprehension.” ↑a“the measures that have scaled have not captured what we desperately need to improve: genuine comprehension.” ↑b
- 6Kaplan et al., Scaling Laws for Neural Language Models, arXiv:2001.08361, submitted 23 Jan 2020, read at source 12 Oct 2026“We study empirical scaling laws for language model performance on the cross-entropy loss.” ↑
- 7Hestness et al., Deep Learning Scaling is Predictable, Empirically, arXiv:1712.00409, submitted 1 Dec 2017, read at source 12 Oct 2026results “show power-law generalization error scaling across a breadth of factors” ↑
Kaplan et al., Scaling Laws for Neural Language Models, arXiv:2001.08361, 23 Jan 2020, read at source 1 Oct 2026.
Hoffmann et al., Training Compute-Optimal Large Language Models, arXiv:2203.15556, 29 Mar 2022, read at source 1 Oct 2026.
Villalobos et al., Will we run out of data?, arXiv:2211.04325, revised 4 Jun 2024, read at source 1 Oct 2026.
The comparison on this page (marathon training) is this site’s own way of explaining, not a claim from these sources.
Free printable AI cheat sheets, and AI news every week.
Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.
SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK