ONLINEAGENT_OPS 2026.Q3 HOME HUBSMODELSBUILDINGCOMPANIESFILMMAKINGSECURITYLIVING WITH ITSKILLSALL HUBS, ONE TERMINAL → ARTICLES CRAFT RECORD BLOG TOOLS SKILLS SEARCH /
HOME›AI MEANINGS›WHAT ARE SCALING LAWS IN AI?
AI MEANINGS · MEANINGBEGINNER

What are scaling laws in AI? Bigger AI gets better, but for how long?

Scaling laws are the measured pattern that AI models predict text better as they get more data, size and computing. Whether it lasts is debated.

READ6 min
WORDS1,216
SECTIONS7
TYPEMEANING
CHECKED1 OCT 26

Scaling laws means the pattern, measured by researchers, that AI language models get steadily better at predicting text as you give them more data, more size and more computing power. Whether that pattern will keep delivering smarter AI is one of the biggest open arguments in the field.

◈ IN PLAIN TERMS

Think of training for a marathon. Your first extra kilometres a week make a big difference to your time. Later, each small gain needs far more training. The improvement is real and fairly predictable, but each step takes more effort, and it is not clear where the limit is.

In a sentence: “The company is betting on scaling laws: train a bigger model on more data and it should get better again.”

What the term means

The plain point: bigger models, trained on more text with more computing, make fewer prediction mistakes, and the improvement follows a smooth curve.

The best-known version is a 2020 paper by Jared Kaplan and colleagues. They found that a model’s error falls in a regular way as three things grow: the model’s size (its number of parameters), the amount of training text, and the computing power used.1

Two words to unpack. Loss is a score for how wrong the model’s guesses about the next word are; lower is better. A power law is a kind of smooth curve where gains keep coming, but each one needs many times more resources than the last.

“Law” here does not mean a rule of nature or a legal rule. It is a pattern seen in experiments, which could stop holding.

WHAT SCALING LAWS SAY · WHAT THEY DO NOT
A measured pattern about prediction mistakes, not a forecast of intelligence.
WHAT THE PATTERN SHOWS
Yes: More size, text and computing: fewer prediction mistakes
Yes: The gains follow a smooth curve
Yes: Each gain needs many times more than the last
WHAT IT DOES NOT SHOW
No: That AI becomes intelligent
No: That the pattern will keep holding
No: That it is a rule of nature
Reasoning, 1 Oct 2026 — summarises this page’s section on what the term means and the takeaway of its debate section.

A simple example

The plain point: the laws tell labs how to spend a fixed budget of computing.

In 2022, a team led by Jordan Hoffmann tested how to split computing between model size and training text. Their answer was to grow both together, and they found many large models of the day had been given too little text.2 Their model built on that recipe was called Chinchilla.

That is what scaling laws are used for in practice: planning the next, bigger training run before committing months of computing to it.

The debate: will scaling keep working?

The plain point: people agree the pattern has held so far. They disagree on how long it lasts and what it really buys.

ViewThe argument
It can keep goingArgument: Epoch AI, a research group, looked at power, chips, data and timing, and judged that much larger training runs are likely possible by 2030.3
The text may run outArgument: the supply of public human writing is finite. One study projected that training sets will match all of it at some point between 2026 and 2032 if trends continue.4
It measures the wrong thingArgument: Gary Marcus, a long-time critic of this approach, argues that better word prediction is not the same as understanding.5
TAKEAWAY

The laws describe lower prediction error, not intelligence. When someone uses them to forecast human-level AI by a date, that is an argument on top of the data, not the data itself.

Why it matters to you

The plain point: much of today’s AI building is a bet that this pattern keeps holding.

  • It shapes the building. If bigger keeps meaning better, ever more chips and power make sense; if it stops, that logic weakens. What AI actually runs on shows what that means physically.
  • It feeds the predictions. Many forecasts about AI’s future lean on scaling continuing. AGI meaning shows how widely those forecasts differ.
  • It links to the data question. If human text runs short, labs may train on AI-made text, which carries its own risk: model collapse.

Where the word comes from

The phrase is the title of the 2020 paper by Jared Kaplan and colleagues, “Scaling Laws for Neural Language Models”, which opens: “We study empirical scaling laws for language model performance on the cross-entropy loss.”6 The pattern had been reported before: in December 2017, Joel Hestness and colleagues published results that “show power-law generalization error scaling across a breadth of factors”.7 “Law” here means a regular pattern found by measuring, which is what “empirical” means, not a rule of nature that must keep holding.

Questions people ask

What is the Chinchilla scaling law?

A 2022 correction to the earlier advice: train on more data, not just a bigger model. The paper found that “current large language models are significantly undertrained” and that “for every doubling of model size the number of training tokens should also be doubled.”2

Will AI run out of training data?

Possibly. One study projects that models “will be trained on datasets roughly equal in size to the available stock of public human text data between 2026 and 2032”.4 See synthetic data meaning for one proposed answer, and model collapse meaning for its risk.

Do scaling laws mean AI will keep getting smarter?

Not necessarily. The laws measure how well a model predicts text. Critics such as Gary Marcus argue that “the measures that have scaled have not captured what we desperately need to improve: genuine comprehension.”5

Learn more on this site

QUICK CHECK · 3 QUESTIONS

Pick an answer to see if you got it. Nothing is recorded.

1. What do scaling laws describe?

  • A rule of nature that AI must always obey
  • A legal limit on how large AI models can be
  • A pattern where more size, text and computing cut mistakes
Show the answer

Answer: A pattern where more size, text and computing cut mistakes. The page says scaling laws are a measured pattern: bigger models trained on more text with more computing make fewer prediction mistakes.

2. What did the team behind Chinchilla find?

  • Model size and training text should grow together
  • Only the model size needs to grow
  • Less computing works better than more
Show the answer

Answer: Model size and training text should grow together. The page says the Chinchilla team found both should grow together, and that many large models had been given too little text.

3. Do scaling laws prove that AI will become more intelligent?

  • Yes, the curve shows intelligence rising each year
  • No, they describe fewer prediction mistakes, not intelligence
  • Yes, but only once the text runs out
Show the answer

Answer: No, they describe fewer prediction mistakes, not intelligence. The page says the laws describe lower prediction error, not intelligence, and forecasts of human-level AI are arguments on top of the data.

Related terms: Pretraining · Reinforcement learning · RLHF · Singularity (technological singularity) · Superintelligence (ASI) · Supervised learning

SOURCES
  1. 1Kaplan et al., Scaling Laws for Neural Language Models, arXiv:2001.08361, submitted 23 Jan 2020, read at source 1 Oct 2026“The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude.” ↑
  2. 2Hoffmann et al., Training Compute-Optimal Large Language Models, arXiv:2203.15556, submitted 29 Mar 2022, read at source 1 Oct 2026“current large language models are significantly undertrained” and “for every doubling of model size the number of training tokens should also be doubled.” ↑a“current large language models are significantly undertrained” and “for every doubling of model size the number of training tokens should also be doubled.” ↑b
  3. 3Epoch AI, Can AI scaling continue through 2030?, 20 Aug 2024, read at source 1 Oct 2026“our analysis suggests that training runs of around 2e29 FLOP are likely possible by 2030.” ↑
  4. 4Villalobos et al., Will we run out of data? Limits of LLM scaling based on human-generated data, arXiv:2211.04325, revised 4 Jun 2024, read at source 1 Oct 2026“models will be trained on datasets roughly equal in size to the available stock of public human text data between 2026 and 2032”. ↑a“models will be trained on datasets roughly equal in size to the available stock of public human text data between 2026 and 2032”. ↑b
  5. 5Gary Marcus, Deep Learning Is Hitting a Wall, Nautilus, 10 Mar 2022, read at source 1 Oct 2026“the measures that have scaled have not captured what we desperately need to improve: genuine comprehension.” ↑a“the measures that have scaled have not captured what we desperately need to improve: genuine comprehension.” ↑b
  6. 6Kaplan et al., Scaling Laws for Neural Language Models, arXiv:2001.08361, submitted 23 Jan 2020, read at source 12 Oct 2026“We study empirical scaling laws for language model performance on the cross-entropy loss.” ↑
  7. 7Hestness et al., Deep Learning Scaling is Predictable, Empirically, arXiv:1712.00409, submitted 1 Dec 2017, read at source 12 Oct 2026results “show power-law generalization error scaling across a breadth of factors” ↑

Kaplan et al., Scaling Laws for Neural Language Models, arXiv:2001.08361, 23 Jan 2020, read at source 1 Oct 2026.

Hoffmann et al., Training Compute-Optimal Large Language Models, arXiv:2203.15556, 29 Mar 2022, read at source 1 Oct 2026.

Villalobos et al., Will we run out of data?, arXiv:2211.04325, revised 4 Jun 2024, read at source 1 Oct 2026.

The comparison on this page (marathon training) is this site’s own way of explaining, not a claim from these sources.

Free printable AI cheat sheets, and AI news every week.

Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.

SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK
aipromptgeneer://$ about this siteAI PROMPT GENEERAI explained in plain English. Every claim links to its source, with the date it was checked.NEWSLETTERWhat changed in AI each week, in plain English.SUBSCRIBE FREE ↗AI PROMPT GENEER · MACHINE ASSISTANCE DISCLOSEDEVERY ARTICLE CARRIES ITS OWN CHECKED DATE