HOW AN AI IS BUILT

It starts as sand. Purified into silicon, sliced into wafers, printed with circuitry at a scale measured in billionths of a metre, then packaged into the specialised processors that do one thing extraordinarily fast: multiply enormous grids of numbers.

A frontier training run occupies tens of thousands of these chips continuously for months. They are not general computers in any everyday sense — they are arithmetic engines, and everything above this layer is that arithmetic wearing a costume.

This layer also explains the energy question. Each chip generates heat that must be removed, which is why AI is a buildings-and-power-grid problem as much as a software one. The International Energy Agency projected global datacentre electricity demand more than doubling by 2030 to around 945 terawatt-hours — slightly more than Japan's total electricity consumption today — in its April 2025 Energy and AI report. The agency has since updated that central projection to about 950 TWh in 2030, roughly double the 485 TWh of 2025.Source: IEA, Energy and AI (World Energy Outlook Special Report), April 2025, read at source 17 Sep 2026 in the report PDF (iea.org itself refuses automated requests): “Data centre electricity consumption is set to more than double to around 945 TWh by 2030. This is slightly more than Japan's total electricity consumption today.” Carbon Brief renders the comparison as “equivalent to the current electricity demand of Japan”; the IEA's own wording is used here. The update was read at source 17 Sep 2026 in the IEA's April 2026 Key Questions on Energy and AI (PDF): “Our updated projections see electricity consumption from data centres roughly doubling from 485 TWh in 2025 to 950 TWh in 2030, accounting for around 3% of global electricity demand by that date.” The same report says the Base Case sees consumption “growing from around 415 TWh in 2024 to around 950 TWh in 2030”, and that data centres “accounted for 1.5% of global electricity consumption in 2025”.

Trillions of words, images and lines of code — scraped from the open web, licensed from publishers, bought, and increasingly generated by other models. Then filtered: duplicates removed, low-quality pages dropped, some categories excluded entirely.

This layer decides what the finished system can possibly know, and it carries most of the field's unresolved fights: who owns the text, whether consent was given, what happens when models begin training on their own output. What goes in decides what can come out — and unlike the layers above it, this one cannot be fixed later.

And the quarry is changing composition. A growing share of what is available to scrape was itself machine-generated — roughly half of newly published articles, by the figures in the quarterly record.Graphite, “AI Now Writes as Many Online Articles as Humans”, May 2026, read at source 17 Sep 2026: “since Q1 2025 the percentage of primarily AI-generated articles has plateaued at roughly 50%”. A count of English-language articles in Common Crawl, not of everything a lab might train on. Research published in Nature in 2024 showed that models trained repeatedly on their own outputs degrade: rare cases vanish first, then variety collapses. In the paper's own terms, early model collapse is the model "losing information about the tails of the distribution", and late collapse is convergence "to a distribution that carries little resemblance to the original one, often with substantially reduced variance". The overall score is the wrong place to look for it: with a tenth of the real data retained, the authors measured "only minor degradation of performance" while the tails were going. One of the authors, Yarin Gal, put it as an AI echo chamber.

This is not a prediction of doom. Labs filter aggressively, license human-written corpora, and use generated data deliberately where it helps. But it does mean the economics of the open web are now a training-data question: if human pages stop being viable to produce, the quarry that every layer above depends on gets worse.Shumailov et al., "AI models collapse when trained on recursively generated data", Nature 631, 755–759 (2024); body read at source in the open-access copy at Edinburgh Research Explorer on 11 Sep 2026: "At first, the tails (low-probability events) begin to disappear". CORRECTED 11 Sep 2026 — this page previously said performance on common cases "can look fine or even improve while the tails rot away", and attributed to an author the line that the surprise is "how fast it starts and how elusive it is". Neither is in the paper: it reports that "Both training regimes lead to degraded performance in our models", and the author quote could not be traced to any source that could be opened. The Gal quote is from Oxford's Department of Computer Science: "Model collapse threatens to create an AI echo chamber." The economic and legal half — including the copyright rulings that now turn on how training data was obtained — on AI and the open web.

Here is the whole mechanism, honestly: the system is shown a piece of text with the next part hidden, guesses what comes next, and has billions of internal numbers nudged slightly whenever it guesses wrong. Repeat for months.

No engineer writes the rules. Grammar, arithmetic, the shape of an argument, the conventions of a recipe — all of it emerges from correction at scale. Which is also why it is so hard to point to where a specific skill lives inside the finished model, and why testing it is genuinely hard rather than merely tedious.

The result is a set of frozen numbers — the weights. That file is the model. Everything you have ever seen an AI do came out of arithmetic performed on numbers that were settled during this stage and have not changed since.

A freshly pretrained model is capable and unusable: it will continue any text at all, helpfully or otherwise. Alignment is the work of shaping behaviour afterwards — people ranking outputs, written principles the model is trained against, red teams paid to make it fail, and evaluations run before anything ships.

Two honest caveats belong here. First, this shapes behaviour, not beliefs — the model has none. Second, it is demonstrably incomplete: people keep finding ways to talk models out of their guidelines, which is why alignment is an active research field rather than a solved checkbox.

The part that gets left out: this layer runs on people. Ranking outputs, writing demonstrations, labelling harmful material and probing for failures are human jobs, done at scale, frequently through outsourced contracts and often by workers in lower-income countries. Some of it involves sustained exposure to the worst material a model might be asked to reproduce.

It is worth naming for two reasons. Practically: the behaviour you experience as a model's "values" was substantially decided by the judgement of people whose names you will never see. Honestly: a technology routinely described as automating work is, at this layer, an employer of a workforce whose conditions are rarely discussed by the people describing it.

Chat windows, APIs, copilots inside other software. Almost nothing here is the model itself — it is packaging: what the system is told before you type, how much of your conversation it can still see, which tools it may reach for, what it refuses.

Worth knowing when a product feels dramatically better or worse than another using the same underlying model: you are usually comparing the wrapping, not the mind. ChatGPT's launch invented nothing fundamental — the architecture was five years old. What changed was a text box anyone could type into.

Systems that do not merely answer but act: browse, write and run code, call other software, take multi-step tasks and pursue them. Structurally this is the same predictor from Layer 03, given tools and a loop.

The reason this layer gets its own name is that the failure modes change. A wrong answer is a wrong answer; a wrong action books the flight, sends the message, deletes the file. Everything unresolved in Layer 04 becomes more consequential here, which is precisely why this floor is still under inspection.