AI biasWhen an AI system works better for some people than others or repeats unfair stereotypes, usually because of the data and human choices behind it.
FULL MEANING →
AI winterA period when disappointment with AI makes funding and interest dry up after a boom of big promises. There have been two: in the 1970s, and from the late 1980s into the 1990s.
FULL MEANING →
AlignmentThe work of making an AI behave the way people intend: helpful, honest and refusing to cause harm.
FULL MEANING →
AttentionThe part of a transformer that weighs how much each word, or part of the input, matters to every other part. It is why the surrounding text changes the answer.
FULL MEANING →
BackpropagationHow a neural network learns from its mistakes: after it makes guesses and measures how wrong they were, it works backwards through its layers to decide which connections to strengthen and which to weaken. It is how gradient descent is carried out in neural networks.
QUICK DEFINITION
Base modelA model that has finished its first big round of training but has not yet been tuned for a particular job. It is the starting point that fine-tuning builds on.
QUICK DEFINITION
CheckpointA saved snapshot of a model at a point in training. What you actually download when you download a model.
FULL MEANING →
Classification and regressionTwo kinds of prediction. Classification picks a category (is this email spam? which language is this sentence in?); regression predicts a number (what will this house sell for? how much rain will fall?).
QUICK DEFINITION
ClusteringGetting an AI to sort examples into groups of similar items, usually from data that has no labels; a person can then decide what each group means.
FULL MEANING →
Computer visionThe part of AI that lets computers make sense of pictures and video, such as spotting objects, faces, text or movement.
FULL MEANING →
Constitutional AIA way of training an AI with a written list of principles (a “constitution”) that the AI uses to critique and improve its own answers, so fewer human ratings are needed. Anthropic developed it for its Claude models.
FULL MEANING →
DatasetA collection of data gathered together, often laid out like a spreadsheet. In AI, part of a dataset is used to train a model and other parts are kept back to check it.
QUICK DEFINITION
DistillationTraining a smaller model to copy a larger one, so it is cheaper and faster to run, usually with some loss in ability.
FULL MEANING →
Emergent abilitiesSkills that larger AI models seem to gain suddenly while smaller ones lack them; researchers still dispute whether the jumps are real or a measuring effect.
FULL MEANING →
EpochOne full pass of training through the whole set of training examples, in which the model has worked through every example once.
QUICK DEFINITION
Fine-tuningContinuing a model training on your own examples so it adopts a style, format, or domain. Heavier than prompting; lighter than pretraining.
FULL MEANING →
Foundation modelA big, general AI model trained on a huge amount of varied data, used as the starting point for many different tools and jobs.
FULL MEANING →
Gradient descentThe step-by-step method used to shrink the loss during training: it nudges the model’s internal numbers (its weights) a little at a time in the direction that makes the errors smaller, again and again.
QUICK DEFINITION
InferenceUsing a trained model to get an answer, as opposed to training it. Every question you ask a chatbot is inference.
FULL MEANING →
Instruction tuningExtra training (a kind of fine-tuning) on many example instructions covering lots of different tasks, so the model gets better at doing what it is asked, even for requests that come with no worked example.
QUICK DEFINITION
Labelled dataExamples that come with the right answer attached, such as emails each marked “spam” or “not spam”. A model learns from labelled examples, then makes guesses about new ones that have no label.
QUICK DEFINITION
Latent spaceThe squeezed-down form a model turns data into on the inside: each item is boiled down to a compact set of numbers that keeps only its most important features, and similar items end up close together.
FULL MEANING →
Loss functionThe scoring rule used while an AI is trained: it works out how far the model’s guesses are from the right answers, giving a lower score (the “loss”) for better guesses. Training tries to make that score as small as possible.
QUICK DEFINITION
Mixture of Experts (MoE)A design where a big model is split into specialist parts and only a few are used for each request, so it answers faster; the whole model still has to fit in memory.
FULL MEANING →
Model collapseAI getting worse when it is trained mostly on AI-made content instead of human-made data. Studies differ on how far keeping real data in the mix prevents it.
FULL MEANING →
Open weightsA model whose trained parameters are downloadable. Licenses vary from fully permissive to research-only; the license, not the adjective open, is what matters.
FULL MEANING →
OverfittingWhen an AI learns its practice examples so exactly that it does well on them but badly on anything new, like memorising answers instead of the subject.
FULL MEANING →
PretrainingThe first and biggest stage of training: the model learns language and facts about the world by predicting the next token across a vast amount of text, before later training teaches it to follow instructions.
FULL MEANING →
RLHFReinforcement learning from human feedback: people rate the AI’s answers, and the model is trained towards the ones they preferred.
FULL MEANING →
Scaling lawsThe pattern researchers measured: models made bigger, with more data and computing power, make fewer prediction errors. Whether that keeps turning into more capability is debated.
FULL MEANING →
Superintelligence (ASI)A hypothetical AI far smarter than the best humans at almost everything. Some AI companies say they are building it; whether and when it could exist is unknown.
FULL MEANING →
Synthetic dataTraining data made by computers rather than collected from people. Useful when used carefully; risky when AI-made content quietly replaces real data.
FULL MEANING →
Training dataThe examples an AI studies before anyone uses it, such as text or pictures, whose patterns shape everything it can later say or do.
FULL MEANING →
Training runOne full session of training a model on a large amount of data, using many specialised computer chips: the long and costly part of making an AI.
FULL ARTICLE →
Transfer learningReusing what a model learned on one task to help it with another, such as carrying knowledge from a task with plenty of example data over to one with very little.
QUICK DEFINITION
Turing testAlan Turing’s 1950 “imitation game”: can a machine chat so much like a person that a judge cannot tell which is which? Passing shows it sounds human, not that it understands.
FULL MEANING →
Unsupervised learningTeaching an AI with data that has no answers attached, so it finds patterns and groups by itself; a person then decides what the groups mean.
FULL MEANING →
Vision-language model (VLM)An AI model that takes in pictures (and often video) as well as text and answers in words, such as describing a photo, reading a document or answering questions about what it sees.
FULL MEANING →
WeightsThe learned numbers that make up a trained model. Apps and chat windows are just ways of reaching them.
FULL MEANING →
World modelAn AI system that has learned how a place or situation behaves well enough to simulate it and predict what happens next. Used to train robots and agents, and to generate worlds people can explore.
FULL MEANING →