Red teaming means deliberately attacking an AI system, with permission, to find out how it can be made to fail before real users or bad actors find out. Testers try to trick it into harmful, false or private answers, and the makers then fix what they find.
Think of a shop that hires someone to try shoplifting, on purpose and with the manager’s blessing. Every trick that works shows a gap in security, and the shop closes it before real thieves arrive. AI red teamers do the same, with words instead of pockets.
In a sentence: “Before the launch, the company paid outside experts to red-team the chatbot and try to make it give dangerous advice.”
What the term means
The name comes from security testing, where the “red team” plays the attacker. The US government’s security glossary, published by NIST, defines a red team as a group of people allowed to play a likely attacker and try its methods against an organisation’s security.1
With AI chatbots, the word has stretched to cover many kinds of probing and testing. Microsoft’s guide says that since large language models arrived, the term has moved “beyond traditional cybersecurity”.2
A simple example
Testing a health chatbot before launch. A team of testers spends a week trying to break it. One pretends to be a doctor to get dangerous dosage advice. Another hides instructions inside a pasted document. A third simply asks normal questions in a worried, rambling way. Every bad answer is written down, fixed, and tested again.
That mix is deliberate. Microsoft advises using experts in the subject, such as medical experts for a health chatbot, alongside ordinary users.3
Why it matters to you
It is a big part of why chatbots refuse some requests. When a tester finds a way through, the gap is patched and tested again. This site’s page on how AI is tested calls red teams the layer most responsible for the guardrails you meet day to day.
“Red-teamed” does not mean “safe”. Testers can only find what they think to try, so new tricks, such as a fresh jailbreak, still turn up after launch. Microsoft says red teaming finds problems but does not measure how common they are.3 When a company says a model was red-teamed, read it as a starting point, not a guarantee.
Often confused with
| Term | How it differs |
|---|---|
| Jailbreaking | Differs: the same tricks, but a jailbreak is done to misuse the AI. Red teaming is done with permission, to fix it. |
| Benchmark | Differs: a benchmark is a fixed test with known answers that measures skill. Red teaming is open-ended attack that hunts for failures. |
| Blue team | Opposite side: in security testing the blue team defends while the red team attacks. |
Where the word comes from
The word comes from military war games. During the Cold War, the RAND Corporation ran military simulations for the US government in which the red team stood for the Soviet Union and the blue team for the United States.4 The same paper traces the habit of playing the enemy back to a tabletop war game, Kriegsspiel, that the Prussian army adopted in 1812.5 Computer security borrowed the term, and with chatbots it now covers many kinds of probing and testing, as Microsoft’s guide notes above.
Questions people ask
What is the difference between a red team and a blue team?
The red team attacks; the blue team defends. The US security glossary published by NIST describes a blue team as “The group responsible for defending an enterprise’s use of information systems by maintaining its security posture”.6
Does red teaming make an AI safe?
No single test can. Microsoft says red teaming “is not a replacement for systematic measurement and rigorous mitigation work”: it shows that a problem exists, not how often it happens.3
Who can be an AI red teamer?
Not only security experts. Microsoft advises mixing subject experts with people “who are ordinary users of your application system and haven’t been involved in its development”, because they notice different problems.3
Learn more on this site
- How AI is tested — benchmarks, red teams, and the questions testing leaves open.
- Prompt injection meaning — one of the main attacks red teamers try.
- Guardrails meaning — the rules red teaming puts to the test.
- The AI glossary (see “Red-teaming”) and AI for beginners.
QUICK CHECK · 3 QUESTIONS
Pick an answer to see if you got it. Nothing is recorded.
1. What makes red teaming different from jailbreaking?
- Red teaming is done with permission, to fix the AI
- Red teaming only uses fixed tests with known answers
- Red teaming is only done after users complain
Show the answer
Answer: Red teaming is done with permission, to fix the AI. The page says a jailbreak uses the same tricks to misuse the AI, while red teaming is done with permission so the makers can fix it.
2. A company says its chatbot was red-teamed. How should you read that?
- As proof that the chatbot is fully safe
- As a score showing how often it fails
- As a starting point, not a guarantee of safety
Show the answer
Answer: As a starting point, not a guarantee of safety. Testers can only find what they think to try, so the page says red-teamed does not mean safe.
3. Why does Microsoft advise using ordinary users as testers too?
- They already know how the AI was built
- They notice different problems from the experts
- They can test much faster than experts
Show the answer
Answer: They notice different problems from the experts. The page says Microsoft advises mixing subject experts with ordinary users who were not involved in building it, because they notice different problems.
Related terms: Jailbreak · Prompt injection · Provenance / C2PA · Shadow AI · Slop · Voice cloning
- 1NIST Computer Security Resource Center, red team – Glossary, citing CNSSI 4009-2022, read at source 1 Oct 2026“A group of people authorized and organized to emulate a potential adversary’s attack or exploitation capabilities against an enterprise’s security posture.” ↑
- 2Microsoft Learn, Planning red teaming for large language models (LLMs) and their applications, 13 May 2026, read at source 1 Oct 2026“With the rise of LLMs, the term has extended beyond traditional cybersecurity and evolved in common usage to describe many kinds of probing, testing, and attacking of AI systems.” ↑
- 3Microsoft Learn, Planning red teaming for LLMs, read at source 1 Oct 2026“if you’re designing a chatbot to help health care providers, medical experts can help identify risks in that domain”, and red teamers “who are ordinary users of your application system and haven’t been involved in its development can bring valuable perspectives”. ↑ared teaming “is not a replacement for systematic measurement and rigorous mitigation work. It is important that people do not interpret specific examples as a metric for the pervasiveness of that harm.” ↑bred teaming “is not a replacement for systematic measurement and rigorous mitigation work. It is important that people do not interpret specific examples as a metric for the pervasiveness of that harm.” ↑cred teamers “who are ordinary users of your application system and haven’t been involved in its development can bring valuable perspectives” ↑d
- 4Majumdar, Pendleton and Gupta, Red Teaming AI Red Teaming, arXiv:2507.05538, submitted 7 Jul 2025, §2.1 “Military Origins”, read at source 12 Oct 2026“the RAND Corporation conducted military simulations for the US government”; “‘red team’ and the color red were used to represent the Soviet Union, while ‘blue team’ and blue represented the United States” ↑
- 5Majumdar, Pendleton and Gupta, Red Teaming AI Red Teaming, arXiv:2507.05538, read at source 12 Oct 2026“The Prussian army adopted Kriegsspiel (literally ‘wargame’ in German) in 1812” ↑
- 6NIST Computer Security Resource Center, blue team – Glossary, citing CNSSI 4009-2022, read at source 12 Oct 2026“The group responsible for defending an enterprise’s use of information systems by maintaining its security posture” ↑
NIST Computer Security Resource Center, red team – Glossary (definition from CNSSI 4009-2022), read at source 1 Oct 2026.
The comparisons and examples on this page (the hired shoplifter, the health chatbot test week) are this site’s own ways of explaining, not claims from these sources.
Free printable AI cheat sheets, and AI news every week.
Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.
SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK