A rate limit is a cap, set by an AI provider, on how much you can ask of its service in a set amount of time, such as requests per minute or tokens per minute. Go over it and your requests are refused for a while, usually with an error that tells your software to wait and try again.
Think of a busy coffee shop that lets each customer order a set number of drinks every few minutes. Order faster than that and the barista says “give me a moment”. The shop is not broken and it has not banned you; it is keeping the queue fair for everyone. A rate limit is that rule, for computers asking an AI for answers.
In a sentence: “My script sent 500 questions at once, hit the rate limit, and had to slow down and resend the ones that failed.”
What does rate limit mean?
The plain point: a limit on speed of use, counted over a time window.
OpenAI’s developer guide puts it simply: “Rate limits are restrictions that our API imposes on the number of times a user or client can access our services within a specified period of time.” An API is the doorway programs use to talk to an AI without a chat window.1
Limits are counted in more than one way at once. OpenAI lists “RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), TPD (tokens per day), IPM (images per minute)”.1 Anthropic measures its main limits in “requests per minute (RPM), input tokens per minute (ITPM), and output tokens per minute (OTPM)”, and Google’s Gemini API uses requests per minute, tokens per minute and requests per day.23 Hitting any one of them is enough to be paused. Tokens are the small chunks of text AI services count.
What is a rate limit often confused with?
The plain point: several limits sound alike but cap different things.
| Term | How it differs |
|---|---|
| Spend limit | Differs: a cap on how much money an account can spend in a month. A rate limit caps speed, not cost. |
| Context window | Differs: the most text a model can read in one go. A rate limit is how much you can send over time. |
| Latency | Differs: how long one answer takes. Being rate-limited can feel like slowness, but it is a refusal to take more work for now. |
Why do AI providers set rate limits?
The plain point: to stop abuse, share capacity fairly and keep the service up.
OpenAI gives three reasons: rate limits “help protect against abuse or misuse of the API”, “help ensure that everyone has fair access to the API”, and “can help OpenAI manage the aggregate load on its infrastructure.”1 Anthropic says its limits “are designed to prevent API abuse, while minimizing impact on common customer usage patterns”, and that they rise by usage tier as an organisation builds a history of use.2
Limits usually belong to the account, not to one person or key. OpenAI says they are set “at the organization level and at the project level, not user level”, and Google says Gemini’s are “applied per project, not per API key.”13
What happens when you hit a rate limit?
The plain point: you get an error, wait, and try again more slowly.
Anthropic says that if you exceed a limit “you will get a 429 error describing which rate limit was exceeded”; 429 is the standard web code for “Too Many Requests”.4 When the reply carries a retry-after header, a short note in the response, the guidance is to “wait that many seconds before retrying.”2 Short bursts count too: Anthropic notes that “a rate of 60 requests per minute (RPM) might be enforced as 1 request per second.”2
The usual fix is exponential backoff. OpenAI describes it as “waiting briefly after an unsuccessful request” and “increasing the delay after each unsuccessful retry”, and warns that “continuously resending a request won’t work” because failed requests count against the limit too.1 Anthropic also points to prompt caching: for most of its models, text read back from the cache does not count toward the input-token limit.2
A rate limit is a speed cap, not a ban. If you build with an AI API, plan for 429 errors from day one: slow down, wait as told, retry with growing gaps, and reuse repeated text so you send less.
Questions people ask
Is a rate limit the same as the message limit in a chat app?
They are close cousins. Rate limits are the developer term for API use; chat apps have their own usage limits by plan. OpenAI says ChatGPT’s Free plan has unlimited everyday text chats “subject to abuse-prevention safeguards”, while file uploads, image generation and other tools “have separate usage limits.”5
What does error 429 mean?
It means too many requests: you went over a limit. Anthropic’s documentation says exceeding a rate limit returns “a 429 error describing which rate limit was exceeded”.2 Wait, then retry with growing gaps.
When do rate limits reset?
It depends on the provider. Anthropic refills capacity continuously rather than at fixed times, while Google says Gemini’s daily request quotas “reset at midnight Pacific time.”23 Anthropic shows each organisation’s current limits in its Claude Console.2
Where to go next
- How programs talk to AI: what is an API?
- What gets counted: token meaning and token economics.
- Sending less, paying less: prompt caching.
- Speed versus limits: latency meaning.
- If you are building an app: the AI app security checklist and the AI glossary.
QUICK CHECK · 3 QUESTIONS
Pick an answer to see if you got it. Nothing is recorded.
1. What is a rate limit?
- A cap on how much you can ask of an AI service in a set amount of time
- The price of each answer
- The longest reply a model can write
Show the answer
Answer: A cap on how much you can ask of an AI service in a set amount of time. The page defines a rate limit as a provider’s cap on requests or tokens over a time window, such as per minute.
2. What should your software do after a 429 error?
- Resend the request again and again straight away
- Wait, then retry with growing gaps between tries
- Switch off and never try again
Show the answer
Answer: Wait, then retry with growing gaps between tries. The page explains exponential backoff and quotes OpenAI warning that constantly resending will not work.
3. How is a rate limit different from a spend limit?
- They are the same thing
- A spend limit caps speed and a rate limit caps money
- A rate limit caps speed of use; a spend limit caps how much money is spent
Show the answer
Answer: A rate limit caps speed of use; a spend limit caps how much money is spent. The page’s table says a spend limit caps monthly cost, while a rate limit caps how fast you can send requests.
Related terms: Orchestrator · Plugin · RAG · Schema / structured output · Semantic search · Subagent
- 1OpenAI, Rate limits, OpenAI API documentation, undated, read at source 19 Oct 2026“Rate limits are restrictions that our API imposes on the number of times a user or client can access our services within a specified period of time.” ↑a“Rate limits use metrics such as RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), TPD (tokens per day), IPM (images per minute)”. ↑b“They help protect against abuse or misuse of the API.” … “Rate limits help ensure that everyone has fair access to the API.” … “Rate limits can help OpenAI manage the aggregate load on its infrastructure.” ↑c“Rate limits are defined at the organization level and at the project level, not user level.” ↑d“Exponential backoff means waiting briefly after an unsuccessful request” … “increasing the delay after each unsuccessful retry” … “Note that unsuccessful requests contribute to your per-minute limit, so continuously resending a request won’t work.” ↑e
- 2Anthropic, Rate limits, Claude API documentation, undated, read at source 19 Oct 2026“The rate limits for the Messages API are measured in requests per minute (RPM), input tokens per minute (ITPM), and output tokens per minute (OTPM) for each model class.” ↑a“Limits are designed to prevent API abuse, while minimizing impact on common customer usage patterns.” … “Organizations are placed on a tier automatically based on usage history and account standing and can move to a higher tier over time as they use the API.” ↑b“If you exceed any of the rate limits you will get a 429 error describing which rate limit was exceeded. When the response has a retry-after header, wait that many seconds before retrying.” ↑c“For instance, a rate of 60 requests per minute (RPM) might be enforced as 1 request per second.” ↑d“For most Claude models, only uncached input tokens count toward your ITPM rate limits.” ↑e“If you exceed any of the rate limits you will get a 429 error describing which rate limit was exceeded.” ↑f“This means that your capacity is continuously replenished up to your maximum limit, rather than being reset at fixed intervals.” ↑g“You can see your organization’s tier and current limits on the Rate limits page” in the Claude Console. ↑h
- 3Google AI for Developers, Rate limits, Gemini API documentation, last updated 15 Oct 2026, read at source 19 Oct 2026“Rate limits regulate the number of requests you can make to the Gemini API within a given timeframe.” Dimensions listed: “Requests per minute (RPM)”, “Tokens per minute (input) (TPM)”, “Requests per day (RPD)”. ↑a“Rate limits are applied per project, not per API key.” ↑b“Requests per day (RPD) quotas reset at midnight Pacific time.” ↑c
- 4MDN Web Docs, 429 Too Many Requests, undated, read at source 19 Oct 2026“The HTTP 429 Too Many Requests client error response status code indicates the client has sent too many requests in a given amount of time.” ↑
- 5OpenAI Help Center, ChatGPT Free Tier FAQ, “Updated: 2 months ago” as shown, as cited on this site’s how to use ChatGPT page, read at source 12 Oct 2026“Free users have unlimited everyday text chats, subject to abuse-prevention safeguards. File uploads, image generation, voice, data analysis, and other tools have separate usage limits.” ↑
The comparisons and examples on this page (the coffee shop, the 500-question script) and the takeaway’s advice are this site’s own ways of explaining, not claims from these sources. Limits change; each provider’s own page is the only current one.
Free printable AI cheat sheets, and AI news every week.
Sixteen one-page AI guides and checklists in one free PDF, no sign-up. And if you want to keep up, the newsletter covers what changed in AI each week, in plain English, with the sources.
SUBSCRIBE FREE ↗ GET THE CHEAT SHEET PACK