• Docs
  • Talk to an expert
Blog
Blog
BlogProductCase studiesNewsInsights
Blog

What is rate limiting?

APIsecurityperformance
Share

TL;DR

  • Rate limiting caps how many requests a person, device, or API key can make in a set time, such as 100 requests a minute.
  • When someone goes over the limit, the server usually refuses with a 429 Too Many Requests error until the window resets.
  • It protects your app from abuse, such as bots, scraping, and password-guessing attacks, and keeps one heavy user from slowing things down for everyone.
  • It also protects your wallet. If your app calls a paid AI or other API for each user request, anyone who can hammer your app can run up your bill.
  • Rate limits can be applied at several layers: a CDN at the edge, a reverse proxy, your app's code, and spending limits with the services you pay for.
  • On Upsun Cloud, you can rate limit with Varnish, in your app, or at the edge with a CDN add-on.

Rate limiting is a rule that limits how often someone can do something. A login page might allow five attempts a minute. An API might allow 1,000 requests an hour per key. Go over the limit, and further requests are refused until enough time passes.

It used to be something mainly large services worried about. Now that many apps call paid AI models behind the scenes, it matters for everyone: a feature that costs a fraction of a cent per use can cost thousands if someone scripts it. This guide explains how rate limiting works, why AI-built apps especially need it, and where to add it.

How rate limiting works

Key takeaway: the server counts requests from each user or key, and refuses them once the count passes the limit for the current time window.

  1. A request arrives, and the server works out who it's from, using an IP address, a logged-in user, or an API key.
  2. It checks how many requests that identity has made in the current time window.
  3. If it's under the limit, the request goes through, and the count goes up.
  4. If it's over, the server refuses the request, usually with a 429 Too Many Requests status code.

A good 429 response tells the client when to try again, using a Retry-After header. Many APIs also send headers showing the limit and how many requests are left, so well-behaved clients can slow down before they hit it.

Common rate limiting methods

Key takeaway: the methods differ in how they handle bursts. Token bucket is a popular middle ground.

MethodHow it worksTrade-off
Fixed windowCounts requests in fixed blocks of time, such as each minuteSimple, but allows bursts at the edges, such as 100 requests at 12:00:59 and 100 more at 12:01:00
Sliding windowCounts requests in the last rolling period, such as the past 60 secondsSmoother, slightly more work to track
Token bucketEach user has a bucket of tokens that refills at a steady rate. Each request uses oneAllows short bursts while holding the average steady
Leaky bucketRequests are processed at a fixed rate, and extras wait or are droppedVery steady output, less tolerant of bursts

Why AI-built apps need rate limiting

Key takeaway: if every request to your app costs you money, anyone who can send lots of requests can run up your bill.

Apps built quickly, often with AI tools, frequently skip rate limiting, because everything works fine in testing. The problems appear once the app is public:

RiskWhat happens
Surprise AI billsYour app calls a paid AI model for each chat message or generation. Someone scripts thousands of requests, and you pay for every one. This is sometimes called a denial-of-wallet attack
Free accounts abusedBots create accounts in bulk to use free credits or free features
Password guessingAttackers try thousands of passwords against your login page
ScrapingBots copy your content or data, using up resources and bandwidth
Runaway loops in your own codeA bug calls a paid API in a loop, and nobody notices until the invoice arrives
One user slowing everyone downA single heavy user, or a misbehaving script, takes capacity from everyone else

The expensive endpoints matter most. Put the strictest limits on anything that costs money or is attractive to attackers: AI features, sign-up, login, password reset, and checkout. Our guide What is an API key? covers the related risk of a leaked key being used to run up your bill.

Where to add rate limiting

Key takeaway: limits work best in layers, from blocking floods at the edge to precise per-user limits in your app.

LayerWhat it's good forExamples
CDN or edgeStopping floods before they reach your servers, by IP address or pathRate limiting rules in a CDN or web application firewall
Reverse proxyLimits per IP address in front of your appNginx, HAProxy, Varnish
Your appPrecise limits per logged-in user, per API key, or per featureFramework tools and libraries
The services you pay forCapping your total spending, whatever happensSpending limits and usage alerts in your AI or API provider's dashboard

Many web frameworks have rate limiting built in or available as a package:

FrameworkRate limiting option
Express (Node.js)The express-rate-limit package
Django (Python)The django-ratelimit package
Laravel (PHP)Built-in rate limiting and the throttle middleware
Symfony (PHP)The RateLimiter component
Rails (Ruby)Built-in rate_limit in controllers, from Rails 7.2

Spending limits deserve special mention. Whatever else you do, set a monthly spending cap and usage alerts with every AI and API provider you pay. It's the safety net if every other layer fails.

When you're the one being rate limited

Key takeaway: if a service returns 429 errors, slow down and retry later, rather than retrying immediately.

Rate limits work in both directions. If your app calls an API that returns 429 errors:

  • Respect the Retry-After header, and wait that long before retrying.
  • Retry with increasing delays, such as one second, then two, then four, rather than immediately.
  • Cache responses that don't change often, so you make fewer requests.
  • Batch requests where the API allows it.
  • Check whether your plan's limits match how much you actually use the service.

Common rate limiting mistakes

Key takeaway: many rate limiting bugs come from counting the wrong thing, or counting in the wrong place.

MistakeWhat happensHow to avoid it
Limiting by the proxy's addressBehind a CDN or proxy, every request seems to come from the same address, so one limit blocks everyoneUse the visitor's real address, which the proxy passes in a header
Counting separately on each serverWith three app instances, each keeps its own count, so the real limit is three times higherKeep counts in a shared store, such as Redis
No limits on expensive featuresA free AI feature gets scripted, and the bill arrivesLimit costly endpoints strictly, per user and per address
Limits so strict they block real usersCustomers on a shared office or mobile network get blocked togetherPrefer limits per logged-in user where you can, and tune limits using real traffic
No spending caps with providersRate limits have a gap, and costs run uncheckedSet spending limits and alerts with every paid service
No monitoringNobody notices abuse until something breaksLog 429 responses and alert on unusual spikes

Rate limiting on Upsun Cloud

Key takeaway: on Upsun Cloud, you can rate limit in your app, with Varnish, or at the edge with a CDN add-on.

Upsun Cloud gives you several places to apply limits:

OptionWhat it doesDetails from the Upsun docs
In your appPrecise limits per user, per API key, or per feature, using your framework's toolsThe router passes each visitor's real IP address to your app in the X-Client-IP header, so limits by address work correctly
VarnishLimits per IP address in front of your app, using Varnish's throttling moduleAdd Varnish as a service, and use the vsthrottle module in your Varnish configuration
Edge-level rate limitingLimits applied at the CDN, before requests reach your app, scoped by path, request type, or IP addressAvailable as an add-on at the CDN layer, arranged through Sales and configured with Upsun Support. No rules are applied by default

The Upsun docs on Varnish include an example that blocks any IP address making more than 20 requests in 10 seconds, for two minutes:

import vsthrottle;

sub vcl_recv {
  # The Upsun Cloud router provides the real client IP as X-Client-IP
  if (vsthrottle.is_denied(req.http.X-Client-IP, 20, 10s, 120s)) {
    return (synth(429, "Too Many Requests"));
  }
  set req.backend_hint = application.backend();
}

The edge-level rate limiting add-on is rule-based. According to the docs, it doesn't identify bots automatically or present challenges such as CAPTCHAs, and Upsun Cloud offers separate integrations for advanced bot protection.

If your app runs several instances, keep app-level rate limit counts in a shared store. A Redis service on Upsun Cloud works well for this. Our guide What is caching? Redis explained for beginners covers adding one.

For hands-on examples, read the Dev Center posts Varnish 102: Protecting your application with rate limiting on Upsun and Rate limiting Rails apps when you can't put Varnish in front.

You can deploy on Upsun Cloud free for 15 days, with no card needed, or see how security and compliance work on Upsun Cloud.

Frequently asked questions

What is rate limiting in simple terms?

A limit on how many requests someone can make in a set time, such as 100 a minute. Requests over the limit are refused until the time window resets.

What does a 429 Too Many Requests error mean?

That you've sent more requests than the server allows in a period of time. Wait, ideally for as long as the Retry-After header says, and try again.

Why does my app need rate limiting?

To stop abuse such as bots, scraping, and password guessing, to keep one user from slowing down everyone else, and to stop people running up your bill if your app calls paid services.

What is a denial-of-wallet attack?

An attack that doesn't try to take your app down, but to make it cost you money, for example by sending thousands of requests to a feature that calls a paid AI model.

Should I rate limit by IP address or by user?

Both, where possible. Limit by IP address for anonymous traffic and login pages, and by user or API key for logged-in features, so people sharing a network aren't blocked together.

Is rate limiting enough to protect against bots?

Not on its own. It limits how fast anyone can send requests, but it doesn't tell bots from people. Combine it with bot protection, spending limits, and monitoring.

Stay updated

Subscribe to our monthly newsletter for the latest updates and news.

Deploy possibility.
Try Upsun for free.

Build with DispatchDeploy on Cloud