Chapter 13 — Rate Limiting

Shreyesh kolhe
July 10, 2026

Chapter 13 — Rate Limiting

Hey everyone! Welcome back to Namaste System Design! 🙏

The internet is full of bad actors — bots scraping your data, attackers hammering your login page, or just a buggy client stuck in a retry loop sending 10,000 requests a second. Left unchecked, they can take your whole system down. The bouncer at the door is rate limiting: capping how many requests a client can make in a window of time.

What we will cover:

  • What rate limiting is (the nightclub bouncer)
  • Why we need it (abuse, cost, fairness)
  • The 4 classic algorithms
  • Token bucket, step by step
  • Where to put the limiter & the 429 response
  • Distributed rate limiting with Redis
  • Interview Questions

1. The Nightclub Bouncer

┌─────────────────────────────────────────────────────────────┐
│   RATE LIMITING = capping how many requests a client may    │
│   make in a given time window.                              │
│                                                             │
│   The bouncer 🕴️ lets in a controlled number of people      │
│   per minute. Too many at once? "Wait outside." This keeps  │
│   the club (your server) from getting dangerously packed.   │
└─────────────────────────────────────────────────────────────┘

Example rule: "Each user may call the API 100 times per minute." The 101st request in that minute gets rejected with a polite 429 Too Many Requests.

2. Why We Need It

✅ STOP ABUSE      → block bots, scrapers, brute-force login attacks
✅ PREVENT OVERLOAD → one client can't flood & crash the system
✅ FAIRNESS        → no single user hogs resources; everyone gets a share
✅ CONTROL COST    → caps usage of expensive operations (e.g. AI calls)
✅ DDoS DEFENSE    → a first line against traffic floods

3. The 4 Classic Algorithms

AlgorithmIdeaTrade-off
Fixed WindowCount requests per fixed clock window (e.g. per minute)Simple, but allows bursts at window edges
Sliding WindowRolling window that moves with timeSmoother & fairer, a bit more work
Token BucketTokens refill at a steady rate; each request spends oneAllows controlled bursts. Most popular ⭐
Leaky BucketRequests drip out at a fixed steady rate (a queue)Smooths output; can add delay

The Fixed-Window Edge Problem

   Limit: 100/minute.
   User sends 100 at 10:00:59  ✅
   ...then 100 more at 10:01:01 ✅
   → 200 requests in 2 seconds! The window RESET let a burst
     slip through. Sliding window fixes this.

4. Token Bucket — Step by Step (The Favorite)

   A bucket holds up to N tokens. Tokens are added at a steady
   rate. Each request must TAKE one token to proceed. No token
   left → rejected.

   Bucket capacity: 10 tokens.  Refill: 1 token/second.

   [🪙🪙🪙🪙🪙🪙🪙🪙🪙🪙]  full → user can burst 10 quick requests
   request → take a token → [🪙🪙🪙🪙🪙🪙🪙🪙🪙] (9 left)
   ...10 fast requests → bucket EMPTY [ ] → 11th REJECTED (429)
   wait 1 second → +1 token → [🪙] → one request allowed again

Why it's loved: it allows short, natural bursts (a user clicking around quickly) while still enforcing a steady long-run average. That matches real human behavior better than a hard fixed cap.

5. Where Does the Limiter Live?

   [ Client ] ─▶ [ API Gateway / Load Balancer ] ─▶ [ servers ]
                        ▲
                 rate limiter HERE (common)
                 → rejects abusive traffic BEFORE it wastes
                   your expensive backend resources

Rejecting early (at the gateway/edge) is best — you don't want a flood reaching your database. When you reject, be polite and helpful:

   HTTP 429 Too Many Requests
   Retry-After: 30          ← "try again in 30 seconds"
   X-RateLimit-Limit: 100
   X-RateLimit-Remaining: 0 ← tells a good client to back off

6. Distributed Rate Limiting

Here's the catch at scale. You have 10 servers behind a load balancer. If each keeps its own count in memory, a user could do 100 requests on each server = 1000 total. Oops. The counts must be shared.

   [ Server 1 ] ─┐
   [ Server 2 ] ─┼──▶ [ Redis ]  ← ONE shared counter per user
   [ Server 3 ] ─┘      "user:42 → 87 requests this minute"

   Every server checks/updates the SAME counter in Redis.
   Redis is fast (in-memory) and central → accurate global limit.

This is a perfect example of why we push state out of servers into a shared store — the exact lesson from Chapter 02 (stateless servers).

Interview Questions — Quick Fire!

Q: What is rate limiting and why is it important?

"Rate limiting caps how many requests a client can make in a time window. It's important for protecting the system from abuse like bots and brute-force attacks, preventing any single client from overloading the system, ensuring fairness among users, controlling cost of expensive operations, and providing a first line of defense against DDoS attacks."

Q: Explain the token bucket algorithm.

"A bucket holds up to N tokens and refills at a steady rate. Each request must take a token to proceed; if the bucket is empty, the request is rejected. It's popular because it allows short bursts — a full bucket lets a user fire several quick requests — while still enforcing a steady average rate over time, which matches real usage patterns well."

Q: What's the problem with the fixed-window algorithm?

"It allows bursts at window boundaries. If the limit is 100 per minute, a client can send 100 requests at the very end of one window and another 100 at the start of the next — 200 requests in a couple of seconds. A sliding window smooths this out by using a rolling time window instead of resetting on a fixed clock boundary."

Q: How do you rate limit across multiple servers?

"You can't keep counts in each server's local memory, because a client could hit the limit separately on each server and exceed the global cap. Instead you use a shared, fast store like Redis to hold a central counter per client that all servers read and update. That gives an accurate global limit regardless of which server handles the request."

Q: What HTTP status code do you return when rate limited?

"429 Too Many Requests, ideally with a Retry-After header telling the client when to try again, and headers like X-RateLimit-Remaining so well-behaved clients can back off proactively."

Key Points to Remember

ConceptKey Takeaway
WhatCap requests per client per time window. Reject extras with HTTP 429.
WhyStop abuse, prevent overload, ensure fairness, control cost, blunt DDoS.
AlgorithmsFixed window · sliding window · token bucket (popular, allows bursts) · leaky bucket.
PlacementAt the gateway/edge — reject before wasting backend resources.
DistributedUse a shared counter in Redis so the limit is global, not per-server.

What's Next?

Season 3 done! In Chapter 14 we open Season 4: Reliability & Scale with the deep topic of consistency & consensus — how independent machines that can crash and lie manage to agree on a single truth.

Keep designing, keep scaling! See you in the next one!