Chapter 06 — Caching (The #1 Speed Trick)
Chapter 06 — Caching (The #1 Speed Trick)
Hey everyone! Welcome back to Namaste System Design! 🙏
If system design had a "most valuable player," it would be caching. It's the single most common way to make a slow system fast. There's even a famous joke: "There are only two hard things in computer science — cache invalidation and naming things." By the end of this chapter you'll understand both the magic and the pain.
What we will cover:
- What a cache is (the kitchen-counter analogy)
- Why RAM caching is 100x faster than disk
- Where caches live (browser → CDN → server → DB)
- Cache strategies: cache-aside, write-through, write-back
- Eviction policies (LRU & friends)
- The hard part: invalidation & stale data
- Interview Questions
1. The Kitchen Counter Analogy
You're cooking. The pantry (database) is in the basement — going there takes 2 minutes. The counter (cache) is right next to you — grabbing something takes 2 seconds.
┌─────────────────────────────────────────────────────────────┐ │ Smart cook: keep the things you use OFTEN on the counter. │ │ │ │ Salt, oil, spices → COUNTER (cache, instant) │ │ Rare items → PANTRY (database, slow trip) │ │ │ │ You don't run to the basement for salt every single time. │ │ That's caching: keep hot data close and fast. │ └─────────────────────────────────────────────────────────────┘
Definition: a cache is a small, fast store (usually in RAM, like Redis or Memcached) that holds copies of frequently-used data so you avoid slow trips to the database.
2. Why It's Such a Big Deal
Remember the latency ladder from Chapter 03:
Read from RAM (cache) ~100 nanoseconds ⚡ ~100,000x faster Read from disk (DB) ~10 milliseconds 🐌 Turning a 50ms DB read into a 1ms cache read means: → users feel instant response → the database handles WAY fewer queries (it survives)
WITHOUT cache: WITH cache:
every request → DB 1st request → DB, store it
DB at 100% 🔥 next 10,000 → cache ⚡
DB barely touched 😌
3. Where Do Caches Live? (Everywhere!)
A single request can hit MANY caches on its way:
[ Browser cache ] → your own device remembers images/CSS
│ miss
[ CDN cache ] → a server near you holds static files (Ch 12)
│ miss
[ Load balancer / API cache ]
│ miss
[ Application cache ] → Redis/Memcached in front of the DB ← the classic
│ miss
[ Database ] → the slow source of truth (last resort)
Each layer that "hits" saves the trip to everything below it.
When people say "add a cache" in an interview, they usually mean the application cache (Redis) sitting between the app servers and the database.
4. Caching Strategies
How does data get INTO the cache, and how do writes work? Three main patterns:
| Strategy | How it works | Trade-off |
|---|---|---|
| Cache-Aside (lazy) | App checks cache first; on miss, reads DB and fills cache | Most common. First read is slow; cache only holds what's used |
| Write-Through | Write to cache AND DB at the same time | Cache always fresh; writes are slightly slower |
| Write-Back (write-behind) | Write to cache now, flush to DB later in batches | Super fast writes; risk of data loss if cache dies |
CACHE-ASIDE (the default you'll describe most often):
──────────────────────────────────────────────────
1. App asks cache: "got user 1?"
HIT → return it ⚡ (done)
MISS → ↓
2. App reads DB, gets user 1
3. App stores user 1 in cache
4. Returns it. Next time → HIT.
5. Eviction: The Cache is Small, So What Do We Kick Out?
RAM is limited. When the cache is full and new data arrives, something must go. The most popular rule is LRU.
┌─────────────────────────────────────────────────────────────┐ │ EVICTION POLICIES │ ├─────────────────────────────────────────────────────────────┤ │ LRU (Least Recently Used) → kick out what hasn't been │ │ used in the longest time. (most common) ⭐ │ │ LFU (Least Frequently Used) → kick out the least- │ │ popular item. │ │ FIFO (First In First Out) → kick out the oldest added. │ │ TTL (Time To Live) → auto-expire after N seconds │ │ (great for freshness) │ └─────────────────────────────────────────────────────────────┘
6. The Hard Part: Stale Data & Invalidation
Here's the catch. The cache holds a copy. If the real data in the DB changes, the cache is now wrong (stale) until it's updated or expires.
DB: price = ₹200 (someone just changed it from ₹100) Cache: price = ₹100 ← STILL showing the old price! 😱 Users see ₹100. This is a STALE cache.
How we fight staleness:
✔ TTL (expiry) → cache auto-dies after N seconds, re-fetches
✔ Write-through → update cache whenever DB updates
✔ Explicit invalidation → on a write, delete the cache key so next
read re-loads fresh data
The eternal trade-off: longer TTL = faster & cheaper but more stale; shorter TTL = fresher but more DB load. How stale can this data be? A stock price: milliseconds. A user's profile photo: minutes are fine. Match the TTL to the need.
Watch out: cache stampede
Popular key expires → 10,000 requests all MISS at once → all 10,000 hammer the DB simultaneously → DB melts. 🔥 Fix: "lock" so only ONE request rebuilds the cache while others wait, or stagger expiry times.
Interview Questions — Quick Fire!
Q: What is caching and why is it used?
"Caching stores copies of frequently accessed data in a fast layer, usually RAM like Redis, so we avoid repeatedly hitting a slower source like a database. It dramatically reduces latency — a RAM read is around a hundred thousand times faster than disk — and it reduces load on the database, letting the system serve far more traffic."
Q: Explain the cache-aside strategy.
"In cache-aside, the application first checks the cache. On a hit, it returns the cached data. On a miss, it reads from the database, stores the result in the cache, and returns it. So the cache is populated lazily — only with data that's actually requested. It's the most common pattern, and the trade-off is that the first read for any item is slow."
Q: What is cache invalidation and why is it hard?
"Cache invalidation is keeping the cache consistent with the source of truth when the underlying data changes. It's hard because the cache holds a copy — if the database updates but the cache doesn't, users see stale data. We handle it with TTL expiry, write-through updates, or explicitly deleting the cache key on writes, but choosing the right approach and timing is genuinely tricky."
Q: What is an eviction policy? Name one.
"Since a cache has limited memory, an eviction policy decides what to remove when it's full. The most common is LRU — Least Recently Used — which evicts the item that hasn't been accessed for the longest time, on the assumption that recently used data is most likely to be used again."
Q: What's a cache stampede and how do you prevent it?
"A cache stampede happens when a popular key expires and many simultaneous requests all miss and hit the database at once, potentially overwhelming it. We prevent it by locking so only one request rebuilds the cache while others wait, or by staggering expiry times so keys don't all expire together."
Key Points to Remember
| Concept | Key Takeaway |
|---|---|
| What | Fast RAM store (Redis) holding hot data to avoid slow DB trips. |
| Why | ~100,000x faster than disk; slashes latency AND database load. |
| Strategies | Cache-aside (default) · write-through (fresh) · write-back (fast, risky). |
| Eviction | LRU is the go-to; TTL auto-expires for freshness. |
| The pain | Stale data. Fight it with TTL / write-through / invalidation. Watch for stampedes. |
What's Next?
Chapter 07 tackles database replication — keeping multiple copies of your data so reads are fast and your system survives a database crash. We'll meet the leader-follower pattern and the dreaded "replication lag."
Keep designing, keep scaling! See you in the next one!
Post a Comment