Chapter 02 — Scaling: Vertical vs Horizontal
Chapter 02 — Scaling: Vertical vs Horizontal
Hey everyone! Welcome back to Namaste System Design! 🙏
Last chapter we saw Sharma Ji's chai stall melt down when 500 customers showed up. Today we answer the very first question every designer asks: "My system is drowning in traffic — how do I make it handle more?" There are only two answers to this question in the entire universe. Learn both and you've unlocked the foundation of everything.
What we will cover:
- Scaling UP (vertical) vs scaling OUT (horizontal)
- The chai-stall analogy for both
- Why the whole internet runs on horizontal scaling
- The one magic word that makes it possible: stateless
- Sticky sessions and why they're a trap
- The Scaling Ladder — 6 stages from 1 server to millions of users
- The hard limits of each approach
- Interview Questions
1. The Two Ways to Grow — A Story
Sharma Ji's chai stall is overwhelmed. He has exactly two options:
┌─────────────────────────────────────────────────────────────┐ │ OPTION A — SCALE UP (Vertical) │ │ Buy ONE giant industrial stove that boils 10x faster. │ │ │ │ OPTION B — SCALE OUT (Horizontal) │ │ Buy 10 normal stoves and hire helpers to run them. │ └─────────────────────────────────────────────────────────────┘
That's the whole chapter. Vertical scaling = a bigger machine. Horizontal scaling = more machines. Everything else is detail.
2. Vertical Scaling (Scale UP)
Definition: make your single server more powerful — add CPU, RAM, faster disk.
BEFORE AFTER (scaled up)
────── ─────────────────
[ Server: 4 CPU, 8GB ] ──▶ [ Server: 64 CPU, 512GB ]
🥵 struggling 😌 breathing room
Same ONE box — just a monster version of it.
REAL EXAMPLE (AWS instance pricing, roughly) t3.medium → 2 vCPU, 4 GB RAM → ~$30 / month r5.4xlarge → 16 vCPU, 128 GB RAM → ~$700 / month 8x the CPU, 32x the RAM... for about 23x the price. That's "the bigger box costs disproportionately more" in real numbers, not just theory.
The good: dead simple. No code changes. Your app doesn't even know. Great for databases (which are hard to split).
The bad:
❌ There's a CEILING — you can't buy an infinitely big machine. ❌ Cost is exponential — a 2x machine can cost 5x. ❌ STILL a single point of failure — one box, one crash = total outage. ❌ Downtime to upgrade — you often reboot to swap hardware.
3. Horizontal Scaling (Scale OUT)
Definition: add more machines and split the work across them.
┌──▶ [ Server 1 ]
[ Users ] ─▶ [ LB ]──▶ [ Server 2 ]
└──▶ [ Server 3 ]
A LOAD BALANCER spreads requests across identical servers.
Need more capacity? Add Server 4, 5, 6... forever. ♾️
REAL EXAMPLE 1 server handles ~1,000 requests/second 3 servers handle ~3,000 requests/second 10 servers handle ~10,000 requests/second Capacity grows roughly LINEARLY with server count. That's the entire appeal of scaling out.
The good:
✅ Near-infinite scale — just keep adding cheap boxes. ✅ Fault tolerant — Server 2 dies? LB routes around it. No full outage. ✅ Cheaper per unit — many commodity machines beat one super-machine.
The bad: it's harder. Your app must be built so any server can handle any request. That leads us to the single most important word in this chapter...
4. The Magic Word: STATELESS
Horizontal scaling only works if your servers are stateless — meaning a server keeps no memory of a user between requests. Every request carries everything needed to handle it.
┌─────────────────────────────────────────────────────────────┐ │ STATEFUL server 🚫 STATELESS server ✅ │ │ ────────────────── ──────────────────── │ │ "I stored your login "I remember nothing. │ │ in MY memory." Your token proves who │ │ you are on every call." │ │ │ │ → request MUST return to → ANY server can handle │ │ the same server ANY request │ └─────────────────────────────────────────────────────────────┘
Why it matters: if Server 1 stored your shopping cart in its own RAM, and the load balancer sends your next click to Server 2 — your cart is gone! So we push all "state" OUT of the servers into a shared place (a database, or a cache like Redis).
[ Server 1 ] ─┐ [ Server 2 ] ─┼──▶ [ Shared Redis / DB ] ← state lives HERE [ Server 3 ] ─┘ (cart, session, etc.) Now servers are interchangeable. THIS is what makes horizontal scaling actually work.
Sticky Sessions — The Tempting Trap
Some teams cheat: they tell the load balancer "always send this user back to the same server" (a "sticky session"). It works... until that server dies and takes every user's session with it, or until traffic is lopsided because one server got all the heavy users. Prefer stateless + shared store over sticky sessions.
5. The Scaling Ladder — From 1 User to 10 Million
Real systems never jump from 1 server straight to 10,000. They climb a ladder, one bottleneck at a time. Each rung exists because the previous setup broke under load. Let's climb it the way a real startup would.
5.1 Stage 0 — One Server Does Everything
The app code and the database live on the exact same machine.
[ Users ] ──▶ [ ONE Server ]
(app code + database, same box)
👉 USE IT WHEN Under ~1,000 users. An MVP. A weekend project.
❌ BREAKS WHEN The app and the database start fighting over
the same CPU and RAM.
5.2 Stage 1 — Separate the Database
Move the database onto its own machine. Now the app server only runs code, and the DB server only stores data.
[ Users ] ──▶ [ App Server ] ──▶ [ DB Server ]
(just code) (just data)
✅ Each machine can be scaled and tuned on its own. 👉 Use it when: one box is maxed out on CPU, or slow queries are dragging down the app too.
5.3 Stage 2 — Add a Cache
Most reads repeat the same data over and over. Put a fast in-memory layer (Redis) in front of the database to catch them.
[ Users ] ─▶ [ App Server ] ─▶ [ Cache ] ─▶ [ DB Server ]
hit → 1ms miss → 50ms
👉 Great for: read-heavy data that doesn't change every second — product pages, user profiles. Full deep dive in Chapter 06.
5.4 Stage 3 — Add a Load Balancer + More App Servers
The single app server is now the bottleneck. Run several, and put a load balancer in front to spread requests.
┌─▶ [ App 1 ] ─┐
[ Users ] ─▶[LB]─┼─▶ [ App 2 ] ─┼─▶ [ Cache ] ─▶ [ DB ]
└─▶ [ App 3 ] ─┘
👉 Requires: stateless app servers (Section 4) — any server must be able to handle any request. Full deep dive in Chapter 04.
5.5 Stage 4 — Add a CDN
Static files (images, JS, CSS) don't need to hit your servers at all. A CDN caches them on servers physically close to the user.
User in Mumbai, NO CDN:
request ──▶ [ Origin server, USA ] ──▶ ~200ms
User in Mumbai, WITH CDN:
request ──▶ [ CDN edge, Mumbai ] ──▶ ~10ms
👉 Use it when: you serve a lot of static assets to users spread across the globe. Full deep dive in Chapter 12.
5.6 Stage 5 — Shard the Database
Eventually one database server — even a vertically-scaled monster one — can't hold or serve all the data. Split it into pieces called shards, each holding a slice of the data.
users A-M ──▶ [ DB Shard 1 ] users N-Z ──▶ [ DB Shard 2 ] Each shard is a smaller, faster database. No single machine holds everything anymore.
👉 Use it when: the database is the bottleneck and vertical scaling has run out of room. This is the last resort — it works, but it adds real complexity (splitting queries, joining across shards). Full deep dive in Chapter 08.
Visual Comparison
| Stage | What we added | What it fixes |
|---|---|---|
| 0 | Single server | Nothing yet — this is the starting line |
| 1 | Separate DB server | ✅ App & DB stop competing for CPU/RAM |
| 2 | Cache (Redis) | ✅ Repeated reads become fast (~1ms) |
| 3 | Load balancer + more app servers | ✅ App tier survives crashes & traffic spikes |
| 4 | CDN | ✅ Static content is fast worldwide |
| 5 | Sharded DB | ✅ Removes the database's scale ceiling |
6. Side by Side
| Vertical (Scale UP) | Horizontal (Scale OUT) | |
|---|---|---|
| How | Bigger machine | More machines |
| Limit | Hard hardware ceiling | Practically unlimited |
| Failure | Single point of failure | Survives node deaths |
| Complexity | Easy (no code change) | Harder (needs stateless design + LB) |
| Cost curve | Exponential | Linear (commodity boxes) |
| Best for | Databases, quick wins | Web/app servers, big scale |
Real-world truth: big systems do both. Scale app servers horizontally, and scale the database vertically (until you're forced to shard it — Chapter 08).
Interview Questions — Quick Fire!
Q: What's the difference between vertical and horizontal scaling?
"Vertical scaling means making a single machine more powerful — adding CPU, RAM, or disk. Horizontal scaling means adding more machines and distributing load across them with a load balancer. Vertical is simpler but has a hardware ceiling and remains a single point of failure; horizontal scales almost infinitely and is fault-tolerant but requires the application to be stateless."
Q: Why is horizontal scaling preferred for large systems?
"Because it has no hard ceiling — you just keep adding commodity machines — and it's fault-tolerant, since one node failing doesn't take the whole system down. Vertical scaling eventually hits the limit of the biggest machine you can buy and still leaves a single point of failure."
Q: What does 'stateless' mean and why does it matter?
"A stateless server keeps no client-specific data in its own memory between requests; each request carries everything needed, like an auth token. It matters because it makes servers interchangeable — any server can handle any request — which is exactly what allows a load balancer to distribute traffic freely and lets us scale horizontally. Shared state is pushed into a database or a cache like Redis."
Q: What are sticky sessions and what's the downside?
"Sticky sessions tie a user to one specific server so their in-memory state stays valid. The downside is that if that server dies, those users lose their sessions, and load can become uneven. The cleaner approach is stateless servers with session state in a shared store."
Q: In what order would you typically scale a growing system?
"I'd start by separating the database from the app server so they stop competing for resources. Next, add a cache in front of the database for repeated reads. Then add a load balancer with multiple app servers, which requires the app to be stateless. After that, push static assets to a CDN so they're served close to the user. Sharding the database is the last resort, once even a vertically-scaled database can't keep up — it's powerful but adds real complexity, so I'd only reach for it when the earlier steps aren't enough."
Key Points to Remember
| Concept | Key Takeaway |
|---|---|
| Vertical | Bigger machine. Simple, but has a ceiling + single point of failure. |
| Horizontal | More machines. Near-infinite scale + fault tolerance, but needs stateless design. |
| Stateless | Servers remember nothing per-user; state lives in a shared store. The key to scaling out. |
| Sticky sessions | A shortcut that reintroduces a single point of failure. Avoid. |
| Scaling ladder | Separate DB → cache → LB + more servers → CDN → shard the DB. Climb it one bottleneck at a time. |
| Reality | Big systems mix both — scale app tier out, DB up (then shard). |
What's Next?
In Chapter 03, we get precise about the words we throw around — latency, throughput, and availability. What does "99.99% uptime" actually mean in minutes? What's the difference between fast and high-capacity? These are the numbers interviewers push on, and by the end you'll speak them fluently.
Keep designing, keep scaling! See you in the next one!
Post a Comment