Chapter 02 — Scaling: Vertical vs Horizontal

Shreyesh kolhe
July 10, 2026

Chapter 02 — Scaling: Vertical vs Horizontal

Hey everyone! Welcome back to Namaste System Design! 🙏

Last chapter we saw Sharma Ji's chai stall melt down when 500 customers showed up. Today we answer the very first question every designer asks: "My system is drowning in traffic — how do I make it handle more?" There are only two answers to this question in the entire universe. Learn both and you've unlocked the foundation of everything.

What we will cover:

  • Scaling UP (vertical) vs scaling OUT (horizontal)
  • The chai-stall analogy for both
  • Why the whole internet runs on horizontal scaling
  • The one magic word that makes it possible: stateless
  • Sticky sessions and why they're a trap
  • The hard limits of each approach
  • Interview Questions

1. The Two Ways to Grow — A Story

Sharma Ji's chai stall is overwhelmed. He has exactly two options:

┌─────────────────────────────────────────────────────────────┐
│   OPTION A — SCALE UP (Vertical)                            │
│   Buy ONE giant industrial stove that boils 10x faster.     │
│                                                             │
│   OPTION B — SCALE OUT (Horizontal)                        │
│   Buy 10 normal stoves and hire helpers to run them.        │
└─────────────────────────────────────────────────────────────┘

That's the whole chapter. Vertical scaling = a bigger machine. Horizontal scaling = more machines. Everything else is detail.

2. Vertical Scaling (Scale UP)

Definition: make your single server more powerful — add CPU, RAM, faster disk.

BEFORE                          AFTER (scaled up)
──────                          ─────────────────
[ Server: 4 CPU, 8GB ]   ──▶   [ Server: 64 CPU, 512GB ]
      🥵 struggling                  😌 breathing room

   Same ONE box — just a monster version of it.

The good: dead simple. No code changes. Your app doesn't even know. Great for databases (which are hard to split).

The bad:

❌ There's a CEILING — you can't buy an infinitely big machine.
❌ Cost is exponential — a 2x machine can cost 5x.
❌ STILL a single point of failure — one box, one crash = total outage.
❌ Downtime to upgrade — you often reboot to swap hardware.

3. Horizontal Scaling (Scale OUT)

Definition: add more machines and split the work across them.

                    ┌──▶ [ Server 1 ]
   [ Users ] ─▶ [ LB ]──▶ [ Server 2 ]
                    └──▶ [ Server 3 ]

   A LOAD BALANCER spreads requests across identical servers.
   Need more capacity? Add Server 4, 5, 6... forever. ♾️

The good:

✅ Near-infinite scale — just keep adding cheap boxes.
✅ Fault tolerant — Server 2 dies? LB routes around it. No full outage.
✅ Cheaper per unit — many commodity machines beat one super-machine.

The bad: it's harder. Your app must be built so any server can handle any request. That leads us to the single most important word in this chapter...

4. The Magic Word: STATELESS

Horizontal scaling only works if your servers are stateless — meaning a server keeps no memory of a user between requests. Every request carries everything needed to handle it.

┌─────────────────────────────────────────────────────────────┐
│   STATEFUL server 🚫              STATELESS server ✅       │
│   ──────────────────              ────────────────────      │
│   "I stored your login            "I remember nothing.      │
│    in MY memory."                  Your token proves who    │
│                                    you are on every call."  │
│                                                             │
│   → request MUST return to        → ANY server can handle   │
│     the same server                 ANY request             │
└─────────────────────────────────────────────────────────────┘

Why it matters: if Server 1 stored your shopping cart in its own RAM, and the load balancer sends your next click to Server 2 — your cart is gone! So we push all "state" OUT of the servers into a shared place (a database, or a cache like Redis).

   [ Server 1 ] ─┐
   [ Server 2 ] ─┼──▶ [ Shared Redis / DB ]  ← state lives HERE
   [ Server 3 ] ─┘        (cart, session, etc.)

   Now servers are interchangeable. THIS is what makes
   horizontal scaling actually work.

Sticky Sessions — The Tempting Trap

Some teams cheat: they tell the load balancer "always send this user back to the same server" (a "sticky session"). It works... until that server dies and takes every user's session with it, or until traffic is lopsided because one server got all the heavy users. Prefer stateless + shared store over sticky sessions.

5. Side by Side

Vertical (Scale UP)Horizontal (Scale OUT)
HowBigger machineMore machines
LimitHard hardware ceilingPractically unlimited
FailureSingle point of failureSurvives node deaths
ComplexityEasy (no code change)Harder (needs stateless design + LB)
Cost curveExponentialLinear (commodity boxes)
Best forDatabases, quick winsWeb/app servers, big scale

Real-world truth: big systems do both. Scale app servers horizontally, and scale the database vertically (until you're forced to shard it — Chapter 08).

Interview Questions — Quick Fire!

Q: What's the difference between vertical and horizontal scaling?

"Vertical scaling means making a single machine more powerful — adding CPU, RAM, or disk. Horizontal scaling means adding more machines and distributing load across them with a load balancer. Vertical is simpler but has a hardware ceiling and remains a single point of failure; horizontal scales almost infinitely and is fault-tolerant but requires the application to be stateless."

Q: Why is horizontal scaling preferred for large systems?

"Because it has no hard ceiling — you just keep adding commodity machines — and it's fault-tolerant, since one node failing doesn't take the whole system down. Vertical scaling eventually hits the limit of the biggest machine you can buy and still leaves a single point of failure."

Q: What does 'stateless' mean and why does it matter?

"A stateless server keeps no client-specific data in its own memory between requests; each request carries everything needed, like an auth token. It matters because it makes servers interchangeable — any server can handle any request — which is exactly what allows a load balancer to distribute traffic freely and lets us scale horizontally. Shared state is pushed into a database or a cache like Redis."

Q: What are sticky sessions and what's the downside?

"Sticky sessions tie a user to one specific server so their in-memory state stays valid. The downside is that if that server dies, those users lose their sessions, and load can become uneven. The cleaner approach is stateless servers with session state in a shared store."

Key Points to Remember

ConceptKey Takeaway
VerticalBigger machine. Simple, but has a ceiling + single point of failure.
HorizontalMore machines. Near-infinite scale + fault tolerance, but needs stateless design.
StatelessServers remember nothing per-user; state lives in a shared store. The key to scaling out.
Sticky sessionsA shortcut that reintroduces a single point of failure. Avoid.
RealityBig systems mix both — scale app tier out, DB up (then shard).

What's Next?

In Chapter 03, we get precise about the words we throw around — latency, throughput, and availability. What does "99.99% uptime" actually mean in minutes? What's the difference between fast and high-capacity? These are the numbers interviewers push on, and by the end you'll speak them fluently.

Keep designing, keep scaling! See you in the next one!