Chapter 04 — Load Balancers
Chapter 04 — Load Balancers
Hey everyone! Welcome back to Namaste System Design! 🙏
In Chapter 02 we drew a mysterious box labelled [ LB ] that magically split traffic across servers. Today we open that box. The load balancer is the traffic cop of the internet — without it, horizontal scaling simply doesn't work. It's on the whiteboard in every single system design interview.
What we will cover:
- What a load balancer does (the receptionist analogy)
- How it makes many servers look like one
- Balancing algorithms: round robin, least connections, hashing
- Health checks — routing around dead servers
- Layer 4 vs Layer 7 load balancing
- "But isn't the LB itself a single point of failure?"
- Interview Questions
1. The Receptionist Analogy
Walk into a big hospital. You don't wander the halls hunting for a free doctor. You go to one reception desk, and the receptionist sends you to whichever doctor is free. 🛎️
┌─────────────────────────────────────────────────────────────┐ │ The LOAD BALANCER is that receptionist. │ │ │ │ Patients (users) all arrive at ONE address. │ │ The receptionist (LB) decides which doctor (server) │ │ handles each one — so no doctor is overwhelmed and │ │ no patient waits in the wrong line. │ └─────────────────────────────────────────────────────────────┘
┌──▶ [ Server 1 ] (30% busy)
[ Users ] ─────▶ [ LB ]──▶ [ Server 2 ] (32% busy)
one public IP └──▶ [ Server 3 ] (28% busy)
Users see ONE address. The LB hides the fleet behind it.
Add/remove servers anytime — users never notice. ✨
2. What the Load Balancer Buys You
✅ SCALABILITY → spread load so no server melts ✅ AVAILABILITY → a server dies? LB stops sending it traffic ✅ FLEXIBILITY → add/remove servers with zero user impact ✅ ONE ENTRY POINT → users hit one address, not 50 IPs ✅ (bonus) SSL termination, compression, sometimes caching
3. How Does It Choose a Server? (Algorithms)
The receptionist needs a rule for picking the next doctor. The common ones:
| Algorithm | How it works | Best when |
|---|---|---|
| Round Robin | Server 1, 2, 3, 1, 2, 3... in a loop | Servers are equal & requests are similar |
| Weighted Round Robin | Beefier servers get more turns | Servers have different power |
| Least Connections | Send to the server with fewest active requests | Requests vary a lot in duration |
| Least Response Time | Send to the fastest-responding server | You want lowest latency |
| IP / Hash | Hash the user's IP → always same server | You need stickiness (sessions) |
ROUND ROBIN (simple, fair for equal servers): req1 → S1, req2 → S2, req3 → S3, req4 → S1 ... LEAST CONNECTIONS (smarter under uneven load): "S2 only has 3 open requests, S1 has 40 → send to S2"
Default answer for an interview: "Round robin is the simplest; I'd use least-connections when request durations vary widely."
4. Health Checks — The Life Saver
How does the LB know a server died? It constantly pings each one.
LB ──"you alive?"──▶ [ Server 2 ] ──"yes! 200 OK"──▶ ✅ keep it LB ──"you alive?"──▶ [ Server 3 ] ──(no reply)────▶ ❌ pull it out Every few seconds. The moment S3 stops replying, the LB STOPS sending it users. When S3 recovers, it's added back. Users never see the dead server. THIS is how "self-healing" and high availability actually happen.
5. Layer 4 vs Layer 7
Load balancers work at two different "depths" of understanding the traffic.
| Layer 4 (Transport) | Layer 7 (Application) | |
|---|---|---|
| Sees | Just IPs & ports (TCP/UDP) | The full HTTP request (URL, headers, cookies) |
| Can it read the URL? | No | Yes |
| Speed | Faster (less to inspect) | Slightly slower (reads content) |
| Smart routing | No | Yes — /images → image servers, /api → API servers |
| Example | AWS NLB | AWS ALB, Nginx |
Simple takeaway: Layer 7 is smarter (it understands HTTP, so it can route by path or do SSL termination); Layer 4 is faster and dumber. Most web apps use Layer 7.
6. "Isn't the LB Itself a Single Point of Failure?"
Brilliant catch — and interviewers love asking it. If all traffic flows through one LB and that LB dies... total outage. So we make the LB redundant too:
┌─▶ [ LB 1 - active ] ─┐
[ Users ] ─DNS─ ├─▶ [ server fleet ]
└─▶ [ LB 2 - standby ] ─┘
Two (or more) load balancers. If the active one dies,
the standby takes over its IP (a "floating IP") in seconds.
Big cloud LBs (AWS ELB) do this for you automatically.
The lesson generalizes: anywhere you have exactly one of something critical, you have a single point of failure — so you duplicate it.
Interview Questions — Quick Fire!
Q: What is a load balancer and why do we need one?
"A load balancer sits in front of a group of servers and distributes incoming requests across them. We need it because horizontal scaling only works if traffic is spread evenly — it prevents any single server from being overwhelmed, provides one entry point that hides the server fleet, and improves availability by routing traffic away from failed servers."
Q: Name a few load balancing algorithms.
"Round robin cycles through servers in order and is simplest for equal servers. Weighted round robin gives more powerful servers more traffic. Least connections sends the next request to the server with the fewest active connections, which is better when request durations vary. IP hash maps a client consistently to the same server, useful for session stickiness."
Q: How does a load balancer know a server is down?
"Through health checks — it periodically sends a request, like a ping to a health endpoint, and expects a healthy response. If a server stops responding, the load balancer removes it from the pool and stops routing traffic to it, then adds it back once it recovers. This is what makes the system self-healing."
Q: What's the difference between Layer 4 and Layer 7 load balancing?
"Layer 4 operates at the transport level, routing based only on IP and port without reading the request content — it's fast but not content-aware. Layer 7 operates at the application level, so it can read the HTTP request and route based on URL path, headers, or cookies, and do things like SSL termination. Layer 7 is smarter; Layer 4 is faster."
Q: Isn't the load balancer itself a single point of failure?
"It can be, so we make it redundant — typically an active-standby or active-active pair. If the active load balancer fails, a standby takes over its IP address within seconds using a floating IP. Managed cloud load balancers handle this redundancy automatically."
Key Points to Remember
| Concept | Key Takeaway |
|---|---|
| Purpose | Spreads traffic across servers; one entry point hiding the fleet. |
| Algorithms | Round robin (simple) · least connections (uneven load) · IP hash (stickiness). |
| Health checks | Pings servers; pulls dead ones out automatically → self-healing. |
| L4 vs L7 | L4 = fast, IP/port only · L7 = smart, understands HTTP paths/headers. |
| LB redundancy | Duplicate the LB (active-standby) so it isn't itself a single point of failure. |
What's Next?
Season 1 done — you now understand scale, the vital-sign numbers, and the traffic cop. In Chapter 05 we start Season 2: Data & Storage with the eternal question — SQL vs NoSQL — but this time through a designer's eyes: which one to pick, and why, for a given system.
Keep designing, keep scaling! See you in the next one!
Post a Comment