Load Balancer: L4 vs L7, algorithms
A Load Balancer spreads incoming requests across many servers so no single server is overloaded and the site keeps working even if one server goes down.
3.1 Why is it used?
- One server cannot handle lakhs of requests. You need 10-100 servers and a single entry point (the LB) in front of them.
- High availability: when a health check fails, the LB stops sending traffic to that server.
- Zero-downtime deploy: take servers out of the LB one by one and update them (rolling deploy).
- Doing SSL/TLS termination at the LB saves CPU on the backend servers.
- With auto-scaling, new servers join the LB's target group automatically.
3.2 How it works
- 1.Users open flipkart.com; DNS gives the IP of the nearest data center's L4 LB (AWS NLB)
- 2.The L4 LB looks only at IP + TCP port, does not peek inside the packet, and spreads TCP connections across many L7 proxies
- 3.L7 proxies (Nginx / Envoy) read the URL: /search -> Search service, /cart -> Cart service, /payment -> Payment service
- 4.Out of the Cart service's 200 pods, the proxy picks the freest pod (Pod B, 2 conn) using 'least connections'
- 5.The slow pod (Pod C) fails its health check 3x -> removed from the pool; when traffic goes 10x, +100 new pods auto-scale
L4 = very fast, L7 = smart routing by URL; together they give both speed and smartness
- L4 LB (Transport layer): looks only at IP + TCP/UDP port and does not peek inside the packet. Very fast, millions of connections.
- L7 LB (Application layer): looks at the HTTP URL, headers and cookies. It can send /api to API servers and /images to image servers.
- Round Robin: one request to each server in turn. Weighted Round Robin: a bigger server gets a bigger share.
- Least Connections: the request goes to the server with the fewest active connections right now. Good for long connections.
- IP Hash / Consistent Hash: the server is fixed by a hash of the client IP or a key. The same user lands on the same server (cache locality).
- Least Response Time / Power of Two Choices: pick 2 random servers and give the request to the less busy one. Popular in Envoy/Nginx.
- Health checks: active (the LB hits /health every few seconds) and passive (removing a server after seeing errors).
- The LB itself must not become a single point of failure: use an active-passive pair (keepalived/VRRP) or a managed cloud LB that is HA by itself.
- Connection draining: before removing a server let its in-flight requests finish and send no new ones (essential in a rolling deploy).
- 1.Round Robin: each in turn (S1 req 1, 4; S2 req 2; S3 req 3); Weighted RR = a bigger server gets a bigger share
- 2.Least Connections: the one with the fewest active conn (S1 8, S2 2, S3 5 -> S2); best for long connections
- 3.IP Hash / Consistent Hash: hash(client IP) % 3 = 2 -> user X always goes to S3, cache locality
- 4.Power of Two Choices: pick 2 random servers (S2, S3) and give it to the less busy one; popular in Envoy / Nginx
3.3 Which tools can be used
| Tool | Type | How it is used |
|---|---|---|
| Nginx | Open-source L7 (and L4 stream) LB | List servers in the upstream block; supports round robin, least_conn, ip_hash, hash. |
| HAProxy | Open-source L4/L7 LB | Very high performance, detailed stats and health checks. A favourite of big on-prem setups. |
| AWS ALB | Managed L7 LB | Path/host based routing, HTTPS termination, WebSocket, gRPC, target groups with EC2/ECS/Lambda. |
| AWS NLB | Managed L4 LB | TCP/UDP/TLS, static IP per AZ, millions of requests per second, ultra-low latency. |
| F5 BIG-IP | Hardware/virtual enterprise ADC | L4-L7 LB in bank and telecom data centers, SSL offload, iRules scripting. |
| Azure Load Balancer / Application Gateway | Managed L4 / L7 (Azure) | Azure LB = L4, Application Gateway = L7 with WAF. |
| Envoy | Open-source L7 proxy | L7 load balancing, retries and circuit breaking inside a service mesh (Istio) and microservices. |
3.4 Comparison: Nginx vs HAProxy vs AWS ALB vs AWS NLB
| Point | Nginx | HAProxy | AWS ALB | AWS NLB |
|---|---|---|---|---|
| Layer | L7 (L4 via the stream module) | L4 + L7 | L7 only (HTTP/HTTPS/gRPC) | L4 only (TCP/UDP/TLS) |
| Speed | Very fast | Very fast, LB-focused | Fast, slightly higher latency (because of L7 parsing) | Fastest, ultra-low latency |
| Routing smartness | Path, host, header based | Path, header, ACL based, very flexible | Path, host, header, query based rules | Only port/protocol based |
| Management | You must install, configure and build HA yourself | You must manage it yourself | Fully managed, auto-scales | Fully managed, static IP |
| Cost | Free (open-source), server cost | Free (open-source), server cost | Per hour + LCU | Per hour + NLCU |
| Extra | Also a web server + cache | Best-in-class stats page | WAF, Cognito auth integration | Static/Elastic IP, preserves client IP |
| Best for | On-prem/VM, both LB and reverse proxy | High-traffic on-prem TCP/HTTP | Web apps and microservices on AWS | Gaming, IoT, non-HTTP, extreme traffic |
3.5 When to choose which
- AWS ALB: a normal web/API app on AWS where you need path based routing.
- AWS NLB: TCP/UDP traffic, when you need a static IP, or a huge number of connections (gaming, MQTT, database proxy).
- Nginx/HAProxy: on-prem or multi-cloud, when you want full control or want to save cost.
- F5: an enterprise data center that has already invested in it and needs advanced SSL/security.
- Common pattern: NLB (L4) in front, with Nginx/Envoy (L7) behind it. You get both speed and smart routing.
3.6 Real-world example
Scenario: sale day traffic on a site like Flipkart
- The user opens flipkart.com; DNS gives the user the IP of the nearest data center's L4 load balancer.
- The L4 LB spreads the TCP connection across many L7 proxies (Nginx/Envoy).
- The L7 proxy looks at the URL: /search -> search service, /cart -> cart service, /payment -> payment service.
- The cart service has 200 pods; the proxy picks the freest pod using 'least connections'.
- One pod becomes slow because of a memory issue; its health check fails 3 times and the LB removes it from the pool.
- Traffic goes 10x, auto-scaling creates 100 new pods, and they pass the health check and join the pool automatically.
- The user does not even notice that servers are coming and going behind the scenes. That is the magic of load balancing.
3.7 Interview tips
- In the interview explain the difference between L4 and L7 and 2-3 algorithms (round robin, least connections, consistent hashing) with examples.
- Mistake: running the LB on a single instance. The LB itself becomes a single point of failure.
- Mistake: health-checking only '/', which returns 200 even when the DB is down. Build a proper /health endpoint that checks dependencies.
- Mistake: losing the client IP behind an L7 LB. Use the X-Forwarded-For header or Proxy Protocol.
- Number: 3 failed health checks = the server is out of the pool; common pattern NLB (L4) + Nginx/Envoy (L7).