Scaling: Vertical vs Horizontal and Auto-scaling
Increasing a system's capacity as load grows: either a bigger server (vertical) or more servers (horizontal), and auto-scaling does this by itself.
6.1 Why is it used?
- 10,000 users on a normal day, 10 lakh on a sale day. Fixed servers will either be too few, or money is wasted on the other days.
- Vertical scaling has a limit. Even the biggest server stops getting bigger after a point, and it is a single point of failure.
- Horizontal scaling gives almost unlimited growth and high availability. If one server dies the others keep running.
- Auto-scaling saves cost by reducing servers at night and adds them by itself on a spike.
- Handling traffic spikes without manual intervention. No need to wake an engineer at 2 AM.
6.2 How it works
- 1.Vertical: Server 4 vCPU / 16 GB -> upgrade -> Bigger Server 32 vCPU / 256 GB RAM; no code change, simple
- 2.Limits of vertical: a max size limit, downtime on upgrade, 1 server = SPOF
- 3.Horizontal: Server 1 ... Server 4 behind an LB, then Server 5 (new) is added; almost unlimited growth + HA
- 4.In horizontal, if one server dies the others keep running, but the app must be stateless
- 1.10 PM: normal day 50 pods, CPU ~40%
- 2.11 PM: 500 pods pre-warmed through scheduled scaling
- 3.12 AM: traffic 20x; Kubernetes HPA (CPU > 70%) takes pods 500 -> 900
- 4.Pods Pending: Cluster Autoscaler / Karpenter brings new nodes (1-2 min); scale the DB separately, grow Redis cache + read replicas in advance
- 5.3 AM: traffic falls, and after the stabilization window pods go 900 -> 600 -> 300 -> 100 (4 AM)
HPA adds pods based on CPU / memory / custom metrics; AWS ASG / Azure VMSS keep min / max / desired for VMs
- Vertical (Scale Up): add CPU/RAM to the same machine (4 vCPU -> 32 vCPU). Simple, no code change, but limited and has downtime.
- Horizontal (Scale Out): add more machines behind an LB. The app must be stateless.
- Auto-scaling policies: Target tracking (keep CPU at 60%), Step scaling (if CPU > 80% then +5), Scheduled (200 servers at 9 AM before the sale).
- Predictive scaling: ML forecasts from the pattern of previous weeks so capacity is ready in advance.
- Metrics: CPU, memory, requests per target, queue length (SQS/Kafka lag), custom metrics.
- Cooldown / stabilization window: wait a little after scaling so it does not keep going up and down (flapping).
- Warm-up time: a new VM takes 1-3 min to boot, a container takes seconds. Scale before the spike or keep a warm pool.
- Database scaling is a separate challenge: read replicas, sharding, caching. Scaling app servers is easy, the DB is not.
6.3 Which tools can be used
| Tool | Type | How it is used |
|---|---|---|
| AWS Auto Scaling Group (ASG) | VM auto-scaling (AWS) | Creates EC2 from a launch template, min/max/desired capacity, target tracking, integrates with the ALB. |
| Kubernetes HPA | Pod auto-scaling | Increases/decreases the number of pods of a Deployment based on CPU/memory/custom metrics. |
| Cluster Autoscaler / Karpenter | Node auto-scaling (Kubernetes) | If pods are Pending it adds new nodes, and it removes empty nodes. Works as a pair with HPA. |
| Azure VMSS | VM auto-scaling (Azure) | A group of identical VMs, with autoscale rules based on a metric or a schedule. |
| GCP Managed Instance Groups | VM auto-scaling (GCP) | Autoscaling from an instance template, autohealing and regional distribution. |
| KEDA | Event-driven autoscaler (Kubernetes) | Scales pods on events like Kafka lag, queue length and cron, even down to zero. |
6.4 Comparison: AWS ASG vs Kubernetes HPA vs Azure VMSS
| Point | AWS ASG | Kubernetes HPA | Azure VMSS |
|---|---|---|---|
| What it scales | EC2 virtual machines | Pods (containers) | Azure virtual machines |
| Scale speed | Minutes (VM boot) | Seconds (if the node has room) | Minutes (VM boot) |
| Metrics | CloudWatch: CPU, ALB requests, custom | CPU, memory, custom/external metrics | Azure Monitor: CPU, memory, custom |
| Policies | Target tracking, step, scheduled, predictive | Target utilization, behavior rules | Metric-based, schedule-based |
| Cloud lock-in | AWS only | On any cloud/on-prem | Azure only |
| Complexity | Medium | Higher (you must manage the cluster) | Medium |
| Best for | VM-based apps on AWS | Microservices, containers | VM-based apps on Azure |
6.5 When to choose which
- Vertical: the early stage, or a database where horizontal is hard. Take a bigger instance first.
- Horizontal + ASG/VMSS: monoliths or VM-based web apps.
- Kubernetes HPA + Cluster Autoscaler/Karpenter: microservices running in containers.
- Scheduled scaling: when you know the time of the spike (sale at 12 AM, IPL toss at 7 PM). Scale in advance.
- KEDA: queue/event based workers where the load depends on the number of messages.
6.6 Real-world example
Scenario: a midnight sale like Big Billion Day
- On a normal day the app runs 50 pods/servers, CPU ~40%.
- The team knows the sale starts at 12 AM, so at 11 PM they pre-warmed 500 pods using scheduled scaling.
- At 12:00 AM traffic is 20x; HPA sees CPU above 70% and adds pods (500 -> 900).
- The nodes run out of room, pods go Pending; Cluster Autoscaler/Karpenter brings new nodes (1-2 min).
- To keep the load on the DB low, the Redis cache and read replicas were scaled up in advance.
- At 3 AM traffic falls; after the stabilization window pods slowly come back down to 100.
- Benefit: the spike was handled and there was no bill for extra servers after the sale.
6.7 Interview tips
- Say the pros/cons of vertical vs horizontal and 'scale out the stateless tier, and for the DB use cache + replica + sharding'.
- Mistake: scaling a stateful app horizontally. If the session is in the server's memory the user will be logged out. Make the app stateless first.
- Mistake: relying only on reactive scaling. By the time the VM boots the spike is gone. Use scheduled/pre-warm for known events.
- Mistake: scaling the app servers but leaving the DB/connection pool the same. The bottleneck shifts to the DB.
- Number: VM boot 1-3 min, container seconds; a typical target tracking value is CPU 60%.