High Availability, Failover, Multi-Region and Disaster Recovery

Even if a server, a data center or a whole region goes down, the website keeps running and there is no data loss.

Hardware fails, data centers have power/network outages, and even a whole cloud region can go down.
On sale day, 1 hour of downtime means a loss of crores and brand damage.
99.9% availability = roughly 8.7 hours of downtime a year, 99.99% = only ~52 minutes. Every extra 9 is very expensive.
Ransomware, a wrong DELETE query or a bug can corrupt data; only a backup saves you.
In sectors like banking and payments, regulators demand DR drills and defined RTO/RPO.

Redundancy: no single point of failure. At least 2 copies of every layer (LB, app, DB), in different Availability Zones.
RTO (Recovery Time Objective): how soon the service must be back after a disaster, for example 15 minutes.
RPO (Recovery Point Objective): the maximum acceptable data loss, for example the last 5 minutes. With sync replication RPO is ~0.
Active-Passive: one region takes traffic, the other is standby. On failover, DNS switches. Cheap, but failover takes time.
Active-Active: both regions take live traffic. RTO is almost zero, but data conflicts and multi-region writes are complex.
DR strategies (cheapest to costliest): Backup & Restore, Pilot Light, Warm Standby, Multi-site Active-Active.
Health checks + automatic failover: the LB or DNS stops sending traffic to an unhealthy instance/region.
Backups: the 3-2-1 rule (3 copies, 2 media, 1 offsite), point-in-time recovery, and regular restore testing.

HA and DR: Active-Passive, Active-Active and the 4 DR strategies

1.A Active-Passive (Warm Standby): Route 53 failover policy + health check; Mumbai ACTIVE (ALB, App AZ-a/AZ-b, DB Primary Multi-AZ sync) -> Singapore PASSIVE (ALB standby, small app fleet, DB Replica via async replication)
2.Mumbai fails -> health check fails -> Route 53 DNS switches to Singapore, replica is promoted; RTO in minutes, a few seconds of data loss possible
3.B Active-Active (Netflix region evacuation): Global users -> DNS / global routing -> us-east-1, us-west-2, eu-west-1 all LIVE; error rate high in one region -> users shifted to healthy regions, data already replicated, only latency increases; failover tested regularly with Chaos Monkey
4.C RPO vs RTO timeline: last backup / replica sync -> DISASTER -> service back; RPO = data loss 5 min, RTO = downtime 15 min
5.D 4 DR strategies (cheapest to costliest, cost rises while RTO falls): Backup & Restore (RTO hours, S3 / AWS Backup, 3-2-1 rule) -> Pilot Light (only the DB replica on, RTO ~1 hour) -> Warm Standby (small fleet live, RTO minutes) -> Multi-site Active-Active (2x full infra, RTO ~0)
6.Uptime: 99.9% = ~8.7 hours downtime / year, 99.99% = only ~52 min / year

The costlier the DR, the lower the RTO/RPO

Tool	Type	How it is used
AWS Route 53	DNS with health checks	Failover, latency and weighted routing; if the primary region is unhealthy, DNS points to the secondary.
AWS Aurora Global Database / RDS Multi-AZ	Managed DB HA	A sync standby across AZs with automatic failover; Aurora Global adds a cross-region replica, typically under 1 second of lag.
Azure Traffic Manager / Front Door	Global traffic routing	Region failover through priority or performance routing; Front Door also gives global L7 load balancing.
GCP Cloud Spanner / Cloud Load Balancing	Global DB + LB	Spanner gives multi-region strongly consistent writes; the global LB takes users to the nearest healthy region through a single anycast IP.
Velero / AWS Backup	Backup tool	Backup of Kubernetes resources and volumes (Velero) and central scheduled backup of cloud resources (AWS Backup).
Patroni / Orchestrator	DB failover manager	Patroni for PostgreSQL and Orchestrator for MySQL perform automatic primary election and failover.

Point	Backup & Restore	Active-Passive	Active-Active
RTO	Hours	Minutes	Seconds or near zero
RPO	Up to the last backup (hours)	Seconds to minutes (async replication)	Near zero, depends on design
Cost	Cheapest	Medium, standby infra sits idle	Costliest, 2x full infra
Complexity	Low	Medium, needs a failover runbook	High, data conflicts, global routing
Data consistency	Simple	Simple, a single writer	Hard, multi-writer conflicts
User latency	Single region	Single region	Users are served from the nearest region
Best for	Internal tools, dev, archives	Most e-commerce / enterprise apps	Netflix, global payments, critical services

Backup & Restore: when a few hours of downtime is acceptable and the budget is low.
Active-Passive / Warm Standby: when you need RTO in minutes and want to keep a single-writer DB.
Active-Active: when you have global users, need no downtime at all, and the team can handle the complexity.
In every case keep Multi-AZ as the minimum; decide on multi-region based on the business need.

Real world: Netflix region evacuation

Netflix runs active-active across multiple AWS regions (according to public engineering blogs).
Scenario: a major problem hits one region and the error rate goes up.
Netflix's traffic team shifts that region's users to other healthy regions by changing DNS and routing; this is called region evacuation.
The other regions are already live and the data is replicated, so users only see slightly higher latency and the service does not stop.
Netflix deliberately kills instances in production with tools like Chaos Monkey so that failover is tested regularly.
Lesson: a DR plan only works if it is drilled regularly, not just written in a document.

Taking backups but never testing a restore; on the day of the disaster you find the backup is corrupt.
Going multi-region for everything but keeping a 1-day DNS TTL: even after failover, users keep going to the old region.
Ignoring split-brain: in a network partition, if both sides think they are primary, data gets corrupted.
Interview tip: explain the difference between RTO and RPO, and the 4 DR strategies in order of cost.