Containers and Orchestration

Pack an app together with all its dependencies into one box (a container), then run, scale and heal thousands of those boxes automatically.

The 'it worked on my laptop' problem goes away: code + libraries + runtime ship together in the container, so it runs the same everywhere.
Compared to a VM a container is light (starts in seconds, uses less RAM), so one server can run more apps.
If a sale day needs 50 to 500 instances, the orchestrator (Kubernetes/ECS) starts the new containers by itself.
If a container crashes, the orchestrator restarts it automatically or moves it to another node (self-healing).
Perfect for microservices: each service gets its own container, its own version and its own scaling.

Docker image: a read-only template built from a Dockerfile (base OS + app + dependencies), stored in a registry (Docker Hub, ECR, ACR).
Container: a running instance of an image; it uses Linux namespaces (isolation) and cgroups (CPU/RAM limits), and shares the kernel with the host.
Kubernetes Pod: the smallest deployable unit, one or more containers sharing the same network/IP.
Deployment + ReplicaSet: say 'I want 10 copies' and K8s always keeps 10 pods running; it also performs rolling updates.
Service + Ingress: pod IPs keep changing, so a Service gives a stable virtual IP/DNS and an Ingress routes outside HTTP traffic in.
HPA (Horizontal Pod Autoscaler): watches CPU or a custom metric (such as QPS) and adds/removes pods; Cluster Autoscaler/Karpenter brings in new nodes.
The control plane (API server, etcd, scheduler) decides which pod goes to which node; the kubelet on each worker node runs the pods.
Serverless (AWS Lambda): no servers or containers to manage, just upload a function; you pay per request, but with cold starts and a 15 min max runtime.

From code to running pods

HPA adds pods and Cluster Autoscaler/Karpenter adds nodes; if a node crashes its pods are rescheduled on another node.

Tool	Type	How it is used
Docker	Container runtime / build tool	Build an image from a Dockerfile, run containers locally, push images to a registry.
Kubernetes (K8s)	Open-source orchestrator	Handles scheduling, scaling, self-healing, service discovery and rolling updates for thousands of containers.
AWS ECS (with Fargate)	AWS managed orchestrator	AWS's simple orchestrator; with Fargate you manage no servers, you only provide a task definition.
AWS EKS	Managed Kubernetes	AWS manages the control plane, and you run standard K8s workloads on worker nodes or Fargate.
Azure AKS	Managed Kubernetes	Azure's managed K8s; the control plane is free/managed and integrates with Azure AD and Azure networking.
AWS Lambda	Serverless compute	For small event-driven functions (image resize, order event processing); an alternative to containers.
Helm	K8s package manager	Package K8s YAML files into charts and install/upgrade a whole app with one command.

Point	Kubernetes (EKS/AKS)	AWS ECS (Fargate)	AWS Lambda
Complexity	Highest; takes effort to learn and run	Medium; simpler than K8s via AWS console/CLI	Lowest; just upload code
Portability	Runs on any cloud or on-prem	Locked to AWS	Locked to AWS
Scaling speed	Pods in seconds, new nodes in 1-2 min	Tasks in ~30-60 sec	Milliseconds, but with cold start delay
Control	Full control: networking, scheduling, sidecars	Limited but enough	Very little; just runtime, memory, timeout
Long running	Yes, best for 24x7 services	Yes	No, max 15 minutes per invocation
Cost model	You pay for nodes whether used or not, plus the EKS control plane fee	vCPU/RAM per second	Per request + execution time; zero when idle
Ecosystem	Huge: Helm, Istio, ArgoCD, Prometheus	Limited to AWS services	AWS events (S3, SQS, API Gateway)
Best for	Large microservice fleets, multi-cloud	AWS-only teams that want simplicity	Spiky, event-driven, small tasks

Kubernetes (EKS/AKS/GKE): when you have 50+ microservices, a platform team, and want to avoid cloud lock-in.
AWS ECS + Fargate: when the whole stack is on AWS and the team is small, without the complexity of K8s.
AWS Lambda: when traffic is very uneven or the work is event-driven (S3 upload, SQS message) and idle cost must be zero.
Plain Docker on a VM: only for dev/test or one or two services; use an orchestrator at production scale.

Kubernetes autoscaling on an e-commerce sale day (an illustrative scenario).

A Flipkart-like site has its product, cart and payment services packed in separate Docker images and running on an EKS cluster.
On a normal day cart-service runs 20 pods, each handling 500 req/sec.
One day before the sale the team raises the HPA minReplicas from 20 to 150 and pre-scales the nodes (pre-warming).
When the sale starts traffic jumps 10x; HPA scales pods from 150 up to 300 once CPU crosses 70%.
If nodes run short for the new pods, Cluster Autoscaler/Karpenter launches new EC2 nodes.
If a node crashes, K8s automatically reschedules its pods on other nodes and users never notice.
Async work such as the order-confirmation email runs on Lambda, which scales itself with the SQS messages.
After the sale HPA scales pods down and the extra nodes are removed, so cost returns to normal.

State the container vs VM difference clearly (shared kernel, namespaces, cgroups); this is what interviewers ask.
Common mistake: not setting CPU/memory requests and limits on pods, so one pod eats a whole node and the others crash.
Skipping liveness and readiness probes: traffic then goes to a pod that is not ready yet.
Putting something stateful like a database into K8s without thought; a managed DB (RDS, Aurora) is usually better. Numbers to remember: Lambda max 15 min, K8s pods scale in seconds and nodes in 1-2 min.