SkillPilotAI

Quick Revision Cheat-Sheet

A one-page cheat-sheet of the whole guide: what each topic does and what its main tools are. Revise from this quickly before an interview.

Part 1: Traffic Entry Gate

#TopicWhat it doesMain tools
1DNS and GeoDNSConverts a domain name (amazon.in) into an IP address, and GeoDNS gives the user the IP of the data center nearest to them.AWS Route 53, Cloudflare DNS, Azure DNS / Traffic Manager, NS1 (IBM), Google Cloud DNS, BIND / CoreDNS
2CDN - Content Delivery NetworkKeeps copies of images, videos and JS/CSS on edge servers around the world so the user gets content fast from nearby.AWS CloudFront, Akamai, Cloudflare CDN, Azure Front Door, Google Cloud CDN, Fastly
3Load Balancer - L4 vs L7, algorithmsSpreads incoming requests across many servers so no single server is overloaded and the site keeps running even if one server is down.Nginx, HAProxy, AWS ALB, AWS NLB, F5 BIG-IP, Azure Load Balancer / Application Gateway, Envoy
4Reverse Proxy and API GatewayA 'receptionist' standing in front of the backend servers that takes all requests, checks them and routes them to the right service.Nginx, Kong, AWS API Gateway, Envoy, Apigee (Google), Azure API Management
5Rate Limiting and ThrottlingPutting a limit on how many requests one user/IP/API key can send in a time window. If it goes over, 429 Too Many Requests.Redis, Nginx limit_req, AWS API Gateway, Envoy, Kong rate-limiting plugin, Cloudflare / AWS WAF rate rules
6Scaling - Vertical vs Horizontal and Auto-scalingIncreasing the system capacity when load grows: either a bigger server (vertical) or more servers (horizontal), and auto-scaling does this by itself.AWS Auto Scaling Group (ASG), Kubernetes HPA, Cluster Autoscaler / Karpenter, Azure VMSS, GCP Managed Instance Groups, KEDA
7Stateless Servers and Session ManagementThe server does not keep user data in its own memory, so any request can go to any server. The session lives either in Redis or in a token (JWT).Redis, Memcached, JWT libraries (jjwt, jsonwebtoken, PyJWT), AWS ALB / Nginx sticky sessions, Amazon ElastiCache / Azure Cache for Redis, Keycloak / AWS Cognito / Auth0
8DDoS Protection, WAF and Bot ProtectionStopping fake traffic floods, hacking attempts (SQL injection, XSS) and bots at the edge, before they reach the website.Cloudflare, AWS Shield, AWS WAF, Akamai (Prolexic, App & API Protector, Bot Manager), Google reCAPTCHA, Azure DDoS Protection + Azure WAF, ModSecurity + OWASP CRS

Part 2: Caching and Database

#TopicWhat it doesMain tools
9Caching - Redis vs Memcached vs Local Cache; Cache-aside, Write-through, Write-back, Read-throughKeep frequently requested data in fast memory so that you do not have to go to the DB at all.Redis, Memcached, Caffeine / Guava (Java), lru_cache (Python), AWS ElastiCache / Azure Cache for Redis / GCP Memorystore, Amazon DynamoDB DAX, CDN (CloudFront, Akamai)
10Cache Eviction, Invalidation, TTL and Cache StampedeWhat to remove when the cache is full, when to remove stale data, and how to stop everything hitting the DB at once. All of it is here.Redis, Memcached, Caffeine (Java), Varnish / Nginx proxy_cache, Debezium + Kafka, CloudFront / Akamai
11SQL vs NoSQL - Which Database WhenChoose the right database by looking at the shape of the data, the scale and the consistency need.PostgreSQL, MySQL, MongoDB, Apache Cassandra, Amazon DynamoDB, Neo4j
12Database Replication and Read ReplicasCopy one DB's data to other servers so reads are shared and, if one server fails, another takes over.MySQL replication, PostgreSQL streaming replication, AWS RDS / Aurora, MongoDB Replica Set, Cassandra, ProxySQL / pgpool-II
13Sharding / PartitioningSplitting a big database into smaller pieces (shards) and placing them on different servers.Vitess, Citus, MongoDB sharding, Amazon DynamoDB, Apache Cassandra, Google Spanner / CockroachDB / YugabyteDB
14Consistent HashingA way of distributing data so that only a little data moves, not all of it, when servers are added/removed.Apache Cassandra, Amazon DynamoDB, Memcached clients (ketama), Nginx / HAProxy, Envoy (ring_hash, Maglev), Akamai / CDNs
15Database Indexing and Query OptimizationMaking the DB up to 100x faster with the right index, the right query and the right connection handling.EXPLAIN / EXPLAIN ANALYZE, pg_stat_statements / MySQL slow query log, PgBouncer, HikariCP, AWS RDS Proxy / ProxySQL, AWS Performance Insights / Azure Query Store / Cloud SQL Insights
16CAP Theorem, PACELC and Consistency ModelsWhen the network breaks in a distributed system, do you want consistency or availability? You cannot get both fully at the same time.Google Spanner / CockroachDB, Apache Cassandra, Amazon DynamoDB, MongoDB, Azure Cosmos DB, etcd / ZooKeeper
17Transactions - ACID and Distributed Transactions (2PC, Saga, Outbox)A multi-step job either completes fully or not at all, whether it is one DB or many microservices.PostgreSQL / MySQL InnoDB, Temporal / Cadence, AWS Step Functions / Azure Durable Functions, Debezium + Kafka, Axon / Eventuate / Seata, Google Spanner / CockroachDB

Part 3: Storage, Search and Data

#TopicWhat it doesMain tools
18Object / Blob Storage vs Block vs File StorageA way to store big unstructured files like images, videos and backups cheaply and almost without limit.AWS S3, Azure Blob Storage, Google Cloud Storage (GCS), MinIO, AWS EBS, AWS EFS / Azure Files
19Image and Video Processing / StreamingConverting an uploaded image/video into different sizes and qualities so it plays smoothly on every device and network.FFmpeg, AWS Elemental MediaConvert, Cloudinary, Azure Media Services / GCP Transcoder API, ImageMagick / libvips (sharp), Video.js / Shaka Player / ExoPlayer
20Search Engine - Full-Text SearchBringing back relevant results in milliseconds from crores of products/documents, based on the words the user typed.Elasticsearch, OpenSearch, Apache Solr, Algolia, Azure AI Search / Vertex AI Search, Meilisearch / Typesense
21Autocomplete / TypeaheadShowing top suggestions as soon as the user types 2-3 letters, within 100 ms on every keystroke.Redis (Sorted Sets), Elasticsearch / OpenSearch Completion Suggester, Custom in-memory Trie service, Algolia, Kafka + Spark/Flink
22Data Warehouse, ETL and Big Data AnalyticsBringing the data of all systems to one place and analysing it for business reports, dashboards and ML.Apache Kafka, Apache Spark, Apache Flink, Snowflake, Google BigQuery, Amazon Redshift, Airflow / dbt / Azure Synapse
23Recommendation System and A/B TestingShowing each user products/content they like, and proving through experiments that a new feature is really better.Amazon Personalize / Vertex AI / Azure ML, Feast / Tecton / SageMaker Feature Store, FAISS / Milvus / OpenSearch kNN, Optimizely, GrowthBook, LaunchDarkly

Part 4: Async Communication and Services

#TopicWhat it doesMain tools
24Message QueueOne service puts a message in the queue, and another service picks it up and processes it at its own speed. Point-to-point, one message to one consumer.RabbitMQ, AWS SQS, Apache ActiveMQ, Azure Service Bus, Google Cloud Tasks
25Pub/Sub vs Message Queue vs Event StreamingIn a queue one message goes to one worker, in Pub/Sub every subscriber gets a copy, and in Event Streaming the message stays saved in a log that anyone can replay any time.Redis Pub/Sub & Streams, Apache Kafka, RabbitMQ, Google Cloud Pub/Sub, AWS SNS, AWS Kinesis / Azure Event Hubs
26Event Streaming with KafkaKafka is a distributed, replicated commit log where events are saved on disk in order and many consumers read at their own speed.Apache Kafka, Confluent Cloud / AWS MSK, AWS Kinesis Data Streams, Apache Pulsar, Azure Event Hubs, Apache Flink / Spark Streaming
27Background Jobs and Task SchedulingWork that is not urgent within the user request (email, report, resize) is done by a background worker, and work that must run at a fixed time is run by a scheduler.Celery, Sidekiq, BullMQ, cron / Kubernetes CronJob, Apache Airflow, AWS EventBridge Scheduler + Lambda
28Real-time Communication - WebSocket vs SSE vs Long PollingIf the server has to push data to the client instantly (chat, live score, location), normal request-response is not enough and you need a persistent connection.Socket.IO, AWS API Gateway WebSocket API, Pusher / Ably, Azure Web PubSub / SignalR Service, Nginx / HAProxy, Firebase Realtime Database / Firestore
29Notification SystemA central service that sends the user a message on the right channel (email, SMS, push, in-app), at the right time, without duplicates.AWS SES, SendGrid, Twilio, Firebase Cloud Messaging (FCM), Apple Push Notification service (APNs), AWS SNS / Amazon Pinpoint
30Monolith vs MicroservicesThe whole app in one deployable unit (monolith) or in small independent services that talk over the network (microservices). In between sits the modular monolith.Kubernetes, Istio, Linkerd, Spring Boot / Spring Modulith, Kong / AWS API Gateway, OpenTelemetry + Jaeger
31API Design and Service Communication - REST vs gRPC vs GraphQLHow services and clients talk to each other: REST (resources over HTTP), gRPC (fast binary RPC) or GraphQL (exactly the data the client asks for).OpenAPI (Swagger), gRPC + Protocol Buffers, Apollo GraphQL / GraphQL Federation, AWS AppSync, Postman, Kong / AWS API Gateway / Apigee
32Service Discovery and Configuration ManagementService discovery tells you 'which IP:port the payment service is on right now', and config management gives all services their settings/secrets from one central place.HashiCorp Consul, etcd, Apache ZooKeeper, Kubernetes DNS (CoreDNS), Spring Cloud Config / Netflix Eureka, AWS Systems Manager Parameter Store / Secrets Manager

Part 5: Reliability and Correctness

#TopicWhat it doesMain tools
33Timeouts, Retries, Exponential Backoff and Circuit BreakerSo that one slow or down service does not bring down the whole system, we add timeouts, smart retries and circuit breakers.Resilience4j, Netflix Hystrix, Polly, Istio / Envoy, AWS SDK / App Mesh, Linkerd
34Idempotency and Exactly-once processingWhether the same request arrives once or 10 times, the result is applied only once. That is idempotency.Redis (SET NX EX), PostgreSQL / MySQL unique constraint, Apache Kafka (idempotent producer + transactions), AWS SQS FIFO, DynamoDB conditional write, Stripe API Idempotency-Key
35High Availability, Failover, Multi-Region and Disaster RecoveryThe website keeps running and there is no data loss even when a server, a data center or a whole region goes down.AWS Route 53, AWS Aurora Global Database / RDS Multi-AZ, Azure Traffic Manager / Front Door, GCP Cloud Spanner / Cloud Load Balancing, Velero / AWS Backup, Patroni / Orchestrator
36Distributed Locking and Leader ElectionDistributed locks and leader election make sure that out of many servers only one does a job at a time, or only one becomes the leader.Redis / Redisson, Apache ZooKeeper / Curator, etcd, Consul, PostgreSQL advisory locks / SELECT FOR UPDATE, DynamoDB lock client / Azure Blob lease
37Unique ID GenerationGiving every order, tweet and message a unique ID across thousands of servers without any clash.Twitter Snowflake (and clones: Sonyflake, Baidu UidGenerator), UUID libraries (java.util.UUID, Python uuid, gen_random_uuid()), ULID libraries, MySQL / PostgreSQL AUTO_INCREMENT / SEQUENCE, Redis INCR, MongoDB ObjectId
38Concurrency Control - Inventory, Flash Sale and Overselling1000 phones and 10 lakh buyers: not a single extra phone should be sold and the system should not fall over either.Redis (DECR, Lua scripts), MySQL / PostgreSQL row locks, Apache Kafka / AWS SQS FIFO, DynamoDB conditional writes, Redisson / ZooKeeper locks, Virtual waiting room (Queue-it, Cloudflare Waiting Room)
39Backpressure, Load Shedding and Graceful DegradationWhen traffic exceeds capacity, instead of the whole system crashing, reject some requests or switch off features so that the core keeps running.Queue-it / Cloudflare Waiting Room / Akamai Visitor Prioritization, LaunchDarkly / Unleash / AWS AppConfig, Envoy / Istio / Nginx, Netflix concurrency-limits / Resilience4j Bulkhead, Kafka / RabbitMQ / Reactor (Reactive Streams), Kubernetes HPA / AWS Auto Scaling
40Payment System DesignThe money is deducted only once, goes to the right account, and every rupee is accounted for. That is the job of a payment system.Razorpay, Stripe, UPI (NPCI), PayU / Cashfree / Juspay, PostgreSQL / MySQL, Kafka, TigerBeetle

Part 6: Security and Observability

#TopicWhat it doesMain tools
41Authentication and AuthorizationAuthentication = who you are (verify identity), Authorization = what you can do (check permissions).Keycloak, Auth0 (Okta), AWS Cognito, Okta Workforce Identity, Microsoft Entra ID (Azure AD), Open Policy Agent (OPA)
42Encryption, HTTPS/TLS and Secrets ManagementEncrypting data on the way (in transit) and on disk (at rest), and keeping passwords/API keys outside the code in a safe vault.HashiCorp Vault, AWS KMS, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager / Cloud KMS, Let's Encrypt + cert-manager
43Centralized LoggingCollecting the logs of all servers and services in one central place and searching, filtering and analysing them.ELK / Elastic Stack, Grafana Loki, Splunk, AWS CloudWatch Logs, Fluent Bit / Fluentd, Azure Monitor Logs / GCP Cloud Logging
44Monitoring, Metrics and AlertingContinuously measuring the numbers of system health (CPU, latency, errors), viewing them in graphs, and alerting the team the moment something goes wrong.Prometheus, Grafana, Datadog, AWS CloudWatch, PagerDuty, Azure Monitor / Google Cloud Monitoring
45Distributed TracingThe whole journey of one request: which services it went through and how much time it took in each. Seen in a single timeline.OpenTelemetry (OTel), Jaeger, Zipkin, AWS X-Ray, Grafana Tempo, Datadog APM

Part 7: Deployment, DevOps and Planning

#TopicWhat it doesMain tools
46Containers and OrchestrationPacking the app with all its dependencies into one box (container), and running, scaling and healing thousands of boxes automatically.Docker, Kubernetes (K8s), AWS ECS (with Fargate), AWS EKS, Azure AKS, AWS Lambda, Helm
47CI/CD and Deployment StrategiesAn automatic pipeline for build, test and deploy from code commit to production, and ways to release a new version without downtime.Jenkins, GitHub Actions, GitLab CI, ArgoCD, AWS CodePipeline / CodeDeploy, LaunchDarkly / Unleash
48Infrastructure as Code (IaC)Creating infra like servers, network, load balancers and databases by writing code and keeping it in Git, not by clicking in a console.Terraform / OpenTofu, AWS CloudFormation, AWS CDK, Pulumi, Ansible, Azure Bicep / ARM
49Load Testing, Performance Testing and Chaos EngineeringBefore the real sale arrives, checking how much load and how much breakage the system can take by injecting fake traffic and deliberate failures.Apache JMeter, k6 (Grafana k6), Gatling, Locust, Chaos Monkey, AWS Fault Injection Service (FIS), Gremlin / Chaos Mesh / LitmusChaos
50Capacity Planning and Back-of-the-envelope EstimationEstimating in advance, with rough maths, how many QPS, how much storage, how much bandwidth and how many servers you need.Spreadsheet (Excel / Google Sheets), Grafana + Prometheus, AWS CloudWatch, AWS Pricing Calculator, AWS Compute Optimizer / Trusted Advisor, Load testing tools (k6, JMeter)

Part 8: Networking, Performance and Advanced Data

#TopicWhat it doesMain tools
51TCP vs UDP, HTTP/1.1 vs HTTP/2 vs HTTP/3 (QUIC), Keep-Alive and Connection ReuseHow data travels on the wire (TCP/UDP) and how the web protocol HTTP evolved: a fast website through fewer handshakes, multiplexing and connection reuse.Nginx, Envoy, Cloudflare, curl --http3, AWS CloudFront / ALB, Chrome DevTools / Wireshark
52Web Performance - Compression, Browser Caching Headers, Lazy Loading, SSR vs CSR vs SSGShowing the page quickly on the user's phone: send fewer bytes (Gzip/Brotli), do not send again (Cache-Control/ETag), load on need (lazy) and choose the right rendering.Next.js, Nginx, Lighthouse, WebPageTest, Cloudflare / CloudFront, Webpack / Vite
53CQRS and Event SourcingCQRS: separate models/DBs for write (command) and read (query); Event Sourcing: storing every change as an event instead of the current state.Axon Framework, EventStoreDB (KurrentDB), Apache Kafka, Debezium, DynamoDB Streams, Elasticsearch / Redis
54Consensus and Cluster Coordination - Raft, Paxos, Quorum (N/R/W), Gossip ProtocolHow many servers agree on one thing (who is the leader, what the latest value is) when some servers are down or the network breaks.etcd, Apache ZooKeeper, HashiCorp Consul, Apache Cassandra, Kafka KRaft, Amazon DynamoDB
55Probabilistic Data Structures - Bloom Filter, HyperLogLog, Count-Min SketchAn approximate answer in very little memory: 'this item may be present or is definitely absent', 'how many unique users', 'how many times an item appeared'.Redis (HyperLogLog), Redis Stack / RedisBloom, Google Guava, Apache Cassandra, Apache DataSketches, BigQuery / Presto approx functions
56Geo-spatial and Location Services - Geohash, Quadtree, S2/H3Indexing latitude/longitude data so that a query like 'which drivers/restaurants are within 2 km of me' runs in milliseconds.Redis GEO, PostGIS (PostgreSQL), Elasticsearch / OpenSearch geo, Uber H3, Google S2 Geometry, MongoDB 2dsphere, AWS Location Service / Google Maps Platform
57Specialised Databases - Time-series, Graph, NewSQL, Wide-column (Polyglot Persistence)Every kind of data has a different nature, so one DB is not right for every job. Choosing the right DB for the right data is polyglot persistence.Prometheus / InfluxDB / TimescaleDB, Neo4j / Amazon Neptune, Google Spanner, CockroachDB / TiDB / YugabyteDB, Apache Cassandra / ScyllaDB, Amazon Keyspaces / Azure Cosmos DB / Bigtable
58Zero-downtime Database Migration and Schema ChangesAdding/renaming a column in a table while the live website keeps running, or moving the whole DB to a new system, without downtime and without data loss.gh-ost (GitHub), pt-online-schema-change (Percona), Flyway, Liquibase, AWS DMS / Azure DMS / GCP Database Migration Service, Debezium, pgroll / PostgreSQL native
59BFF (Backend for Frontend) and Mobile BackendA thin backend layer for each client (Android, iOS, web, TV) that stitches microservice data together according to what that screen needs.Apollo GraphQL / GraphQL Federation, Node.js (Express/NestJS) / Spring Boot, Firebase, AWS Amplify / AppSync, Azure API Management / Kong / AWS API Gateway, Supabase
60Large File Upload and DownloadUploading/downloading GB-sized files (video, backups, documents) in pieces, in a resumable way, straight to object storage, without the app server in the middle.Amazon S3, Azure Blob Storage (SAS), Google Cloud Storage, tus (tusd server, Uppy client), ClamAV / cloud malware scanning, CloudFront / Azure Front Door / Cloud CDN, MinIO

Part 9: Business, Compliance and Cost

#TopicWhat it doesMain tools
61Fraud Detection and Risk EngineChecking every order, payment, login or signup in real time and giving a risk score: allow, challenge (OTP/CAPTCHA) or block.AWS Fraud Detector, Sift, Drools, Apache Flink, Redis / Feature Store (Feast), Neo4j
62Compliance and Data Privacy (DPDP, GDPR, PCI-DSS, Data Localisation)Collecting, storing, protecting and deleting users' personal and payment data as the law requires, and keeping proof (audit).HashiCorp Vault, AWS Macie, Microsoft Purview (Azure Purview), AWS KMS / Azure Key Vault, Card network / PG tokenisation (Visa, Mastercard, Razorpay, PayU), AWS CloudTrail / Azure Monitor + S3 Object Lock
63Multi-tenancy and Data IsolationRunning many customers (tenants) on the same software/infra while keeping each tenant's data and performance separate and safe from the others.PostgreSQL Row-Level Security, Separate DB per tenant (RDS / Azure SQL elastic pools), Citus (distributed PostgreSQL), Kubernetes Namespaces, AWS IAM + separate accounts (AWS Organizations)
64Cloud Cost Optimization / FinOpsUnderstanding, tracking and reducing the cloud bill without hurting performance, with engineering, finance and business together.AWS Cost Explorer, Azure Cost Management + Billing, Kubecost / OpenCost, AWS Compute Optimizer, Karpenter / Cluster Autoscaler, S3 Lifecycle / Intelligent-Tiering