Case Study 7: IRCTC Tatkal Ticket Booking under Extreme Spike

At 10 AM lakhs of people rush for limited Tatkal seats at the same moment; the system must stay fair, fast and bot-proof.

C.1 Requirements

Functional requirements:

• The user logs in, searches trains and sees seat availability.
• Accept booking requests the moment the Tatkal window opens (AC 10:00, Non-AC 11:00).
• Virtual waiting room: keep excess users in a queue and let them in when their turn comes.
• Temporarily lock the seat until payment completes (for example 10-15 min).
• Take payment through a payment gateway / UPI and generate the PNR on success.
• On payment failure or timeout, release the seat lock and return the seat to the pool.
• Stop misuse by bots and agents (CAPTCHA, rate limit, per-user booking limit).
• Send the PNR confirmation by SMS / email.

Non-functional requirements:

• Handle 10-15 lakh concurrent users at peak without crashing (graceful degradation via a queue).
• Zero overselling: one seat to exactly one passenger; strong consistency on inventory.
• Fairness: first come, first served (FIFO queue); bots must not get an advantage.
• Seat lock + booking API p99 latency < 500 ms once the user is let in.
• Exactly-once booking with payment: no double charge or double PNR.
• 99.95% availability during the Tatkal window.

C.2 Estimation (back-of-the-envelope)

C.3 API design

C.4 Data model

C.5 High-level architecture

• Edge: CDN static pages + cached queue status; the WAF + Bot Manager blocks bots at the edge and gives suspicious users a CAPTCHA.
• LB -> Virtual Waiting Room: a token in a Redis ZSET; it admits at a fixed rate of ~5,000/s and issues a signed JWT admit-token.
• The API Gateway verifies the token and passes the request to Booking Service; seat lock via Redis Lua with a 15 min TTL, the hold in the Inventory DB.
• Payment Service takes payment via the gateway; on the webhook, BOOKED + PNR in the Bookings DB, with notification and reconciliation through Kafka.
• The Hold Expiry Worker sets the seat back to FREE when the Redis key expires.

C.6 Main flows

Getting in through the waiting room:

1. At 9:58 the user logs in; the WAF + CAPTCHA do the bot check.
2. At 10:00 the user does POST /queue/join and a token with the arrival time is added to the Redis ZSET.
3. Pre-window arrivals are randomized so those who refreshed early get no unfair advantage.
4. The client polls the status every 5-10 sec (a lightweight, CDN-cacheable endpoint).
5. Every second the admission controller marks the top N tokens ADMITTED (N = backend capacity).
6. The admitted user gets a short-lived signed admit-token (for example valid for 10 min).
7. The API Gateway verifies the admit-token on every booking call; requests without a token are rejected.

Seat lock and payment flow:

1. The user selects train, class and passengers and sends POST /bookings/hold with an Idempotency-Key.
2. Booking Service runs a Redis Lua script: check free berths + mark HELD + SET the hold key with a 15 min TTL, all atomic.
3. The hold record is saved in the DB in the PENDING_PAYMENT state.
4. The user goes to the payment page; Payment Service creates an order at the gateway.
5. The gateway webhook arrives: verify the signature, dedup on payment_id.
6. On success, in a DB transaction the seats become BOOKED, the PNR is generated and the hold is deleted.
7. A booking_confirmed event goes on Kafka and the Notification Worker sends SMS/email.
8. If the TTL expired and the payment arrived late, an auto-refund is triggered.

Payment timeout flow:

1. The user does not complete payment within 15 min.
2. The Redis key expires; a keyspace event or the Hold Expiry Worker detects it.
3. The worker marks the hold EXPIRED in the DB and returns the seats to the inventory as FREE (with a version check).
4. The availability cache is updated and the next admitted users see the seat.
5. If a late payment success arrives afterwards, the reconciliation job initiates a refund.

C.7 Deep dives

Problem: thundering herd at 10:00, backend crash. The virtual waiting room absorbs the traffic and admits at a fixed rate. Static pages come from the CDN, queue status is cacheable. The backend never gets more load than its capacity.

Problem: overselling / the same berth to two people. The check-and-set in the Redis Lua script is atomic. The DB also has optimistic locking on the status + version column. Inventory is sharded by train_id+date so lock contention stays limited to one shard.

Problem: fairness and bot protection. Bot Manager (fingerprint, behaviour), CAPTCHA, per-user and per-IP rate limits, limited bookings per user per window, and randomizing pre-window arrivals. The admit-token is bound to the user and cannot be shared.

Problem: payment succeeds but the booking fails (or the reverse). Idempotency key + a payment state machine. Both the webhook and the client callback are idempotent. Daily reconciliation matches the gateway settlement file and auto-refunds on a mismatch.

Problem: a hot train (everyone piles onto one popular train). Put that train's inventory on a separate Redis shard. Coach-level partitioning spreads the locks. Read availability from an eventually consistent cache; only the hold gets a strong check.

C.8 Trade-offs

C.9 Interview tips

• Say first that the problem is not read-heaviness but write contention and a spike; that is why the waiting room is the most important piece.
• Explain the seat hold as a state machine: FREE -> HELD -> BOOKED / EXPIRED.
• Two layers against overselling: a Redis atomic lock + DB optimistic locking.
• Be sure to mention the payment timeout and late-payment refund edge case.
• Present bot protection as a combination of WAF, CAPTCHA, rate limits and a per-user limit.