Case Study 2: URL Shortener (bit.ly / TinyURL)

Turn a long URL into a 7-character short link, redirect in milliseconds, and show analytics for every click.

C.1 Requirements

Functional requirements:

• Give a long URL, get a short URL (for example sho.rt/aB3xK9p).
• Opening the short URL redirects the user to the original long URL.
• Custom alias option: the user picks their own name (sho.rt/diwali-sale) if it is free.
• Expiry: a link can expire after a date; the user can delete/disable the link.
• Analytics: total clicks, unique clicks, country, device, referrer, time-wise graph.
• Optional dedup for the same user and the same long URL, so a new code is not created again and again.

Non-functional requirements:

• Redirect latency p99 < 50 ms (the user must not feel any delay).
• Availability 99.99% for redirects; the create API can be slightly less critical.
• Read-heavy: read:write ratio ~100:1.
• The short code must be unique (never a collision) and must not look guessable/sequential.
• Scale: 10 crore (100 million) new URLs/month, store 5 years of data, analytics eventually consistent (a few seconds of delay is OK).

C.2 Estimation (back-of-the-envelope)

C.3 API design

C.4 Data model

C.5 High-level architecture

C.6 Main flows

Short URL creation flow:

1. The client sends POST /v1/urls {long_url}; the LB hands the request to Shortener Service.
2. The service validates the URL format and checks the malware/phishing blacklist.
3. If there is a custom alias, do a conditional insert in Cassandra (IF NOT EXISTS); if it is taken, return a 409 error.
4. Otherwise the server takes the next number from its local ID range (for example 125000001).
5. The number is converted to base62 to make the code; to avoid a sequential look, the ID can first go through a bijective shuffle/XOR.
6. The record {code, long_url, user_id, expires_at} is saved in Cassandra and can also be put in Redis.
7. The response returns short_url = https://sho.rt/{code}.

Redirect flow:

1. The user opens sho.rt/aB3xK9p; DNS routes the request to the nearest LB/edge.
2. Redirect Service checks the Bloom filter first; if the code does not exist, return 404 immediately.
3. Look up the code in Redis; on a hit the long URL is found (~1 ms).
4. On a miss, read from Cassandra, then set it in Redis with a TTL (cache-aside).
5. If expired or disabled, 410 Gone; otherwise a 302 response with Location: long_url.
6. After the response is sent, the click event (code, IP, user-agent, referrer) is published to Kafka fire-and-forget.
7. The browser goes to the long URL; the analytics pipeline processes the event async.

Analytics flow:

1. The Kafka topic "clicks" is partitioned by code.
2. The Click Aggregator consumes events and derives the country from the IP (GeoIP) and the device from the user-agent.
3. Per-code per-minute counts are inserted into ClickHouse in batches.
4. Use HyperLogLog for unique clicks (an approximate unique count in little memory).
5. The Stats API runs a group-by query on ClickHouse and gives the data to the dashboard.

C.7 Deep dives

Problem: how to generate a unique short code (no collisions). A hash (first 7 chars of MD5) can collide and needs a retry. Better: counter-based ID + base62. Ranges are allocated by ZooKeeper, so servers do not coordinate on every request.

Problem: sequential codes are guessable (privacy). Before base62-encoding the counter ID, apply a reversible permutation (Feistel cipher/XOR with a secret); the code looks random but stays unique.

Problem: 301 vs 302 redirect. 301 is permanent and the browser caches it, so server load is lower but analytics will miss clicks. If you need analytics, use 302 (or 301 with a short cache).

Problem: hot key on a viral link. If one link gets lakhs of clicks/sec, a single Redis node becomes hot. Add a local in-memory LRU cache (a few seconds TTL) in the Redirect Service and Redis read replicas.

Problem: random code scanning / abuse. The Bloom filter keeps fake codes away from the DB; add IP-based rate limiting and a Safe Browsing-style check for malicious long URLs at create time. Note: a Bloom filter can give false positives (but never false negatives), and custom aliases must be added to the filter too.

Problem: analytics must not slow the redirect. Click logging is async on Kafka; if Kafka is slow, use a local buffer and a drop policy, so the redirect is never blocked. Analytics is eventually consistent.

C.8 Trade-offs

C.9 Interview tips

• Show the base62 math: 62^7 = ~3.5 trillion, so 7 chars are enough.
• Make the read path (redirect) super fast and cache-first; the write path QPS is very low.
• Compare ID generation options: hash, DB auto-increment, Snowflake, ZooKeeper ranges.
• Mentioning the 301 vs 302 and analytics trade-off is a strong signal.
• Also mention abuse (spam, phishing links) and expiry cleanup.