Notification System
A central service that sends messages to the user on the right channel (email, SMS, push, in-app), at the right time, without duplicates.
29.1 Why is it used?
- OTP, order shipped, payment failed, offer alert: every app has to send crores of notifications; if every team writes its own code, it becomes chaos.
- Third-party providers (SMS gateway, FCM) can be slow or down; you need queue + retry + fallback.
- Respecting user preferences (DND, unsubscribe, language) and rate limits is necessary for both legal reasons and UX (in India, TRAI DLT rules apply to SMS).
- Delivery tracking: message sent, delivered, opened; needed for analytics and debugging.
29.2 How it works
- Flow: Services -> Notification API -> validation + preferences check -> channel-wise queue (email/SMS/push) -> workers -> providers.
- Templates: 'Hi {{name}}, your order {{id}} has shipped'; content is stored separately, in multiple languages.
- Priority: OTP/transactional on a separate high-priority queue; marketing bulk on low priority, throttled.
- Push notification: the app registers a device token; the server sends the token + payload to FCM (Android/web, and iOS too) or APNs (Apple).
- Retry + fallback: SMS provider A fails -> backoff retry -> provider B; push fails -> SMS fallback for critical messages.
- Dedup + idempotency: a notificationId / idempotency key ensures the same event does not send a message twice.
- Rate limiting + quiet hours: max N promos per user per day, no promos from 10 pm to 9 am.
- Status tracking: the DB is updated from provider webhooks (delivered, bounced, clicked); bounced emails go to a suppression list.
- 1.Order Service: 'OrderShipped' event on Kafka (order-events)
- 2.Notification Service: validate + user preferences + dedup + rate limit + template (order_shipped_hi, notificationId)
- 3.Channel queues: push (high), sms (high, OTP), email (normal), promo (low, throttled)
- 4.Workers: Push worker -> FCM / APNs, SMS worker -> provider A (2 retries on 5xx, then provider B), Email worker -> AWS SES
- 5.Provider webhooks: delivered / bounced / clicked -> notification log; bounce -> suppression list, invalid token -> delete from DB
Rules run alongside: max N promos / user / day, quiet hours 10 pm to 9 am, TRAI DLT approved template for SMS
29.3 Tools
| Tool | Type | How it is used |
|---|---|---|
| AWS SES | Email service (AWS) | Cheap bulk + transactional email; bounces/complaints arrive on SNS; domain verification (DKIM, SPF) is required. |
| SendGrid | Email API (Twilio) | Templates, analytics, deliverability tools; for both marketing and transactional. |
| Twilio | SMS / voice / WhatsApp API | Global SMS, WhatsApp Business messages, OTP via the Verify API; in India MSG91 and Gupshup are also common. |
| Firebase Cloud Messaging (FCM) | Push notifications (Google) | Free; Android, iOS (through APNs) and web push, topics and device groups. |
| Apple Push Notification service (APNs) | Push for Apple devices | The official channel for iOS/macOS; token-based (.p8 key) auth over HTTP/2. |
| AWS SNS / Amazon Pinpoint | Multi-channel (AWS) | SNS fans out SMS and mobile push; Pinpoint for campaigns and segmentation (AWS is now moving it to End User Messaging). |
29.4 Comparison: Email (SES/SendGrid) vs SMS (Twilio) vs Push (FCM/APNs)
| Point | Email (SES / SendGrid) | SMS (Twilio / MSG91) | Push (FCM / APNs) |
|---|---|---|---|
| Cost per message | Very cheap (SES ~0.10 USD per 1000) | Most expensive, charged per SMS | Free (FCM/APNs themselves are free) |
| Speed | Seconds to minutes | Seconds, depends on the operator | Usually seconds, near real-time |
| Reach | Needs an email address, spam folder risk | Every phone, no internet needed | Only app installed + permission ON |
| Content | Rich HTML, attachments, long content | 160 chars per segment, plain text | Title + short body + image + deep link |
| Reliability | Bounce/spam filters, DKIM/SPF required | High; in India DLT template registration is required | Best effort; delayed or collapsed if the device is offline |
| Best for | Invoices, statements, newsletters | OTP, critical alerts, non-smartphone users | Order updates, engagement, re-engagement |
29.5 When to choose which
- SMS: OTP and critical alerts where delivery is needed at any cost, and higher cost is acceptable.
- Push (FCM/APNs): cheap, fast updates to app users; order status, offers.
- Email (SES for cost, SendGrid for features): long content, invoices, legal records.
- Multi-provider setup: keep at least 2 providers on critical channels (SMS) for failover.
29.6 Real-world example
Scenario: E-commerce order lifecycle notifications
- The Order Service publishes an 'OrderShipped' event on Kafka; the Notification Service consumes it.
- The Notification Service checks user preferences: push ON, SMS ON, email ON, language Hindi.
- The template 'order_shipped_hi' is rendered and three messages go to separate queues (push, sms, email) with the same notificationId.
- The push worker sends to FCM with the device token; if the token comes back invalid (app uninstalled), it deletes the token from the DB.
- The SMS worker sends to provider A with a DLT-approved template; on a 5xx error it retries 2 times, then goes to provider B.
- The email worker sends the invoice through SES; when a bounce webhook arrives, it puts the email on the suppression list.
- On sale day, promo notifications are throttled in the low-priority queue so OTP and order updates do not slow down.
29.7 Interview tips
- Sending a notification synchronously inside the request: if the provider is slow, checkout itself becomes slow.
- Not keeping idempotency in retries: the user gets the same SMS 5 times and money is wasted too.
- Keeping OTP and marketing in the same queue: the OTP gets delayed by 10 minutes behind a bulk campaign.
- Interview tip: preferences, rate limit, templates, retry/fallback and tracking; be sure to draw these 5 components.