Designing a Notification Service

The whiteboard for this problem is in excalidraw/3. Designing a Notification Service.excalidraw. This file is the full write up of that board.

1. Problem Statement

Design a notification service that sends notifications to users across channels. The system has to be horizontally scalable and support very high fan out, meaning one trigger can turn into millions of individual sends.

Internal teams, such as a product manager or another backend service, should be able to define a message once and then send it to one user or to everyone.

2. Requirements

2.1 Functional Requirements

A template looks like this:

Hello {{user.name}}, order from XYZ and get {{discount}} off!

2.2 Non Functional Requirements

1. Horizontally scalable. Adding capacity means adding queues and workers, not rewriting anything.
2. High fan out. One request such as "notify everyone" must expand to millions of sends without the caller waiting.
3. Load isolation. A large marketing campaign must not starve transactional notifications.
4. Reliable. A message accepted by the system is eventually delivered or ends in a dead letter queue where someone can see it.
5. Provider outage tolerant. A provider being down must not lose messages.

2.3 Out of Scope

Building the email and SMS infrastructure itself. We call providers such as SES, Mailgun, OneSignal, Twilio and msg91.

3. Start Simple: Templates

We need a UI and a small backend so an internal team can create templates. The number of templates is small, a few thousand at most, and each one has clear structure with relationships to channels and variables. That fits on a single node, so we start with a relational database.

The flow for a team that wants to send something:

1. Create a template in the control service.
2. Declare the variables the template needs.
3. Note the template id, and pass it when triggering a send.

4. Channels and Providers

A user can be reached over several channels:

1. Email
2. Android push
3. Apple push
4. SMS

Each channel has providers that expose an API, and we call those APIs to actually deliver. The servers that trigger real sends have to configure each provider SDK.

The important observation: these are expensive network calls with high latency. One machine cannot make millions of concurrent outbound calls. That single fact drives the whole rest of the design.

5. Evolving the Architecture

5.1 Synchronous, One User

The control service renders the template and calls the provider directly. This works for one user and a manual trigger, and nothing more.

5.2 What Breaks

If a team wants to trigger thousands of notifications at once:

• Triggering one notification per user is painful for the caller.
• The control service becomes the bottleneck, because each provider call takes real time.
• A provider outage blocks the caller and loses messages.

This is the classic place to make things asynchronous.

5.3 Asynchronous with a Queue

Each queue message carries what one send needs:

{ user_id: u1, notification_id: n1, channel: email, body: rendered }

Workers stay dumb on purpose. A worker picks a message, renders if needed, calls the provider, acknowledges. Nothing else. That makes them trivially scalable and safe to kill.

This architecture gives us retries and stops the control service from being overwhelmed.

5.4 Bulk Notifications

The queue works well for moderate traffic and one to one triggers. The typical hard case is "notify everyone".

The experience we want: the caller submits one job and the system handles the rest. Someone has to walk the user table and emit one message per user, and that someone must not be the control service.

Two queues with two different jobs:

• SQS 1, the emitter queue. One message equals one notification to one user. Single sends go straight here.
• SQS 2, the job queue. Anything that needs processing first. Bulk sends go here.

The iterator reads from a read replica of the user table, applies the filter criteria, and pushes one emitter message per matching user. It reads the replica so a campaign scan never competes with production writes.

5.5 Priority Lanes

Some notifications matter more than others. An appointment reminder is far more important than a marketing push.

With one emitter queue, a single marketing campaign of ten million messages keeps every worker busy and everything else starves behind it.

Fix: replace the single emitter queue with several priority queues, each with its own worker pool.

A message is now (notification_id, user_id, priority), for example (n1, u1, p1).

What this buys us:

• Horizontal scalability, add queues and add workers.
• Load isolation, a P3 marketing flood cannot starve P1 traffic.

Typical mapping: P1 for OTP, security alerts and appointment reminders, P2 for transactional updates such as order shipped, P3 for marketing.

5.6 Deduplication

Priority lanes do not solve duplicates. Receiving a marketing notification is already annoying, receiving the same campaign three times is worse.

So we track what was already sent, in a key value store.

key:   <campaign_id>:<user_id>
value: sent_at

Sizing it:

100M users x (4 B user id + 4 B campaign id) = 800 MB
5 concurrent campaigns  ~ 4 GB

5.7 Where to Check, and the Optimization

The naive version checks in the worker. That means the iterator pushes every user into a priority queue, workers pull each message, and only then discover the notification was already sent. We paid for the enqueue, the storage and the dequeue to learn nothing.

Better: the iterator consults the tracker before enqueueing. Already sent users never enter the queue at all.

The tracker needs to be:

1. Lightweight and shardable.
2. In memory with periodic persistence.
3. Able to support a Bloom filter, which is space efficient.

Redis satisfies all three, so the tracker is Redis.

A Bloom filter answers "have we sent this already" in constant time and tiny space. It can say yes when the answer is actually no, which here means we skip a send that we could have made. For marketing that is acceptable. For a P1 OTP we skip the filter and use an exact key, because a missed OTP is a real failure.

The worker still writes the tracker entry after a successful send, so the filter stays truthful for the rest of the campaign.

6. API Design

6.1 Template Management

{
  "name": "diwali_discount",
  "channel": "EMAIL",
  "subject": "Your Diwali offer",
  "body": "Hello {{user.name}}, order from XYZ and get {{discount}} off!",
  "variables": ["user.name", "discount"],
  "category": "MARKETING",
  "defaultPriority": "P3"
}

6.2 Sending

Single send:

{
  "templateId": "t_881",
  "userId": "u_1729",
  "channel": "SMS",
  "priority": "P1",
  "variables": { "discount": "20 percent" },
  "idempotencyKey": "order-5512-shipped"
}

The idempotencyKey makes a retry by the caller safe. Two calls with the same key produce one notification.

Bulk send:

{
  "templateId": "t_881",
  "campaignId": "c_diwali_2026",
  "priority": "P3",
  "filter": { "country": "IN", "lastActiveWithinDays": 30, "plan": "free" },
  "channels": ["EMAIL", "PUSH"],
  "dedupe": true
}

Response is a job id and 202 Accepted. The caller does not wait for millions of sends.

{ "jobId": "j_4410", "status": "QUEUED", "estimatedRecipients": 8400000 }

Job progress:

{
  "jobId": "j_4410",
  "status": "RUNNING",
  "scanned": 3100000,
  "enqueued": 2870000,
  "skippedDuplicate": 230000,
  "sent": 2650000,
  "failed": 1200
}

7. Data Model

The meta database is relational because templates, channels, providers and preferences all have real relationships and low volume. Delivery records are high volume and append only, so they go in a wide column or document store.

7.1 Tables

templates

template_versions

Versions exist so an in flight campaign keeps rendering the body it started with, even if someone edits the template mid run.

channel_providers

user_notification_preferences

Composite key of (user_id, channel, category) so a user can accept transactional SMS while refusing marketing SMS.

notification_jobs

The cursor is what makes a bulk job resumable. An iterator that dies at user three million restarts from three million, not from zero.

notification_deliveries (Cassandra or DynamoDB, not relational)

This table is the high volume one, tens of millions of rows a day. It is partitioned by user_id because the query we actually run is "show me this user's recent notifications".

7.2 Relationships

In words:

• One template has many versions, one version belongs to one template.
• One template is used by many jobs, one job uses one template.
• One job produces many deliveries, one delivery belongs to at most one job.
• One user receives many deliveries and sets many preference rows, one per channel and category pair.
• One provider handles many deliveries.

7.3 The Tracker Store

Not a table. Redis, per campaign:

Exact form:  SETNX  dedupe:{campaign_id}:{user_id}  1   with TTL of campaign length
Bloom form:  BF.ADD dedupe:{campaign_id}  {user_id}

Sharded by campaign id so one huge campaign spreads across nodes, and the whole namespace can be dropped when the campaign ends.

8. Key Flows

8.1 Single Transactional Send

8.2 Bulk Campaign

9. Deep Dives

9.1 Why Two Queue Types

The job queue and the emitter queue carry different units of work. A job message is one row and expands into millions. An emitter message is one send. Mixing them in one queue means a single fat message blocks a worker for hours while every other message waits.

9.2 Provider Rate Limits

Every provider caps throughput. Workers hold a shared token bucket per provider, in Redis, and block on it before calling out. Without this we get throttled by the provider, and provider throttling usually means dropped messages rather than a clean retry.

9.3 Retries and Dead Letters

Failures split into two kinds:

Retries add jitter so a provider recovering from an outage does not get hit by every worker at the same instant. After the final attempt, the message goes to a dead letter queue where an on call engineer can inspect and replay it.

9.4 Idempotency

Queues deliver at least once, so a worker will occasionally see the same message twice. Two guards:

1. The tracker check, which is already there for campaigns.
2. A per notification key from idempotencyKey, checked with SETNX before the provider call.

Both are cheap Redis operations, and both are far cheaper than a duplicate SMS.

9.5 Quiet Hours and Preferences

Checks happen at two points. The iterator filters coarse criteria such as country and plan, because that reduces enqueue volume. The worker checks quiet hours right before sending, because a message may sit in the queue long enough for local time to change. A marketing message inside quiet hours is rescheduled rather than dropped.

10. Failure Handling

11. Tradeoffs

12. Interview Talking Points

• Start synchronous, then show exactly what breaks. The argument for queues is more convincing when the failure is named.
• The line "these are expensive network calls with high latency, one machine cannot make millions of them" is the sentence that motivates the whole design.
• Explain the two queue types and why one queue is not enough.
• Priority lanes need their own worker pools. A priority field on a single queue does not prevent starvation.
• The dedupe placement optimization, checking at the iterator rather than the worker, is the detail that shows you are thinking about cost and not just correctness.
• Say the Bloom filter tradeoff out loud: a false positive means one skipped marketing message, which is fine, and that is why P1 does not use it.