Design a Chat App

Everyone has used a chat app, so everyone thinks they know how it works. Then the interviewer asks "Phone A is connected to server 17 and phone B is connected to server 203. How does the message get from one to the other?" and the room goes quiet. This note is about answering that question calmly, and the ten questions that come after it.

1. Problem Statement

Design a chat app like WhatsApp or Telegram. Two people can talk one to one. A group of people can talk in one shared chat. Messages show whether they were sent, delivered or read. If your friend's phone is off, the message waits and arrives when the phone comes back. You can see who is online. You can send photos and short videos.

The hard part is not storing text. Text is tiny. The hard part is that millions of phones keep a connection open all day, and a message has to find the one server out of hundreds that is holding the other phone's connection.

Think of a big post office where every customer stands at one of 400 counters, and nobody knows which one. A letter arrives for Ben. Someone has to know that Ben is at counter 203.

2. Questions to Ask First

Ask these out loud. It takes one minute and it shows you do not jump to boxes before you know the problem.

Notice what I left out: voice and video calls, stories, bots, search inside messages. Say it out loud so the interviewer knows it was a choice.

3. Requirements

3.1 Functional Requirements

3.2 Non Functional Requirements

1. Delivery is fast. Under 200 ms between two online phones in one region.
2. No message is ever lost once the sender sees the first tick.
3. Messages in one chat appear in the same order for everyone.
4. The app stays up when a single server dies. Phones reconnect by themselves.
5. It scales by adding machines, not by buying a bigger one.
6. Battery friendly. A phone must not wake up every two seconds to ask "anything new?".

3.3 Out of Scope

Voice and video calls, stories, message search, bots, channels with millions of readers, multi device sync. Each of these is its own design.

4. Back of the Envelope Numbers

Do the arithmetic in small steps and say each step aloud.

Messages per day:

100M users x 40 messages = 4 billion messages per day

Messages per second:

4,000,000,000 / 86,400 seconds = about 46,000 per second
Peak is about 3 times the average = about 140,000 per second

How many deliveries? One to one messages go to one person. Group messages go to many. Say 70 percent are one to one and 30 percent are group messages with 10 members on average.

70% x 1 recipient   = 0.7
30% x 10 recipients = 3.0
average              = 3.7 deliveries per message

4B x 3.7 = 14.8 billion deliveries per day
14.8B / 86,400 = about 171,000 per second
Peak x3 = about 500,000 deliveries per second

Open connections. Not everyone is online at once. Say 20 percent of daily users are connected at the busiest moment.

100M x 20% = 20 million open WebSocket connections

One tuned server can hold about 100,000 idle connections, but I would plan for 50,000 so a server has room to breathe.

20M / 50,000 = 400 gateway servers
Memory per connection is about 10 KB
50,000 x 10 KB = 500 MB per server, which is nothing

What if we skipped WebSockets and polled every 2 seconds?

20M phones / 2 seconds = 10 million requests per second, nearly all of them empty

That one number is the whole argument for keeping a connection open. It also kills the battery.

Message storage. A text message with its ids, times and sender is about 200 bytes.

4B x 200 bytes = 800 GB per day
800 GB x 365 = about 292 TB per year
Three copies = about 880 TB per year

Media. Say 2 percent of messages carry a photo or video, averaging 300 KB after the phone compresses it.

4B x 2% = 80 million media files per day
80M x 300 KB = 24 TB per day
24 TB x 365 = about 8.8 PB per year

So text is a big but ordinary database. Media is the real storage bill, and it goes to cheap object storage, never to the message database.

Presence heartbeats. If every phone said "I am alive" every 30 seconds:

20M / 30 = about 670,000 heartbeats per second

We will see in the deep dive that this number is why presence is more expensive than it looks.

5. High Level Design

A walk through, box by box.

The load balancer only matters at the moment a phone connects. It picks a gateway, and from then on the phone talks to that one gateway for as long as the connection lives. After that the load balancer is not in the message path at all.

The gateways are the servers that hold WebSocket connections. A WebSocket is a normal web connection that stays open after the first handshake, and either side can send at any time. That is what lets the server push a message to a phone without being asked. Gateways do very little: they keep the connection alive, check who the user is when they connect, pass incoming messages to the chat service, and write outgoing messages to the right socket. I keep them dumb on purpose. A dumb server is easy to replace when it dies.

The chat service is stateless and holds the real logic. It checks the sender belongs to the chat, gives the message its sequence number, saves it, and then finds every recipient's gateway and hands the message over. It can be scaled by simply adding copies.

The counter database hands out the next sequence number for each chat. It is a small table. I explain why it exists in section 9.2.

The message store holds every message ever sent. It is write heavy and append only, so I use a wide column store (Cassandra or ScyllaDB style) partitioned by chat. Reading the latest 50 messages of a chat is one partition read.

The metadata database holds users, chats, members, and who has read up to where. It is small, relational, and sharded by chat id.

The session registry is a Redis table that answers one question: "which gateway is user 42 connected to right now?". It is the post office board that says Ben is at counter 203. About 20 million entries of 100 bytes each is 2 GB, so it fits in memory easily.

Kafka carries the work that does not need to happen inside the send path. The main one is push notifications for phones that are offline. If the push system is slow, chatting is not slow.

The push worker reads from Kafka and calls Apple's APNs or Google's FCM. These two services are the only way to reach a phone whose app is closed, so every chat app uses them.

The media API gives phones a temporary upload link straight to object storage. A CDN serves the files back. Photos never travel through the chat service, and I will explain why in the key flows.

5.1 Why WebSocket and not something else

I pick WebSocket. The price is that gateways are stateful: each one remembers which phones are connected to it. Most of the rest of this design is about living with that price.

6. API Design

Chat traffic goes over the WebSocket. Everything that is not real time goes over normal HTTPS.

6.1 WebSocket frames

Each frame is a small JSON object with a type.

Phone to server, sending a message:

{
  "type": "send_message",
  "clientMsgId": "7c1f0a52-5a1b-4c0e-9d3a-2a6d9d1d4f10",
  "conversationId": "c_81723",
  "kind": "text",
  "body": "Reached the station, 5 mins"
}

The clientMsgId is made up by the phone. It is the thing that makes retries safe, and I come back to it in 9.2.

Server to the sender, once the message is stored:

{
  "type": "message_ack",
  "clientMsgId": "7c1f0a52-5a1b-4c0e-9d3a-2a6d9d1d4f10",
  "conversationId": "c_81723",
  "seq": 4812,
  "state": "sent"
}

Server to a recipient:

{
  "type": "message",
  "conversationId": "c_81723",
  "seq": 4812,
  "senderId": "u_204",
  "kind": "text",
  "body": "Reached the station, 5 mins",
  "sentAt": "2026-10-10T08:15:03Z"
}

Other frames, in short:

6.2 HTTP endpoints

History uses a cursor (beforeSeq) and not page numbers. If new messages keep arriving, page numbers shift and the user sees the same message twice. A cursor does not shift.

Upload link request:

{ "mime": "image/jpeg", "sizeBytes": 312044 }

Response:

{
  "mediaId": "m_55012",
  "uploadUrl": "https://storage.example.com/up/m_55012?sig=...",
  "expiresInSeconds": 300
}

7. Data Model

Five stores, each picked for how it is read.

7.1 Message store (wide column, partitioned by chat)

messages

Why the bucket? A very busy group might get 10,000 messages a day at 200 bytes, which is 2 MB a day and 60 MB a month. Partitions in this kind of store are happy up to roughly 100 MB. One month per partition keeps even the busiest chat under that. The app reads the current bucket first, then the previous one if the user keeps scrolling.

7.2 Counter database (relational, sharded by chat)

conversation_counters

One statement hands out the next number:

UPDATE conversation_counters
SET last_seq = last_seq + 1
WHERE conversation_id = $1
RETURNING last_seq;

The row is locked for a moment, so two messages in the same chat can never get the same number. At a 140,000 per second peak and about 5,000 per second per shard, I need about 30 shards. Different chats never touch each other's rows.

7.3 Chat metadata (relational, sharded by chat)

conversations

For a direct chat I use an id made from the two user ids sorted, so the same two people always land on the same chat and nobody ends up with two.

conversation_members

The delivery ticks are not stored per message. They are worked out from these two numbers. If Ben's last_read_seq is 4812, then every message from seq 4812 down is read. That is one small row per member, not one row per message per member. For a group of 256 and 4 billion messages a day, the difference is enormous.

user_conversations (the chat list)

The same fact is written in two places (members and user_conversations) because the two reads are different. "Who is in this chat?" and "which chats is this person in?" need two different keys. I accept the duplication.

7.4 Pending deliveries and device tokens

pending_deliveries

A row is written only when the recipient is offline, and deleted once their phone acknowledges delivery. So the table only holds the backlog of people who are away, not the whole history.

device_tokens

7.5 Session registry and gateway leases (Redis)

session:{userId}    ->  "gw-17"          set when the phone connects, deleted when it disconnects
gateway:{gatewayId} ->  "alive"          expires after 10 seconds, each gateway renews it every 3 seconds

To find out whether a user is online, read session:{userId}, then check that the gateway it names still has a live lease. If the gateway crashed, its lease expired, and every user pointing at it counts as offline. Nobody has to clean up 50,000 keys one by one. I like this trick because the bad case (a crash) fixes itself.

7.6 Media

One small media table: id as primary key, uploader_id, object_key (where the file sits in object storage), thumb_key, mime, size_bytes and created_at. It is looked up by id only, so any key value store would do.

8. Key Flows

8.1 Sending a message when both people are online

Asha sends "Reached the station" to Ben. Asha is on gateway 17, Ben is on gateway 203.

Two details matter here.

The message is stored before anyone is told about it. If the chat service crashed between the store and the push, Ben still gets the message later from storage. If it was the other way round, Ben could see a message that does not exist. Store first, always.

The first tick means "the server has it safely", not "Ben got it". That honest meaning is why Asha can close the app after one tick and trust the message will go.

When Ben opens the chat, his phone sends read with the highest seq he has seen. The chat service updates last_read_seq, and Asha's phone turns the ticks blue. Reading ten messages sends one read frame, not ten.

8.2 Ben is offline

The push notification is only a nudge. It does not carry the message. It says "you have something" so the user opens the app, and the app then fetches the real messages over the WebSocket. This also means a lost push does no harm: the messages are still waiting, the user just finds out a bit later.

Three rules keep push from being annoying:

• The collapse key is the chat id, so ten messages in one chat show as one notification, not ten.
• Muted chats write the pending row but send no push.
• If the same user got a push for this chat in the last 30 seconds or so, skip the new one.

8.3 Sending a photo

The chat message itself only carries a media id and a 1 to 2 KB blurred preview. The big file never touches the chat service or a WebSocket. If it did, one 5 MB video on a gateway would block hundreds of tiny text messages sharing that same connection. The receiver sees the blurred preview instantly, then downloads the real file from the CDN with a signed link that expires after a while, so a leaked link stops working.

9. Deep Dives

9.1 How a message finds the right server

The problem. Ben's phone is connected to exactly one of 400 gateways. The chat service runs on a different set of machines. When a message for Ben arrives, something must tell the chat service which gateway holds Ben's connection.

Option A: a lookup table (the registry). When a phone connects, its gateway writes session:{userId} = gw-203 to Redis. To deliver, the chat service reads that key and then calls gateway 203 directly. When the phone disconnects, the key is removed.

Option B: work it out from the user id. Take hash(userId) mod 400 and always send each user to that gateway. Nobody needs a table, because everyone can do the sum.

Option B looks neat and falls apart on the first deployment. The day you go from 400 to 450 servers, the answer for almost every user changes, so almost every phone has to be disconnected and reconnected somewhere else. Even a gentler version (consistent hashing) forces connections to move whenever the fleet changes, and moving a live connection is not free. Option B also stops you draining one gateway for an upgrade, because the phone must go back to the same place.

I choose option A. The cost is one Redis read per recipient, about 0.3 ms inside one region. At 500,000 deliveries per second at peak that is 500,000 reads per second, which a Redis cluster of 6 to 8 nodes handles. The benefit is that the load balancer can send a phone anywhere, and I can add, remove or drain gateways freely.

What about the gap between "I read the table" and "I called the gateway"? Ben may have disconnected in those few milliseconds. The call to gateway 203 then fails or finds no socket. The chat service treats that exactly like an offline user: write a pending row, send a push. The message already sits in the store, so nothing is lost. I do not try to make the table perfectly accurate, I make the fallback safe.

9.2 Ordering, retries and exactly one copy

The problem. A phone on a bad network sends a message, never hears back, and sends it again. Two people send at the same moment into the same group. Servers have clocks that disagree by a few hundred milliseconds. Yet every person must see the same order, and nobody wants to see a message twice.

Option A: order by time. Use the sender's phone clock, or the server clock, as the sort key.

Phone clocks are wrong, sometimes by minutes, and some people set them wrong on purpose. Server clocks drift by milliseconds, which is enough to flip two messages sent at nearly the same moment, and two chat servers will flip them differently. Time alone gives you an order that can differ between viewers, which is the one thing a chat must not do.

Option B: a number per chat. Each chat has its own counter, and the message gets the next number when it is stored. Order is simply that number. Everyone sorts by it, so everyone sees the same order, and nothing depends on a clock.

I choose B. It costs one counter update per message, on a small sharded table, which I sized at about 30 shards. The chat's counter is like the numbered ticket machine at a bank: it gives out 1, 2, 3 with no repeats, and who arrived first is decided by the machine, not by anybody's watch.

Honest detail: numbers may have gaps. If the counter hands out 4812 and then the message write fails, 4812 is never used. The phone must not assume numbers are contiguous. It sorts by seq and does not count them. Gaps are harmless. Trying to remove them would mean a distributed transaction between the counter and the message store, which is a lot of pain for a cosmetic problem.

Now retries. Every message carries clientMsgId, made by the phone before the first attempt. The chat service keeps a short lived key dedupe:{conversationId}:{clientMsgId} in Redis for 24 hours, holding the seq it assigned. On a retry:

1. The key exists, so return the old ack with the same seq. Do not store again.
2. The key does not exist, so this is a new message. Take a number, store, then set the key.

There is a narrow race when two copies of the same message arrive at the same instant on two chat servers. I close it by setting the key with "only if not already set" before taking a number, and the loser waits for the winner's result. And since the store has client_msg_id on each row, a nightly check can spot any duplicates that still slipped through.

This gives you what people call at least once delivery on the wire (the phone retries until it hears an ack) plus dedupe on the server, which behaves like exactly once to the user. Real exactly once over an unreliable network does not exist. At least once plus an id is what everyone really does.

For groups, nothing special is needed. Group messages go through the same counter, so the group has one order. The only extra cost is that after storing, the chat service fans out to up to 255 other members. I store the message once and send to many. I do not copy the message into each member's inbox, because with 4 billion messages a day that would multiply storage by the average group size.

9.3 Presence, the cost nobody counts

The problem. People love the green dot and "last seen 8:12 pm". It feels like a tiny feature. It is actually the most expensive thing in the app per byte of value.

Option A: the obvious way. Every phone sends a heartbeat every 30 seconds. The server stores online with an expiry of 60 seconds in Redis. When someone's status changes, push it to all their friends.

Check the numbers from section 4. 20 million online phones every 30 seconds is about 670,000 Redis writes per second just to say "still here". Then the fan out: say 100 million people flip between online and offline about 10 times a day (opening the app, backgrounding it).

100M x 10 changes = 1 billion changes per day
1B / 86,400 = about 11,600 changes per second
Each has about 200 friends: 11,600 x 200 = 2.3 million pushes per second

2.3 million pushes per second, for green dots. That is more than four times the traffic of real messages (500,000 per second at peak). Presence would cost more than the chat.

Option B: do less, on purpose. Three changes.

First, no heartbeat writes to Redis. The WebSocket connection already tells us the phone is alive, so we write only when a phone connects or disconnects. Those are the same session registry writes from section 7.5, and the 10 second gateway lease handles crashes. The 670,000 writes per second drop to the 11,600 per second of real status changes.

Second, do not push to all friends. Push a change only to friends who are looking right now: people who have the chat open, or who have this person on the first screen of their chat list. A phone tells the server "I am watching these 20 people" when its screen changes, and the server pushes only to those. If 5 percent of friends are watching, that is 10 pushes, not 200.

11,600 changes per second x 10 watchers = about 116,000 pushes per second

That is 20 times less than 2.3 million.

Third, everyone else is fetched on demand. Opening a chat calls GET /v1/presence for that one person. And round "last seen" to the nearest minute, or let people hide it. Exact seconds make constant change that nobody needs.

I choose B. What I give up is a promise that a friend's dot is right to the second. Users will not notice.

9.4 Side box: end to end encryption, kept honest

End to end encryption means the messages are scrambled on the sender's phone and unscrambled only on the recipient's phone. The server in the middle carries and stores messages it cannot read.

Most apps that do this use the Signal protocol. Each phone creates a set of keys, hands the public half to the server, and two phones use those public halves to agree on a secret nobody else knows. The secret changes with every message, so a stolen key only opens a small slice of history.

What it changes in this design:

• The message store holds opaque bytes. The body column is ciphertext.
• Push notifications cannot show the message text, because the server does not have it. The phone's own app decrypts after waking.
• The server cannot search messages, scan links, or build previews. All of that moves to the phone.
• Groups are harder. Every time a member leaves, the group's keys must change so the leaver cannot read the future. That is real work for a group of 256.
• A new phone cannot read old chats unless the user made an encrypted backup. This is the number one support complaint in apps that do this.

I can describe it at this level. I could not design the key exchange from scratch in an interview, and I would say that plainly. The correct move is to say "the protocol exists and is well studied, I would use it and not invent one", then show how it changes the rest of the design. That is what the list above is for.

10. Failure Handling

One rule runs through all of this: once the sender sees the first tick, the message is in the store. Everything after that point can be delayed or retried, and none of it loses data.

11. Tradeoffs

12. Interview Talking Points

Say these in this order. Each one sets up the next.

• Start with the questions and assumptions: 100M daily users, 40 messages each, groups up to 256, one device. Then say what you leave out.
• Do the three numbers that matter: about 46,000 messages per second (140,000 at peak), about 20 million open connections which means about 400 gateways, and about 800 GB of text per day against 24 TB of media per day.
• Explain why WebSocket, with the polling number (10 million empty requests per second) as the proof. Then admit the price: gateways are stateful.
• Answer the opening question. A registry in Redis says which gateway holds each user. Say why hashing the user id is worse, and say that a stale registry entry falls back to the offline path, so it is safe.
• Walk the send path: store first, then ack, then push. Explain that the first tick means "stored". Show that delivered and read are two numbers per member, not a row per message.
• Explain ordering with a counter per chat, and say gaps are fine. Explain retries with clientMsgId. Say "at least once plus dedupe" and do not claim exactly once.
• Cover the offline path: pending rows, a push that only nudges, and fetching on reconnect. Then media going straight to object storage with a signed link.
• Finish with the parts people skip: presence costs 2.3 million pushes per second if done naively and about 116,000 if you only tell watchers, and end to end encryption in an honest paragraph about what it changes and what you would not try to invent.