36. Design WhatsApp / Chat at Scale

Tests: long-lived connections, fan-out for messages, message ordering, presence, end-to-end encryption, group chat.

36.1 Requirements

Functional

1:1 and group messaging.
Message delivery (sent / delivered / read receipts).
Online presence.
Media (photo, video, voice).
Group chat (up to ~1000 members).
End-to-end encryption.
Multi-device.

Non-functional

~2B users; 100B+ messages/day.
Low latency: < 1s for delivery.
Reliable: messages must not be lost.
Encrypted in transit + E2E.
Mobile-first (poor networks, battery).

36.2 Estimates

100B messages/day ÷ 86,400 = ~1.2M msgs/sec average; peak ~5M/sec.
Avg message ~100 bytes → ~10 TB/day text only.
Media: 10-100x more.
Concurrent connections: ~500M peak online.

36.3 High-level architecture

[Mobile app]──ws/long-conn──→[Gateway pool]──→[Pub/sub bus (Kafka/NATS)]
                                                   │
                                       [Message svc]──→[Message store (Cassandra)]
                                       [Presence svc]──→[Presence cache (Redis)]
                                       [Group svc]
                                       [Media svc]──→[Object store + CDN]
                                       [Push svc]──→[APNS/FCM]

36.4 Persistent connections

Hundreds of millions of concurrent WS / XMPP / custom-binary connections.

Each gateway holds 1-2M connections (with kernel tuning).
Need 250-500 gateway nodes.
Map: (user_id → gateway_id) in fast KV (Redis).

When sending to user X: look up gateway holding X's connection, deliver via that gateway.

WhatsApp historically used Erlang/OTP for massive concurrency. Discord uses Elixir/Erlang. The actor model fits.

36.5 Sending a message (1:1)

1. Alice sends msg via her gateway connection.
2. Gateway → Message svc.
3. Message svc:
   - Generate msg_id (Snowflake).
   - Persist to Cassandra: (chat_id, msg_id, sender, payload, ts).
   - Lookup recipient's gateway.
   - Push to recipient's gateway → recipient app via WS.
   - If recipient offline: send push notification (APNS/FCM).
4. Acks flow back: delivered, read.

End-to-end delivery in < 1s when both online.

36.6 Message store

Sharded by chat_id (1:1 chats have a deterministic chat_id from sorted user pair).

Cassandra (or HBase) wide-column:

chat_id (partition) | msg_id (clustering, time-sorted) | sender | payload | timestamps

Query "give me last 50 messages" = single partition, top-N range scan. Fast.

Retention:

Recent (last 6 months): hot storage.
Older: cold archive.
E2E-encrypted means server can't read content; only metadata.

36.7 Group chats

Group = chat_id with multiple members.

Send to group:

Sender → gateway → Message svc.
Persist once to message store.
Look up all online members.
Fan out to each member's gateway.
Offline members: push notification + retrieve on next online.

For groups up to ~1000 members (WhatsApp's limit): fan-out is feasible per message.

For bigger broadcasts (channels): different architecture (one-to-many publish, with smart fan-out).

36.8 Ordering

Within a chat:

Messages must appear in order.
Snowflake msg_id (time-ordered) gives global-ish order; final order on receipt.
Receiver may re-order on display by msg_id.

For group chats, single partition (chat_id) gives consistent order.

36.9 Delivery semantics

At-least-once + idempotent (msg_id dedup at receiver).

State machine per message:

SENT (server received it)
DELIVERED (recipient device acked)
READ (recipient saw it)

Acks travel back via the same WS / gateway.

36.10 Presence

"Online" / "last seen" status.

Each connection maintains heartbeat.
Presence cache: user_id → (online, last_seen).
TTL on heartbeat absence (e.g., 30 sec) → marks offline.
Updates fanned out to friends/contacts (subscription model).

Cost: presence updates can be high-volume. Throttle / batch.

WhatsApp has privacy options for who sees presence; honor in fan-out filter.

36.11 End-to-end encryption (Signal protocol)

Server never sees plaintext.

Signal protocol (used by WhatsApp, Signal, Messenger Secret):

X3DH (Extended Triple Diffie-Hellman): key agreement.
Double Ratchet: forward secrecy + post-compromise security.

Each user has long-term identity key + medium-term signed prekey + ephemeral one-time prekeys, registered with server.

To start a chat:

Sender fetches recipient's public keys.
Performs X3DH → derives shared session key.
Sends first message encrypted; ratchet advances.

Server's role: store public keys (key directory), relay encrypted blobs.

For group chat:

Pairwise sessions (every pair has one) — Signal's original design; doesn't scale to large groups.
Sender keys: each sender encrypts once with a group key; group members have shared key.
MLS (Messaging Layer Security): IETF standard for scalable secure group messaging.

36.12 Multi-device

User has phone + desktop + web. All must receive messages.

WhatsApp's old model: phone was authoritative; web was a relay. Newer: each device has its own keys; sender encrypts separately to each device of recipient.

Synchronization across devices: one device sees a message, others see read state.

36.13 Media

Sender encrypts media with random AES key → uploads encrypted blob to object storage → sends key + URL via Signal channel.

Recipient downloads blob, decrypts with key.

Server cannot decrypt media. Object storage holds encrypted bytes.

Thumbnails included encrypted in the message metadata so previews work without download.

Retention: encrypted media expires after fixed period unless persisted by user.

36.14 Push notifications

When recipient offline, server triggers APNS/FCM push.

For E2E encrypted messages, server can't include content; pushes "You have a new message." App fetches and decrypts when opened.

36.15 Failure modes

Gateway dies: clients reconnect to another. Map updated.
Message store partition unavailable: writes queued; eventual reconciliation.
Push provider down: messages still flow when receiver reconnects.
Network partition between datacenters: cross-region routing falls back; possibly delays delivery.

36.16 Sketch for interview

[Mobile]──ws──→[Gateway]──→[Pub/sub]──→[Msg svc]──→[Cassandra: chat msgs]
                                              ↓
                                  routing to recipient's gateway
                                              ↓
[Gateway]──ws──→[Mobile]
                                              ↓ if offline
                                  [Push svc]──→APNS/FCM

[Presence svc]←─heartbeats──[Gateways]
[Group svc] manages membership
[Media svc]──→[Object store + CDN]

36.17 What an interviewer wants

Persistent connection model (WS / custom binary).
Gateway-to-user routing via shared map.
Pub/sub bus to decouple gateway pool from message processing.
Message store sharded by chat_id; time-ordered IDs.
Presence + heartbeats.
E2E encryption awareness (Signal protocol high-level).
Push notifications for offline receivers.

36.18 Common pitfalls

Single gateway as SPOF: many gateways, replicate routing map.
Persistent connection state held in memory only: a gateway crash wipes routing; need shared map.
Sync delivery to all devices: doesn't scale; async fan-out.
No idempotency: retries cause dupes.
Trying to encrypt in the server: defeats E2E.

Key takeaways

Long-lived connections at hundreds of millions: many gateways with shared routing.
Pub/sub bus decouples connection layer from message logic.
Cassandra for chat history, sharded by chat_id.
E2E encryption: Signal protocol; server is a blind relay.
Presence is high-volume; throttle and subscribe selectively.
Push notifications fill the gap when receiver offline.