Design a Chat System (WhatsApp / Messenger) — System Design

🟠 High-Level Design · Senior → 🔴 the connection-routing deep dive

"Design WhatsApp / Messenger / Slack." A chat system looks simple — send a message, it appears — but the moment you have millions of users spread across hundreds of servers, a hard question appears: when Alice sends Bob a message, which server is Bob even connected to? That routing problem, plus offline delivery and group fan-out, is what this interview is really testing. Let's design it from the ground up.

Step 1 — Requirements

• Functional: 1-on-1 messaging and group chats; real-time delivery; online/last-seen status; delivery & read receipts; message history; offline users get messages when they reconnect.
• Non-functional: low latency (chat must feel instant), highly available, and reliable — messages must not be lost and should arrive in order within a conversation.

Step 2 — Why WebSockets

Plain HTTP can't let the server push to the client, and chat is inherently two-way and instant. So each client opens a persistent, full-duplex WebSocket to a chat server and keeps it open. Messages flow both directions over that one socket — no polling, minimal latency. That decision drives everything else.

Step 3 — The core challenge: who holds the connection?

🔴 This is the heart of the problem. With millions of users, you run hundreds of chat servers behind a load balancer, and each user's WebSocket lives on one of them. When Alice (on Server 3) messages Bob, Server 3 has no idea Bob is connected to Server 47. You need a routing layer.

The standard solution: a connection registry — a fast store (like Redis) mapping user_id → the server holding their connection. When a user connects, their server writes this mapping; on disconnect, it clears it. Now delivery is: look up the recipient's server, forward the message there, and that server pushes it down the recipient's socket.

Step 4 — The message flow

🟠 Putting it together, sending a message does two things in parallel:

1. Persist first: write the message to the database (this guarantees it's never lost, powers history, and enables offline delivery). Store it with a per-conversation sequence so ordering is well-defined.
2. Deliver in real time: look up the recipient in the connection registry. Online? Forward to their chat server, which pushes it down their socket instantly. Offline? It's already saved — deliver it when they reconnect and their client fetches "what did I miss?"

This "store-and-forward" pattern is the key reliability idea: persistence makes delivery guaranteed, and the real-time push makes it feel instant. Receipts flow the same way — the recipient's client sends back "delivered"/"read" events that route to the sender.

Step 5 — Data model

messages:  msg_id, conversation_id, sender_id, content,
           seq_no, created_at, status
conversations: conv_id, type (1:1 | group), member_ids
connection registry (Redis): user_id → server_id  (who holds the socket)

🔴 Message volume is enormous and write-heavy, so this is a great fit for a horizontally-scalable store like Cassandra (an LSM-tree DB built for high write throughput), sharded by conversation so a chat's messages stay together and ordered.

Step 6 — Group chat (fan-out again)

🔴 A group message must reach every member. For normal groups you fan out: look up each member in the registry and push to their server (or drop to their offline store). Just like the news feed, huge groups/broadcast channels are the pain point — you cap group size, or for broadcast-scale use a pull/subscribe model rather than pushing to millions at once. Often a message queue sits between the sender and the fan-out so a slow delivery never blocks the sender.

Step 7 — Online presence

🟠 Online/last-seen is powered by the same connection state: a user is "online" while their WebSocket is alive (with periodic heartbeats so a dead connection is detected). Presence changes are pushed to that user's contacts. Because exact presence for everyone is expensive, it's usually eventually consistent — a few seconds of lag on "last seen" is fine.

The interview-ready walkthrough

"Each client holds a WebSocket to a chat server. The core challenge is routing: with many servers, I keep a connection registry (Redis) mapping user → server, so to deliver a message I look up the recipient's server and push it down their socket. I use store-and-forward — persist every message first (to a write-optimised store like Cassandra, sharded by conversation) so nothing is lost and offline users get it on reconnect, then deliver in real time if they're online. Group chats fan out to members via a queue, capping huge groups. Presence and receipts ride the same connection with heartbeats." That covers real-time delivery, the routing problem, reliability, and scale — the full picture."

What to read next

• Design a Notification System — delivering to users across channels & devices
• WebSockets, Long Polling & SSE — the real-time transport underneath
• Message Queues & Kafka — decoupling the fan-out
• ← The complete System Design guide (hub)

← Design a news feed · Design a notification system →