---

1-Hour Interview Roadmap

Time	Topic
0 – 5 min	Requirements clarification
5 – 10 min	Capacity estimation
10 – 15 min	High-level architecture + core components
15 – 25 min	Real-time communication — WebSockets
25 – 35 min	Messaging flow — send, deliver, acknowledge
35 – 45 min	Database design + media storage
45 – 55 min	Scaling + caching + offline delivery
55 – 60 min	Security (E2EE) + trade-offs + failure scenarios

---

What Are We Building?

A real-time messaging platform supporting:

• One-to-one messaging
• Group messaging (up to 1,024 members)
• Media sharing (images, video, documents)
• Online/offline status and typing indicators
• Message delivery and read receipts
• End-to-end encryption

Scale reference: WhatsApp serves 2 billion+ users, delivers 100 billion messages/day, and maintains under 50ms message delivery latency.

---

Step 1 — Requirements

Functional Requirements

#	Requirement
1	Send and receive text messages (one-to-one and group)
2	Show message status: Sent → Delivered → Read
3	Show online/offline status and last seen
4	Show typing indicators in real time
5	Share media — images, video, audio, documents
6	Deliver messages to offline users when they come back online
7	Push notifications when the app is in background
8	Support group chats with up to 1,024 members
9	Message history — load previous messages (pagination)

Non-Functional Requirements

#	Requirement
1	Message delivery latency < 100ms for online users
2	High availability — 99.99% uptime
3	Messages must not be lost — at-least-once delivery
4	Messages must not be duplicated — idempotent delivery
5	End-to-end encryption — server never sees plaintext
6	Horizontally scalable — billions of users, trillions of messages
7	Eventual consistency acceptable for read receipts and presence

Out of Scope

• Voice and video calls
• Payments
• Stories / Status updates
• WhatsApp Business API

---

Step 2 — Capacity Estimation

Assumptions

Metric	Value
Daily Active Users (DAU)	500 million
Messages per user per day	40
Total messages per day	20 billion
Media messages (%)	30%
Average text message size	100 bytes
Average media file size	1 MB
Message retention	Forever (user's device)

Message Throughput

20 billion messages/day
= ~230,000 messages/second (sustained)
= ~700,000 messages/second (peak)

Storage

Text:  20B × 70% × 100 bytes  = ~1.4 TB/day
Media: 20B × 30% × 1 MB       = ~6 PB/day  (after deduplication: ~600 TB/day)

Key Insight

This is a write-heavy, latency-critical system. The primary design challenge is not storage — it is delivering 700K messages/second with sub-100ms latency while maintaining correct ordering and exactly-once delivery semantics.

---

Step 3 — High-Level Architecture

flowchart TD
    Client --> AG[API Gateway\nAuth + Rate Limiting]
    AG --> WSG[WebSocket Gateway]
    AG --> US[User Service]
    AG --> MS[Media Service]

    WSG --> MSG[Messaging Service]
    MSG --> KF[Kafka\nMessage Queue]
    MSG --> RD[(Redis\nPresence + Sessions)]
    KF --> DLV[Delivery Service]
    DLV --> CS[(Cassandra\nMessage Store)]
    DLV --> NS[Notification Service]
    NS --> FCM[FCM / APNs\nPush Notifications]

    MS --> S3[Object Storage\nS3 / GCS]
    S3 --> CDN[CDN]

Component Responsibilities

Component	Responsibility
API Gateway	Authentication (JWT), rate limiting, routing
WebSocket Gateway	Maintain persistent connections for all online clients
Messaging Service	Validate, route, and enqueue messages
Kafka	Durable message queue — decouple send from delivery
Delivery Service	Consume from Kafka, deliver to online clients or store for offline ones
Redis	Track online presence, active WebSocket sessions, typing indicators
Cassandra	Persistent message storage — optimized for time-series write-heavy workload
Media Service	Accept uploads, store in S3, return CDN URL
Notification Svc	Send push notifications via FCM (Android) and APNs (iOS)

---

Step 4 — Real-Time Communication: Why WebSockets?

HTTP vs WebSocket

Factor	HTTP (Polling)	WebSocket
Connection	New connection per request	Single persistent connection
Latency	High — poll interval delay	Very low — server pushes instantly
Overhead	Full HTTP headers on every request	Minimal framing overhead after setup
Server push	Not possible without long-polling hacks	Native — server pushes any time
Battery on mobile	Poor — constant polling drains battery	Good — idle connection uses minimal power
Scale	Many connections = many threads	Event-loop model handles millions

Conclusion: WebSockets are the only viable option for real-time messaging at scale.

WebSocket Connection Lifecycle

flowchart LR
    App -->|HTTP Upgrade| WSG[WebSocket Gateway]
    WSG -->|Authenticate JWT| Auth[Auth Service]
    Auth -->|OK| WSG
    WSG -->|Register connection| RD[(Redis\nuser_id → server_id)]
    WSG -->|Connection open| App
    App -->|Disconnect| WSG
    WSG -->|Remove from Redis| RD

• On connect: User's user_id → server_id mapping stored in Redis
• On disconnect: Mapping removed; last-seen timestamp updated
• Multiple devices: Each device gets its own WebSocket connection

---

Step 5 — Messaging Flow

One-to-One Message Send Flow

sequenceDiagram
    participant Alice
    participant WSG as WebSocket Gateway
    participant MSG as Messaging Service
    participant KF as Kafka
    participant DLV as Delivery Service
    participant RD as Redis
    participant CS as Cassandra
    participant Bob

    Alice->>WSG: Send message {to: Bob, text: "Hello"}
    WSG->>MSG: Forward message
    MSG->>MSG: Assign message_id (Snowflake)
    MSG->>CS: Persist message (status: SENT)
    MSG->>KF: Publish to topic "messages"
    MSG-->>Alice: ACK {message_id, status: SENT}

    KF->>DLV: Consume message
    DLV->>RD: Is Bob online? Which server?
    alt Bob is online
        DLV->>WSG: Route to Bob's server
        WSG->>Bob: Deliver message
        Bob-->>DLV: DELIVERED ACK
        DLV->>CS: Update status → DELIVERED
        DLV->>Alice: DELIVERED notification
    else Bob is offline
        DLV->>CS: Store for offline delivery
        DLV->>NS: Send push notification
    end

Message Status States

SENT → DELIVERED → READ
 ✓         ✓✓        ✓✓ (blue)

Status	Trigger
SENT	Server received and persisted the message
DELIVERED	Message reached the recipient's device
READ	Recipient opened the conversation

Offline Message Delivery

When a user comes back online:

1. WebSocket connection re-established
2. Device sends last seen message_id to server
3. Delivery Service queries Cassandra for all messages after that ID
4. Undelivered messages are pushed in order
5. Status updated to DELIVERED

---

Step 6 — Group Messaging

Challenge

A message to a group of 1,024 members must be delivered to up to 1,024 WebSocket connections — potentially spread across hundreds of servers.

Design

flowchart TD
    Alice -->|Send to group| MSG[Messaging Service]
    MSG -->|Persist once| CS[(Cassandra)]
    MSG -->|Publish one event| KF[Kafka\ngroup-messages topic]
    KF --> FAN[Fan-out Service]
    FAN -->|Lookup members| GS[Group Service\nRedis]
    FAN -->|Deliver per member| DLV[Delivery Service]
    DLV --> Members

Key Design Decisions

Decision	Choice
Message stored once	Store single copy in Cassandra; each member has a pointer (receipt)
Fan-out strategy	Async fan-out via Kafka — do not block sender
Large groups	Fan-out happens in background; recipient may see slight delay
Membership cache	Group member list cached in Redis to avoid DB lookup per message

---

Step 7 — Database Design

Why Cassandra?

Factor	Why Cassandra Wins
Write throughput	Millions of writes/second — LSM tree, append-only
Read pattern	Load chat history = range scan by (chat_id, timestamp)
Scale	Linear horizontal scaling — add nodes, capacity grows
Availability	Tunable consistency — AP system, no single point of failure
Time-series data	Partition by chat ID, cluster by message time — perfect fit

Message Table Schema

Table: messages

Partition key:   chat_id           VARCHAR
Clustering key:  message_id        BIGINT (Snowflake — time-ordered)
                 created_at        TIMESTAMP

Columns:
  sender_id     BIGINT
  content       TEXT (encrypted)
  content_type  VARCHAR  (TEXT | IMAGE | VIDEO | AUDIO | DOC)
  media_url     TEXT
  status        VARCHAR  (SENT | DELIVERED | READ)
  is_deleted    BOOLEAN

Query pattern: SELECT * FROM messages WHERE chat_id = ? AND message_id > ? LIMIT 50

This maps exactly to Cassandra's partition + clustering key design — efficient range reads with no full scans.

User Table

Table: users

  user_id        BIGINT   PRIMARY KEY
  phone_number   VARCHAR  UNIQUE
  display_name   VARCHAR
  avatar_url     TEXT
  created_at     TIMESTAMP
  last_seen      TIMESTAMP
  public_key     TEXT  (for E2EE)

Chat Table

Table: chats

  chat_id       VARCHAR  PRIMARY KEY  (UUID for 1:1, group_id for groups)
  chat_type     VARCHAR  (ONE_TO_ONE | GROUP)
  created_at    TIMESTAMP
  last_message  TEXT
  last_activity TIMESTAMP

Group Member Table

Table: group_members

  group_id    VARCHAR
  user_id     BIGINT
  joined_at   TIMESTAMP
  role        VARCHAR  (ADMIN | MEMBER)

---

Step 8 — Media Storage

Design Principle

Never send media through the messaging pipeline. Media is large, and routing it through WebSocket gateways and Kafka would overwhelm the system.

Media Upload Flow

flowchart LR
    Alice -->|1. Request upload URL| MS[Media Service]
    MS -->|2. Pre-signed S3 URL| Alice
    Alice -->|3. Upload directly to S3| S3[Object Storage]
    S3 -->|4. Upload complete| MS
    MS -->|5. Return CDN URL| Alice
    Alice -->|6. Send message with media_url| MSG[Messaging Service]
    Bob -->|7. Click to view| CDN[CDN Edge]
    CDN --> S3

Why Pre-signed URLs?

• Client uploads directly to S3 — no media bytes travel through your backend
• Your servers handle only metadata, never raw file bytes
• CDN serves media at the edge — Bob in Tokyo gets the file from a Tokyo edge node

Media Deduplication

Before uploading, compute SHA-256 of the file:

• If the hash already exists in S3 → return the existing CDN URL (zero upload cost)
• If new → upload and store hash

This is how WhatsApp avoids storing the same viral video millions of times.

---

Step 9 — Caching Strategy

Redis Usage

Data	Key Pattern	TTL	Purpose
Online presence	presence:{user_id}	30 sec	Is user online right now?
WebSocket server	ws_server:{user_id}	Session	Which server is this user connected to?
Typing indicator	typing:{chat_id}:{user_id}	5 sec	Show typing bubble (auto-expires)
Group membership	group_members:{group_id}	10 min	Avoid DB lookup on every group message
Recent message cache	recent:{chat_id}	1 hr	Last 20 messages for instant load
Session tokens	session:{user_id}:{device}	30 days	Active device sessions

Presence Detection

flowchart LR
    Client -->|Heartbeat every 20s| WSG[WebSocket Gateway]
    WSG -->|SETEX presence:user_id 30| RD[(Redis)]
    RD -->|TTL expires = offline| Presence[Presence Service]
    Presence -->|Broadcast to contacts| DLV[Delivery Service]

• Client sends a heartbeat every 20 seconds
• Redis key TTL is 30 seconds — if no heartbeat, key expires → user is offline
• Contacts are notified of status change asynchronously

---

Step 10 — Scaling

Connection Scaling

Each WebSocket server can handle ~50,000–100,000 concurrent connections.

500M DAU × 30% online at peak = 150M concurrent connections
150M / 75,000 per server = 2,000 WebSocket servers

All servers are stateless (connection state in Redis) — add or remove servers without any migration.

Message Throughput Scaling

flowchart LR
    MSG[Messaging Services\n× 100 pods] -->|publish| KF[Kafka Cluster\n100+ partitions]
    KF --> DLV1[Delivery Pod 1]
    KF --> DLV2[Delivery Pod 2]
    KF --> DLV3[Delivery Pod N]

• Kafka partitioned by chat_id — messages in the same chat always go to the same partition, preserving order
• Add Delivery pods to scale consumer throughput linearly

Database Scaling

Layer	Strategy
Cassandra	Consistent hashing — shard by chat_id across nodes
Read replicas	3x replication factor — reads served from nearest replica
Multi-region	Each region has its own Cassandra cluster; cross-region async replication
Redis	Redis Cluster — shard by key hash; sentinel for failover

---

Step 11 — End-to-End Encryption (E2EE)

How It Works

WhatsApp uses the Signal Protocol for E2EE. The server never has access to plaintext messages.

flowchart LR
    Alice -->|Encrypted with Bob's public key| Server[WhatsApp Server]
    Server -->|Passes encrypted blob| Bob
    Bob -->|Decrypts with own private key| Plaintext
    Server -->|Never sees plaintext| X[❌]

Key Exchange

1. Each device generates a key pair (public + private)
2. Public key is uploaded to WhatsApp servers
3. Before first message, Alice fetches Bob's public key
4. Alice encrypts: Encrypt(message, Bob_public_key)
5. Server stores and forwards the encrypted blob
6. Bob decrypts: Decrypt(blob, Bob_private_key)

Key Points

Point	Detail
Server visibility	Server sees only encrypted bytes — cannot read messages
Key storage	Private keys never leave the device
Group messages	Each member has their own encryption key pair; message encrypted once per recipient
Backup concern	Cloud backups (Google Drive, iCloud) are NOT E2EE by default unless opted in

---

Step 12 — Push Notifications

When a user is offline, messages must be delivered via push notification.

flowchart LR
    DLV[Delivery Service] -->|user is offline| NS[Notification Service]
    NS -->|Android users| FCM[Firebase Cloud Messaging]
    NS -->|iOS users| APNs[Apple Push Notifications]
    FCM --> AndroidDevice
    APNs --> iOSDevice
    Device -->|User opens app| WSG[WebSocket Gateway]
    WSG --> DLV2[Full message delivery]

• Push notification only carries a badge/alert — not the message content (E2EE)
• Actual message fetched when user opens the app and establishes WebSocket connection

---

Step 13 — Failure Scenarios

Failure	Impact	Mitigation
WebSocket server crashes	All users on that server disconnected	Clients auto-reconnect; Redis removes stale session; no messages lost (Kafka still has them)
Redis goes down	Presence and routing unavailable	Delivery falls back to broadcast or push notification; Redis Sentinel auto-failover
Kafka goes down	Message queue unavailable	Messages buffered in Messaging Service memory; Kafka cluster has 3 replicas
Cassandra node fails	Some partition reads/writes rerouted	Replication factor 3 — two other nodes handle it; eventual rebalance
Media service down	Media uploads fail	Text messages unaffected; media upload retried by client with backoff
Delivery Service down	Messages not consumed from Kafka	Kafka retains messages (7-day retention); another consumer picks up

---

Final Architecture

flowchart TD
    Mobile[Mobile Clients] --> LB[Load Balancer]
    LB --> AG[API Gateway\nAuth + Rate Limiting]
    AG --> WSG[WebSocket Gateway Cluster\n2,000+ servers]
    AG --> US[User Service]
    AG --> GS[Group Service]
    AG --> MS[Media Service]

    WSG --> MSG[Messaging Service\n× 100 pods]
    MSG --> KF[Kafka Cluster\nchat_id partitioned]
    MSG --> RD[(Redis Cluster\nPresence + Sessions)]
    KF --> DLV[Delivery Service\n× 100 pods]
    DLV --> CS[(Cassandra Cluster\nMessage Store)]
    DLV --> NS[Notification Service]
    NS --> FCM[FCM]
    NS --> APNs[APNs]
    MS --> S3[Object Storage\nS3 / GCS]
    S3 --> CDN[CDN\nEdge Delivery]

---

Technology Stack

Layer	Technology
Client protocol	WebSocket (persistent) + HTTPS (media upload)
API Gateway	Kong / AWS API Gateway / NGINX
Messaging Service	Go / Spring Boot
Real-time gateway	Netty / Go net/http (event-loop, non-blocking)
Message queue	Apache Kafka (partitioned by chat_id)
Presence cache	Redis Cluster
Message storage	Apache Cassandra
Media storage	Amazon S3 / Google Cloud Storage
CDN	CloudFront / Cloudflare
Push notifications	Firebase Cloud Messaging + Apple APNs
Monitoring	Prometheus + Grafana + ELK + Jaeger
Deployment	Kubernetes across multiple regions

---

Key Trade-Offs

Decision	Option A	Option B	Choice & Reason
Client protocol	HTTP Polling	WebSocket	WebSocket — persistent connection, server push, low latency
Message storage	MySQL/PostgreSQL	Cassandra	Cassandra — write-heavy, time-series, linear horizontal scale
Message delivery	Synchronous	Async via Kafka	Async — decouple send from delivery; durable; handle offline users
Media routing	Through app servers	Direct S3 + CDN	Direct S3 — never route large files through your own servers
Consistency (messages)	Strong	Eventual	Strong for ordering; eventual for receipts and presence
Encryption	Server-side	End-to-End (Signal Protocol)	E2EE — privacy, regulatory compliance, zero server visibility
Group fan-out	Sync (block sender)	Async fan-out	Async — large groups (1,024) cannot block the sender

---

Common Interview Mistakes

• ❌ Designing with HTTP polling instead of WebSockets
• ❌ Not explaining how presence (online/offline) detection works
• ❌ Routing media through WebSocket connections — never do this
• ❌ No offline message delivery strategy
• ❌ Forgetting message ordering — Kafka partitioned by chat_id preserves order
• ❌ No duplicate message prevention (idempotency keys)
• ❌ Storing group messages N times (once per member) — store once, fan-out receipts
• ❌ Not mentioning E2EE — it is a core WhatsApp feature
• ❌ No push notification strategy for background/offline users

---

Interview Questions

1. Why does WhatsApp use WebSockets instead of HTTP?
2. Walk me through how a message travels from Alice to Bob.
3. How do you deliver a message when the recipient is offline?
4. How does WhatsApp detect whether a user is online?
5. Why is Cassandra chosen over PostgreSQL for message storage?
6. How does group messaging fan-out work at scale?
7. How do you prevent duplicate message delivery?
8. How is message ordering guaranteed in a distributed system?
9. How does End-to-End Encryption work at a high level?
10. How is media shared without overloading your servers?
11. What happens if a WebSocket server crashes?
12. What happens if Redis goes down?
13. How do you scale to 700,000 messages per second?
14. How does Kafka help decouple the messaging pipeline?
15. What is the difference between FCM and APNs push notifications?

---

Summary

Concern	Solution
Real-time delivery	Persistent WebSocket connections
Message queue	Kafka partitioned by chat_id — preserves order
Online presence	Redis with heartbeat-based TTL
Message storage	Cassandra — write-heavy, time-series, horizontally scalable
Offline delivery	Store in Cassandra + push notification via FCM/APNs
Media sharing	Pre-signed S3 upload + CDN delivery
Privacy	Signal Protocol — End-to-End Encryption
Group messaging	Store once, async fan-out per member
Scale	Stateless services + Kafka partitioning + Cassandra sharding
High availability	Multi-region, Redis Sentinel, Kafka replication factor 3

The core principle: Keep the message path fast, async, durable, and encrypted. Presence and receipts are eventually consistent — message delivery must not be.

---