Design a Chat System

By the Syncfusion DevTutorials team · Last updated October 20, 2026

Build customer–rider chat: WebSockets, sizing gateways by connections, routing through pub/sub, saving before delivering, and splitting messages by conversation.

The rider is outside, but the gate code does not work. The customer is upstairs. They need to talk now, not in thirty seconds. QuickBite is adding chat between customers and riders.

Chat is different from everything so far. Until now, the phone sent a request and the server sent back a response. In chat, the server must send a message to a phone immediately, even when that phone did not ask for anything.

That one requirement changes the whole design.

The main idea: keep a connection open to every phone, know which server holds each connection, and route every message to the right phone.

What You Will Learn

In this case study, you will:

1. Write the requirements for one-to-one chat.
2. Estimate connections, message volume and storage.
3. Compare polling, long polling and WebSockets.
4. Route messages using pub/sub.
5. Store messages in order and scale writes.
6. Build and test the design with 400,000 phones online.

Step 1: Requirements

What It Must Do (Functional)

• One-to-one chat between a customer and the rider for an active order.
• Delivered and read ticks, so users know when messages are received and viewed.
• Message history when a user opens the chat again.
• Offline delivery, so messages sent while a user is disconnected are delivered when they reconnect.

How Well It Must Do It (Non-Functional)

• Real time: messages arrive within 200 ms (p99).
• In order: messages appear in the correct order within a conversation.
• Never lost: once a sender sees a tick, the message is safely stored.

Out of Scope

The first version does not include:

• Group chat
• Voice messages
• File sharing
• End-to-end encryption

These can be added later on top of the same architecture.

Step 2: Estimate

At dinner peak, about 400,000 phones are online with an active chat.

The gateway count comes from connections, not messages. Each gateway holds about 100,000 open connections:

What to notice:

• Messages are not the primary challenge.
• Open connections are the real challenge.
• A gateway can process thousands of messages per second.
• Holding 100,000 active WebSocket connections requires significant memory.

Now storage. QuickBite sends about 60 million messages a day, and each one takes about 200 bytes (text, ids, time and status):

60,000,000 × 200 bytes × 365 days = 4.4 TB for one year.

Storage is manageable. The difficult part is writing thousands of messages every second while keeping them in order.

Step 3: How Does the Server Reach the Phone?

There are three common approaches.

Polling

The phone asks the server, "Do I have new messages?" every few seconds.

Most requests return nothing, which wastes resources and adds delay.

Long Polling

The phone sends a request and the server keeps it open until a message arrives.

This is better than polling, but every received message requires another request.

WebSockets

The phone opens one connection and keeps it open.

Both the phone and server can send messages whenever they want.

Step through polling and then WebSockets for QuickBite's 400,000 phones, with a poll every 2 seconds:

What to notice:

• Polling generates huge numbers of unnecessary requests.
• Most requests return no messages.
• Users wait before receiving updates.
• WebSockets provide the best real-time experience.

QuickBite uses WebSockets. The phone connects once, through the load balancer, to one Chat Gateway, and keeps the connection open throughout the order.

Step 4: Routing a Message to the Right Gateway

With multiple gateways, the sender and receiver are usually connected to different servers.

Step through one message from the customer to the rider:

The flow:

1. The customer sends a message to Gateway 1 through a WebSocket.
2. Gateway 1 forwards the message to the Chat Service.
3. The Chat Service validates and stores the message.
4. The sender receives a sent tick because the message is now safely stored.
5. The Chat Service publishes the event using pub/sub.
6. Gateway 3, which holds the rider's connection, receives the event.
7. Gateway 3 pushes the message to the rider immediately.

If the rider is offline:

• No gateway currently owns the connection.
• The message remains safely stored.
• The rider receives it after reconnecting.
• A push notification can remind the rider to open the app.

How often does a message need to cross to another gateway? With phones spread evenly, the receiver is on the same gateway only 1 time in (number of gateways). The chart shows the rest:

What to notice:

• Most messages travel between gateways.
• Pub/sub becomes more important as gateway count increases.
• Gateways rarely contain both chat participants.

Step 5: Storing Messages in Order

One Leader Is Not Enough

Every message creates a database write.

If every write goes to one database leader, that leader eventually becomes overloaded.

More read replicas do not help because messages are write-heavy workloads.

Split by Conversation

Partition messages using the conversation id.

Each conversation always maps to the same shard.

This provides two benefits:

• Writes are distributed across multiple shards.
• Messages remain ordered within a conversation.

Each shard gives every message in a conversation the next sequence number: message 1 gets sequence 1, message 2 gets sequence 2, and so on.

Phones sort messages using sequence numbers rather than timestamps.

Step through three messages and watch where they land:

Here is the same idea in code:

Each conversation always lands on the same shard

function shardFor(conversationId, shards) {
  let h = 0;
  for (const ch of conversationId) h = (h * 31 + ch.charCodeAt(0)) >>> 0;
  return h % shards;
}

for (const id of ["order-8812", "order-8813", "order-8812"]) {
  console.log(`${id} -> shard ${shardFor(id, 4)}`);
}

Expected output:

order-8812 -> shard 0
order-8813 -> shard 1
order-8812 -> shard 0

Step 6: Build It on the Canvas and Test It

Open the design sandbox.

The canvas starts with:

• 2 load balancers
• 4 chat gateways
• 4 chat service servers
• A message store with 2 copies and one leader

The simulator sends 12,000 messages per second. A message counts as delivered only when it reaches the recipient's gateway.

What you will probably find:

• One database leader cannot handle all writes.
• Without pub/sub, gateways cannot deliver most messages.
• Four gateways provide no spare capacity during failures.

• A passing design uses:

  • 2 load balancers
  • 5 chat gateways
  • 5 chat service servers
  • Pub/Sub
  • 4 message shards

This design costs about $2,230 per month and delivers messages with p99 latency around 23 ms.

Common Mistakes

Practice

What to Remember

• Chat requires the server to push messages to users.
• WebSockets keep a real-time connection open.
• Size gateways by connection count, not message count.
• Use pub/sub to route messages between gateways.
• Save first, then acknowledge delivery.
• Store messages by conversation id.
• Use sequence numbers instead of phone timestamps.
• Keep gateways lightweight and move business logic into the Chat Service.

Further Reading

These are the sources this lesson was checked against:

• Slack Engineering: Real-time Messaging: gateway servers, channel servers and routing messages to the right connection.
• MDN: The WebSocket API: how a browser or app keeps a two-way connection open.
• Discord: How Discord stores trillions of messages: partitioning messages by channel, and why the database choice matters.

Next: Notification System, where QuickBite sends order updates through push notifications, SMS and email.