Design CanvasOpen the canvas

Design a Chat Service

People send short messages to friends and groups over a live connection. Every message must reach the other side in the order it was sent, be kept in the conversation history, and, if a recipient is offline, trigger a push notification through an outside provider. Evenings are much busier than the rest of the day.

Constraints

  • Accepting a message must take under 150 ms
  • About 5,000 messages per second at peak, and three times that on a busy evening
  • Messages in one conversation are delivered in the order they were sent
  • Every message is kept in the conversation history
  • About 3 in 10 recipients are offline and get a push notification from an external provider
  • Keep infrastructure under $6,000 a month
  • About 5 billion messages are stored, and people scroll back through long conversations

Scenarios to survive

Typical peak

Steady messaging at the usual peak.

Interviewer asks: Which parts of sending a message must finish before you acknowledge it, and which can happen after?

Busy evening (3x)

Evening traffic triples the message rate.

Interviewer asks: What gives first when the rate triples, and how do you add capacity without breaking per-conversation order?

Delivery workers fail

Whatever runs delivery loses an instance.

Interviewer asks: A delivery worker dies in the middle of a conversation. How do you avoid losing or reordering messages?

Push provider outage

The external push provider stops answering.

Interviewer asks: You cannot control the push provider. What happens to messages and to offline users when it is down?

What a good design must do

  • Messages are accepted within the latency target at peak
  • The busy evening passes without losing messages
  • Total monthly cost stays within $6,000
  • Messages travel through a Stream, so order is kept per conversation
  • Something delivers messages in order
  • Something saves the history
  • Something sends push notifications
  • Messaging keeps working when the push provider is down
  • The design explains how order is kept per conversation, and what the partition key is
  • Every query is answered through a key or an index, never by reading a whole table