Design a Notification Service
Other teams call this service to send emails, push messages and texts to users. Requests arrive in sharp bursts, such as a marketing blast. Users can opt out, so every request checks their preferences first. The actual sending is done by an external provider that is slow and rate limited, and a message must never be delivered twice, even if a worker retries.
Constraints
- Accepting a send request must take under 200 ms
- About 5,000 send requests per second at peak, and bursts of three times that
- Opted-out users must never be messaged
- A message is delivered at most once, even when a delivery is retried
- Delivery goes through an external provider with a rate limit you buy by the plan
- Old delivery logs are purged by a background job
- Keep infrastructure under $6,500 a month
Scenarios to survive
Typical peak
Steady send requests at the usual peak.
Interviewer asks: Which parts of sending a message must happen before you answer the caller, and which can wait?
Marketing blast (3x)
A campaign triples the request rate for a few minutes.
Interviewer asks: Where does the burst pile up, and how do you keep the provider's rate limit from becoming your outage?
Delivery workers crash
Whatever runs delivery loses an instance.
Interviewer asks: Workers die mid-send. How do you make sure the message goes out exactly once?
Preference lookups slow down
Whatever runs the preference check becomes 10x slower.
Interviewer asks: You cannot skip the opt-out check. What do you do instead to keep requests fast?
What a good design must do
- Requests are accepted within the latency target at peak
- At least 98% of requests are accepted during the blast
- Total monthly cost stays within $6,500
- Delivery is decoupled from accepting requests (queue plus worker)
- The hand-off to delivery is an async connection
- Something delivers messages
- Something purges old logs
- The design explains how opt-outs are enforced