Katkat
Back

Design a global real-time notification system

GoogleSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

This question assesses the ability to design a distributed, global-scale notification platform that handles massive throughput, multi-channel delivery, and reliability requirements.

Problem Statement

Design a global real-time notification system capable of sending billions of messages daily across multiple channels, including mobile push notifications, email, and SMS. The system must support low-latency delivery, high availability, and must ensure that users do not receive duplicate notifications even in the event of retries.

Functional Requirements

  • Support multiple delivery channels (Push, Email, SMS).
  • Allow clients/services to request notifications via a unified API.
  • Ensure message delivery even if a provider (e.g., Twilio for SMS, Firebase for Push) is temporarily down.

Non-Functional Requirements

  • Scale: Handle 100,000 requests per second (RPS) peak load.
  • Latency: End-to-end delivery latency for push notifications should be under 500ms for 95th percentile (p95).
  • Reliability: At-least-once delivery guarantee.
  • Data Consistency: Maintain idempotency across retries.

Requirements for your Design

  1. High-level architecture diagram (components and their interactions).
  2. API definition for the notification gateway.
  3. Strategy for handling message persistence, queuing, and background processing.
  4. How to manage multi-region deployment and global load balancing.
  5. Discussion on observability, rate limiting, and failure handling (e.g., fallback mechanisms).

Hints

Hint 1

Consider using a message queue or distributed streaming platform to decouple the ingestion layer from the delivery worker layer.

Hint 2

Address the idempotency requirement by assigning a unique message ID to every request and using a distributed cache or database to track processing state.

Hint 3

For global scale, utilize regional ingestion points and a message-bus architecture that routes traffic based on provider latency or availability.

Follow-up questions

  • How would you implement user preferences (e.g., 'Do not disturb' mode or notification silencing) without introducing latency bottlenecks?
  • If the SMS provider's API suddenly starts returning 5xx errors, how would you implement an automated circuit breaker and fallback strategy?
  • How would you measure the performance and 'success rate' of deliveries across disparate third-party APIs?