Katkat
Back

Accept Notification Bursts With Low Latency

AmazonSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

Design a high-throughput, low-latency notification ingestion and delivery service capable of absorbing sudden 10x traffic spikes while maintaining strict delivery SLOs and downstream rate limits.

Scenario

During major promotional events (e.g., Prime Day, flash sales), internal Amazon microservices generate sudden, massive bursts of transactional and marketing notifications (SMS, Email, Push). You need to design a centralized Notification Ingestion Service that accepts incoming notification requests with minimal latency (< 50ms p99) and guarantees durable delivery without overwhelming downstream notification gateways (e.g., Apple APNs, Google FCM, telecom aggregators).

Requirements

Functional Requirements

  • Send Notification: Allow client services to submit single or batch notification requests specifying recipient, channel, priority, and payload.
  • Multi-channel Delivery: Dispatch notifications via push (APNs/FCM), SMS, and email.
  • Deduplication & Idempotency: Prevent duplicate dispatches if callers retry identical requests.
  • Status Tracking: Provide a way for callers or internal dashboards to check delivery status.

Non-Functional Requirements

  • Ingestion Latency: p99 < 50ms for accepting and durably acknowledging requests.
  • Burst Absorption: Handle standard load of 10,000 requests/sec with sudden bursts up to 100,000 requests/sec within seconds.
  • Durability: Zero message loss once acknowledged.
  • Isolation & Prioritization: Critical alerts (e.g., OTP, fraud alerts) must not be delayed by bulk marketing bursts.
  • Downstream Protection: Adhere to strict downstream third-party rate limits via controlled concurrency and backpressure.

Scale Assumptions

  • Normal traffic: 10,000 requests/sec.
  • Peak burst: 100,000 requests/sec (payload size: ~2 KB per request).
  • Target delivery completion: High-priority messages delivered within 5 seconds; standard messages within 2 minutes.

What Your Design Should Cover

  1. API Design: Core endpoints for ingestion and status retrieval.
  2. High-Level Architecture: Decoupling the ingestion path from the delivery path.
  3. Buffering & Queueing Strategy: Partitioning, topic topology, and worker auto-scaling.
  4. Overload Control & Backpressure: Priority queueing, token-bucket rate limiting, and shedding policies.
  5. Idempotency & Failure Handling: Retry mechanisms, dead-letter queues (DLQ), and circuit breaking.

Hints

Hint 1

Decouple the synchronous ingestion tier from downstream dispatchers using a horizontally scalable, partitioned distributed log (e.g., Apache Kafka or Amazon Kinesis).

Hint 2

Separate message streams into distinct priority tiers so high-priority traffic bypasses bulk message backlogs during traffic spikes.

Hint 3

Implement distributed idempotency using unique client-provided request IDs stored in a low-latency key-value store with TTLs to prevent duplicate deliveries across retries.

Follow-up questions

  • How would you ensure that a downstream provider outage (e.g., APNs failing) doesn't exhaust queue capacity or stall delivery to other channels?
  • How does your design prevent a 'noisy neighbor' service from consuming all burst capacity at the ingestion layer?
  • If workers crash mid-delivery after calling the third-party provider, how do you handle idempotency on the downstream provider side?