Katkat
Back

Design High-Throughput Messaging Infrastructure

OktaFrontend Engineer · Technical Screen
MediumFrontend EngineerTechnical ScreenSystem Design

Last updated

This problem tests your ability to design a scalable, reliable messaging system by balancing throughput, latency, and data persistence in a high-concurrency environment.

Scenario

You are asked to design a high-throughput messaging infrastructure similar to a real-time chat or notification system. The system must support millions of concurrent users sending and receiving messages with sub-100ms latency.

Requirements

  • Functional: Users must be able to send messages, receive messages in real-time, and view their message history.
  • Non-functional: High availability, low latency, and guaranteed message delivery for critical notifications.
  • Scale: 1 million concurrent connections, 50,000 messages per second (MPS).

Your Task

Describe a high-level architecture that can handle these requirements. Address the following components:

  1. Message Broker Choice: How will you handle message buffering and distribution (e.g., Kafka, RabbitMQ, or custom Redis-based solutions)?
  2. Database Strategy: How do you handle message storage? Compare SQL vs. NoSQL approaches given the write-heavy workload.
  3. Connection Management: How will you maintain 1 million concurrent connections? Discuss how you handle connection state and load balancing.
  4. Handling Backpressure: How does the system behave when a consumer cannot keep up with the producer's volume?

Hints

Hint 1

Consider the difference between push-based and pull-based models for message delivery.

Hint 2

Think about partition strategies for your message broker to prevent bottlenecks on a single consumer or broker node.

Hint 3

Evaluate whether you need a single source of truth for real-time delivery versus historical data retrieval.

Follow-up questions

  • How would you handle global delivery if your users are distributed across different continents?
  • What strategies would you implement to ensure messages are not lost if a broker instance crashes?
  • How would you change your schema if the system requirement shifted to support 'read receipts' for every message?