Katkat
Back

Design a Chat Application with a Partitioned Cache

DatabricksSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

Design an end-to-end direct and small-group messaging system with strict delivery guarantees, message ordering, and an explicitly partitioned multi-tier caching layer.

Scenario

You are tasked with designing the messaging backend for a real-time collaboration tool supporting 1:1 direct messages and small group chats (up to 50 participants). The system must guarantee durable storage, low-latency online delivery, consistent ordering per conversation, seamless offline catch-up, and deduplication of retried messages.

A central focus of this interview is the caching strategy: you must decide what data lives in local server memory versus a distributed shared cache, how cache keys are partitioned, and how the system preserves correctness during cache invalidation, node failovers, and hot conversation spikes.

Requirements

Functional Requirements

  • 1:1 and Small Group Messaging: Users can send text messages to individuals or small groups (<= 50 members).
  • Delivery & Ordering: Messages within a conversation must be presented to all participants in a strictly monotonic order.
  • Offline Catch-Up: Users who come back online can sync missed messages without receiving duplicates.
  • Deduplication: Client-side retries due to network timeouts must not produce duplicate messages.

Non-Functional Requirements

  • Scale: 50 million Daily Active Users (DAU); 500 million messages sent per day (average ~6,000 writes/sec, peak ~20,000 writes/sec).
  • Latency: Real-time message delivery under 100 ms at p99 for online recipients; history fetch under 150 ms at p99.
  • High Availability & Durability: Zero message loss once acknowledged to the sender (RPO = 0).

Key Deliverables

  1. Core Architecture & Data Flow: Diagram or describe message ingestion, storage persistence, real-time fan-out via persistent connections (e.g., WebSockets), and offline inbox synchronization.
  2. Data Model & Ordering: Define how message IDs / sequence numbers are generated per conversation to maintain monotonic ordering and handle idempotency keys.
  3. Caching Architecture:
    • Distinguish the role of local in-memory caches (gateway/connection nodes) versus distributed shared caches (e.g., Redis/Memcached).
    • Define the cache partitioning strategy (e.g., partitioned by conversation_id, consistent hashing).
    • Detail eviction policies, time-to-live (TTL), and write-through vs. write-around semantics.
  4. Resilience & Edge Cases: Explain how your design handles cache misses, stale reads after a partition rebalance, cache stampedes on hot group chats, and network partitions between the cache and persistent store.

Hints

Hint 1

Assign monotonically increasing sequence numbers scoped to each `conversation_id` using a lightweight coordinator or atomic database sequence rather than relying on distributed wall-clock timestamps.

Hint 2

Partition shared cache instances by `conversation_id` using consistent hashing so recent messages and participant metadata for a given thread reside on the same cache node.

Hint 3

Prevent cache stampedes on active group chats by employing distributed read-through locks (single-flighting) and seeding newly created cache entries with a short TTL before relying on append operations.

Follow-up questions

  • How does your caching and fan-out design change if group size limits are increased from 50 to 5,000 members?
  • If a cache node storing a hot conversation abruptly crashes, how do you prevent the underlying database from being overwhelmed while the partition is rebalanced?
  • How do you handle client clock skew and offline out-of-order retries while keeping the conversation sequence monotonic?