Katkat
Back

Design Chat APIs, Storage, and Core Message Flows

DatabricksSoftware Engineer · Onsite
HardSoftware EngineerOnsiteSystem Design

Last updated

Design a highly available and consistent backend chat system focusing on APIs, storage schema, idempotent message publishing, per-conversation sequencing, and versioned group membership.

Scenario

Design the backend core for an enterprise messaging platform (similar to Slack or Microsoft Teams). The system must support direct (1:1) and group chats, reliable message delivery, strictly ordered conversation timelines, and precise access boundaries when conversation membership changes over time.

Requirements

Functional Requirements

  • Direct & Group Conversations: Support 1:1 direct messages and group conversations with up to 2,000 participants.
  • Idempotent Message Ingestion: Ensure clients retrying on network failures do not create duplicate messages.
  • Strict Per-Conversation Ordering: Messages within any single conversation must have a total, monotonic order visible to all participants.
  • Versioned Membership & Visibility: Users added to a conversation should only view messages from the point they were added (or according to policy), and removed users must immediately lose access to subsequent messages.
  • History & Pagination: Efficiently retrieve paginated message history backward and forward from a specific message or timestamp.

Non-Functional Requirements & Scale

  • Scale: 50 million Daily Active Users (DAU), averaging 500 million messages per day (~6,000 msgs/sec average, 25,000 msgs/sec peak).
  • Latency: P99 delivery latency under 100 ms for online participants; read history queries under 50 ms.
  • Durability & Consistency: Zero message loss once acknowledged. Per-conversation linearizability for sequence assignment.

What Your Design Should Cover

  1. API Design: Define the RPC or HTTP/WebSocket contracts for SendMessage, GetMessages, and UpdateConversationMembership.
  2. Storage Architecture & Data Model: Detail the schema for conversations, messages, and membership. Explain your database choice (e.g., distributed SQL, wide-column NoSQL, or hybrid KV + doc store) and partitioning strategy.
  3. Message Sequencing & Idempotency: Explain how unique, monotonic sequence IDs are allocated per conversation, how client-side retry tokens prevent duplicates, and how concurrent senders are coordinated.
  4. Membership Lifecycle & Visibility: Describe how membership changes (join, leave, kick) are recorded and how read paths enforce historical visibility boundaries without expensive real-time joins.
  5. Delivery & Fan-out Flow: Detail the write path from ingestion to persistent storage and real-time push to online recipients.

Hints

Hint 1

To guarantee gap-free or monotonically increasing message order per conversation, separate identity generation from physical storage by using a lightweight coordinator, distributed counter, or partition-level lock.

Hint 2

For message visibility across membership changes, model membership as an append-only timeline of intervals (e.g., joined at sequence X, departed at sequence Y) rather than a mutable list.

Hint 3

Use client-generated idempotency keys combined with conversation IDs stored in an in-memory cache or transactional index to reject retried requests before assigning sequence numbers.

Follow-up questions

  • How does your architecture handle a 'hot conversation' (e.g., an all-company channel with 50,000 users) without overwhelming connection brokers during synchronous fan-out?
  • If a network partition isolates a replica across regions, how do you prevent split-brain message ordering within the same conversation?
  • How would you implement tombstoning and compliance-mandated permanent message deletion (GDPR/retention policies) across your active storage and archive tiers?