Katkat
Back

Design a Scalable Catalog Service

AttentiveSoftware Engineer · Onsite
MediumSoftware EngineerOnsiteSystem Design

Last updated

Design a high-throughput, read-heavy catalog service supporting draft/publish lifecycles, item versioning, search projections, and resilient rollback mechanisms.

Scenario

Design a multi-tenant product catalog service for an e-commerce marketing platform. Merchants manage product inventories, create draft updates, and publish coordinated product drops (e.g., flash sales). Downstream messaging triggers and public storefront integrations generate massive read traffic against published items.

Requirements

Functional Requirements

  • Item Lifecycle & Versioning: Merchants can create and edit draft items, publish specific drafts, and immediately roll back to any previous published version.
  • Coordinated Releases: Support atomic or coordinated publishing of multiple items simultaneously (e.g., seasonal launches).
  • Search & Browse: Fast querying by SKU, category, tag, or full-text keywords for currently published items only.
  • Schema Evolution: Support dynamic attributes (e.g., size, color, brand-specific metadata) with backward-compatible schema changes.

Non-Functional Requirements

  • Scale: 100,000 active merchants, 100 million total SKUs, 50,000 read QPS (peak 100,000 QPS), and 500 write QPS.
  • Latency: P99 read latency < 25 ms; publish propagation to read layer < 2 seconds.
  • Availability: 99.99% for public reads; 99.9% for merchant writes.
  • Consistency: Strong consistency for write/publish validation; eventual consistency acceptable for search projections within the latency SLA.

Key Areas to Cover

  1. Data Model & Versioning: How drafts, published states, and historical versions are represented in storage without duplicating full catalog trees unnecessarily.
  2. Architecture & CQRS: Separation of write-side (authoring) and read-side (serving/search projections) data stores.
  3. Publishing Pipeline & Idempotency: How writes are processed safely, handling retry storms, partial publish failures, and out-of-order updates.
  4. Projection & Cache Management: How read projections (e.g., search index, distributed cache) are updated, invalidated, or rebuilt without downtime or stale reads.
  5. Rollback & Failure Recovery: Mechanism for reverting an item or collection to a prior state without data corruption.

Hints

Hint 1

Separate the transactional authoring store (e.g., relational DB with immutable version rows) from the read projection store (e.g., document store or search index) using a CQRS pattern.

Hint 2

Model item versions as immutable snapshots with a state pointer (e.g., `current_published_version_id`) to make rollbacks an O(1) metadata update.

Hint 3

Use Change Data Capture (CDC) via an event log (such as Kafka) to asynchronously update search indices and invalidate edge caches while preserving publish order.

Follow-up questions

  • How would you handle a merchant importing an updated catalog of 500,000 items via bulk CSV without starving real-time write requests from other merchants?
  • If the event consumer feeding the search index crashes and lags by 15 minutes, how do you prevent users from browsing and purchasing out-of-stock items?
  • How would you implement zero-downtime schema evolution when adding a new required filterable attribute across millions of existing items?