Katkat
Back

Design a Book Price Aggregator

DatabricksSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

Tests the ability to architect a resilient aggregation system interfacing with heterogeneous, unreliable third-party APIs under strict latency, rate-limiting, and cache consistency constraints.

Scenario

Design a real-time book price aggregator service (similar to BookFinder or Google Shopping for books). The platform enables users to search for books by ISBN or title and view aggregated listings—including base price, shipping cost, condition (new/used), and vendor rating—across dozens of third-party merchants (e.g., Amazon, Barnes & Noble, AbeBooks, eBay).

Requirements

Functional Requirements

  1. Search & Compare: Given an ISBN or keyword query, return a ranked list of available offers sorted by total cost (price + shipping).
  2. Multi-Vendor Fetching: Aggregate offers from 30+ external merchant APIs and scrapers.
  3. Price Freshness: Balance real-time price accuracy with external API quota limits. Cached prices should typically not exceed a 4-hour staleness window.
  4. Partial Availability: If one or more external vendors fail or time out, return partial results quickly without failing the overall user request.

Non-Functional Requirements

  1. Latency: p99 < 800ms for cached/hybrid queries; outbound vendor timeouts capped strictly at 1.5s.
  2. Throughput: 5,000 search queries per second (QPS) at peak.
  3. Catalog Scale: 50 million unique book titles/ISBNs.
  4. Resilience & Fault Isolation: Prevent cascading failures caused by slow third-party partners; avoid thundering herd on popular releases.

Key Considerations & Scope

Your design should address:

  • API & Interface: Endpoints for querying book prices and ingestion/refresh.
  • Data Model: Schema for book metadata, merchant offers, and historical pricing.
  • Aggregation Pipeline: Push (background crawler/polling) vs. Pull (on-demand query) architectures, or a hybrid approach.
  • Reliability & Concurrency: Rate limiting against merchant APIs, circuit breakers, request coalescing (single-flight execution), and distributed caching strategy.

Hints

Hint 1

Consider a hybrid approach: serve hot catalog items from a pre-warmed cache/database while triggering asynchronous background refreshes or bounded on-demand fetches for cold items.

Hint 2

Think about how to handle the thundering herd problem when thousands of users query the same bestseller simultaneously while its cache is expired.

Hint 3

Evaluate resilience patterns such as circuit breakers, fallbacks, and bulkheads to ensure high latency or outages in one merchant API do not deplete the connection pool for others.

Follow-up questions

  • How would you coordinate rate limits across a distributed pool of worker instances querying a merchant API with a strict quota of 100 requests/second?
  • How would you implement request coalescing (single-flight) across distributed nodes to guarantee only one external call is made for a given ISBN during a cache miss?
  • How would you extend this system to support price-drop alerts for 10 million registered user subscriptions efficiently?