Katkat
Back

Design a Top-K trending service

eBaySoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

This problem evaluates your ability to design a distributed system that aggregates high-velocity event streams to identify the most frequent items within a sliding time window.

Problem Statement

Design a service that tracks the top-K most frequently viewed items on an e-commerce platform in real-time. The system must support a dashboard that updates every minute to show the trending items based on the last hour of activity.

Requirements

  • Functional: Provide an API endpoint to retrieve the current top-K trending items.
  • Scale: The platform processes 100,000 views per second across millions of unique product IDs. High availability and low latency for read requests are required.
  • Consistency: Eventual consistency is acceptable, but the rankings should be reasonably accurate within a few seconds of lag.

Expectations

Your design should address:

  1. Data Ingestion: How to handle the high volume of incoming view events.
  2. Aggregation Strategy: How to count frequencies efficiently across a distributed set of servers (e.g., MapReduce-style batching vs. stream processing).
  3. Storage: How to store counters and ranking results for fast retrieval.
  4. Scalability & Reliability: Handling data spikes and ensuring the system does not crash if a node fails.

Provide a high-level architecture diagram (described in text) and explain the flow of data from the client to the final API response.

Hints

Hint 1

Consider using a distributed message queue like Kafka to buffer incoming traffic and decouple ingestion from processing.

Hint 2

Think about using the 'Count-Min Sketch' algorithm if memory constraints make tracking every unique product ID in a hash map infeasible.

Hint 3

Decide whether to calculate top-K rankings on-demand during reads or push pre-computed results to a fast-access cache like Redis.

Follow-up questions

  • How would you modify your design if you needed to support 'top-K' for specific geographic regions or categories?
  • What specific trade-offs would you make if the business suddenly required 100% accuracy rather than approximate results?
  • How would you handle a 'hot key' problem where one specific product accounts for 50% of all traffic?