Katkat
Back

Design systems for global request detection and labeling

AmazonApplied Scientist · Onsite
HardApplied ScientistOnsiteML System Design

Last updated

This problem tests the architecture of a high-throughput streaming system capable of detecting anomalies in real-time, managing extreme class imbalance for labeling, and maintaining low-latency inference at a global scale.

Problem Statement

Design a global request detection and labeling system for a major e-commerce platform that processes millions of requests per second. The system must identify suspicious or anomalous request patterns (e.g., bot activity or fraudulent spikes) in real-time and route relevant events to a labeling pipeline.

Requirements

  • Functional: Detect anomalous patterns within a 500ms latency window and route potential candidates to an asynchronous labeling system.
  • Scale: Handle 5 million requests/second globally. The anomaly rate is extremely low (less than 0.01% of total traffic), creating a severe class imbalance problem.
  • Non-Functional: Maintain high availability and data consistency. Describe how you would handle stale model versions and regional data fragmentation.

Expected Deliverables

  1. High-Level Architecture: Diagram or describe the ingestion layer, processing engine, and model inference services.
  2. Feature Engineering: Explain how to perform time-windowed aggregations (e.g., requests per IP per minute) at scale.
  3. Labeling Strategy: Detail an adaptive sampling strategy to handle the imbalance, ensuring the human-in-the-loop pipeline receives high-entropy samples for retraining.
  4. Data Consistency: How do you ensure the labeling pipeline reconciles events with the features used during real-time inference?

Hints

Hint 1

Consider using a distributed streaming platform like Apache Kafka or Kinesis and partitioning keys to ensure localized state for time-windowed aggregations.

Hint 2

Think about using a multi-stage inference approach: a low-latency 'coarse' filter for feature-based heuristics followed by a 'fine' heavy-weight model for suspicious requests.

Hint 3

Address class imbalance by implementing a stratified reservoir sampling mechanism within the streaming pipeline to prioritize rare, high-uncertainty events for human annotation.

Follow-up questions

  • How would you measure the drift in detection accuracy if the distribution of 'normal' traffic patterns shifts significantly over time?
  • If the labeling process takes several hours, how do you handle the feedback loop to update the real-time inference model without inducing training-serving skew?
  • How would you implement a 'shadow mode' deployment to test a new model version against live traffic without impacting the primary production detection pipeline?