Katkat
Back

Design a fraud detection system

AmazonSoftware Engineer · Technical Screen
HardSoftware EngineerTechnical ScreenML System Design

Last updated

This problem evaluates your ability to design a scalable, low-latency fraud detection system for an e-commerce platform, focusing on handling class imbalance, feature engineering, and the challenges of delayed feedback loops.

Problem Statement

Design a fraud detection service for a global e-commerce marketplace that authorizes payments in real-time. The system must process thousands of transactions per second with a latency budget of under 100ms per request.

Requirements

  • Functional: Provide a binary classification score (Fraud/Legit) for every transaction attempt.
  • Non-Functional: Ensure high availability, low latency, and a mechanism to handle model updates without downtime.
  • Data Challenges: Address the extreme class imbalance (fraud is rare) and the "delayed label" problem (fraud is often confirmed by users or banks days after the transaction).

Discussion Points

Your solution should cover:

  1. Data Pipeline: How do you handle real-time feature extraction vs. historical batch training data?
  2. Model Serving: How would you implement a shadow mode or A/B testing strategy for new model deployments?
  3. Feedback Loop: Since labels are delayed, how do you train and validate the model effectively?
  4. Adversarial Handling: How do you detect and mitigate feature drift caused by attackers changing their tactics?

Hints

Hint 1

Consider using a feature store to bridge the gap between low-latency online serving and high-throughput batch training.

Hint 2

Think about how to balance the precision-recall trade-off given that the cost of a false positive (denying a legit user) is high, but a false negative (allowing fraud) is also costly.

Hint 3

For the delayed feedback issue, explore techniques like importance sampling or partial label learning to utilize 'uncertain' transactions during training.

Follow-up questions

  • How would you re-architect the system if we needed to provide 'explainability' for why a specific transaction was flagged as fraud?
  • If an attacker starts generating high volumes of low-value 'legit' transactions to confuse the model, how would you detect this 'data poisoning'?
  • How would you implement a 'circuit breaker' to bypass the ML model if latency spikes or the model starts outputting abnormal scores?