Katkat
Back

Design Personalized Promotion Recommendations

CreditkarmaMachine Learning Engineer · Technical Screen
MediumMachine Learning EngineerTechnical ScreenML System Design

Last updated

This question tests the ability to design an end-to-end recommendation system for personalized financial promotions, focusing on handling sparse conversion data, real-time feature requirements, and feedback loops.

Design a Personalized Promotion Recommendation System

Design a system for a financial platform that recommends credit products (e.g., credit cards, personal loans) to users. The objective is to maximize the click-through rate (CTR) and the conversion rate (application completion) for these promotions.

Requirements

  • Scale: The system must support 50 million daily active users with sub-200ms latency for recommendations.
  • Data: User profiles, historical credit application data, credit score snapshots, and interaction logs (impressions, clicks, dismissals).
  • Feedback: Conversion data (loan approval/disbursement) is often sparse and delayed, occurring days or weeks after the initial recommendation.

Your design should address:

  1. Candidate Generation & Ranking: How to select a small subset of relevant offers from a catalog of thousands and how to rank them effectively.
  2. Feature Engineering: What features are most predictive for financial product propensity, and how to handle real-time vs. batch feature updates.
  3. Labeling Strategy: How to handle the "delayed feedback" problem where a user might apply for a product long after the recommendation.
  4. System Architecture: High-level components (API, Feature Store, Model Service, Data Pipeline).
  5. Evaluation: How to validate the model offline and perform A/B testing in production.

Hints

Hint 1

Consider using a two-stage architecture with a light retrieval layer (like collaborative filtering or embedding-based lookup) followed by a heavy deep learning ranking model.

Hint 2

Address the sparsity of conversions by using intermediate signals like 'clicks' or 'application start' as proxy labels during training.

Hint 3

To handle delayed feedback, explore techniques like building a time-weighted loss function or implementing a waiting period for positive labeling.

Follow-up questions

  • How would you ensure that the system does not recommend credit products for which the user is highly likely to be rejected?
  • How would you implement an exploration strategy to show new or less popular products without significantly hurting user experience?
  • What monitoring metrics would you track to detect data drift, given that financial market conditions can change rapidly?