Katkat
Back

Design an ad recommendation and ranking system

MetaData Scientist · Onsite
MediumData ScientistOnsiteMachine Learning

Last updated

This question evaluates end-to-end ad recommendation and ranking design skills, covering machine learning modeling, recommender systems, causal and counterfactual evaluation, online learning, and production ML engineering within the Machine Learning domain, at a high-level system-design and modeling abstraction.

You are building an ad recommendation/ranking system for a content feed (e.g., short-form videos). At each feed position, you may show either an organic item or an ad.

Design the end-to-end approach:

  1. Define the objective(s) (e.g., revenue, user experience) and how you would combine them (constraints vs. weighted objective).
  2. Propose modeling choices for predicting key events (CTR, CVR, p(revenue), long-term value), including labels, features, and handling delayed feedback.
  3. Explain how you would address common issues:
  • selection bias / counterfactual evaluation (logged policy)
  • exploration vs. exploitation
  • cold start (new ads, new users)
  • advertiser budget/pacing constraints
  • fairness/diversity constraints (optional but discuss if relevant)
  1. Provide an evaluation plan: offline metrics + online A/B metrics and guardrails.