Katkat
Back

How to design Shop ad ranking

MetaData Scientist · Technical Screen
HardData ScientistTechnical ScreenMachine Learning

Last updated

This problem tests your ability to design a multi-objective ranking system for e-commerce ads, balancing short-term engagement, advertiser conversion, and platform revenue while mitigating feedback loops.

Problem Statement

You are tasked with designing an ad ranking system for a social media "Shop" tab. The platform displays sponsored product listings interspersed with organic content. Your objective is to design a model that ranks these ads to maximize total platform value.

Requirements

  1. Objective Formulation: Define a composite utility function that balances user experience (e.g., ad relevance, click-through rate), advertiser ROI (e.g., conversion rate, add-to-cart), and platform revenue.
  2. Feature Engineering: Propose features that capture user intent, ad-content relevance, and cross-session signals.
  3. Handling Constraints: Propose strategies to address the "cold-start" problem for new advertisers and ensure fair exposure for smaller brands.
  4. Feedback Loops: Discuss how you would prevent popularity bias and feedback loops where high-exposure items receive more clicks, potentially starving new, high-quality inventory.

Discussion Points

  • Explain how you would perform offline evaluation (e.g., log-based metrics) and online evaluation (e.g., A/B testing strategy).
  • Describe the loss function structure to handle multi-task learning (e.g., predicting CTR and CVR simultaneously).
  • Address how you would measure the trade-off between short-term revenue and long-term user retention.

Hints

Hint 1

Consider framing the objective as a weighted sum of probabilities of different user actions (click, cart, purchase) and explain why these weights might be dynamic.

Hint 2

Think about using a multi-task learning architecture like ESMM (Entire Space Multi-Task Model) to handle the sample selection bias inherent in conversion prediction.

Hint 3

For feedback loops, consider exploration-exploitation trade-offs like Multi-Armed Bandits or positional bias correction techniques during training.

Follow-up questions

  • How would you adjust your model if the advertiser landscape shifts from high-frequency, low-cost goods to low-frequency, luxury items?
  • If you detect that your model is favoring 'clickbait' ads that rarely lead to purchases, how would you re-engineer your target labels or loss function?
  • How does your design change if the latency budget for the ranking inference is strictly under 50ms?