Katkat
Back

Design a search-to-ads ranking pipeline

SnapchatMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteSystem Design

Last updated

This question tests the ability to design a high-throughput, low-latency ranking system that balances user experience with monetization goals through multi-stage retrieval and bidding mechanics.

System Design: Search-to-Ads Pipeline

Design a ranking pipeline for a mobile app (like Snapchat) where user queries trigger both organic search results and sponsored advertisements. The system must deliver a blended feed that optimizes for user engagement (e.g., clicks, time spent) and platform revenue.

Requirements

  1. Functional:

    • Retrieve relevant content and ads based on a search query.
    • Rank and mix items to balance organic relevance and ad monetization.
    • Incorporate real-time ad bidding/pacing constraints.

  2. Non-Functional:

    • Latency: End-to-end latency must be under 200ms.
    • Throughput: Handle 50k queries per second.
    • Scalability: Ad inventory is massive; search results are dynamic.

Expected Response

  • Architecture: Define the stages of the funnel (e.g., Candidate Generation, Scoring, Re-ranking/Mixing).
  • Modeling: Describe how you handle the different objectives (engagement vs. revenue) and what features are essential.
  • Ad Auction: Explain how the auction mechanism integrates into the ranking flow, including how you ensure ads don't overwhelm the user experience.
  • Performance: Discuss strategies for low-latency retrieval (e.g., vector search, caching, model quantization).

Hints

Hint 1

Consider a multi-stage funnel approach where the initial retrieval stage is broad and high-recall, followed by increasingly complex models as the candidate pool shrinks.

Hint 2

Think about the 'Mixing Problem': how do you decide the interleaving ratio of organic to paid content while maintaining user relevance?

Hint 3

Address how budget pacing—ensuring advertisers don't spend their entire daily budget in the first hour—can be incorporated into the ranking score.

Follow-up questions

  • How would you handle cold-start problems for new ads that have no historical performance data?
  • If latency spikes during peak hours, what is your strategy for gracefully degrading the system?
  • How would you design an A/B testing framework to evaluate a change to the ranking algorithm without hurting long-term user retention?