Katkat
Back

Design a Large-Scale Personalized Recommendation System End to End

MetaMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteML System Design

Last updated

An ML system design question asking you to design a large-scale personalized recommendation system end to end, from candidate retrieval and multi-stage ranking to training data, evaluation, serving and monitoring. It tests problem framing, latency budgeting, cold start, feedback loops and online experimentation.

Design a recommendation system end to end. When a user opens a product surface, the system picks a small, ordered set of items to show out of a very large and constantly changing catalog, and it keeps learning from how users respond.

Cover the whole loop: how the problem is framed as a machine learning problem, how candidates are retrieved and ranked, what data and features the models use, how they are trained and evaluated, how the system serves requests within its latency budget, and how it is monitored.

Constraints and Clarifications

  • This was the system design round of an onsite for a machine learning engineer working on search, ads and recommendations.
  • The item type, product surface, scale and success metric are not given. Ask for them or state your assumptions.

Clarifying Questions

  • What is being recommended (posts, videos, products, accounts to follow), and on which surface (a home feed, related items, notifications)?
  • What is the business goal (time spent, meaningful interactions, purchases, long-term retention), and which guardrails matter, such as integrity, diversity or fairness to creators?
  • How many users, items and requests per second, and what is the latency budget per request?
  • How quickly must new items become recommendable: within minutes, or within a day?
  • Is there logged interaction history to train on?

Hints

Hint 1

The catalog is too big to score: The most accurate model cannot score every item in the catalog for every request in time. Decide how to divide the work between cheap and expensive stages, and how many items each stage passes on.

Hint 2

Your data comes from your own system: Users can only react to items the system chose to show them. Think about what that does to your training labels, to offline evaluation, and to new items.

Follow-up questions

  • A new item has no interactions yet. How does it get its first impressions, and how do you keep it from being buried?
  • Offline metrics improve but the online A/B test shows no gain. What are the likely causes, and how do you investigate?
  • How would you combine several objectives, such as clicks, watch time, shares and hides, into one ranking?
  • How do you keep the feed from narrowing each user's interests over time?