Katkat
Back

ML System Design: Movie Recommendation Model and Pipeline (AI-Assisted Round)

TubiMachine Learning Engineer · Technical Screen
MediumMachine Learning EngineerTechnical ScreenML System Design

Last updated

This question tests ML system design competency, specifically the ability to architect end-to-end recommendation pipelines covering retrieval, ranking, feature engineering, and low-latency serving at scale. It assesses practical knowledge of two-stage recommender systems, implicit feedback modeling, cold-start handling, and critical evaluation of AI-generated code — skills central to machine learning engineering interviews.

ML System Design: Movie Recommendation Model and Pipeline (AI-Assisted Round)

This is a hands-on, AI-assisted coding round: you are allowed to use AI coding tools (e.g., an AI pair-programmer in your editor) while you build. Design and sketch the implementation of a movie recommendation model and its data/training/serving pipeline for a streaming service. You should both reason about the system design and produce working scaffolding code for the core pieces — the data preparation, the model, the training loop, and the serving/inference interface — explaining your choices as you go.

Concretely, build toward a system that, given a user, returns a ranked list of movies they are likely to watch. Cover the full pipeline: ingesting interaction data, generating features/labels, training a recommendation model, evaluating it, and serving recommendations at low latency, including how you handle retraining and cold start.

Constraints & Assumptions

  • Catalog on the order of $10^5$ movies; tens of millions of users.
  • Primary interaction signal is watch events (plus optional explicit ratings, search, browse).
  • Online recommendations must return within a low-latency budget (tens of milliseconds) for a few hundred items.
  • The model is retrained on a regular cadence (e.g., daily/weekly) on fresh interaction logs.
  • New users and new movies appear continuously (cold start).
  • Because AI coding tools are permitted, the interviewer expects you to move quickly to runnable scaffolding and to critically review the AI-generated code rather than accept it blindly.

Clarifying Questions to Ask

  • What surface is this for — the personalized home feed, "because you watched X," or post-play "up next"? It changes the candidate set and latency budget.
  • What is the optimization objective — predicted watch probability, expected watch time, or a multi-objective blend with diversity/freshness?
  • What interaction signals are available, and are there explicit ratings or only implicit watch/skip events?
  • What is the online latency and QPS budget, and is there an existing feature store / serving infra to reuse?
  • How fresh must recommendations be — do we need near-real-time updates from a session, or is daily/weekly retraining acceptable?
  • Since this is AI-assisted, are you evaluating the final design, my code-review judgment over AI output, or both?

Follow-up questions

  • Walk through your two-stage design's latency budget end to end: where does the time go in retrieval vs ranking, and what do you precompute offline vs compute per request?
  • For implicit feedback, how exactly do you construct negatives, and what goes wrong if you sample them naively (e.g., uniformly over the whole catalog)?
  • You used an AI tool to generate the training loop. Show me a specific bug or train/serve-skew risk you would specifically check the generated code for, and how.
  • How do you prevent a feedback loop where the recommender keeps recommending what it already recommends, starving exploration and new content?
  • A movie is added to the catalog today with zero interactions. Trace exactly how it can still appear in someone's recommendations within minutes.