Katkat
Back

Design an Advertising or Recommendation Model End to End

SnapchatMachine Learning Engineer · Onsite
HardMachine Learning EngineerOnsiteML System Design

Last updated

Design a recommendation model from labels and sampling through features, loss, ranking metrics, online experiments, and production debugging.

Design an advertising or recommendation model end to end, then defend the data, features, sampling, model, loss, tuning, and evaluation choices. Begin by choosing one concrete prediction and ranking objective for the discussion.

Constraints & Assumptions

  • The report describes an open-ended ad/recommendation case without a specific product or scale. State the chosen case as a model-answer assumption.
  • Keep eligibility rules and the candidate supply separate from model ranking unless the interviewer expands the scope.
  • Explain metric definitions and their trade-offs, including which online outcome validates the offline model.

Clarifying Questions to Ask

  • What action or user value should the ranking improve, and what guardrails constrain it?
  • Which candidates and user/context features are available at serving time?
  • Are labels delayed, selectively observed, or affected by the previous ranking policy?

Hints

Hint 1

Tie every metric to a decision: AUC, log loss, ranking quality, and product outcomes answer different questions. State which failure each metric would detect and which it could miss.

Follow-up questions

  • What changes if negative sampling improves training cost but distorts predicted probabilities?
  • How would you investigate an offline improvement that fails in an online experiment?