Katkat
Back

Predict Click Probability for Music Recommendations

AmazonMachine Learning Engineer · Technical Screen
MediumMachine Learning EngineerTechnical ScreenML System Design

Last updated

Design music recommendation CTR prediction with song embeddings, sparse-feature handling, model comparison, calibration, and online evaluation.

Design a machine-learning system that predicts the probability that a user will click a recommended song. Explain how you would represent songs, handle extremely sparse features, choose a prediction algorithm, and evaluate the resulting click-through-rate predictions.

Requirements and Constraints

For this exercise, assume the system receives a user, a candidate song, and recommendation context, then outputs a click probability for that impression. Candidate retrieval is outside the initial scope. Training data contains logged impressions and whether each impression produced a click within a defined attribution window. These are explicit task assumptions; the attribution window, serving latency target, and available song metadata must be clarified.

Clarifying Questions

  • What counts as a click, and how are repeated impressions or delayed clicks attributed?
  • Which user history, song metadata, interaction data, or audio-derived features are available at prediction time?
  • Are sparse features high-cardinality identifiers, infrequently observed categories, or mostly missing measurements?
  • Must the probability be calibrated, used only for ranking candidates, or both?

Hints

Hint 1

Distinguish an unseen song from an unobserved feature: A song with little interaction history may still have usable metadata or audio information. Decide how its representation is formed before it has enough clicks for a reliable learned identifier embedding.

Follow-up questions

  • How would you score a newly released song with useful metadata but no historical interactions?
  • What could cause a model with better offline AUC to produce poorly calibrated click probabilities?
  • How would a change in which songs are displayed affect the training labels collected by the system?