Katkat
Back

Design a Comment-Likelihood Prediction System with a Focus on Feature and Model Serving

RedditMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteML System Design

Last updated

Design a machine-learning system that predicts how likely a user is to comment on a post, covering label definition, features and model choice, with most of the depth on feature serving and model serving. Tests online and offline feature consistency, feature freshness, latency budgets and safe model deployment.

Design the machine-learning system behind comment likelihood on a large discussion platform where posts belong to communities: given a user and a post, estimate the probability that the user will comment on that post. Cover the problem framing, training data, features and model, but expect most of the discussion to be about feature serving and model serving: how every feature the model needs is computed, stored and fetched at request time, and how the model is deployed and scored within the request's latency budget.

Clarifying Questions

  • Where is the score consumed: ranking a personalized feed, ranking posts inside one community, or choosing which posts to send in notifications? This changes the number of candidates and the latency budget.
  • What counts as a positive: a top-level comment only, or any reply in the post's thread, and within how long after the post was shown?
  • What latency budget does the ranking stage have, and how many candidates does it score per request?
  • Which infrastructure already exists: an event stream, a feature store, a model-serving platform?

Hints

Hint 1

Sort features by freshness: Some signals change once a day and others change every few seconds while a post is taking off. Decide how fresh each feature must be; the pipeline and the store that produce it follow from that.

Hint 2

Score the whole candidate list at once: A single request asks for scores on many posts for the same user. Look for work that can be done once per request instead of once per post.

Follow-up questions

  • How would you prevent training-serving skew when a feature's streaming computation differs slightly from its batch backfill?
  • A trending post's comment count changes many times per second and is read by many concurrent requests. How do you serve it fresh without overloading the online store?
  • How would you roll out a new model version safely, and how would you detect a calibration regression after launch?
  • How do you score a brand-new user, or a post that was created a minute ago?