Katkat
Back

Build cold-start restaurant ratings

UberData Scientist · Technical Screen
MediumData ScientistTechnical ScreenMachine Learning

Last updated

This problem tests the ability to design an end-to-end machine learning system for predicting restaurant ratings when historical user-item interactions are scarce, focusing on feature engineering, model selection, and bias mitigation.

Scenario

Uber Eats wants to improve the discoverability of new restaurants. When a restaurant is added to the platform, it has no customer ratings. A low or missing rating can lead to a 'cold-start' penalty, where users are less likely to order from new options. Your task is to design a model that predicts an expected rating for new restaurants to display until sufficient real user data is collected.

Objective

Propose an approach to estimate the 'expected rating' for a new restaurant. Your response should address:

  1. Data Sources & Features: What data would you use to represent a new restaurant before it receives its first order? Think about metadata, location, and metadata-derived embeddings.
  2. Target Definition: Since you don't have historical ratings, what is your proxy variable for the target? How do you handle the transition from predicted ratings to actual user ratings?
  3. Model Architecture: Would you use a simple regression, a gradient-boosted tree, or a hybrid approach (e.g., matrix factorization with side information)?
  4. Bias and Fairness: How do you prevent 'popularity bias' from causing the model to consistently underrate niche restaurants, and how do you ensure the model doesn't over-promote low-quality new entrants?
  5. Validation: How would you evaluate the success of this model in a live marketplace environment?

Hints

Hint 1

Consider using 'lookalike' features by grouping the new restaurant with similar existing establishments based on menu item prices, cuisine type, and neighborhood density.

Hint 2

Think about using a Bayesian approach where the 'prior' is the predicted rating and the 'posterior' is updated as real user reviews arrive.

Hint 3

Address the cold-start problem by discussing how to handle the 'exploration-exploitation' trade-off when deciding which new restaurants to boost in the feed.

Follow-up questions

  • How would you handle a scenario where a restaurant has extremely high predicted ratings but consistently receives poor user feedback after launch?
  • If you were to implement an A/B test for this model, what specific metrics would you monitor beyond just the rating accuracy (RMSE/MAE)?
  • How would your approach change if the model needed to provide personalized ratings for different user segments rather than a global rating?