Katkat
Back

Design photo and listing quality models

AirbnbMachine Learning Engineer · Onsite
MediumMachine Learning EngineerOnsiteMachine Learning

Last updated

This question tests the end-to-end design of a machine learning system to evaluate the quality of listing photos and overall listing attractiveness to improve search ranking and conversion.

Problem Statement

Airbnb aims to improve user experience by ensuring that high-quality listings appear at the top of search results. You are tasked with designing a two-part machine learning system:

  1. Photo Quality Model: A system to score individual photos based on aesthetic appeal and utility (e.g., clarity, lighting, composition).
  2. Listing Quality Model: A system that consumes photo scores and other listing attributes (price, reviews, location) to predict the overall attractiveness of a listing to guests.

Requirements

  • Goal: Maximize conversion rate (bookings) and user engagement (clicks/saves).
  • Data: You have access to millions of raw photos, user click logs, booking history, and listing metadata.
  • Design Components:
    • Define the target variable for both models.
    • Propose a feature engineering strategy, specifically for the unstructured image data.
    • Explain the training approach, including handling the potential class imbalance if using a binary classification approach.
    • Define metrics to evaluate the models offline and online.
    • Address how to handle "cold start" listings with few or no reviews.

Constraints

  • The system must scale to millions of listings.
  • Consider the latency requirements for real-time ranking updates.
  • Address potential biases (e.g., photos of luxury properties vs. budget-friendly rooms) and ensure fairness in your quality assessment.

Hints

Hint 1

Consider using pre-trained convolutional neural networks or vision transformers as feature extractors to generate embeddings for the photos.

Hint 2

Think about how to use 'click-through rate' (CTR) as a proxy for listing quality when explicit labels like 'high quality' are unavailable.

Hint 3

Address the feedback loop where popular listings gain more clicks, potentially creating a bias in the training data.

Follow-up questions

  • How would you integrate a 'diversity' constraint so that a listing with 20 identical high-quality photos doesn't get an unfair ranking boost?
  • If the model starts penalizing budget listings because they lack professional photography, how would you adjust the model or the data collection to maintain platform inclusivity?
  • Describe a strategy for monitoring model performance in production; what metrics would alert you that the quality model is degrading?