Katkat
Back

Should Uber double member discounts?

UberData Scientist · Technical Screen
MediumData ScientistTechnical ScreenStatistics & Math

Last updated

This problem tests your ability to design a controlled experiment for a two-sided marketplace and evaluate whether a discount-doubling strategy is statistically and economically viable.

The Problem

Uber is considering doubling the discount offered to loyalty program members for rides in a specific metropolitan market. As a Data Scientist, you are tasked with evaluating the impact of this change before a full rollout.

The Scenario

  • Current state: Loyalty members get a 5% discount on rides.
  • Proposed change: Increase discount to 10%.
  • Primary objective: Determine if this change increases Net Revenue (Total Fare - Discounts) or if the volume increase (higher ride demand) is offset by the margin compression.

Your Task

  1. Experimental Design: Describe how you would set up an A/B test to measure the impact of this change. Specifically, address how you would handle the two-sided marketplace dynamics (riders and drivers) to avoid interference/spillover effects.
  2. Metric Definition: Define a 'North Star' metric and at least two 'Guardrail' metrics to monitor during the experiment.
  3. Power Analysis: Suppose you expect a 2% lift in ride volume. If your current standard deviation of daily ride volume is 10%, qualitatively explain the relationship between the sample size required, the significance level (alpha), and the desired power (1-beta).
  4. Decision Framework: If the test results show a statistically significant increase in ride volume but a neutral impact on Net Revenue, what further considerations would you weigh before recommending the roll-out?

Hints

Hint 1

Consider whether randomization should happen at the user level or the geographic cluster level to minimize network spillover.

Hint 2

Think about the impact of increased ride demand on driver wait times and surge pricing, and how that affects the 'neutral' revenue scenario.

Hint 3

Guardrail metrics should focus on operational health, such as driver availability or fulfillment rates, to ensure the discount doesn't break the marketplace balance.

Follow-up questions

  • How would you account for 'cannibalization,' where members might have taken a ride anyway, effectively losing revenue on those bookings?
  • If this experiment shows a positive impact, how would you test for heterogeneous treatment effects (e.g., does the discount work better for suburban commuters versus city-center users)?
  • What statistical methods would you use to control for external shocks, such as a major local event or a competitor's price drop during the test period?