Katkat
Back

Evaluate Promotions for Uber Eats Users

UberData Scientist · Technical Screen
MediumData ScientistTechnical ScreenMachine Learning

Last updated

This question tests the ability to design an A/B test for promotions in a two-sided marketplace, focusing on identifying interference (network effects) and selecting appropriate metrics to measure causal impact.

Scenario

Uber Eats is considering launching a '15% off delivery fees' promotion for users in a specific city to increase order volume. You are tasked with designing an experiment to measure the causal impact of this promotion on total gross bookings.

However, there is a concern: in this marketplace, if more users order food, delivery partners might get busier, potentially increasing wait times for everyone. This is a classic 'interference' or 'network effect' problem.

Your Task

  1. Propose an experimental design that accounts for the potential interference between treated and control users.
  2. Define a primary metric to measure success and at least two 'guardrail' metrics to ensure the promotion does not negatively impact the ecosystem.
  3. Explain how you would account for the two-sided nature of the marketplace when estimating the incremental gain of the promotion.

Hints

Hint 1

Consider geographic or time-based randomization instead of user-level randomization to isolate treatment effects.

Hint 2

Think about what happens to courier earnings and restaurant throughput when order volume shifts suddenly.

Hint 3

Consider if the 'total gross bookings' metric captures potential cannibalization or displacement effects.

Follow-up questions

  • If your experiment shows a 5% increase in orders but a 2% increase in average delivery time, how would you determine if the promotion is net-positive?
  • How would you adjust your analysis if the promotion had a 'spillover effect' where non-promoted users were discouraged by long wait times caused by the promotion?
  • What statistical methods would you use to verify that your randomization strategy successfully minimized selection bias?