Katkat
Back

Evaluate business value of lower ETA

UberData Scientist · Technical Screen
MediumData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

This question tests the ability to design an experiment that isolates the causal impact of reduced ETA on user retention and to translate these metrics into business value.

Problem Statement

At a ride-hailing company, the engineering team has developed an optimization algorithm that reduces the average Estimated Time of Arrival (ETA) for riders by 10% in a specific pilot city. As a Data Scientist, you are tasked with evaluating whether this technical improvement translates into increased long-term business value.

Your Tasks

  1. Experimental Design: Propose an experimental setup to measure the impact of the lower ETA on user behavior. Define the target population, randomization unit, and key metrics.
  2. Causal Inference: How would you handle potential spillover effects (e.g., improved supply efficiency for one rider affecting the ETA for another)?
  3. Business Valuation: If the experiment shows a 2% increase in "Completed Rides" and a 1% increase in "Weekly Active Riders," describe a framework to calculate the Net Present Value (NPV) of this feature rollout for the company.

Key Considerations

  • How do you distinguish between short-term noise (e.g., seasonal holidays) and long-term behavioral changes?
  • What trade-offs might occur on the driver side (e.g., earnings per hour) when you optimize strictly for rider ETA?
  • Consider how you would account for the cannibalization of other ride tiers.

Hints

Hint 1

Consider using cluster-based randomization (e.g., by city zone or time-window) rather than individual-level randomization to minimize interference.

Hint 2

Think about decomposing 'business value' into its constituent parts: volume (number of rides), retention (lifetime value), and marketplace balance (supply-side costs).

Hint 3

Use a difference-in-differences (DiD) approach to compare treatment and control groups before and after the intervention to control for underlying trends.

Follow-up questions

  • If we see an increase in rider demand but no corresponding increase in driver supply, what happens to the marketplace, and how should we measure that?
  • How would you adjust your analysis if the treatment effect on ETA is heterogeneous across different neighborhood densities (urban vs. suburban)?
  • What specific guardrail metrics would you implement to ensure that reducing ETA does not inadvertently hurt driver safety or satisfaction?