Katkat
Back

Analyze Cancellation Change with Statistics

UberData Scientist · Technical Screen
MediumData ScientistTechnical ScreenStatistics & Math

Last updated

This question evaluates statistical inference for binary outcomes, regression adjustment for confounding, experiment design and power analysis, and nonparametric uncertainty estimation using bootstrapping, and is targeted at Data Scientist roles within the Statistics & Math domain.

Uber introduces a small product tweak to reduce cancellations. You observe:

  • Before: n1 = 50,000 requested trips, cancel rate p1 = 7.2%.
  • After: n2 = 48,000 requested trips, cancel rate p2 = 6.6%. Assume independent Bernoulli outcomes per request to start. Answer:
  1. Choose an appropriate hypothesis test for the change in cancellation rate and compute a 95% CI for (p_after − p_before). State H0/HA and interpret the CI practically.
  2. Now consider confounding by city and hour-of-day. Specify a logistic regression that adjusts for these (and an interaction term you deem important). Write the model formula, list key assumptions, and describe how you would check calibration and overdispersion.
  3. Power/MDE: What per-arm sample size would you need to detect an absolute change of 0.5 percentage points at α=0.05 with 80% power (ignore clustering first)? Then discuss how an intraclass correlation of 0.02 at the driver level would inflate the required N under a cluster-robust design; show the design effect and the new N.
  4. Nonparametric check: Describe a bootstrap procedure to form a CI for the rate difference stratified by city (stratified resampling) and when its CI might be preferable to the z-approximation.