Katkat
Back

Should DoorDash add bicycle dashers?

DoorDashData Scientist · Technical Screen
MediumData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

This problem tests the ability to design an A/B test for a marketplace intervention, identify success metrics, and account for potential cannibalization and external factors in urban delivery.

The Scenario

DoorDash is considering introducing bicycle-based dashers in a highly congested, high-density urban downtown area. Currently, all deliveries are handled by car/motorcycle drivers. You are tasked with determining whether this rollout will improve marketplace health and delivery efficiency.

The Task

Design an experimental framework to evaluate the impact of introducing bicycle dashers. Address the following:

  1. Metric Selection: Define primary and secondary metrics to measure success, including both operational efficiency and business performance.
  2. Experimental Design: Describe how you would conduct an A/B test. How would you handle the geographic nature of delivery zones to avoid spillover effects?
  3. Causal Considerations: Identify at least two potential confounding variables (e.g., weather, time of day) and how you would mitigate their impact on your results.
  4. Marketplace Dynamics: Discuss how you would measure potential cannibalization, specifically whether bicycle dashers are stealing deliveries from existing drivers or creating new demand by reducing wait times.

Constraints & Requirements

  • Provide a clear methodology for randomization (e.g., cluster randomization vs. user-level randomization).
  • Ensure the solution accounts for the differences in travel speeds and delivery radii between bicycles and motorized vehicles.
  • Your answer should focus on the logic and trade-offs rather than writing code.

Hints

Hint 1

Consider the 'Stable Unit Treatment Value Assumption' (SUTVA) and how the proximity of bike lanes affects delivery zones.

Hint 2

Think about how to differentiate between 'incremental demand' (new customers ordering) and 'substitution' (existing customers switching to faster delivery).

Hint 3

Identify if metrics like 'Time to First Dash' or 'Average Delivery Time' are sufficient, or if you need to measure 'Driver Utilization' and 'Delivery Cost per Order'.

Follow-up questions

  • If the experiment shows high performance in the downtown core, how would you design a 'staggered rollout' strategy to expand to other neighborhoods?
  • How would you account for seasonal impact if the bicycle rollout happens in the summer and you want to predict performance for winter?
  • Suppose the bike dashers significantly increase order volume, but driver satisfaction scores among car drivers drop. How do you reconcile these two conflicting business metrics?