Katkat
Back

Investigate Declining Successful Orders

DoorDashData Scientist · Technical Screen
HardData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

This question tests your ability to decompose a high-level drop in a key business metric into granular segments and formulate testable hypotheses to explain the underlying root causes.

Scenario

You are a Data Scientist at a food delivery platform. You notice that the 'Order Success Rate' (defined as the ratio of completed orders to total orders placed) has dropped by 5% week-over-week. This metric is a critical North Star for the business.

Task

  1. Diagnostic Strategy: How would you systematically investigate this decline? Break down the funnel and identify which segments (e.g., geography, restaurant type, time of day, customer cohort) you would examine to isolate the issue.
  2. Hypothesis Generation: Propose three distinct hypotheses for why this decline might be occurring, focusing on different sides of the marketplace: the customer, the courier (Dasher), and the restaurant.
  3. Validation & Testing: Choose one of your hypotheses. How would you design an A/B test or an observational study to confirm that this is the primary driver of the decline? Define your primary metric and guardrail metrics for this test.

Constraints

  • Assume you have access to logs for orders, deliveries, courier status, and restaurant availability.
  • Keep your analysis focused on actionable levers that the product or operations teams can control.

Hints

Hint 1

Start by visualizing the order lifecycle funnel: from checkout to restaurant acceptance, courier assignment, and final delivery.

Hint 2

Consider external factors versus internal product changes, such as recent software releases, marketing promotions, or regional weather events.

Hint 3

For the experiment design, ensure your guardrail metrics protect against cannibalization or negative impacts on other parts of the marketplace.

Follow-up questions

  • If the data shows the drop is concentrated in 'Busy' hours, how might you distinguish between a supply-side shortage and a surge-pricing failure?
  • How would you account for 'seasonal noise' when comparing week-over-week performance in a marketplace that has strong day-of-week trends?
  • If you find that the decline is caused by an increase in restaurant cancellations, what features or data signals would you look at to predict restaurant-side churn?