Katkat
Back

Diagnose a One-Day Drop in Successful Deliveries

DoorDashData Scientist · Technical Screen
MediumData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

Tests the candidate's structured metric troubleshooting, funnel decomposition, and root cause analysis for an acute drop in core marketplace volume. Evaluates the ability to separate count effects from conversion rate changes and isolate partner-specific vs. market-wide anomalies.

Context

On a recent Tuesday, the core operations team at DoorDash observed that completed deliveries in a major metropolitan market dropped by 18% compared to the trailing 4-week Tuesday average (from ~100,000 to ~82,000 deliveries). Leadership wants to know what caused this decline, whether it reflects a systemic platform issue or an isolated operational event, and what immediate actions to take.

Prompt

Outline a structured investigation to diagnose this one-day volume drop. In your response, address the following sections:

  1. Data Integrity & Metric Validation: How do you confirm this drop is real before debugging business logic? Identify at least two data pipeline or tracking anomalies that could create a false drop.
  2. Funnel Decomposition: Break down the end-to-end delivery lifecycle into a conversion funnel (from app open through checkout, merchant acceptance, courier dispatch, pickup, and completion). Clearly explain how you distinguish an absolute volume/count shock (e.g., fewer top-of-funnel users) from an efficiency/rate degradation (e.g., lower checkout conversion or Dasher cancellation spikes).
  3. Hypothesis Generation & Prioritization: Brainstorm and rank plausible drivers across five buckets:
    • Data & Tracking Quality
    • Customer Demand (e.g., app bugs, marketing pauses)
    • Merchant Operations (e.g., store closures, menu outages)
    • Courier / Dasher Supply (e.g., Dasher supply shortages, dispatch latency)
    • External Factors (e.g., weather, local holidays, competitor promotions)
  4. Merchant Concentration Analysis: Suppose you suspect a large enterprise merchant (e.g., a major national chain) temporarily went offline or paused delivery. Define a quantitative formula to compute the exact contribution of a single merchant's decline to the total market-level decline. How do you assess whether non-chain merchants also experienced spillover effects?
  5. Synthesis & Next Steps: What would your threshold be to escalate an incident to engineering versus operations, and how would you monitor recovery over the next 48 hours?

Hints

Hint 1

Check tracking pipeline latency and partition arrival times before assuming consumer behavior actually shifted.

Hint 2

Express completed deliveries as the product of top-of-funnel traffic and step-by-step conditional conversion rates to isolate whether the issue is supply/demand volume or execution failure.

Hint 3

Use a shift-share or attribution formula where total volume change equals the change in the specific merchant plus the change in the rest of the market, controlling for organic cross-merchant substitution.

Follow-up questions

  • If checkout conversion dropped 15% but Dasher acceptance rate remained flat, which logs and app releases would you inspect first?
  • How would you statistically test whether orders that would have gone to the offline merchant substituted to other merchants in the same zone?
  • How does your troubleshooting framework change if this drop occurred on a Friday evening peak hour versus a Tuesday afternoon?