Diagnose Decline in Successful Orders

You are a Data Scientist at a food-delivery marketplace. In one geographic market, the number of successful orders has declined over the past 4 weeks.

Assume a successful order is an order that is placed, accepted, delivered, and not later refunded or canceled due to marketplace failures. The business wants to know:

1. What could be causing the decline?
2. What metrics and slices would you examine first?
3. How would you distinguish between demand-side, supply-side, operational, and measurement issues?
4. What analyses and A/B tests would you propose to validate root causes and improve the metric?

Please structure your answer as if you were leading the investigation for a marketplace product team.

You may assume the platform has the following relevant data available:

• Customer sessions / app opens
• Search and menu views
• Add-to-cart and checkout events
• Order placement events
• Merchant acceptance / rejection
• Dasher assignment and delivery times
• Cancellations, refunds, and support contacts
• Pricing, fees, promos, ETAs, and stockout signals
• Customer, merchant, and dasher attributes by market and time

In your answer, define a metric framework for diagnosing the drop. Consider factors such as:

• Seasonality, holidays, weather, outages, and local events
• Changes in demand, conversion, merchant availability, courier supply, ETAs, pricing, and service quality
• Mix shift across customer segments, cuisines, merchants, or time of day
• Data quality / instrumentation issues
• Short-term fixes versus longer-term product interventions

Finally, describe one or two experiments you would run, including:

• Primary metric
• Guardrail metrics
• Likely sources of bias or confounding
• How you would interpret ambiguous results