Diagnose a Decline in Order Acceptance Rate

Scenario

Over the past two weeks, the marketplace operations team at DoorDash observed that Dasher Order Acceptance Rate (AR)—defined as the ratio of accepted dispatch offers to total dispatch offers sent to Dashers—dropped by 4.2 percentage points (from 78.5% to 74.3%) nationwide. At the same time, customer-facing delivery times have crept up by 3.5 minutes, and unfulfilled order cancellations increased by 18%.

As the Data Scientist embedded within the Dasher Dispatch & Logistics team, you are tasked with leading the root-cause diagnosis and proposing a data-driven path forward.

Tasks

1. Funnel & Instrumentation Audit

• Define the dispatch funnel from order placement to pickup. Clarify how Acceptance Rate is mathematically defined at the offer level versus the order level.
• What initial instrumentation, logging, and data pipeline sanity checks would you perform to verify this is a genuine behavioral shift rather than a telemetry or logging defect?

2. Metric Decomposition (Mix Shift vs. Within-Segment Decline)

• Outline a quantitative framework (e.g., Oaxaca-Blinder or standard rate decomposition) to separate the overall drop into:
  1. Mix effect: Shifts in the proportion of offer types (e.g., changes in vehicle type, geography, order distance, or batching ratio).
  2. Within-segment rate effect: Declines in acceptance probability within specific, constant cohorts.
• Suppose the mix shifted toward long-distance deliveries (> 7 miles) from 15% of all offers to 28%, while acceptance rates for long-distance deliveries remained flat at 45% and short-distance (< 7 miles) stayed flat at 84%. Quantify how much of the 4.2 percentage point drop is explained purely by this distance mix shift.

3. Downstream Ecosystem Impact & Trade-offs

• Map out how a lower AR directly impacts downstream marketplace health metrics: Dasher hourly earnings, assignment latency (time-to-first-accept), order completion rate, and merchant food freshness.
• When might a lower AR actually be healthy or acceptable for the platform?

4. Experimentation & Intervention

• Propose an algorithmic or product intervention (e.g., dynamic acceptance incentives, adjusted dispatch radius, or bundle re-architecting) to address the root cause.
• Detail the experimentation design: unit of randomization, primary metric, guardrail metrics, and how you would address network interference (spatial-temporal spillovers).