Katkat
Back

Evaluate Dasher Initiatives with A/B Testing and Metrics

DoorDashData Scientist · Onsite
HardData ScientistOnsiteAnalytics & Experimentation

Last updated

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Evaluate Dasher Initiatives with A/B Testing and Metrics states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Scenario

As a product/analytics lead at a food-delivery marketplace you must evaluate several Dasher-facing initiatives (Top-Dasher prioritization, Extra-Pay incentives, and switching pay model from per-order to per-time) before deciding whether to launch them.

Question

How would you assess whether the Top-Dasher program should be launched? 2) For an "extra pay" incentive aimed at improving Dasher engagement: a) what primary success metric(s) would you track? b) design an A/B test (including treatment, control, experiment length, sample-size, and guardrail metrics). 3) The company is considering switching Dasher compensation from per-order to per-time. What are the key pros & cons of each model, and how would you experimentally validate which model is better for marketplace health?

Hints

Discuss causal identification, experiment vs. quasi-experiment trade-offs, KPI definition (accept rate, fulfillment time, retention), supply-demand balance, cost impact, and possible negative externalities.