Katkat
Back

Investigate Pop-up Impact on Partner Referral Conversions

DoorDashData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Investigate Pop-up Impact on Partner Referral Conversions states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Scenario

Partner-referral channel suddenly shows a sharp fall in first-time purchasers after a new splash pop-up asking visitors to download the mobile app was launched.

Question

How would you investigate and confirm that the pop-up caused the drop in conversions from partner links? Assuming online traffic is unchanged, how would you quantify the trade-off between app-download uplift and lost first-purchase conversions? What metrics and experiment design would you propose to decide whether to keep, modify, or remove the pop-up?

Hints

Map user funnel, build pre/post or A/B test, measure incremental value of app installs vs. lost orders, control for seasonality and partner mix.