Katkat
Back

Decide and test a 20% discount strategy

DoorDashData Scientist · Onsite
HardData ScientistOnsiteAnalytics & Experimentation

Last updated

This question tests the ability to design a controlled experiment to evaluate a promotional discount strategy, focusing on causality, profit optimization, and long-term customer value impact.

Scenario

DoorDash is considering launching a platform-wide 20% discount on delivery fees for a subset of users to increase order frequency. As the lead data scientist, you are tasked with designing an experiment to determine if this strategy is net-profitable and identifying which user segments respond most favorably.

Your Task

  1. Experiment Design: Propose an experimental framework to measure the causal impact of the 20% discount on incremental gross profit. How will you account for potential cannibalization or spillover effects?
  2. Metrics & Heterogeneity: Define your primary and secondary metrics. Explain how you would identify heterogeneous treatment effects to ensure we aren't just subsidizing users who would have ordered anyway (i.e., identifying 'persuadables').
  3. Long-term Impact: How do you incorporate 90-day LTV into your evaluation? Discuss the tension between short-term conversion gains and long-term retention.
  4. Execution: Describe the power analysis approach for determining the sample size and duration needed, given that purchase behavior is high-variance and seasonal.

Constraints & Considerations

  • Assume a user base of 10 million active monthly users.
  • Consider the risk of 'discount hunting' behavior where users shift their order timing to align with promotional windows.
  • Briefly outline a monitoring plan for the post-experiment rollout to ensure the business model remains sustainable.

Hints

Hint 1

Consider using a Cluster-Randomized Trial (CRT) or a Geo-split instead of simple user-level randomization to mitigate spillover effects between users.

Hint 2

Think about using a uplift modeling approach or a CATE (Conditional Average Treatment Effect) estimator to distinguish between 'sure things', 'persuadables', and 'lost causes'.

Hint 3

Account for the 'novelty effect' by evaluating whether the increase in order frequency persists after the 20% discount period expires.

Follow-up questions

  • How would you handle the interference problem if providing discounts to some users changes the labor supply (Dasher availability) for the entire zone?
  • If the experiment shows high incremental profit for new users but low profit for long-term power users, how would you adjust the targeting strategy?
  • What statistical methods would you use to control for 'selection bias' if the discount was offered via an 'opt-in' coupon rather than a platform-wide nudge?