Design and power an incentive experiment
UberData Scientist · Onsite
Last updated
This question evaluates experimental design, causal inference, A/B testing and statistical power calculation skills, including defining eligibility, unit of randomization, treatment arms, stratification, metric selection with guardrails, handling noncompliance and interference, and interpreting heterogeneous treatment effects.
Offer timing and efficacy of benefits: Design an experiment to test whether a bonus (e.g., cash or queue priority) increases the share of paperwork-complete candidates who complete their first action within 14 days. Specify eligibility/exclusions; unit of randomization; treatment arms (control, bonus-now, bonus-after-first-action, staged bonus); stratification to separate bonus-dependent vs. intrinsically motivated users; primary/secondary metrics and guardrails (quality, fraud, downstream retention); handling noncompliance, spillovers, and interference; stopping rules and decision criteria. Compute the minimum per-arm sample size to detect a +3 percentage-point lift from a 20% baseline at 90% power, α=0.05 using a two-proportion z-test—show formula and a numeric result. Explain when to roll out benefits broadly vs. target narrowly based on heterogeneous treatment effects.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.