Evaluate impact without randomized experiments
UberData Scientist · Technical ScreenLast updated
This question evaluates a data scientist's competence in causal inference and identification strategies for estimating promotion effects from observational longitudinal data, emphasizing challenges such as staggered rollout, interference (spillovers), seasonality, and uncertainty quantification.
If you cannot run an experiment, propose and justify at least three identification strategies to estimate the promotion’s causal effect, including assumptions, diagnostics, and failure modes: (1) Difference‑in‑Differences/event‑study with staggered rollout using modern estimators (avoid naive TWFE bias), parallel‑trends checks, and placebo tests; (2) Synthetic Control at the geo level with pre‑period fit metrics and sensitivity to donor pool/regularization; (3) Instrumental Variables exploiting exogenous variation (e.g., eligibility rules, random delivery outages), with relevance/exclusion checks and weak‑IV diagnostics; (4) Regression Discontinuity if eligibility thresholds exist (e.g., tenure ≥ 30 days), with bandwidth/functional‑form sensitivity and McCrary density tests; (5) Propensity‑score or doubly‑robust learners (IPW/AIPW, causal forests) with overlap checks and covariate balance. Explain how you would report ATE/ATT with uncertainty, account for interference/network effects, and control for seasonality and concurrent campaigns.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.