Choose alternatives when randomization fails
MetaData Scientist · Onsite
Last updated
This question evaluates proficiency in causal inference and experimentation analytics for a Data Scientist role, focusing on selecting appropriate identification strategies, designing placebo and pre‑trend tests, and specifying robustness and sensitivity analyses for non‑randomized feature adoption.
A feature autoloads for opted‑in power users only, preventing clean randomization. a) Choose among Difference‑in‑Differences, Synthetic Control, Instrumental Variables, or Propensity Score Matching; justify the choice and its identifying assumptions. b) Design placebo and pre‑trend tests and specify robustness checks (e.g., varying windows, alternative controls). c) Propose sensitivity analyses (negative controls, Rosenbaum bounds). d) Communicate the directional bias risks compared to a randomized A/B.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.