Reduce variance with covariate adjustment
MetaData Scientist · Onsite
Last updated
This question evaluates a data scientist's competency in experimental design, variance reduction techniques, and regression-based covariate adjustment (CUPED/ANCOVA) within randomized A/B tests, including understanding of statistical power and diagnostic checks.
A pre‑period covariate X explains 36% of the variance in outcome Y. a) Derive how CUPED/regression adjustment changes Var(Δ) and quantify the expected sample‑size savings. b) List assumptions that invalidate CUPED (e.g., post‑treatment leakage, mis‑timed covariates) and diagnostics you’d run. c) Would you prefer stratified randomization or covariate adjustment here, and why?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.