Estimate variance for ratio metrics
MetaData Scientist · Onsite
Last updated
This question evaluates statistical inference and experimental-analysis skills for data scientists, including variance estimation for ratio metrics via the delta method, covariance estimation from sample moments, model-based comparisons (ANCOVA/log models with offsets), bootstrap and t-test inference considerations, and robustness techniques for heavy-tailed revenue such as winsorization and robust M-estimators. It is commonly asked to assess both theoretical derivation and practical estimation trade-offs in A/B or experimental settings; the category is Statistics & Math and the level of abstraction spans conceptual understanding and practical application.
Your KPI is ARPU = Revenue/ActiveUsers, computed per bucket with corr(Revenue, ActiveUsers)=0.6. Derive an approximate Var(ARPU) using the delta method, including the covariance term, and state how you would estimate inputs from sample moments. Compare testing ARPU directly vs. modeling log‑Revenue with log‑ActiveUsers as a covariate (or using ANCOVA). When is a t‑test acceptable, when would you prefer a bootstrap, and how would you handle heavy‑tailed revenue (e.g., winsorization or robust M‑estimators)?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.