Detect and address Simpson’s paradox
MetaData Scientist · Onsite
Last updated
This question evaluates understanding of aggregation bias and Simpson's paradox, proficiency with stratified or mixed-model estimators, heterogeneity testing, and experiment decisioning within Analytics & Experimentation for a Data Scientist role.
An experiment shows +1.2pp overall conversion uplift, yet the segment‑level effects are −0.5pp (New) and +2.0pp (Returning). a) Construct a concrete example showing how a shift in segment mix can create Simpson’s paradox. b) Pre‑register a stratified estimator (or MMRM) that protects against aggregation bias. c) Specify an interaction test for heterogeneity and a decision rule when heterogeneity is material (e.g., segment‑specific rollout vs. global ship).
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.