Katkat
Back

Design experiments under network interference

MetaData Scientist · Onsite
HardData ScientistOnsiteAnalytics & Experimentation

Last updated

This question evaluates a data scientist's competence in experimental design and causal inference under network interference, assessing understanding of clustering strategies, exposure and exclusion definitions, contamination control, design effect and sample-size implications, estimation of intra-cluster correlation, and mixed-effects analysis for evolving clusters. Commonly asked in Analytics & Experimentation interviews to probe the ability to preserve internal validity while managing operational feasibility in two-sided marketplaces, it tests both conceptual understanding of interference and practical application skills in design and analysis planning.

Design an A/B test for a search‑ranking change in a 2‑sided marketplace with cross‑seller spillovers and consumer social contagion. Choose between cluster randomization (e.g., by seller or geography) and ego‑cluster randomization; precisely define exposure and exclusion criteria. Given mean cluster size m=50 and intra‑cluster correlation ρ=0.02, compute the design effect and the required sample‑size inflation versus individual randomization; explain how you would estimate ρ pre‑experiment from historical data. How will you prevent treatment contamination during reassignment and validate success using invariant metrics and SSRM under clustering? If clusters evolve over time, propose a rehashing schedule and an analysis plan using mixed‑effects models with cluster random intercepts.