Katkat
Back

How would you use propensity score matching here

GoogleData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

Evaluates proficiency with propensity score matching and broader causal inference concepts for observational treatment effect estimation, including identification assumptions, propensity score estimation and covariate selection, matching algorithms and common support, balance diagnostics, choice of estimand (ATE vs ATT) and uncertainty quantification, plus sensitivity checks and alternative methods. Common in Analytics & Experimentation interviews for Data Scientist roles because it probes understanding of confounding and identification in nonrandomized settings and requires intermediate-to-advanced applied statistics reasoning about methodological trade-offs.

You want to estimate the causal effect of a new recommender feature on 7-day retention.

The feature was not randomized: users “opt in” after seeing a prompt, and more engaged users are more likely to opt in. You have observational data with columns:

  • user_id
  • opt_in (0/1)
  • pre-period covariates (e.g., prior watch time, sessions, country, device)
  • outcome: retained_7d (0/1)

Explain how you would use propensity score matching (PSM) to estimate the treatment effect. Address:

  1. Identification assumptions (and what can break them).
  2. How to estimate propensity scores and choose covariates.
  3. Matching method and common support.
  4. Balance diagnostics.
  5. How to estimate ATE vs ATT and compute uncertainty.
  6. Sensitivity analyses and alternatives (e.g., weighting, doubly robust, IV) if PSM is weak.