Katkat
Back

Estimate Redesign Impact Using Propensity Score Matching

LinkedInData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

Estimate Redesign Impact Using Propensity Score Matching evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Scenario

A mobile app is redesigned, but the new version is only adopted by users who choose to upgrade; the team needs to measure performance impact.

Question

Without a forced A/B test, how would you estimate the causal impact of the redesign? How would you define comparable treatment and control groups of ‘new-version’ and ‘old-version’ users? Which user features beyond engagement/behavior would you include to ensure similarity? What statistical or causal-inference methods would you apply, and how would you validate assumptions?

Hints

Discuss propensity scores, matching, weighting, diff-in-diff, covariate balance checks, sensitivity analyses.