Katkat
Back

Estimate Causal Impact Using Synthetic Control Methods

AirbnbData Scientist · Technical Screen
HardData ScientistTechnical ScreenAnalytics & Experimentation

Last updated

Evaluates causal impact estimation after a full product rollout with no holdout. Strong answers use interrupted time series, synthetic control, Bayesian structural time series, exposure variation, and placebo tests to build and validate a credible counterfactual.

Scenario

A feature has already been launched to 100% of traffic; no control or holdout group exists.

Question

How would you estimate the causal impact of the launch? Outline possible methodologies, required data, assumptions, and how you would communicate uncertainty.

Hints

Discuss pre-post analysis, synthetic controls, difference-in-differences, or propensity scoring; emphasize validation and sensitivity checks.