Katkat

Diagnose Causes and Test Hypotheses for Metric Drop

AmazonData Scientist · Technical Screen

Last updated

Evaluates incident-style analysis of a sudden purchase-conversion drop in a consumer product. Strong answers list plausible causes, validate with concrete cuts and logs, design an A/B test, and define metrics.

Scenario

The product’s key metric suddenly drops. Stakeholders want a structured diagnosis and experiment plan.

Question

List plausible causes for the performance drop, describe analyses you would run to validate each cause, and design an A/B test to confirm the main hypothesis. Which primary and guardrail metrics would you track and why?

Hints

Think segmentation, funnel breakouts, external factors, and metric hierarchy (north-star vs. health).