Diagnose Causes and Test Hypotheses for Metric Drop
AmazonData Scientist · Technical ScreenLast updated
Evaluates incident-style analysis of a sudden purchase-conversion drop in a consumer product. Strong answers list plausible causes, validate with concrete cuts and logs, design an A/B test, and define metrics.
Scenario
The product’s key metric suddenly drops. Stakeholders want a structured diagnosis and experiment plan.
Question
List plausible causes for the performance drop, describe analyses you would run to validate each cause, and design an A/B test to confirm the main hypothesis. Which primary and guardrail metrics would you track and why?
Hints
Think segmentation, funnel breakouts, external factors, and metric hierarchy (north-star vs. health).
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.