Investigate Sudden Metric Changes and Design A/B Test
LyftData Scientist · Technical Screen
Last updated
Investigate Sudden Metric Changes and Design A/B Test evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
Scenario
A key business metric suddenly spikes or drops; leadership asks for a root-cause analysis and later wants to run an A/B test on a proposed fix.
Question
How would you systematically investigate the cause of a sudden change in a core metric? 2. Outline the essential steps, assumptions, and success criteria for designing and analyzing an A/B experiment to validate a potential solution.
Hints
Think root-cause frameworks (segmentation, funnel, time), define control/treatment, randomization, power, sample size, success metric.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.