Katkat
Back

Design an A/B Test for Dashboard Engagement Impact

AmazonData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

Amazon data scientist experimentation prompt on designing an A/B test for an energy-usage dashboard redesign, including hypotheses, metrics, guardrails, power, test duration, seasonality, and analysis methods.

Scenario

Product team is launching a redesigned energy-usage dashboard and needs to measure its impact.

Question

How would you design an A/B test to determine whether the new dashboard increases user engagement? Which metrics and statistical methods would you use, and how would you decide the test duration?

Hints

State hypothesis, randomize, compute sample size, discuss p-values, guardrail metrics.