Katkat
Back

Design A/B Test for Marketing Campaign Impact Evaluation

Capital OneData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design A/B Test for Marketing Campaign Impact Evaluation states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Scenario

Analyst case study – evaluate the business impact of a marketing campaign waiving first-year annual fees.

Question

Design an A/B test to measure incremental card acquisitions and spend. Specify primary metrics, guardrail metrics, sample-size calculation, and post-experiment analysis steps.

Hints

Conversion rate, revenue lift, CUPED or stratification to increase power.