Katkat
Back

Design and Analyze A/B Test for Cashback Program

PayPalData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Design and Analyze A/B Test for Cashback Program states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Scenario

A/B testing and product analytics discussions

Question

Walk me through how you would design, run, and analyze an A/B test for a new checkout feature. Case study: PayPal plans to launch a cashback program. How would you evaluate its success, what metrics would you track, and what data would you need?

Hints

State hypothesis, choose primary & guardrail metrics, randomization, sample-size, test duration, segmentation, analysis plan.