Katkat
Back

Master A/B Testing: Key Concepts and Methodologies Explained

PayPalData Scientist · Onsite
MediumData ScientistOnsiteAnalytics & Experimentation

Last updated

Evaluates practical A/B testing and causal inference fundamentals for online products. Strong answers cover p-values, errors, power, workflow, Simpson's paradox, metrics, and observational causal methods.

Scenario

Data scientist is interviewed on A/B-testing know-how for an online product.

Question

Explain what a p-value represents; define Type I and Type II errors; outline the end-to-end experimentation workflow; describe Simpson's paradox and how to detect it; propose primary/secondary metrics; name two causal-inference methods useful when randomization is impossible and when you would apply them.

Hints

Cover hypothesis, sample-size, segmentation, lift vs variance, DAGs or matching, and practical examples.