Katkat
Back

Compute an A/B test p-value by hand

AmazonData Scientist · Technical Screen
MediumData ScientistTechnical ScreenStatistics & Math

Last updated

This question evaluates understanding of hypothesis testing for proportions, specifically the two-proportion z-test, p-value computation and interpretation of statistical significance in an A/B test with binary outcomes.

In an A/B test on a game feature, you measure conversion rate (binary outcome).

  • Control: n₁ = 1000 users, x₁ = 120 conversions
  • Treatment: n₂ = 980 users, x₂ = 145 conversions

Tasks:

  1. Compute the two-sided p-value for the null hypothesis H₀: p₁ = p₂ using a two-proportion z-test (show formulas and intermediate values).
  2. State whether the result is significant at α = 0.05.
  3. (Implementation) Describe how you would compute this p-value in Python without using SciPy (you may use math only).