Katkat
Back

Test whether two user populations differ

AmazonApplied Scientist · Onsite
MediumApplied ScientistOnsiteMachine Learning

Last updated

This question evaluates competency in statistical inference and multivariate analysis, covering hypothesis formulation about equal distributions, preprocessing and normalization considerations, choice of statistical tests or modeling approaches, and interpretation of significance and effect size.

Problem

You are given two groups of users:

  • Group A: North America users
  • Group B: Europe users

Each user has a vector of continuous features (e.g., session duration, click-through rate, purchase conversion rate, etc.).

Task

Describe how you would determine whether the two groups differ significantly in their overall multivariate distribution.

Your answer should cover:

  • Hypotheses (what does “same distribution” mean?)
  • Preprocessing/normalization considerations
  • One or more statistical tests or modeling approaches
  • How you would report significance and effect size
  • Pitfalls (multiple testing, confounding, non-IID, heavy tails, missing data)