Katkat
Back

Infer distribution and choose robust statistics

GoogleData Scientist · Technical Screen
MediumData ScientistTechnical ScreenStatistics & Math

Last updated

This problem tests the ability to diagnose non-normal data distributions (specifically zero-inflated, heavy-tailed data) and select appropriate statistical metrics for hypothesis testing.

Scenario

You are analyzing a dataset of 10,000 session revenue observations from an e-commerce platform. The summary statistics are provided as follows:

  • Count: 10,000
  • Mean: $12.50
  • Standard Deviation: $45.00
  • 50th percentile (Median): $0.00
  • 75th percentile: $0.00
  • 90th percentile: $15.00
  • 99th percentile: $250.00

Task

  1. Describe the underlying distribution. Given the high percentage of zeros and the large gap between the mean and the median, what type of statistical distribution does this represent?
  2. Why would a standard t-test for comparing the mean revenue between two variants be inappropriate for this dataset?
  3. You are asked to report a 'typical' user value. Why is the mean potentially misleading here, and what alternative metric would you provide to stakeholders?
  4. Propose a transformation or a robust statistical test that could effectively compare two groups of this revenue data in an A/B test setting.

Constraints

  • Assume the revenue values are non-negative.
  • Consider the impact of outliers on the variance.
  • Ensure your chosen test accounts for the high frequency of zero-value observations.

Hints

Hint 1

Consider that 65% of your observations are zero; a distribution that combines a point mass at zero with a continuous distribution for positive values is called a 'Zero-Inflated' model.

Hint 2

The t-test relies on the assumption of normality of the sample mean, which fails here due to extreme positive skewness and the spike at zero.

Hint 3

Since the mean is heavily influenced by a few high-value customers in the 99th percentile, consider using a non-parametric test like the Mann-Whitney U test or a hurdle model approach.

Follow-up questions

  • How would you calculate a confidence interval for the median in this dataset since the median is $0?
  • If you were to model this using a 'Hurdle Model,' what two sub-models would you use to represent the zero-count and the positive revenue values?
  • How would you communicate the presence of these 'whales' (the top 1% of spenders) to a product manager who wants to prioritize feature development based on the average user?