Explain p-value and choose correct test
CVS HealthData Scientist · Technical ScreenLast updated
This question evaluates a candidate's understanding of statistical inference, hypothesis testing, and robust test selection, focusing on p-value interpretation and choosing between paired vs independent and parametric vs rank-based tests.
Part A — Plain-language p-value: Explain a p-value to a non-scientist without using the phrase 'probability the null is true.' Use a concrete everyday analogy and clarify common misinterpretations (e.g., p≠effect size, p is conditional on the test and assumptions).
Part B — Wilcoxon vs t-test: For each scenario, select and justify the appropriate test, assumptions, and an effect size measure.
- Paired data: 12 patients' systolic BP measured pre/post a low-sodium diet; the distribution of paired differences is skewed with outliers. Choose between a paired t-test and a Wilcoxon signed-rank test. State assumptions, how you'd check them, how ties/zeros are handled, and report an effect size (e.g., Cohen's dz vs rank-biserial or matched-pairs r).
- Independent samples: Compare length of stay between two independent clinics (n=18 and n=25) with unequal variances and non-normal heavy tails. Choose between Welch's t-test and a Wilcoxon rank-sum (Mann–Whitney) test. Discuss what the nonparametric test estimates (probability of superiority) vs the mean-difference focus of Welch's test, when each is preferable, and how you'd complement with confidence intervals and a robust effect size (e.g., Hedges' g with HC3 SEs or Cliff's delta).
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.