Katkat
Back

Reading a Regression Slope: P-values, Intervals, and Causality

GoogleData Scientist · Onsite
MediumData ScientistOnsiteStatistics & Math

Last updated

Use a fitted slope from a one-predictor regression to explain precisely what a p-value and a confidence interval mean, what assumptions they rest on, how confidence and prediction intervals differ, and why significance is not causation. Tests statistical precision under interview pressure.

A product analyst at a meal-kit startup regresses weekly new subscribers (y) on weekly podcast-ad spend in $100s (x) across 27 weeks and reports:

y = 41.2 + 0.84 x        se(slope) = 0.30      residual sd = 9.5
"p = 0.0097, 95% CI for slope: [0.22, 1.46]"

The marketing lead asks you to explain what this means before they triple the podcast budget.

Part 1: The model and the test

Write down the simple linear regression model, its parameters, and the null and alternative hypotheses behind the reported p-value. Show how the test statistic is built from the slope and its standard error, and which reference distribution is used (with its degrees of freedom).

Part 2: Interpreting the numbers

  • State in one careful sentence what p = 0.0097 means, and give two common wrong interpretations.
  • State what the 95% interval means under repeated sampling, and what it does not say.
  • Explain how the two-sided test at α = 0.05 relates to whether the interval contains 0.

Part 3: Assumptions

List the assumptions your inference relies on. For each, say what goes wrong if it fails and how you would check it on this weekly data.

Part 4: Mean vs new outcome

The lead asks: "If we spend $2,000 next week, how many subscribers will we get?" Contrast a confidence interval for the mean response at x = 20 with a prediction interval for next week's count, and explain which is wider and why.

Part 5: Causality and practical significance

Explain why neither the small p-value nor the positive slope shows that more podcast spend causes more subscribers. Name concrete threats in this setting and propose how to get a causal estimate. Also explain how a tiny slope can still have a tiny p-value.

Hints

Hint 1

The test statistic is (estimate - hypothesized value) / standard error. With two estimated coefficients, the degrees of freedom are n - 2.

Hint 2

A p-value is a probability computed assuming the null hypothesis and the model are true. It is not the probability that the null is true.

Hint 3

A prediction interval adds the noise of a single new observation on top of uncertainty in the fitted line.

Hint 4

Ask what else changes week to week together with ad spend.

Follow-up questions

  • How would adding a second predictor (e.g., email spend) change the interpretation of the podcast slope?
  • If the residuals are strongly autocorrelated week to week, how would you adjust the inference?
  • How would you explain the same result in Bayesian terms, and what would you need to add?
  • If you ran 20 channel regressions and reported only this one, what is the problem and how do you correct for it?