Katkat
Back

Explain Linear Regression Assumptions

DatabricksData Scientist · Technical Screen
HardData ScientistTechnical ScreenStatistics & Math

Last updated

This question tests the candidate's mastery of the Gauss-Markov theorem and the consequences of violating Ordinary Least Squares (OLS) assumptions in real-world data science workflows.

The Problem

Ordinary Least Squares (OLS) linear regression is a cornerstone of statistical modeling, yet its validity relies on several core assumptions. You are building a predictive model for a client and suspect that your model's performance might be compromised.

Task

  1. Define the core assumptions of the Classical Linear Regression Model (CLRM) regarding the error term and the relationship between variables.
  2. Explain the consequences of violating the assumption of homoskedasticity. How does this impact coefficient estimates versus their standard errors?
  3. Describe the 'Endogeneity' problem. If a model is intended for causal inference rather than pure prediction, why is endogeneity a fatal flaw, and what diagnostic tool would you use to detect it?
  4. Suppose you detect significant multicollinearity in your feature set. How does this influence your ability to interpret individual regression coefficients compared to the model's overall predictive power (R-squared)?

Considerations

  • Focus on the distinction between BLUE (Best Linear Unbiased Estimator) properties and the validity of inference (p-values/confidence intervals).
  • Contrast scenarios where high predictive accuracy masks poor statistical validity (e.g., in the presence of omitted variable bias).

Hints

Hint 1

Recall the acronym LINE (Linearity, Independence, Normality, Equal Variance) and consider which are necessary for unbiasedness versus which are required for hypothesis testing.

Hint 2

Consider the impact of the Gauss-Markov theorem on the variance of your estimators when homoskedasticity is absent.

Hint 3

Think about the difference between correlated independent variables and variables correlated with the error term (unobserved confounding).

Follow-up questions

  • How do you distinguish between influential outliers and high-leverage points using Cook's Distance?
  • If you find heteroskedasticity, would you prefer using Weighted Least Squares (WLS) or robust standard errors, and why?
  • Can you explain how a high VIF (Variance Inflation Factor) affects the stability of your model coefficients during cross-validation?