Explain Linear Regression Assumptions, Evaluation, and Feature Addition

Explain the problem solved by linear regression, how the model is fit, which assumptions matter, and how you would evaluate its quality. Then explain what happens to R-squared when new features are added.

Part 1 — Fit and Interpret the Model

State a linear-regression objective and describe how to estimate its coefficients.

What This Part Should Cover

• The response, predictor matrix, intercept, and least-squares objective.
• A numerically appropriate fitting method and the role of rank or collinearity.
• The difference between assumptions for a useful conditional-mean model, coefficient inference, and prediction.

Part 2 — Evaluate Regression Quality

Choose evaluation metrics and diagnostics and explain what they reveal.

What This Part Should Cover

• Error measures and R-squared on an appropriate validation split.
• Residual patterns, outliers, and limitations of relying on one metric.
• The difference between fitting the observed data and generalizing to new observations.

Part 3 — Add Predictors

Does adding features make R-squared higher, lower, unchanged, or does it depend?

What This Part Should Cover

• The exact fitting and evaluation conditions under which training R-squared cannot decrease.
• Why the enlarged model includes the old model as a special case.
• The different behavior of adjusted R-squared and out-of-sample R-squared.

Clarifying Questions

• Are we discussing unregularized least squares with an intercept on the same observations?
• Is the metric measured on training data or held-out data?
• Is the goal prediction, interpretation of coefficients, or causal interpretation?