Explain Training R-Squared After Adding a Random Feature
C3 AIData Scientist · Online Assessment
Last updated
Prove that ordinary training R-squared cannot decrease when adding a feature, while adjusted and test-set R-squared can.
Explain Training R-Squared After Adding a Random Feature
For an ordinary least-squares linear regression fitted on the same training observations, what happens to training R-squared after adding a completely random feature: can it increase, remain the same, or decrease? State the fitting assumptions and distinguish the result from adjusted R-squared and test-set performance.
Hints
Hint 1
Reuse the old solution: The expanded model can reproduce every old fitted value by assigning zero to the new coefficient.
Follow-up questions
- When does the added feature leave training R-squared exactly unchanged?
- Why can adjusted R-squared decrease even when ordinary training R-squared increases?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.