Katkat
Back

Diagnose overfitting from error curves

Zest AIData Scientist · Onsite
HardData ScientistOnsiteMachine Learning

Last updated

This question evaluates the ability to recognize and diagnose model generalization problems—specifically overfitting—by interpreting training, validation, and test error curves and understanding validation procedures and sources of variance.

You are evaluating a supervised machine learning model. You are shown a plot where the x-axis is training epoch or model complexity and the y-axis is prediction error. Training error keeps decreasing, while validation error and test error stop improving and begin to increase, creating a widening gap between training and out-of-sample performance.

Answer the following:

  1. What is the most likely issue shown by this plot?
  2. What are common root causes?
  3. How would you confirm the diagnosis without misusing the test set?
  4. What practical steps would you take to fix or reduce the issue?