Katkat
Back

ML Concepts: Overfitting, Bias-Variance Trade-off, Class Imbalance, Missing Data

YahooMachine Learning Engineer · Technical Screen
EasyMachine Learning EngineerTechnical ScreenMachine Learning

Last updated

A machine learning fundamentals round covering overfitting and underfitting, the bias-variance trade-off, class imbalance, and missing data. It tests whether a candidate can define each concept precisely, diagnose it from training and validation evidence, and choose remedies while weighing their side effects.

A machine learning knowledge round consisted of standard concept questions: overfitting and underfitting, the bias-variance trade-off, class imbalance, and how to handle missing data. Answer each part as you would aloud in the interview: define the concept precisely, explain how you would detect the problem in practice, and give concrete remedies together with their side effects.

Clarifying Questions

  • Should the answers assume a particular model family, such as linear models, tree ensembles or neural networks, or cover several?
  • Is the setting classification, regression, or both?

Part 1 — Overfitting, underfitting and the bias-variance trade-off

Define overfitting and underfitting. How do you tell which of the two a model suffers from, and what do you do about each? Then explain the bias-variance trade-off and how it connects to the two failure modes.

What This Part Should Cover

  • Definitions of both failure modes in terms of training and validation error
  • Diagnosis with learning curves or validation curves, using a held-out set or cross-validation
  • Remedies for each failure mode, with what each remedy costs
  • The bias-variance decomposition, and how capacity, regularization and data volume move each term

Part 2 — Class imbalance

One class makes up only a small fraction of the labeled data. What goes wrong if you train and evaluate the usual way, and how do you handle it?

Clarifying Questions for this Part

  • How rare is the minority class, and how many minority examples exist in absolute terms?
  • Are false positives and false negatives equally costly, and is the output used as a ranking, a yes/no decision or a probability?

What This Part Should Cover

  • Why accuracy misleads, and which evaluation metrics stay informative
  • Resampling and class weighting, with their effects on calibration and overfitting
  • Choosing the decision threshold from the cost of each error type
  • Splitting and validating so that evaluation reflects the true class distribution

Part 3 — Missing data

Some features have missing values. How do you handle them?

Clarifying Questions for this Part

  • Are values missing in the same way at prediction time as in the training data?

What This Part Should Cover

  • Missingness mechanisms and why they change the right treatment
  • The main options: dropping rows or features, simple and model-based imputation, missing-value indicators, and models that handle missing values natively
  • Avoiding leakage, and keeping training and serving consistent
  • How to check whether the chosen treatment helped

Hints

Hint 1

Compare two curves: Put training error and validation error side by side, and watch how each one moves as model capacity or the amount of training data grows.

Hint 2

Split the error into parts: Think of the expected test error of a regression model as a sum of separate terms, and ask which term each remedy you named actually reduces.

Hint 3

Start from the metric: Ask what a model that always predicts the majority class would score on accuracy, and which metrics would expose it.

Hint 4

Data, loss or threshold: Remedies can act on the training data, on the loss function, or on the decision threshold after training. Consider what each one does to the predicted probabilities.

Hint 5

Why is it missing?: Before choosing a fix, ask whether the chance that a value is missing depends on the value itself, on other observed features, or on nothing at all.

Hint 6

Where the fill-in values come from: Whatever statistic or model you use to fill the gaps, think about which data it may be computed from, and what happens to it at prediction time.

Follow-up questions

  • You increase regularization strength step by step, and validation error first falls and then rises. Explain both phases in bias-variance terms.
  • After oversampling the minority class, the model's predicted probabilities for that class are too high. Why, and how do you correct them?
  • A feature is missing far more often for one class than for the other. Is that a problem or a signal, and how do you treat it?
  • A very large neural network fits its training data perfectly yet still generalizes well. How does that square with the classical trade-off?