Katkat
Back

How would you choose a classification threshold?

ByteDanceData Scientist · Technical Screen
MediumData ScientistTechnical ScreenMachine Learning

Last updated

This question evaluates understanding of threshold selection for probabilistic binary classifiers, testing competencies in validation-set decision threshold choice, cost-sensitive trade-offs between false positives and false negatives, effects of class imbalance, and probability calibration.

You trained a binary classifier that outputs a probability score p(y=1|x). You must choose a decision threshold t to convert probabilities into class labels.

Answer the following:

  • What is your general approach to selecting t on a validation set?
  • How do you incorporate different costs of false positives vs false negatives?
  • How do class imbalance and probability calibration affect threshold choice?
  • Give one example where you would prefer a high threshold and one where you would prefer a low threshold.