Katkat
Back

Choose Between Precision and Recall for a Classification Decision

CitadelData Scientist · Technical Screen
EasyData ScientistTechnical ScreenMachine Learning

Last updated

Review precision and recall for imbalanced classification and learn how both change with the decision threshold. Choose an operating point from error costs, capacity, calibration, uncertainty, and segment performance instead of treating either metric as universally best.

A binary classifier will support a real decision, and the model produces a probability score for each observation. Explain precision and recall, how they change as the decision threshold moves, and how you would choose an operating point when false positives and false negatives have different costs.

Constraints & Assumptions

  • The positive class may be uncommon.
  • A downstream action is taken only when the score exceeds a selected threshold.
  • The business costs of mistakes can be discussed qualitatively if exact dollar values are unavailable.

Clarifying Questions to Ask

  • What does the positive class represent, and what action follows a positive prediction?
  • What are the consequences of a false positive and a false negative?
  • Is there a fixed review, outreach, or intervention capacity?
  • Are probability calibration and performance consistency across user groups important?

Follow-up questions

  • Why can accuracy look strong when recall for the positive class is poor?
  • When would a precision requirement be treated as a hard constraint?
  • How would you respond if two groups have different precision at the same threshold?
  • What changes if the score is well ranked but poorly calibrated?