Katkat
Back

Build and evaluate bad-link classifier

GoogleData Scientist · Technical Screen
MediumData ScientistTechnical ScreenMachine Learning

Last updated

This question evaluates proficiency in applied machine learning classification, including feature design, training a logistic regression, handling severe class imbalance, selecting evaluation metrics and calibration, choosing thresholds under asymmetric costs, and planning offline-to-online validation and monitoring.

You have 1,000 URLs labeled as bad or good and a much larger unlabeled pool, with bad links rare. Design features and train a logistic regression. Explain your evaluation plan under class imbalance: stratified K-folds, ROC-AUC vs PR-AUC, calibration (reliability curves), and why accuracy is misleading. Choose a decision threshold by minimizing expected misclassification cost given asymmetric costs. Discuss class weighting or resampling, leakage checks, monitoring for dataset shift between labeled and production traffic, and an offline-to-online validation plan with shadow or canary deployment.