Katkat
Back

Answer ML fundamentals and diagnostics questions

TikTokMachine Learning Engineer · Online Assessment
HardMachine Learning EngineerOnline AssessmentMachine Learning

Last updated

This problem tests your theoretical understanding of model diagnostics, ensemble learning mechanics, and the trade-offs between optimization strategies in machine learning.

Part 1: Diagnostic Metrics

Given a binary classifier that predicts if a video is 'high-quality' or 'low-quality', explain the trade-offs in selecting a threshold when the cost of a False Positive (tagging a low-quality video as high-quality) is 10 times greater than the cost of a False Negative. How would you adjust the decision threshold relative to the standard 0.5 probability cutoff, and how does this affect the Precision-Recall curve compared to the ROC curve?

Part 2: Ensemble Learning

You are tasked with selecting between a Random Forest and a Gradient Boosted Decision Tree (GBDT) for a real-time recommendation system.

  1. Contrast the impact of increasing the 'number of trees' on training-loss behavior for both models. What is the risk of overfitting in each case as the number of estimators grows?
  2. Explain the difference between Gini Impurity and Entropy when evaluating splits in a decision tree. Under what data distribution characteristics would one be preferred over the other?

Part 3: Regularization

Describe how L1 (Lasso) and L2 (Ridge) regularization affect the weight vectors of a logistic regression model. Mathematically, why does L1 regularization encourage sparsity in the weight vector while L2 does not?

Hints

Hint 1

For Part 1, consider that increasing the threshold for a positive prediction decreases the False Positive Rate but increases the False Negative Rate.

Hint 2

For Part 2, remember that Random Forests train trees independently in parallel to reduce variance, while GBDTs train sequentially to reduce bias.

Hint 3

For Part 3, visualize the constraint regions (a diamond shape for L1 vs. a circular shape for L2) and where the loss function contours typically touch these constraints.

Follow-up questions

  • How would you handle a highly imbalanced dataset where the positive class represents less than 0.1% of the total samples?
  • If your GBDT model exhibits high variance, what specific hyperparameters would you tune to regularize it?
  • Explain the 'Bias-Variance Tradeoff' in the context of increasing tree depth in a Random Forest.