Katkat
Back

Compare Random Forests and Boosted Trees: Bias, Variance, Speed

TikTokData Scientist · Technical Screen
MediumData ScientistTechnical ScreenMachine Learning

Last updated

Evaluates practical trade-offs between Random Forests and gradient-boosted trees for tabular ML. Strong answers compare bias, variance, speed, interpretability, overfitting, production fit, and feature scaling needs.

Scenario

Product-facing data-science interview on choosing and configuring tree-based ensemble models. The team wants to understand the trade-offs between Random Forests and Gradient-Boosted Decision Trees and whether any feature scaling is required for tree-based algorithms.

Question

Compare Random Forests with Gradient-Boosted Decision Trees such as XGBoost. Specifically:

  1. Contrast them on bias/variance, interpretability, training and inference speed, and robustness to overfitting, explaining how ensemble construction (bagging vs. sequential boosting) drives each difference.
  2. When would you prefer one over the other in a production setting? Consider accuracy ceiling, tuning effort, latency/throughput, robustness to noise, calibration, and distribution drift.
  3. Do tree-based models require feature standardization or normalization? Explain the theoretical reason and any practical exceptions.
Hints

Focus on ensemble construction, sequential vs. parallel learning, split criteria, overfitting control knobs, and why splits are invariant to monotonic transformations of the features.