Katkat
Back

Compare Logistic Regression and Random Forest in Python

Capital OneData Scientist · Technical Screen
MediumData ScientistTechnical ScreenMachine Learning

Last updated

Design, evaluate, and contrast a Logistic Regression baseline and a Random Forest model on an imbalanced credit default prediction dataset in Python. Walk through pipeline design, validation strategy, probability calibration, decision thresholding, and production trade-offs.

Scenario

You are tasked with building a binary classification model for a credit card portfolio to predict whether an account will default within the next 90 days (y = 1) or remain in good standing (y = 0). The historical dataset contains 500,000 observations with an event rate of approximately 2% (severe class imbalance).

The features include:

  • Continuous metrics: debt-to-income ratio, revolving credit utilization, age, and payment-to-minimum-due ratio (some values missing, skewed distributions, outliers).
  • High-cardinality categorical: merchant category code and state of residence.
  • Ordinal categorical: credit tier rating (A through G).

Tasks

1. Preprocessing and Pipeline Architecture

Describe the required preprocessing steps for both Logistic Regression and Random Forest. Explain how you would structure these transformations inside a unified scikit-learn pipeline to prevent data leakage during cross-validation.

2. Feature Interactions and Non-Linearity

  • How does Logistic Regression handle non-linear boundaries and feature interactions, and what manual engineering steps (e.g., splines, interaction terms, weight of evidence) are required?
  • How does Random Forest inherently capture high-order interactions and non-linearities without explicit manual feature creation?

3. Validation Strategy and Class Imbalance

  • Specify an appropriate cross-validation scheme given the temporal nature of financial credit records and class imbalance.
  • Compare handling class imbalance in both models (e.g., class_weight='balanced', downsampling the majority class, SMOTE) and explain the impact on predicted probabilities.

4. Calibration and Threshold Tuning

  • Why are raw scores from an uncalibrated Random Forest often uncalibrated (pushing probabilities away from 0 and 1), whereas Logistic Regression minimizes log-loss directly?
  • If the business loss for a false negative is $5,000 (charged-off balance) and the cost of a false positive is $150 (lost transaction fees and customer friction), how do you determine the optimal decision threshold instead of relying on the default 0.50 cutoff?

5. Production and Regulatory Considerations

Contrast both models along the following operational dimensions:

  • Inference latency (p99 SLA < 15ms).
  • Adverse Action notices (FCRA/ECOA regulatory requirements for model explainability).
  • Retraining cadence and drift monitoring.

Hints

Hint 1

To prevent leakage, fit imputation and scaling transformers exclusively on the training split of each cross-validation fold.

Hint 2

Cost-sensitive thresholding can be formulated by minimizing the expected cost function `C = C_FN * P(y=1) * (1 - Recall) + C_FP * P(y=0) * FPR` over a grid of candidate cutoffs.

Hint 3

Tree ensembles compress probabilities toward the center due to variance reduction across trees, meaning post-hoc calibration like isotonic regression or Platt scaling is often necessary before applying business thresholds.

Follow-up questions

  • How does the behavior of L1 (Lasso) vs. L2 (Ridge) regularization in Logistic Regression change when two features are highly collinear compared to how a Random Forest splits on collinear features?
  • If you must use Random Forest under strict fair lending guidelines, how would you generate legally compliant reason codes equivalent to Logistic Regression model coefficients?
  • Suppose the default rate increases from 2% to 6% during an economic downturn without changes in underlying borrower characteristics. How would you adjust the models' probability outputs without full retraining?