Katkat
Back

Reason About an Intercept Shift in Logistic Regression

LyftMachine Learning Engineer · Technical Screen
MediumMachine Learning EngineerTechnical ScreenMachine Learning

Last updated

Tests understanding of how an additive shift to the intercept of a logistic regression model impacts predicted probabilities, ranking metrics (ROC-AUC), thresholded classification metrics (Recall, FPR, Precision), and calibration.

Problem Statement

At Lyft, you maintain a production binary logistic regression model that predicts whether a passenger will cancel a matched ride (Y = 1 for cancellation, Y = 0 otherwise). The model computes log-odds using feature weights w and intercept b:

z(x) = w^T x + b

The predicted probability is given by the standard logistic sigmoid:

p(x) = 1 / (1 + exp(-z(x)))

By default, a ride is flagged for intervention if p(x) >= 0.5 (which corresponds to z(x) >= 0).

During a deployment update, an engineer accidentally updates the model configuration by adding a strictly positive constant delta > 0 to the intercept, so the new log-odds are z'(x) = w^T x + b + delta. The feature weights w and the labeled evaluation dataset remain completely unchanged.

Address the following theoretical and practical questions regarding the impact of this change on the evaluation dataset:

1. Ranking and Discrimination

How does this intercept shift affect the relative ranking of instances by predicted score? Specifically, what happens to the Area Under the ROC Curve (ROC-AUC)?

2. Classification Metrics at Default Threshold

Assume the downstream system continues using the fixed probability threshold t = 0.5 (classifying as positive when p'(x) >= 0.5):

  • Does True Positive Rate (Recall) increase, decrease, or remain unchanged? Explain why.
  • Does False Positive Rate (FPR) increase, decrease, or remain unchanged? Explain why.
  • What happens to Precision? Can precision increase, or does it always decrease? Justify your answer.

3. Threshold Compensation

Find the exact probability threshold t' such that applying p'(x) >= t' yields the exact same binary classification decisions as the original model under p(x) >= 0.5.

4. Calibration and Log-Loss

Assuming the original model was trained via Maximum Likelihood Estimation (MLE) on an uncorrupted dataset drawn from the same distribution as the evaluation set, what happens to the model's calibration and binary cross-entropy (log-loss)?

Hints

Hint 1

Because the shift delta is added uniformly to all logits, consider whether the relative ordering of any two data points can ever change.

Hint 2

Express the decision rule p'(x) >= 0.5 in terms of the original logit z(x) and delta to see how the set of predicted positives expands.

Hint 3

To find the compensatory threshold t', equate the boundary condition z'(x) >= logit(t') with the original boundary condition z(x) >= 0.

Follow-up questions

  • In what real-world scenarios is an intentional intercept correction (offset) applied without retraining the entire model?
  • If the dataset suffered from label shift (e.g., rider cancellations doubled across all segments due to bad weather), how would you analytically compute the optimal delta?
  • How would your conclusions differ if the base model were a Gradient Boosted Decision Tree (GBDT) instead of a linear logistic regression model?