Katkat

Derive logistic regression objective and gradients

UpstartData Scientist · HR Screen

Last updated

This question evaluates understanding of logistic regression, maximum likelihood estimation, cross-entropy (negative log-likelihood), L2 regularization, and the ability to compute model gradients for binary classification.

Binary logistic regression with dataset {(x_i, y_i)}_{i=1}^m, y_i ∈ {0,1}. Using the sigmoid σ(z)=1/(1+e^{-z}) and linear score z_i = w^T x_i + b: a) Write the exact maximum-likelihood optimization objective (state clearly whether it is a maximization or minimization) both without and with L2 regularization of strength λ≥0. b) Write the explicit negative log-likelihood (cross-entropy) L(w,b). c) Derive the gradients ∂L/∂w and ∂L/∂b. d) Clarify the distinction among 'objective', 'loss', and 'regularized objective' in this context.