Derive logistic regression objective and gradients
UpstartData Scientist · HR ScreenLast updated
This question evaluates understanding of logistic regression, maximum likelihood estimation, cross-entropy (negative log-likelihood), L2 regularization, and the ability to compute model gradients for binary classification.
Binary logistic regression with dataset {(x_i, y_i)}_{i=1}^m, y_i ∈ {0,1}. Using the sigmoid σ(z)=1/(1+e^{-z}) and linear score z_i = w^T x_i + b: a) Write the exact maximum-likelihood optimization objective (state clearly whether it is a maximization or minimization) both without and with L2 regularization of strength λ≥0. b) Write the explicit negative log-likelihood (cross-entropy) L(w,b). c) Derive the gradients ∂L/∂w and ∂L/∂b. d) Clarify the distinction among 'objective', 'loss', and 'regularized objective' in this context.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.