Katkat

Improve low R² without p‑hacking

InstacartData Scientist · Onsite

Last updated

This question evaluates competence in statistical modeling and causal inference, covering regression diagnostics, feature engineering and interactions, appropriate error distributions and link functions, leakage detection, model selection and validation, and the trade-off between predictive accuracy and valid effect estimation.

A linear regression predicting contribution per order has R²=0.07. a) List concrete steps to raise predictive performance without invalidating inference: feature transformations (e.g., splines for basket size), interactions (treatment×daypart), appropriate error distribution (Gamma/log link), and leakage checks. b) Will simply adding another covariate reliably increase R² out‑of‑sample? Demonstrate with CV why/why not and propose alternatives (GAMs, quantile regression, gradient boosting) while keeping effect‑estimation goals in mind. c) Show how you’d use nested cross‑validation and target‑leakage tests to guard against p‑hacking while iterating. d) Explain when a low R² is acceptable for unbiased ATE but unacceptable for accurate individual predictions.