Impute missing values without leakage
Capital OneData Scientist · HR ScreenLast updated
This question evaluates proficiency in data imputation, strict train/validation leakage prevention, temporal per-user propagation rules, and categorical aggregation for model-ready features.
Given a DataFrame df with columns: user_id, event_date (datetime), country (categorical), device_type (categorical), age (numeric), income (numeric), last_purchase_days_ago (numeric), session_length (numeric), is_active_30d (binary label). Implement code to impute missing values for model training with strict no‑leakage. Requirements: 1) Split into train/validation indices; all statistics/models for imputation must be fit on train only and then applied to validation. 2) Numeric: age → median within country (train‑only medians); income → train a ridge regression imputer on train rows using predictors [age_imputed, country, device_type, last_purchase_days_ago, session_length] (one‑hot encoded), then predict income for both train/validation; do not use the label. 3) Time‑ordered within user: for last_purchase_days_ago and session_length, sort by event_date per user_id and forward‑fill gaps up to 14 days; if the gap between consecutive event_date exceeds 14 days, do not propagate; after sequence fills, fill remaining NaNs with the global train median for that feature. 4) Categoricals: device_type and country → per‑user mode; break ties with the global train mode. 5) Deliver: functions fit_imputers(df, train_idx) and transform_impute(df, imputers, idx) where imputers holds all train‑fit objects/statistics; include assertions that no value derived from validation data was used to compute train statistics.
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.