Katkat
Back

Design a model for subscription adoption prediction

StripeData Scientist · Online Assessment
HardData ScientistOnline AssessmentMachine Learning

Last updated

This question evaluates a candidate's ability to design and productionize a supervised machine-learning pipeline for predicting subscription adoption, testing competencies in labeling strategy, leakage identification and prevention, feature engineering from transactional and merchant metadata, model selection and calibration, evaluation with time-based validation, and post-deployment monitoring. It is commonly asked in the Machine Learning domain for Data Scientist roles because it assesses practical application and production-ready design along with conceptual understanding of time-based splits, class imbalance, performance metrics, and drift detection.

You must predict which non-Subscription merchants will adopt Subscription within the next 60 days. Using only the data available up to 2025-07-03 to predict adoption by 2025-09-01, design a production-ready classification approach. Answer concisely: (a) Labeling: precisely define positives/negatives and the observation and outcome windows; handle merchants already using Subscription and cold-start merchants. (b) Leakage: list at least five concrete leakage risks specific to this data (e.g., using product_type after the label cutoff, refund rates influenced by post-adoption behavior) and how you’d prevent them via time-based feature windows and proper splits. (c) Features: propose 15–25 high-signal, computable features from transactions (recency/frequency/monetary, 28–35 day repeat patterns, customer concentration, card_fingerprint diversity, weekend share, chargeback/refund rates, growth rates) and from merchant metadata (age, vertical, geo). (d) Modeling: choose two models (e.g., regularized logistic vs. gradient-boosted trees); discuss class imbalance handling (weights vs. downsampling), calibration, and interpretability for a sales handoff. (e) Evaluation: specify time-based cross-validation, primary metrics (PR-AUC, precision@K, recall@K), and how you would select a threshold to deliver a list of 1,000 merchants with expected precision ≥ 0.60. (f) Monitoring: define post-deployment drift and performance checks (data drift on feature distributions, label drift, calibration drift) and how you’d retrain without contaminating future labels.