Build a Data-Cleaning and Machine-Learning Pipeline Without Leakage
Quantbot TechnologiesData Scientist · Online AssessmentLast updated
Design schema-aware cleaning, appropriate data splits, training-only fitted transformations, reusable inference state, and validation for missing or unseen inputs.
Design a data-cleaning and machine-learning pipeline that can be trained on a dataset and then applied consistently to held-out data and future inputs. Explain the responsibilities of each stage and how you would prevent information leakage.
Constraints
The dataset schema, target, missing-value patterns, and model are unspecified. State any illustrative assumptions and choose cleaning steps based on the actual data contract. Do not automatically drop every incomplete row or fit preprocessing on the full dataset.
Clarifying Questions
- What is one observation, and are rows independent, grouped, or time-ordered?
- Which fields are features, identifiers, labels, or unavailable at prediction time?
- How should missing, invalid, duplicate, and unseen categorical values be handled?
- Which preprocessing statistics must be learned from training data?
Hints
Hint 1
Preserve fit versus transform: A pipeline stage may need to learn a value from training data and then reuse that exact value during evaluation and prediction.
Follow-up questions
- How would a time-based prediction task change the validation split?
- What should happen when an inference row contains a category unseen during training?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.