Keep GPUs Fed with an Efficient Data-Preparation Pipeline
Mistral AISoftware Engineer · Technical Screen
Last updated
Diagnose GPU input starvation across storage, CPU preprocessing, batching, transfer, and synchronization, then improve bounded overlap while preserving data correctness.
A model-training job frequently leaves its GPUs idle while waiting for input batches. Explain how you would diagnose and improve data preparation and loading.
Constraints & Assumptions
The source asks about CPU, I/O, preprocessing, and batch preparation. No framework or storage system is prescribed. Preserve data semantics and reproducibility while improving throughput.
Clarifying Questions
Where are the samples stored? Which transformations run per example? Are batch shapes variable? Is the job distributed? Are GPU gaps caused by input starvation or synchronization elsewhere?
Follow-up questions
- When can adding loader workers make performance worse? What benefits and costs come from caching, prefetching, pinned memory, and sequence bucketing? How would you prevent duplicate samples across workers?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.