Design device telemetry pipeline for real-time and batch
MicrosoftSoftware Engineer · Online Assessment
Last updated
This question evaluates a candidate's competency in distributed systems architecture, data engineering, and operational trade-offs involved in designing telemetry pipelines that support both near-real-time and batch analytics.
Design a distributed system that ingests telemetry from millions of devices and supports both:
- Real-time analytics (near-real-time dashboards/alerts)
- Batch analytics (historical queries, aggregates, offline jobs)
Cover:
- Data ingestion (protocols, auth, buffering)
- Storage choices for hot vs cold data
- Stream processing vs batch processing
- Schema/partitioning strategy
- Scalability, reliability, and cost considerations
- A minimal set of APIs/queries you want to support
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.