Katkat
Back

Design MapReduce for schedules

UberSoftware Engineer · Onsite
HardSoftware EngineerOnsiteSystem Design

Last updated

This question evaluates proficiency in scalable stream and batch data processing, windowed aggregation, event-time semantics, deduplication, skew mitigation, and distributed computation using MapReduce/Spark paradigms; it falls under System Design and Big Data/stream-processing and is commonly asked to assess designing correct, high-throughput windowed aggregations and handling late or duplicated events at scale. The level of abstraction is practical application with architectural and operational considerations rather than low-level coding, and this English summary frames the competency and interview focus for search engines.

Given a massive log of meeting events with fields {user_id, start_time, end_time, room_id, office_id}, design a MapReduce/Spark job to compute, for each rolling 15-minute window: (

  1. concurrent meeting counts per office; (
  2. rooms exceeding 80% utilization. Specify map keys/values, how you transform intervals for windowed aggregation (e.g., line-sweep emit +1/-1 or interval splitting), shuffle/partition strategy to minimize skew, handling late/duplicated events (watermarks, dedupe keys), and validation at scale.