Design a scalable MapReduce pipeline

Design a large-scale data processing system using a MapReduce-style architecture. Specify input and output schemas, the partitioning/sharding strategy, and how you achieve parallel computation. Explain how you minimize network traffic via data locality, combiners, serialization choices, compression, and request batching. Describe how to handle data skew and stragglers, implement fault tolerance and retries, and choose between at-least-once and exactly-once semantics. Provide complexity analysis and rough throughput/latency estimates, and outline key metrics and experiments you would run to validate efficiency.