Compute a Global Mode Across Distributed Partitions
AnthropicSoftware Engineer · Onsite
Last updated
Compute an exact distributed mode with local frequency combining, key-based reducers, global winner selection, communication accounting, and skew-aware recovery.
Compute the mode of data partitioned across several workers using supplied send/receive-style communication primitives. Explain the protocol and how you reduce network traffic without losing the exact global answer.
Constraints & Assumptions
The source describes distributed computation rather than a single-process function. Assume a fixed snapshot of finite records and reliable worker identity; clarify transport failures, value domain, ties, and output ownership. Use smallest-value tie-breaking only as one explicitly chosen example policy, not a source requirement.
Clarifying Questions
How large and skewed are partitions? How many distinct values occur locally and globally? Are duplicates common? Is an exact mode required? What happens on worker failure or repeated messages?
Follow-up questions
- Why is sending only each worker's local mode incorrect? What if almost every value is distinct? How would you prevent one frequent key or a skewed hash partition from dominating a reducer?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.