Design a Distributed Scheduler with At-Most-Once Runs
RobinhoodSoftware Engineer · Onsite
Last updated
Design a distributed scheduler around strict at-most-once run starts, durable identities, ambiguous crashes, SLA failures, queue recovery, logs, and notification outboxes.
Design a distributed background-job scheduler with one-time and recurring jobs, an execution SLA, logs, cancellation, and subscriber notifications. The central requirement is at-most-once execution per scheduled run.
Constraints & Assumptions
Keep that requirement explicit when discussing crashes and retries. A run that may have started cannot simply be re-executed under a new lease while still claiming strict at-most-once execution. Explain the availability and completion tradeoff, and clarify whether the product actually means at-most-once execution or deduplicated effects.
Clarifying Questions
Does “execution” mean starting application code or committing an external effect? Is a lost or unknown run acceptable? Can a retry be a new scheduled run? Does the SLA measure start or completion? Must subscribers recover missed notifications?
Follow-up questions
- What happens if a worker dies immediately before or after beginning execution? What if the queue loses messages or the database commit times out? How do you push or broadcast updates to subscribers without losing durable status history?
Solution walkthrough is for members
Get the full walkthrough with reference code and complexity analysis, plus Kitty Coach at 300 messages a day. Hints, the code runner and community answers stay free.