Katkat
Back

Design a Scalable Job Scheduler

NextdoorSoftware Engineer · Technical Screen
MediumSoftware EngineerTechnical ScreenSystem Design

Last updated

This question evaluates system design skills focused on job scheduling, distributed systems concerns (state modeling, retries, idempotency), API and data-model design, fault tolerance, observability, and scalability within the System Design domain.

Design a job scheduler for a small startup, then explain how you would evolve it to support roughly 100x more scheduled tasks.

The system should:

  • Allow users or internal services to create one-time and recurring jobs.
  • Run jobs at the scheduled time with reasonable accuracy.
  • Support retries for failed jobs.
  • Support cancellation and rescheduling.
  • Track job states such as pending, dispatched, running, succeeded, and failed.
  • Provide basic observability for operators.

Discuss:

  1. API design and core data model.
  2. How jobs are stored and selected for execution.
  3. How workers execute jobs safely.
  4. How to handle duplicate execution, retries, and idempotency.
  5. Failure handling when the scheduler node or worker crashes.
  6. How the initial design for a low-scale startup would work.
  7. What architectural changes you would make to scale to 100x more jobs.