About Us Contact Us Write for Us Advertise
Home > System Design > High-Level Design > Design a Distributed Job Scheduler (High-Level Design)
System Design › High-Level Design

Design a Distributed Job Scheduler (High-Level Design)

High-level design of a distributed job scheduler (cron-as-a-service): jobs indexed by nextRunTime, a leader scheduler that enqueues due jobs, stateless workers, and the reliability story — leader election, at-least-once + idempotency, visibility timeouts, retries and sharding.

Shiv Pandey
Shiv Pandey
Nov 04, 2026 | 70 views
Design a Distributed Job Scheduler (High-Level Design)

🔴 High-Level Design · Case Study · Super-senior

"Design a distributed job scheduler" (think cron-as-a-service, or the engine behind AWS Lambda schedules, CI runners, or email-send jobs) is a favourite because it forces you to reason about reliability: run every job on time, exactly the right number of times, even when workers crash. It's a masterclass in queues, retries, and idempotency. Let's design it.

The approach in one line: store jobs with their next-run time, a scheduler polls for jobs that are due and pushes them onto a queue, and a pool of stateless workers pulls and executes them — with retries, idempotency, and leader election so nothing is lost or double-run.

Step 1 — Requirements

  • Functional: schedule a job to run once at time T or on a recurring schedule (cron); execute it; retry on failure; let users view status/history; cancel.
  • Non-functional: run jobs on time (low delay), scale to millions of jobs, at-least-once execution (never silently drop a job), no thundering herd, survive worker/scheduler crashes.

Step 2 — The classes / data model

🟠 A Job row: id, payload, schedule (cron or runAt), nextRunTime, status, retryCount. The key field is nextRunTime — the whole system is "find jobs whose nextRunTime ≤ now and run them." Index on nextRunTime so that query is cheap even with millions of rows.

Step 3 — Architecture

Submit API Job store (DB)idx nextRunTime Scheduler(leader, polls due) Queue Worker 1 Worker 2 Worker N on success: set next run / status
  1. Scheduler polls the DB every few seconds for jobs with nextRunTime ≤ now, and enqueues them onto a message queue.
  2. Workers (stateless, horizontally scalable) pull jobs off the queue and execute the payload.
  3. On success the job's nextRunTime is advanced (for recurring jobs) or it's marked done; on failure it's retried with backoff.

Decoupling scheduling from execution via a queue is the core move — it absorbs bursts and lets you scale workers independently.

Step 4 — Reliability: the hard part

🔴 This is where the question is really scored:

  • Don't double-schedule: if two scheduler instances both pick the same due job, it runs twice. Fix with leader election (only one active scheduler via consensus/a lock), or have workers atomically claim a job (compare-and-set status PENDING→RUNNING) so only one wins.
  • At-least-once + idempotency: queues guarantee at-least-once, so a job can run twice on retry. Make execution idempotent (dedup on job-run ID) so re-running is safe — turning "at-least-once" into effectively "exactly-once."
  • Crash recovery / visibility timeout: if a worker dies mid-job, the queue re-delivers after a visibility timeout so the job isn't lost. Store a heartbeat/lease so stuck jobs are reclaimed.
  • Retries & dead-letter: retry N times with exponential backoff, then move to a dead-letter queue for inspection.

Step 5 — Scaling

🔴 A single scheduler polling one table won't scale to billions of jobs. Shard jobs by ID or by time bucket, with a scheduler per shard. Avoid the thundering herd when thousands of jobs share the same run time by jittering execution. For very-far-future jobs, a time-bucketed design (hierarchical timing wheels, or a "next N minutes" hot table) keeps the due-query tiny. Partition workers by job type so a slow job class doesn't starve others.

The interview-ready walkthrough

"Jobs live in a store indexed by nextRunTime. A scheduler (single leader, or with atomic job-claiming) polls for due jobs and enqueues them; stateless workers pull from the queue and execute, then advance nextRunTime or mark done. Reliability is the crux: leader election / atomic claim to avoid double-runs, at-least-once delivery + idempotent execution for effectively-once, visibility timeouts so crashed jobs re-run, and retries with backoff into a dead-letter queue. I'd shard by time bucket and jitter to avoid thundering herds." That covers the pipeline and every failure mode they'll poke at."

What to read next

← Design an ad click aggregator · Design a leaderboard →

Related Articles

Design an E-commerce Platform (High-Level Design)
System Design › High-Level Design

Design an E-commerce Platform (High-Level Design)

Design a Leaderboard (High-Level Design)
System Design › High-Level Design

Design a Leaderboard (High-Level Design)

Design an Ad Click Aggregator (High-Level Design)
System Design › High-Level Design

Design an Ad Click Aggregator (High-Level Design)

Design Google Maps (High-Level Design)
System Design › High-Level Design

Design Google Maps (High-Level Design)