Built for locked-down containers. Runs unmodified under restricted
Kubernetes PodSecurity: non-root, read-only root filesystem,
RuntimeDefault seccomp, every Linux capability dropped (see
Production container deployment).
Prebuilt for practically everything. Multi-architecture images on GHCR
and Docker Hub plus self-contained binaries for Linux (glibc and musl),
macOS (signed and notarized) and Windows, so Python on the host is optional
(see Installation).
"Crontab" is in YAML format; classic crontab files are accepted as-is too
(see Classic crontab files)
Business-day schedules: LW (the month's last weekday), L-3 (three
days before month-end), 15W (the weekday nearest the 15th), and 5#3
(the third Friday) express payroll/billing-style cadences directly, and
Quartz day expressions largely paste straight in (see the
Business-Day Schedules
wiki page)
Built-in schedule linting: dead schedules that can never fire again are
called out loudly (never silently dropped), and common footguns (AND day
semantics, uneven */n steps, day-31-in-April, schedules that DST skips or
repeats) are flagged at config load, in the dashboards, and over the API
(see Schedule linting)
Builtin sending of Sentry, Mail, and webhook (Slack-compatible)
notifications when cron jobs fail
End-to-end encrypted push notifications: a fifth reporter seals each
alert to a paired device's own key (libsodium sealed box), so the relay
that forwards it to the platform push service never sees job names,
hostnames, or log lines; pairing is a dashboard QR scan or one API call,
and an opt-in Bonjour/mDNS advert lets a companion app find the daemon on
the LAN (see Push notifications and the
Push Notifications
and LAN Discovery
wiki pages)
Flexible configuration: you decide how to determine if a cron job fails or not
Designed for running in Docker, Kubernetes, or 12 factor environments:
Runs in the foreground
Logs everything to stdout/stderr
Production-ready for locked-down corporate container platforms: runs as a
non-root user, under a restricted seccomp profile, with a read-only root
filesystem, an fsGroup-mounted config, and all Linux capabilities
dropped, so no writable paths or elevated privileges are required (see
Production container deployment)
Option to automatically retry failing cron jobs, with exponential backoff
Per-job SLA monitoring: an sla: block declares thresholds for the runs
that did not happen: too long without a success, a due slot that never
started, a run exceeding its runtime bound. A breach fires a dedicated
onLate reporting hook once (mail, Sentry, shell, webhook), gauges and
counters land in the metrics, and the dashboards badge the job OVERDUE
(see Late-run detection and the
Late-Run Detection
wiki page)
Runtime pause/resume: pause any job's scheduled fires for a bounded
window (an hour by default, thirty days at most) over the API, the
dashboards, or MCP, without touching the config. Skipped slots are recorded
visibly, pending retries defer, catch-up owes nothing for the window, and
with a state: store the pause survives restarts and is honored by every
node (see Pause and resume a job and the
Pausing Jobs
wiki page)
Opt-in durable state: point a single state: config block at a local
directory (or an Amazon S3 Files / EFS mount to share it fleet-wide) and jobs
gain durability across restarts: missed-run catch-up after downtime and
retries that survive a daemon restart. The same store is handed to the jobs
themselves over a loopback endpoint, so a job command can reach for durable
key/value, an ETL cursor/watermark, a fleet-wide mutex or semaphore,
idempotency keys, a shared artifact store and run-scoped secrets with
cronstable state|cursor|lock|artifact|idempotent|secret (see the
Durable State wiki
page); without it, cronstable stays stateless as before
Opt-in orchestration DAGs: a dags: block turns the scheduler into a
small, durable workflow engine: tasks with dependsOn edges, cross-task
data hand-off (XCom), dynamic fan-out/mapping, sensors, human approval gates,
whole-DAG backfill, and crash-resume of a partial graph, all on the same
state store and coordinated across a fleet under a single lease so a task
never double-launches (see the
Orchestration and DAGs
wiki page)
Optional HTTP REST API, to fetch status, start jobs, cancel running jobs, and
read per-job run history on demand
Native TLS on the listeners: web.listen accepts https:// addresses
served from a web.tls block, mixed freely with plaintext and unix-socket
entries on one runner, and an optional clientCa makes the listener require
a client certificate signed by your own CA (mutual TLS), so it authenticates
its callers rather than merely encrypting them. A web certificate replaced in
place is picked up without a daemon restart. The job-facing state API gains
the same block as state.jobApi.tls, and cronstable tui / cronstable mcp
gain --cacert, --client-cert, --client-key and --insecure (see
Serving the API over TLS and the
Listener TLS
wiki page)
iCal calendar export: subscribe any calendar app to GET /calendar.ics
(or one job's /jobs/{name}/calendar.ics) and the fleet's upcoming fires,
enumerated by the scheduler's own engine, land on the on-call engineer's
calendar; the dashboard draws the same data as a seven-day week
calendar (see the
Calendar Export
wiki page)
Optional MCP server for
AI agents (Claude, Cursor, VS Code Copilot). An agent can observe
cronstable, author and debug schedules with the daemon's own engine
(validate/explain an expression, explain field-by-field why a job did not
run at a timestamp), and control it when you opt in. It is read-only by
default and exposes tools, resources, and triage prompts covering jobs,
DAGs, the cluster/fleet, metrics, and durable state. It is served at
POST /mcp on the web listeners and through a cronstable mcp stdio
bridge, and is hand-rolled in pure Python with no new dependencies
Native Prometheus metrics at /metrics (plus per-job statsd push
metrics), covering run outcomes, durations, retries, schedules, and cluster
health (see Metrics)
Opt-in per-job resource monitoring: one monitorResources: true samples
each run's CPU time and peak memory across its whole process tree, live and
per run, in the dashboard, the metrics, and the failure reports (see
Resource monitoring)
A job-set id: an order-independent fingerprint of every job's effective
configuration, so replicas deployed from the same config can confirm they
hold an identical set of jobs (see Job-set id)
Opt-in clustering and leader election: optionally have instances confirm
over mutual TLS that a configured set of peers is running the same job set, and
elect a leader so several replicas can run from one config without
double-running jobs (see
Clustering and leader election)
Arbitrary timezone support
Optional live control panel: watch every job's status,
tail its logs in real time, run or cancel jobs on demand, review run history
and success rates, drive DAG runs and approvals, and keep an eye on the whole
cluster, from one self-contained page with ten themes and a shortcut for
everything and a terminal twin
(cronstable tui) with the same keys
> The tour above is a single animated image (WebP; GIF fallback). Every frame is a real running fleet; see the web dashboard section for the annotated stills.
Quick start
Sixty seconds to a running scheduler with a live dashboard. Install it (see
Installation for Docker, Homebrew, and no-Python binary
options):
pip install cronstable
Describe your first job in a cronstable.yaml:
jobs:
- name: hello
command: echo "hello from cronstable on $(hostname)"
schedule: "* * * * *" # every minute
captureStdout: true
web:
listen:
- http://127.0.0.1:8080 # optional: the REST API + dashboard
Run it (always in the foreground, in true 12-factor spirit):
cronstable -c cronstable.yaml
That's it. Open and watch hello fire once a minute,
with its output tailing live in the dashboard. From here,
each of these is a few lines away:
Never miss a silent failure: retries with backoff and a Slack/mail/Sentry
report when a job ultimately fails (tutorial).
Survive restarts: a one-line state: block makes history, retries and
missed-run catch-up durable (tutorial).
Chain jobs into a pipeline: a durable DAG with data hand-off and an
approval gate (tutorial).
Run replicas safely: leader election so two copies never double-fire
(tutorial).
See it all at once: docker compose -f example/grand-tour/docker-compose.yml up --build boots a nine-node cluster running every feature together
(example gallery).
Already have a crontab? You don't have to translate it:
cronstable -c /etc/crontab runs the classic format as-is (see
Classic crontab files).
Installation
Run with Docker
Prebuilt, multi-architecture (linux/amd64, linux/arm64, linux/386,
linux/arm/v7, linux/ppc64le, linux/s390x and linux/riscv64) images are
published on every release to two registries: the GitHub Container Registry
and Docker Hub. The images are identical; pull from whichever you prefer. The
default image is built on Debian (slim); if you would rather match a particular
base, Alpine, Ubuntu, RHEL/UBI and other variants are
published from the same release too. Mount your crontab and go:
The image runs as a non-root user and reads its configuration from
/etc/cronstable.d by default. For production, pin a specific version instead of
latest (e.g. ghcr.io/ptweezy/cronstable:1.0.14 or ptweezy/cronstable:1.0.14) and
see Production container deployment for the
hardened Kubernetes/Docker setup.
Distro variants
The default latest (and ``) image is built on Debian (slim). The
same release is also published on several other bases, so you can match a
specific one to your environment: a familiar userland, an image-provenance
policy that mandates a particular vendor, or the smallest possible image. Each
variant adds a - suffix to the tag (and the default Debian image is
also available explicitly as -debian):
Tag suffix
Base image
Python
Notes
(none) / -debian
python:3.14-slim
3.14
Default. Widest architecture coverage.
-alpine
python:3.14-alpine
3.14
musl libc; smallest image.
-ubuntu
ubuntu:24.04
3.12
Ubuntu LTS userland.
-rhel
UBI 9 (ubi-minimal)
3.12
Red Hat base for RHEL / OpenShift.
-fedora
fedora:41
3.13
Leading-edge RPM userland.
-opensuse
opensuse/leap:15.6
3.11
SUSE / SLES family.
-amazonlinux
amazonlinux:2023
3.11
AWS-centric deployments.
-distroless
gcr.io/distroless/python3
3.11
No shell or package manager; minimal attack surface.
# e.g. the Alpine variant, pinned to a version:
docker run --rm \
-v "$PWD/cronstable.yaml:/etc/cronstable.d/cronstable.yaml:ro" \
ghcr.io/ptweezy/cronstable:1.0.14-alpine
cronstable is a pure-Python app that supports any Python >= 3.10, so behavior is
identical across variants. The Debian default covers the most architectures; each variant covers the arches
its base image publishes (Alpine matches Debian's full set; RHEL, Fedora,
openSUSE and distroless cover amd64, arm64, ppc64le and s390x; Amazon
Linux covers amd64 and arm64). All variants share the same non-root,
read-only-friendly hardening as the default image.
Install using pip
cronstable requires Python >= 3.10 (for systems with older Python, use the binary instead). It is advisable to install it in a Python
virtual environment, for example:
pipx automates creating a virtualenv and installing a python program in the
newly created virtualenv. It is as simple as:
pipx install cronstable
Install using Homebrew
On macOS or Linux, install from the cronstable
Homebrew tap:
brew install ptweezy/tap/cronstable
This installs the self-contained release binary for your platform (signed and
notarized on macOS; glibc amd64/arm64 on Linux via Homebrew on Linux), so no
Python is required. Upgrade later with brew upgrade cronstable.
This installs the self-contained release binary (amd64 or arm64, matching
your system), so no Python is required. Upgrade later with
winget upgrade ptweezy.cronstable.
Install using binary
Alternatively, a self-contained binary can be downloaded
from github: . Every release
automatically attaches binaries for Linux (amd64, arm64, i686, armv7, armv6,
ppc64le, s390x and riscv64), macOS (amd64 and arm64) and Windows (amd64 and
arm64):
Linux: glibc builds (cronstable-linux-) for the mainstream distros,
working on any system post glibc 2.39 (e.g. Ubuntu 24.04) on the matching CPU,
plus musl builds (cronstable-linux--musl) for Alpine and other musl-based
systems. `` is one of amd64, arm64, i686 (32-bit x86), armv7
(32-bit ARM, e.g. older Raspberry Pi), armv6 (musl only), ppc64le (POWER),
s390x (IBM Z) or riscv64 (64-bit RISC-V).
Python is not required on the target system (it is embedded in the executable):
# pick the asset for your OS and architecture (glibc amd64 Linux shown; append
# -musl on Alpine, or use cronstable-macos- on a Mac)
curl -fsSL -o cronstable \
https://github.com/ptweezy/cronstable/releases/latest/download/cronstable-linux-amd64
chmod +x cronstable
./cronstable --version
The macOS binaries are signed and notarized by Apple.
The standalone binary is a self-extracting executable: on each start it unpacks
its embedded Python runtime into a temporary directory and loads shared
libraries from there. It therefore needs a temporary directory that is both
writable and executable. On an ordinary system the default /tmp already
satisfies this, so no extra setup is required.
This only matters when you run the binary under a read-only root filesystem
(for example, a hardened container). With the root filesystem read-only, /tmp
is read-only too, and the binary aborts at startup: Could not create temporary directory, or Error loading shared library …: Operation not permitted. Give
it a small writable and executable temp mount and it runs fine:
# Note `exec`: Docker's --tmpfs defaults to `noexec`, but the binary must be
# able to execute the libraries it unpacks.
docker run --rm --read-only \
--tmpfs /tmp:rw,exec,nosuid,nodev,size=64m \
-v "$PWD/cronstable.yaml:/etc/cronstable.d/cronstable.yaml:ro" \
your-image-with-the-binary -c /etc/cronstable.d
On Kubernetes, mount an emptyDir at /tmp (an emptyDir is writable and
executable by default; use medium: Memory for a tmpfs). Alternatively, point
the binary at another writable, executable directory with TMPDIR=/path.
This requirement is unique to the standalone binary. The published container
image (and pip/pipx installs) run cronstable as a normal Python package with
the interpreter on disk, so they never self-extract and need no writable temp
directory. See Production container deployment.
Running on Windows
cronstable runs natively on Windows (x64 and ARM64), in addition to Linux and
macOS. Install it with pip install cronstable, or download the self-contained
cronstable-windows-amd64.exe / cronstable-windows-arm64.exe from the
releases page (no Python
required). Everything else, like the YAML crontab, scheduling, reporting, retries,
the HTTP API and the web dashboard, works the same as on
POSIX. A few platform details differ:
Default config location. When -c is omitted, cronstable looks in
%APPDATA%\cronstable (e.g. C:\Users\you\AppData\Roaming\cronstable), the
Windows analog of /etc/cronstable.d. Point it anywhere with -c:
cronstable -c C:\path\to\cronstable.yaml
Default shell. A string command with no explicit shell runs through
the native command processor (%ComSpec%, i.e. cmd.exe), mirroring the
/bin/sh default on POSIX. For PowerShell, or any other interpreter, set
shell: or pass command as a list (which bypasses the shell entirely):
Graceful shutdown. Press Ctrl-C (or Ctrl-Break) to stop cronstable; it
shuts down after the currently running jobs finish, just as SIGTERM does on
POSIX.
Not supported on Windows. Per-job user/group switching (there is no
setuid/setgid equivalent) is rejected with a clear configuration error,
and unix:// web listeners are skipped with a warning. Use an http://
listener instead.
Production container deployment
cronstable is built to run unmodified under the hardened security contexts that
corporate and enterprise Kubernetes / container platforms enforce. At runtime
the daemon only reads its configuration and secrets and writes its output to
stdout/stderr. It never needs a writable working directory, temp files, or log
files, so it slots cleanly into a locked-down pod:
Non-root user: cronstable needs no special privileges to run, so the whole
daemon can run as an unprivileged UID. Only the optional per-job
user/group switching (see Change to another
user/group) requires running as root; if you
don't use that feature, drop root entirely.
Seccomp profile: cronstable makes no exotic syscalls, so the
RuntimeDefault seccomp profile (or an equivalently strict custom profile)
works out of the box.
Read-only root filesystem: no runtime writes are required by the
published image (or a pip/pipx install). Mount your crontab config
read-only. (If you enable the optional HTTP
interface on a Unix socket, point the socket at a
small writable emptyDir volume rather than the root filesystem. And if you
deploy the standalone binary instead of the image, it additionally needs a
small writable, executable temp mount; see Install using
binary.)
fsGroup and dropped capabilities: config and secret volumes can be
mounted with an fsGroup so the non-root process can read them, and you can
drop all Linux capabilities and forbid privilege escalation.
The published image (ghcr.io/ptweezy/cronstable and docker.io/ptweezy/cronstable)
is already built this way (non-root, with cronstable -c /etc/cronstable.d as its
entrypoint and no writable paths required), so for most deployments you can use
it directly and mount your crontab read-only. If you would rather bake the
configuration into your own image, base it on the published image:
FROM ghcr.io/ptweezy/cronstable:latest
# The base image already runs as the non-root user 65534.
COPY cronstable.yaml /etc/cronstable.d/cronstable.yaml
And a corresponding Kubernetes Deployment with a fully restricted security
context:
cronstable ships with a built-in web dashboard: one self-contained page (no
build step, no external assets, no database) served straight from the daemon.
Point a browser at the HTTP listener and you have a keyboard-driven control
room for every job, and, when you use them, for the cluster, the DAGs, and the
durable state store too.
The overview shows every job with its live status, a countdown to its
next run, the last run's duration and exit-code badge, and a sparkline of
recent runs; jobs with resource monitoring add live
CPU and memory chips while they run, and a cluster adds each job's
owner node. Everything is sortable, filterable, and searchable, and when
something is failing a verdict bar correlates the failures into one
headline ("4 share exit=69, likely one cause"). Click any job (or press
Enter) to open its detail drawer:
Live log tail
Run history
Schedule, explained
Follow a running job's output live over Server-Sent Events, with ANSI color, in-log grep (plain text or regex), per-line timestamps, line-wrap, and one-click download.
Success rate plus average / min / max duration over the retained history, with a color-coded per-run chart; with resource monitoring on, CPU time and peak memory per run and in the stats.
A plain-English reading of the cron expression and a timezone-aware preview of the next run times, computed live in the browser.
Everything is one keypress away: a fuzzy command palette (Ctrl-K / ⌘K)
runs any action or jumps to any job, ? lists every shortcut, / filters,
j/k move the cursor, r runs the selected job and x cancels it. You can
run a single job, or every failing job at once, on demand, with a click.
Fuzzy command palette
Keyboard-first, with a shortcut for everything
Orchestration, live
DAGs get
their own card and drawer: trigger or backfill a run, watch the task graph
advance node by node, inspect per-task attempts, XCom values and logs, and
decide approval gates with a click, from any node in the fleet.
The task graph
A human approval gate
A data-quality-gate diamond: fan-out checks that reconverge on a certify task, colored by state as the run advances.
A release train parked on a human: the build succeeded, the approval gate is awaiting, and the sensor and publish tasks queue behind your decision.
The whole fleet on one page
With clustering on, a cluster panel
shows the quorum math, this node's role, per-peer attestation status, and,
with cluster.observability, every node's whole-host CPU and memory. The
fleet view goes further: a jobs × nodes matrix of the entire fleet's runs,
assembled from data that piggybacks on the gossip the nodes already exchange,
so any node can serve the single pane of glass.
Cluster panel
Fleet view
Nine nodes, 8/8 agreed, quorum met, per-node load meters and per-node owns counts under distribution: spread.
Every node's state for every job, one glance: ok / failing / running cells with ages, per-column node health, and a failing only filter.
Built for the 3 a.m. incident
When things break, the dashboard leans in: the verdict bar's incident
timeline lays out every job's most recent finish, newest first, with the
correlated blast-radius set highlighted; the mitigate console start/cancels
the failing set in bulk and copies a Markdown incident summary for your ticket;
and the multi-tail console merges up to four jobs' live logs into one
pane, like tailing a set of pods.
Incident timeline
Merged multi-tail
"What happened, in what order": relative times, outcome glyphs, failure reasons, exit codes, durations, and a failing only filter.
Four streams, one pane: identity-colored prefixes, end of run output markers, auto re-attach on each job's next run.
Wallboards, heatmaps, and the state store
Press w for a full-screen wallboard built for a TV: worst-first tiles,
an incident stamp when something is failing, a NO SIGNAL banner when the
data goes stale (never a stale all-green), and a zen screensaver that
takes over when everything is healthy. The activity heatmap turns run
history into a punchcard (worst outcome per bucket, shaded by volume), and the
opt-in state inspector shows the durable state store's
health: record counts by kind, op latencies and errors, locks, cursors,
counters, artifacts, and quarantine.
Wallboard / TV mode
Activity heatmap
Durable-state inspector
Make it yours
Ten themes: carolina (the default, a Carolina-blue CRT phosphor),
amber and green phosphor, and flat modern and standard looks, each in
a dark (phosphor) and a light (paper) variant. Cycle hues with t, flip
light/dark with T. CRT glow, scanlines, compact density, desktop failure
notifications, audible cues, and the polling interval are all toggles,
remembered per browser, and the CRT effects honor prefers-reduced-motion:
(One board, ten themes, two interface fonts, animated: WebP, GIF. The four stills below are pulled from it.)
Amber phosphor CRT
Green phosphor CRT
Flat modern theme
Carolina, on paper (light)
Readability and accessibility. Beyond the themes, the dashboard is built to
be read comfortably by everyone (all remembered per browser):
Interface font: swap the terminal monospace for a proportional
sans-serif for easier reading (both shown, per theme, in the animation
above); log output and cron strings stay monospace where alignment matters
(in Settings, or "Toggle readable font" in the command palette).
UI scale: zoom the whole interface to 100 / 110 / 125 / 140 % for
larger text without touching your browser zoom.
Colour-vision modes: deuteranopia- and tritanopia-safe palettes
re-ink the success/failure/pending status colours (and the paper themes get
their own tuned variants) so state never rides on red-vs-green alone.
Reduced motion: the CRT glow, scanlines, flicker and the pendulum mark
all honor prefers-reduced-motion (and a manual toggle; the mark parks into
a still pose that stays honest about daemon state: upright when live,
hanging when not), and status is always carried by glyphs and text, not
colour or animation alone.
And because a control room deserves a proper power-on: an optional (on by
default, once per 12 hours) BIOS-style boot self-test that checks the
daemon, job set, cluster, and schedules for real while it types:
Settings
Startup self-test
Even the wordmark is on duty: the l in the header's "cronstable" is a
live cart-and-double-pendulum simulation balanced by a real LQR controller
whose gains are computed in your browser at page load, not a canned
animation. While the daemon is live it stands upright, riding out little
gusts; sweep your cursor through it to brush it aside, right-click to knock
it clean over. Lose the daemon and its motor cuts: the letter collapses out
of the word and swings, and when the signal returns a planner threads the
swing-up back into a verified catch; the word heals. Stability is the
product; the logo has to earn it too.
Run history and live logs are kept in memory only (unless you opt into the
durable state store), and the page is served with a strict
Content-Security-Policy. Turn it on with a one-line web: block: the
web dashboard tour
in the wiki is the full walkthrough, and
Remote web/HTTP interface below shows how to
enable it.
Try it:docker compose -f example/zen-demo/docker-compose.yml up boots a single node with a demo job set, and docker compose -f example/cluster/docker-compose.yml up boots a 3-node cluster (cronstable-a/cronstable-b/cronstable-c) so you can open each node's dashboard and watch the cluster panel and leader election live. For every feature at once (a 9-node mutual-TLS cluster sharing one durable state store and running the classic job set, durable-state jobs, orchestration DAGs and second-level probes together, with all four failure reporters wired to live sinks), run docker compose -f example/grand-tour/docker-compose.yml up --build (the grand tour; see its README). More one-command demos are in the example gallery.
Terminal dashboard
The dashboard has a TUI sibling: cronstable tui opens the board in
your terminal, over SSH, in a tmux pane, or on a box where a browser is
not an option. It is a client of the same HTTP control API (nothing extra
to enable on the daemon), and it keeps the web page's muscle memory;
the shortcut table is the same one: j/k move, Enter opens a job's
drawer, r runs, x cancels, / filters, g refreshes, i opens the
incident timeline, w the wallboard, Ctrl-K the fuzzy command
palette, and ? lists everything (terminal-only extras like quit,
sort/filter cycling, and drawer tabs are grouped separately in that
overlay). It is hand-rolled on the stdlib + the core aiohttp dependency,
so it adds nothing to the install. Every screenshot below is the real
TUI against the same live 9-node grand tour fleet
the web-dashboard shots use, same staged incident and all:
Press Enter on any job for its drawer, the same three tabs as the
web page, plus resources for monitored jobs:
Live log tail
Run history
Schedule, explained
The same SSE stream the browser tails: ANSI colors re-inked per theme, / search with n/N, follow/wrap/timestamps toggles, d saves to a file.
Success rate and duration stats over the retained history; monitored jobs add CPU time and peak memory per run.
The plain-English reading and next-run preview come from the daemon's own cron engine, so the TUI can never disagree with the scheduler.
Fuzzy command palette
Keyboard-first, with the web page's keys
DAGs get the same drawer as the browser (runs, a task graph, per-task
states, XCom, task logs, trigger and backfill), and approval gates
are decided with a keypress:
The task graph, mid-flight
A human approval gate
The data-quality-gate diamond as topological layers with edges, each node colored by its state as the run advances.
A release train parked on a human: the build succeeded, the gate is awaiting, and a approves / R rejects without leaving the keyboard.
With clustering on, the cluster panel and the full fleet matrix
render in the terminal too, every node's state for every job:
Cluster panel
Fleet view
When things break, the same incident kit leans in: the verdict bar's
correlation headline, the timeline, the mitigate console's bulk
start/cancel with a Markdown writeup, and a multi-tail that merges
up to four live logs:
Incident timeline
Merged multi-tail
w flips the whole terminal into the wallboard (worst-first tiles,
a NO SIGNAL banner when data goes stale, the zen screensaver when all
is calm); the heatmap turns history into a punchcard and the
state inspector opens the durable store:
Wallboard / TV mode
Activity heatmap
Durable-state inspector
The same ten themes as the browser (t cycles the hue, T flips
phosphor ↔ paper), with the same colour-vision-safe remaps and an
--ascii glyph mode:
Amber phosphor
Green phosphor
Flat modern
Carolina, on paper (light)
And yes, the control room still gets its power-on: the same BIOS-style
boot self-test, probing the daemon for real, next to the settings
sheet where the theme, refresh, zen, and cues live:
Startup self-test
Settings
cronstable tui # local daemon on :8080
cronstable tui --url http://prod-node:8080 --token-env CRONSTABLE_WEB_TOKEN
cronstable tui --tv # straight to the wallboard
cronstable tui --job nightly-backup # deep-link a job's drawer
Web-only physics (CRT glow, scanlines, the pendulum wordmark) stay in
the browser; the terminal gets honest glyphs, the same status colours,
and a bell instead of desktop notifications. The
Terminal Dashboard
wiki page is the full reference (options, every key, the panel tour).
Tutorials
Four short, copy-paste-runnable walkthroughs, each built on the
quick start config and each ending where the wiki goes deeper.
Tutorial 1: Alert when a job fails, then retry it
Classic cron mails root and hopes. Instead: retry with exponential backoff,
and page a Slack channel only if the job ultimately fails.
jobs:
- name: nightly-backup
command: /usr/local/bin/backup --incremental
schedule: "0 3 * * *"
captureStderr: true # include stderr in the report
onFailure:
retry:
maximumRetries: 5
initialDelay: 5 # 5s, 10s, 20s, 40s, ... capped at 300s
maximumDelay: 300
backoffMultiplier: 2
onPermanentFailure: # fires once, after the last retry is spent
report:
webhook:
url:
fromEnvVar: SLACK_WEBHOOK_URL
By default a job fails when it exits non-zero or writes to a captured
stderr; tune that per job with failsWhen. The webhook's
default body is Slack-compatible (Mattermost and Teams work as-is), and mail,
Sentry, and a shell command are equally one block away, with jinja2 templating
over the run's name, output, and exit code. Deeper:
Failure Detection and Retries
and Reporting in the wiki.
Tutorial 2: Survive restarts, catch up what was missed
Stateless is the default, and a feature. But when a deploy or a reboot lands
mid-schedule, one state: block gives jobs a memory:
state:
path: /var/lib/cronstable # a local dir, or a shared mount for a fleet
jobs:
- name: hourly-invoice-emit
command: python -m billing.emit_hourly
schedule: "0 * * * *"
onMissed: run-all # replay each hour missed while we were down
startingDeadlineSeconds: 21600 # ...unless the slot is older than 6h
onFailure:
retry:
maximumRetries: 10
initialDelay: 30
maximumDelay: 600
backoffMultiplier: 2
With just the state.path line, run history survives restarts (the dashboard
rehydrates it), armed retries re-arm at their absolute deadlines, @reboot
means once per boot rather than once per daemon start, and Prometheus
counters stop resetting to zero. onMissed adds catch-up on top: run-once
coalesces any number of missed slots into one launch, run-all replays each
one, bounded by startingDeadlineSeconds. The same store also hands your job
commands durable primitives (key/value, cursors, fleet-wide locks,
idempotency keys, artifacts, run-scoped secrets) over a loopback endpoint:
cronstable state|cursor|lock|idempotent|artifact|secret. Deeper:
Durable State.
Tutorial 3: Your first DAG, a durable pipeline
A dags: block turns the scheduler into a small, durable workflow engine.
This one builds, waits for a human, then publishes:
state:
path: /var/lib/cronstable # DAGs live on the state store
dags:
- name: release-train # no schedule: manual-only
tasks:
- id: build
command: make dist
- id: approve
type: approval # parks the graph on a human decision
dependsOn: [build]
- id: publish
dependsOn: [approve]
command: make publish
retries: 2 # task-level retries, DAG-owned
retryDelaySeconds: 60
Trigger it and approve the gate (or click Approve in the dashboard's DAG
drawer):
Every transition is durable: restart the daemon mid-run and the run resumes
exactly where it was, and across a fleet the run advances under a lease so a
task never launches twice. Scheduled DAGs add catch-up and backfill over a
date range; tasks can pass data with cronstable xcom push/pull, fan out
dynamically over a list an upstream task produced, and poll for conditions
with type: sensor. Deeper:
Orchestration and DAGs.
Tutorial 4: Two replicas, zero double-runs
Run the same config on two (or nine) hosts that share a POSIX mount, and let
them elect a leader through a fenced lease file, with no certificates and no
coordination service:
state:
path: /mnt/shared/cronstable/state # shared durable state (optional but natural here)
cluster:
backend: filesystem
filesystem:
path: /mnt/shared/cronstable # the mount is the election store
nodeName: node-a # unique and stable per replica!
electLeader: true
jobs:
- name: charge-subscriptions
command: python -m billing.charge
schedule: "0 6 * * *"
clusterPolicy: Leader # the default: exactly the leader runs it
Only the elected leader fires Leader jobs; stop it and a follower adopts the
lease within its TTL. Per job, clusterPolicy picks the trade-off:
Leader (never double-runs, may skip when quorum is lost), PreferLeader
(never skips, may double-run under a partition), or EveryNode (genuinely
per-node work). No shared mount? The gossip backend elects over mutual TLS
with no shared store at all, kubernetes uses a coordination.k8s.io Lease,
and etcd a lease-bound key; distribution: spread load-balances job
ownership across the fleet instead of concentrating it on one leader. Deeper:
Clustering and Leader Election.
Example gallery
Every example in example/ is a self-contained, annotated,
runnable project; each compose file lives in its example's folder (the demo
quickstart uses the root docker-compose.yml). Highlights:
The minimal "add cronstable to your own image" recipe.
Usage
Configuration is in YAML format. To start cronstable, give it a configuration file
or directory path as the -c argument. For example:
cronstable -c /tmp/my-crontab.yaml
This starts cronstable (always in the foreground!), reading
/tmp/my-crontab.yaml as configuration file. If the path is a directory,
any *.yaml or *.yml files inside this directory are taken as
configuration files, along with any classic crontabs (*.crontab, *.cron,
or a file named crontab; see
Classic crontab files).
Configuration basics
This configuration runs a command every 5 minutes:
The command can be a string or a list of strings. If command is a string,
cronstable runs it through a shell, which is /bin/bash in the above example, but
is /bin/sh by default.
If the command is a list of strings, the command is executed directly, without a
shell. The ARGV of the command to execute is extracted directly from the
configuration:
The schedule option can be a string in the classic crontab format (5, 6 or 7 fields; ranges, steps, lists, jan/mon names, and Quartz's ? standing alone in a day field), parsed by cronstable's built-in cron engine; see Schedules and Timezones for the full dialect. Expressions in other dialects (Quartz #/W, the seconds-first 6-field layout) fail with an error naming the dialect and how to convert.
Additionally @reboot can be included , which will only run the job when cronstable is initially
executed. Further schedule can be an object with properties. The following configuration
runs a command every 5 minutes, but only on the specific date 2017-07-19, and
doesn't run it in any other date:
Every schedule is linted at config load. The linter flags legal expressions
that probably do not mean what they say: a schedule with no future
occurrence (a fixed past year, 0 0 30 2 *), day-of-month and day-of-week
both restricted (a day must match BOTH here, unlike Vixie cron's either),
*/n steps that do not divide their field (*/7 minutes fires at :56 and
then :00), day values no selected month is long enough to reach, Feb 29
schedules that only fire in leap years, and wall times a DST transition in
the job's timezone skips or repeats. Findings are logged per job at load
time, served as schedule_findings (plus a never_fires flag) on the
/jobs and /status endpoints, and shown in the dashboards' cron
sandboxes. A dead schedule stays a loud warning rather than an error, so a
parked job (year: "2020") cannot fail an upgrade; prefer enabled: false
to say what you mean. GET /schedule/preview runs the same
parse/describe/preview/lint for any expression before it becomes a job.
See Schedule Linting.
Hashed schedules: H
H * * * * runs a job every hour at a minute hashed from the job's name
(Jenkins-style; H, H(a-b), H/n, and H(a-b)/n work in any field but
the year). Every job lands on its own stable slot, so a fleet of hourly
jobs spreads across the hour instead of stampeding at :00, and because the
slot is a pure function of the name rather than random jitter, it survives
restarts, reloads and replicas, and "was this run late?" stays answerable.
In day-of-month, rangeless H forms (bare H and H/n) hash over 1 to 28
so short months are never skipped; renaming a job re-hashes its slots. The linter notes the concrete
values each H resolved to, and /jobs serves them as schedule_resolved.
See Hashed Schedules.
Schedule pressure: the fleet's collision heatmap
GET /schedule/pressure enumerates every enabled schedule's fires over the
next 24 hours with the scheduler's own engine (timezone- and DST-exact) and
buckets them into an hour-by-minute grid: "37 jobs fire at :00, minute 23
is empty", as data. The web dashboard draws it as a heatmap card (with a
compact strip on the wallboard), the TUI has the same panel as an overlay,
and the cron_schedule_pressure MCP tool serves it to agents. See
Schedule Pressure.
Duplicate-schedule detection
GET /schedule/duplicates groups jobs whose schedules fire on the
identical instants, using the engine's semantic equality (*/5 equals
0-59/5, @hourly equals 0 * * * *) and the resolved timezone, so
"these 14 jobs all share 0 0 * * *" surfaces before midnight proves it.
Shown in the pressure card and TUI overlay. See
Duplicate Schedule Detection.
Suggest a slot
GET /schedule/suggest?period=hourly|daily recommends the least-loaded
minute (or hour:minute) for a new job from the fleet's real fires,
deterministically, with ties breaking away from the busiest slot; the
response includes runners-up and the H spelling that would keep future
jobs spreading themselves. Also available as one-click buttons in the
pressure card and via the cron_suggest_slot MCP tool. See
Suggest a Slot.
Why didn't it run?
GET /schedule/why?job=&at= decomposes the scheduler's
own match test field by field for one job and one instant: "minute
matched; day-of-week Tuesday is not in Monday and Friday", with each
field's accepted values rendered as prose, the nearest real fire on each
side of the timestamp, and notes when the answer is genuinely surprising
(the day-field AND rule where Vixie cron would have fired, and DST wall
times that were skipped or repeated). Aware timestamps convert into the
job's timezone, naive ones read as wall time there, and @reboot,
disabled, hashed-H and dag: schedules all answer honestly.
Agents get the same explainer as the cron_why_no_run MCP tool,
alongside cron_validate_schedule and cron_explain_schedule for
checking an expression before it becomes a job. See
Why Didn't It Run?.
Second-level schedules
Schedules are minute-granular by default, but cronstable can also run jobs at
second granularity. There are two equivalent spellings:
a full seven-field crontab string, where the first field is the second
(second minute hour dayOfMonth month dayOfWeek year); or
the object form with a second: property.
Both of the jobs below run every 15 seconds (at seconds 0, 15, 30 and 45 of
every minute):
jobs:
- name: every-15s-string
command: echo "tick"
schedule: "*/15 * * * * * *" # 7 fields: the leading field is seconds
- name: every-15s-object
command: echo "tick"
schedule:
second: "*/15"
The second field accepts the same syntax as the others (*, */5, 0,30,
10-20, ...). second: "*" (or * * * * * * *) fires every second.
While any enabled job specifies seconds, the scheduler wakes once per second
instead of once per minute; minute-granular jobs are unaffected and still fire
exactly once in their scheduled minute. If no job uses seconds, cronstable keeps
its original once-a-minute cadence, so there is no overhead for the common case.
Second-level scheduling is a YAML feature: classic crontab files
keep their standard five-field, minute-granular format. (A six-field string
is read as the classic five fields plus a trailing year column, not as
seconds; seconds require the full seven fields.)
For a runnable end-to-end example, see
example/pulse-monitor, a small real-time uptime / SLA
monitor that probes a service every few seconds
(docker compose -f example/pulse-monitor/docker-compose.yml up), and its clustered sibling
example/pulse-cluster, which fans the probes across a
three-node leader-electing cluster
(docker compose -f example/pulse-cluster/docker-compose.yml up).
Important: by default all time is interpreted to be in UTC, but you can
request to use local time instead. For instance, the cron job below runs
every day at 19h27 local time because of the utc: false option:
The env file must be a list of KEY=VALUE pairs. Empty lines and lines starting with # will be ignored.
Variables declared in the environment option will override those found in the env_file.
Classic crontab files
Already have a crontab? cronstable runs it as-is. A file named *.crontab,
*.cron, or just crontab (so -c /etc/crontab works) is read in the
classic Vixie format, whether passed directly to -c, dropped into a config
directory next to YAML files, or pulled in with include::
Comments, NAME=value environment lines (position-sensitive, SHELL and
CRON_TZ honored), the @reboot/@daily/... nicknames, and \% escapes
all work as in man 5 crontab. Each entry becomes an ordinary cronstable job
named :, configured to cronstable's standard defaults rather than
an emulation of cron's environment: schedules run in UTC unless the
crontab sets CRON_TZ, failure means a non-zero exit or stderr output (no
MAILTO mail), and the %-as-stdin feature is a load-time error instead of
a silent surprise (\% still gives a literal %). When an entry needs
retries, reporting, timeouts, or any other per-job option, move it to YAML.
The full mapping and every deviation are documented in the
Classic Crontabs
wiki page, and a runnable example (a config directory mixing a crontab with
YAML and the dashboard) lives in example/crontab.
Specifying defaults
There can be a special defaults section in the config. Any attributes
defined in this section provide default values for cron jobs to inherit.
Although cron jobs can still override the defaults, as needed:
Note: if the configuration option is a directory and there are multiple configuration files in that directory, then the defaults section in each configuration file provides default options only for cron jobs inside that same file; the defaults have no effect beyond any individual YAML file.
Reporting
Cronstable has builtin support for reporting jobs failure (more on that below) by
email, Sentry, shell command, and HTTP webhook (Slack-compatible out of the
box):
- name: test-01
command: |
echo "hello" 1>&2
sleep 1
exit 10
schedule:
minute: "*/2"
captureStderr: true
onFailure:
report:
sentry:
dsn:
value: example
# Alternatively:
# fromFile: /etc/secrets/my-secret-dsn
# fromEnvVar: SENTRY_DSN
fingerprint: # optional
- cronstable
- "{{ environment.HOSTNAME }}"
- "{{ name }}"
extra:
foo: bar
zbr: 123
level: warning
environment: production
mail:
from: example@foo.com
to: example@bar.com
smtpHost: 127.0.0.1
# optional fields:
username: "username1" # set username and password to enable login
password:
value: example
# Alternatively:
# fromFile: /etc/secrets/my-secret-password
# fromEnvVar: MAIL_PASSWORD
tls: false # set to true to enable TLS
starttls: false # set to true to enable StartTLS
shell:
shell: /bin/bash
command: ...
webhook:
url:
fromEnvVar: SLACK_WEBHOOK_URL
Here, the onFailure object indicates that what to do when a job failure
is detected. In this case we ask for it to be reported both to sentry and by
sending an email.
The captureStderr: true part instructs cronstable to capture output from the the
program's standard error, so that it can be included in the report. We could
also turn on standard output capturing via the captureStdout: true option.
By default, cronstable captures only standard error. If a cron job's standard error
or standard output capturing is not enabled, these streams will simply write to
the same standard output and standard error as cronstable itself.
Both stdout and stderr stream lines are by default prefixed with
[{job_name} {stream_name}], i.e. [test-01 stdout], if for any reason you
need to change this, provide the option streamPrefix
with your own custom string.
It is possible to customize the format of the report. For
mail reporting, the option subject indicates what is the subject of the
email, while body formats the email body. For Sentry reporting, there is
only body. In all cases, the values of those options are strings that are
processed by the jinja2 templating engine. The following variables are
available in templating:
name(str): name of the cron job
success(bool): whether or not the cron job succeeded
The shell reporter executes a user given shell command in
the specified shell. It passes all environment variables from the python
executable and specifies some additional ones to inform about the state of the
job:
The webhook reporter sends an HTTP POST to a URL of your choice. The default
body is a Slack-compatible {"text": ...} JSON payload, so pointing it at a
Slack, Mattermost, or Teams incoming-webhook URL works with no further
configuration; method, contentType, headers, timeout, and the jinja2
body template are all configurable for other services (Discord, ntfy, or
your own endpoint). Like the other secrets, the URL can come from value,
fromFile, or fromEnvVar:
See Reporting in the wiki
for all webhook options and per-service examples.
It is possible to send emails formatted as html, by adding
the html: true property. For example, here the standard output of a shell
command is captured and interpreted as html and placed in the email message:
- name: test-01
command: echo "hello <b>world</b>"
schedule: "@reboot"
captureStdout: true
onSuccess:
report:
mail:
from: example@foo.com
to: example@bar.com, zzz@sleep.com
html: true
smtpHost: 127.0.0.1
smtpPort: 1025
subject: This is a cron job with html body
Push notifications
A fifth reporter, push, delivers end-to-end encrypted alerts to paired
devices. Each alert is sealed to the device's X25519 public key (a libsodium
sealed box) before it leaves the daemon; the hosted relay that forwards it to
the platform push service (APNs) sees only ciphertext and routing metadata,
never job names, hostnames, or log lines. It needs the push extra
(pip install "cronstable[push]"), a daemon-global push: section, and an
opt-in on the reporting hooks; a config that enables push without any of
those refuses to start rather than silently not alerting:
Setting web.bonjour: true (with the discovery extra installed)
additionally advertises the web API as a _cronstable._tcp mDNS service on
the local network, so a companion app finds the daemon without a typed URL;
see the
LAN Discovery
wiki page.
See
Push Notifications
in the wiki for the report options, pairing and revocation, storage, size
limits, and the relay trust model.
Metrics
Cronstable natively exposes Prometheus metrics whenever the
HTTP REST API is enabled;
no exporter sidecar needed:
web:
listen:
- http://127.0.0.1:8080
GET /metrics then serves job run outcomes, duration histograms, retries,
next-run times, config-reload health, and cluster/leader-election state, in
both the Prometheus text format and OpenMetrics. See
Metrics with Prometheus
for the full metric reference, scrape configuration, and example alert rules.
Cronstable also has builtin support for pushing per-job metrics to
Statsd:
With this config Cronstable will write the following metrics over UDP
to the Statsd listening on my-statsd.example.com:8125:
my.cron.jobs.prefix.test01.start:1|g # this one is sent when the job starts
my.cron.jobs.prefix.test01.stop:1|g # the rest are sent when the job stops
my.cron.jobs.prefix.test01.success:1|g
my.cron.jobs.prefix.test01.duration:3|ms|@0.1
Resource monitoring
Ever wondered which cron job is eating the box?! Turn on per-job resource
accounting with a single flag (or once under defaults: for every job):
While the job runs, cronstable samples its whole process tree (children and
shell-outs included) with psutil, and
the run ends with its total CPU time (user + system) and peak resident
memory. The numbers surface everywhere the run does:
live on the dashboard job row and drawer while it runs (cpu 61% · 288 MiB);
per run and aggregated (avg/max CPU, peak memory) in the dashboard
History tab and GET /jobs/{name}/runs;
as CPU/memory charts in the dashboard's Resources tab (a live
view of the running instance, the recorded profile of any recent run, and
per-run trend strips), plus a node-wide history chart behind the header
meter (GET /jobs/{name}/resources, GET /node/history);
as Prometheus families on GET /metrics
(cronstable_job_cpu_seconds_total, cronstable_job_last_run_max_rss_bytes, ...)
and over statsd when the job has a sink;
in the durable run record's resources object when a
state store is
configured, so it survives restarts;
in report templates (cpu_seconds / max_rss_bytes) and the shell
reporter's environment (CRONSTABLE_CPU_SECONDS / CRONSTABLE_MAX_RSS_BYTES),
so a failure page can say how big the run was when it died.
It is observability only (it never changes a run's verdict), it is off by
default with zero overhead when off, and the numbers are sampled, so
short-lived runs are approximate while the long, heavy runs that matter are
sampled many times. The map form tunes the sampling cadence and how many
chart points each run keeps (monitorResources: { interval: 0.5, history: 240 }); series are downsampled in place so even a days-long run stays a few
KB. DAG tasks accept the same flag; their usage lands in the
task record of the dag_run document. On a cluster,
cluster.observability additionally shares each node's whole-host
CPU/memory so the dashboard's cluster panel and fleet view show where the
load actually is. The full semantics live in the
Configuration Reference.
Handling failure
By default, cronstable considers that a job has failed if either the process
returns a non-zero code or if it generates output to standard error (and
standard error capturing is enabled, of course).
You can instruct cronstable how to determine if a job has failed or not via the
failsWhen option:
The above settings tell cronstable to retry the job up to 10 times, with the delay
between retries defined by an exponential backoff process: initially 1 second,
doubling for every retry up to a maximum of 30 seconds. A value of -1 for
maximumRetries will mean cronstable will keep retrying forever, this is mostly
useful with a schedule of "@reboot" to restart a long running process when it
has failed.
Retries are in-memory by default: a daemon restart forgets an armed retry. With
a state: section configured, armed retries survive restarts and resume where
they left off; see Durable State
in the wiki.
If the cron job is expected to fail sometimes, you may wish to report only in
the case the cron job ultimately fails after all retries and we give up on it.
For that situation, you can use the onPermanentFailure option:
Failure hooks only see runs that happened. An sla: block watches for the
runs that did not: each job can declare up to three independent thresholds,
evaluated once per minute by an in-process monitor, with a dedicated
onLate reporting hook that fires once when a threshold is breached and
takes the same report block (mail, Sentry, shell, webhook) as onFailure:
- name: nightly-etl
command: python -m etl.run
schedule: "0 4 * * *"
sla:
maxTimeSinceSuccessSeconds: 129600 # no success for 36h
lateAfterSeconds: 900 # a due slot not started within 15min
maxRuntimeSeconds: 7200 # a run still going after 2h
onLate:
report:
webhook:
url:
fromEnvVar: SLACK_WEBHOOK_URL
Breaches latch: one report per breach, not one per minute, with a recovery
log line and no report when the check clears. maxRuntimeSeconds observes
and never kills (use executionTimeout to enforce a limit); paused and
disabled jobs are excused; under leader election only the job's owning node
evaluates, so one breach pages once. Breaches surface as an OVERDUE badge in
both dashboards, an sla object on GET /jobs, and
cronstable_job_late{job_name, check} /
cronstable_job_sla_breaches_total{job_name, check} in the metrics. The
monitor runs inside the daemon and cannot report its own death, so pair it
with an external Prometheus staleness alert. See the
Late-Run Detection
wiki page.
Concurrency
Sometimes it may happen that a cron job takes so long to execute that when the moment its next scheduled execution is reached a previous instance may still be running. How cronstable handles this situation is controlled by the option concurrencyPolicy, which takes one of the following values:
Allow
: allows concurrently running jobs (default)
Forbid
: forbids concurrent runs, skipping next run if previous hasn't finished yet
Replace
: cancels currently running job and replaces it with a new one
Execution timeout
If you have a cron job that may possibly hang sometimes, you can instruct cronstable
to terminate the process after N seconds if it's still running by then, via the
executionTimeout option. For example, the following cron job takes 2
seconds to complete, cronstable will terminate it after 1 second:
When terminating a job, it is always a good idea to give that job process some
time to terminate properly. For example, it may have opened a file, and even if
you tell it to shutdown, the process may need a few seconds to flush buffers and
avoid losing data.
On the other hand, there are times when programs are buggy and simply get stuck,
refusing to terminate nicely no matter what. For this reason, cronstable always
checks if a process exited some time after being asked to do so. If it hasn't,
it tries to forcefully kill the process. The option killTimeout option
indicates how many seconds to wait for the process to gracefully terminate
before killing it more forcefully. In Unix systems, we first send a SIGTERM,
but if the process doesn't exit after killTimeout seconds (30 by default)
then we send SIGKILL. For example, this cron job ignores SIGTERM, and so cronstable
will send it a SIGKILL after half a second:
You can request that Cronstable change to another user and/or group for a specific
cron job. The field user indicates the user (uid or userame) under which
the subprocess must be executed. The field group (gid or group name)
indicates the group id. If only user is given, the group defaults to the
main group of that user. Example:
Naturally, cronstable must be running as root in order to have permissions to
change to another user.
This feature is POSIX-only (it relies on setuid/setgid). On Windows, a job
with user or group set is rejected with a configuration error; see
Running on Windows.
Remote web/HTTP interface
If you wish to remotely control cronstable, you can optionally enable an HTTP REST
interface, with the following configuration (example):
With the web interface enabled, cronstable also serves the web dashboard
(showcased near the top of this README) at the root path (/) of any http://
listener. Open in the example above, and see the
full dashboard tour in
the wiki. It is a single self-contained page (no build step or external assets)
that watches every job's status, tails its logs live, runs or cancels jobs on
demand, and shows run history and a plain-English schedule preview. Logs are
shown for the streams a job captures, so enable captureStdout /
captureStderr on jobs whose output you want to watch here.
The run history and logs are kept in memory only. Nothing is written to
disk, so the dashboard does not change cronstable's read-only-filesystem
deployment story. History resets when cronstable restarts.
If you have enabled bearer-token authentication for the web API (the
web.authToken option), the dashboard page itself loads without a token, then
prompts you for one and stores it only in that browser tab; every data request
it makes is authenticated with that token.
To disable the dashboard and expose only the REST API, set ui: false:
web:
listen:
- http://127.0.0.1:8080
ui: false
Serving the API over TLS
web.listen also accepts https:// addresses, served from a web.tls block.
Each entry keeps its own transport, so one runner can serve the same API and
dashboard in plaintext on loopback and over TLS on a routable interface;
unix:// listeners are always plaintext, where the socket's own permissions
(socketMode) are the access control.
web:
listen:
- http://127.0.0.1:8080 # loopback, plaintext
- https://0.0.0.0:8443 # served with the material below
tls:
cert: /etc/cronstable/web.pem
key: /etc/cronstable/web.key
clientCa: /etc/cronstable/callers-ca.pem # optional: require client certs
clientCa turns the listener into mutual TLS: a caller that presents no
certificate, or one signed by another CA, is refused at the handshake. That CA
file is therefore the caller allowlist (a server does no hostname verification,
so any certificate that CA ever signed is accepted), so point it at a CA minted
for this purpose rather than a shared organisational one. Because mTLS does
authenticate the caller, mcp.enabled on such a listener no longer requires
web.authToken.
Web certificates rotate in place. cronstable fingerprints the files and
restarts the listener when they change on disk, so a cert-manager, Vault or
Kubernetes secret refresh needs no daemon restart. That restart drops the
connections open at the time, including the SSE log streams the dashboard and
the TUI hold, which reconnect on their own; a live log tail blips.
The job-facing state API gets the same block: state.jobApi.listen accepts an
https:// URL served from state.jobApi.tls.cert and .key, and
state.jobApi.tls.ca is the trust anchor handed to every job as
CRONSTABLE_STATE_CACERT, so cronstable state|cursor|lock|artifact inside a
job can verify an internally-issued certificate. That listener builds its
context once at startup, so rotating its certificate takes a daemon restart.
The clients take the same four options, as flags or environment variables
(cronstable tui, cronstable mcp):
Flag
Environment
Effect
--cacert PATH
CRONSTABLE_WEB_CACERT
verify against this CA instead of the system trust store
--client-cert PATH
CRONSTABLE_WEB_CLIENT_CERT
certificate to present to a clientCa listener
--client-key PATH
CRONSTABLE_WEB_CLIENT_KEY
private key for --client-cert
--insecure
CRONSTABLE_WEB_INSECURE
skip verification; warns, because the bearer token is still sent
Client hostname verification is on, so the certificate has to cover the name
actually dialled: https://127.0.0.1:8443 needs an IP SAN for 127.0.0.1, and
https://localhost:8443 needs a DNS SAN.
This is a teaser: issuing the certificates, the mTLS trust model and how it
interacts with web.authToken, the rotation mechanics and what they do not
cover, the job state API's trust anchor, and the full client flag surface are
covered in depth in the
Listener TLS
guide in the wiki.
Now you have the following options to control it (using HTTPie as example):
Get the version of cronstable
$ http get http://127.0.0.1:8080/version
HTTP/1.1 200 OK
Content-Length: 22
Content-Type: text/plain; charset=utf-8
Date: Sun, 03 Nov 2019 19:48:15 GMT
Server: Python/3.7 aiohttp/3.6.2
0.10.0b3.dev7+g45bc4ce
Get the status of cron jobs
$ http get http://127.0.0.1:8080/status
HTTP/1.1 200 OK
Content-Length: 104
Content-Type: text/plain; charset=utf-8
Date: Sun, 03 Nov 2019 19:44:45 GMT
Server: Python/3.7 aiohttp/3.6.2
test-01: scheduled (in 14 seconds)
test-02: scheduled (in 74 seconds)
test-03: scheduled (in 14 seconds)
Sometimes it's useful to start a cron job right now, even if it's not
scheduled to run yet, for example for testing:
$ http post http://127.0.0.1:8080/jobs/test-02/start
HTTP/1.1 200 OK
Content-Length: 0
Content-Type: application/octet-stream
Date: Sun, 03 Nov 2019 19:50:20 GMT
Server: Python/3.7 aiohttp/3.6.2
Cancel a running job
POST /jobs/{name}/cancel terminates any currently-running instances of a job
(the same graceful SIGTERM-then-SIGKILL sequence, honoring the job's
killTimeout, that cronstable uses elsewhere). A job cancelled this way is recorded
in its history with the outcome cancelled; unlike a failure it is not
reported and does not trigger retries. It returns 409 Conflict if the job
is not currently running, and 404 Not Found for an unknown job.
$ http post http://127.0.0.1:8080/jobs/test-03/cancel
HTTP/1.1 200 OK
Pause and resume a job
POST /jobs/{name}/pause holds a job's scheduled fires until a deadline,
without touching the config: an hour by default, or a durationSeconds /
until window of up to thirty days, plus optional note and by audit
fields. While paused, each due slot is recorded as a skipped run (so the
history shows the deliberate gap), pending retries wait and fire after the
resume, missed-run catch-up owes nothing for the window, and manual start
still works. POST /jobs/{name}/resume ends the pause early; every pause
expires on its own. With a state: store configured the pause survives
restarts and is honored by every node sharing the store. The dashboards
toggle it with one click (the p key), and paused jobs read
cronstable_job_paused 1 in the metrics.
See the Pausing Jobs
wiki page for the full semantics.
Get detailed job info (used by the dashboard)
GET /jobs returns a JSON array describing every job: its schedule and
timezone, whether it is enabled/running, the time until its next scheduled run, a
summary of its most recent finished run (outcome, exit code, start/finish times
and duration), and a compact history of recent outcomes for the trend
sparkline. This is what the web dashboard polls.
GET /jobs/{name}/runs returns the job's retained run history (oldest first,
bounded and in memory only) together with aggregate statistics. Each run carries
the same fields as last_run above; stats summarizes them. The success_rate
is computed over runs that ran to completion (cancellations are excluded).
Returns 404 Not Found for an unknown job.
GET /jobs/{name}/logs is a
Server-Sent Events
stream of a job's captured output: the most recent buffered lines first, then
new lines live as a running job produces them, and finally an end event when
the run finishes. Each line arrives as an event: line whose data is a JSON
object {"stream": "stdout"|"stderr", "line": "..."}. Only output from the
streams a job captures (captureStdout / captureStderr) is available here.
$ curl -N http://127.0.0.1:8080/jobs/test-01/logs
event: line
data: {"stream": "stdout", "line": "foobar"}
event: end
data: {}
Subscribe to the schedule as an iCal calendar
GET /calendar.ics serves the fleet's upcoming fires as a standard
iCalendar feed (one event per fire, enumerated by the scheduler's own
engine in each job's timezone), and GET /jobs/{name}/calendar.ics serves
one job's. Subscribe a calendar app to the URL and overnight maintenance
jobs show up on the on-call engineer's week; the dashboard's ◫ week
button draws the same data as a seven-day calendar in the browser.
?days= widens the window (default 14, max 60) and ?per_job= caps
events per job. Event descriptions carry the schedule, its plain-English
reading, and the typical runtime, never the command line. With
web.authToken set, the .ics paths (only) also accept the token as
?token=, since calendar clients cannot send a bearer header. See
Calendar Export.
The job-set id is an order-independent fingerprint of the set of jobs a
cronstable instance is running. Two instances produce the same id if and only if
they hold the same set of jobs, which lets several replicas deployed from the
same configuration confirm they are running the same thing, or detect that one
has drifted from the others.
The id is taken over the effective (post-merge) configuration of every job,
which gives it some useful properties:
it is independent of job order, and of whether a setting was written
inline on each job or hoisted into a defaults block;
equivalent schedule spellings match: the minute:/hour: object form
fingerprints the same as the equivalent five-field crontab string;
it covers every behavior-affecting field (command, schedule, shell, the
names of environment variables, capture flags, failsWhen,
retry/reporting policy, timezone, enabled, and so on), so any meaningful
change to a job changes the id;
user/group are fingerprinted as configured (e.g. www-data), not as
the resolved numeric uid/gid, which can differ host to host;
secret/value material is never embedded: inline reporting secrets
(Sentry DSN, mail password, webhook URL and header values) are redacted,
and only the names of
environment variables are hashed, not their values (env commonly holds
secrets, and a per-host value, e.g. from env_file, would otherwise make
identical configs differ across hosts). The id is safe to log and serve, and
rotating a secret or changing an env value does not change it.
Because it reflects effective config, it also reflects platform-dependent
defaults (the default shell is /bin/sh on POSIX, cmd.exe on Windows), so
compare instances running on the same platform, which replicas are. The scheme
is versioned with a v1: prefix; ids are only comparable within a scheme
version.
It is available three ways:
CLI: print it and exit (handy in scripts / health checks):
HTTP: GET /job-set-id on the web interface
(also application/json), and shown in the dashboard header:
$ http get http://127.0.0.1:8080/job-set-id
v1:b834d7565aee0da50cd017f666651a5ba3b2e6b161daf0cb6e430f23f51ce90b
$ http get http://127.0.0.1:8080/job-set-id Accept:application/json
{"job_set_id": "v1:b834d7…51ce90b", "jobs": 3}
Logs: it is logged once at startup, and again whenever a config reload
changes it.
Clustering and leader election
By default cronstable runs as a single instance and every replica runs every job.
An optional cluster section lets several replicas coordinate: each node serves
a small GET /peer endpoint over mutual TLS and periodically polls its
configured peers, comparing job-set ids so they can confirm they
are running the same set of jobs (cluster peer attestation). Turning on
electLeader promotes that same attestation into a quorum-gated leader
election, so you can run more than one replica from one config without
double-running scheduled jobs:
cluster:
listen: "0.0.0.0:8443" # the mTLS listener for this node
tls:
ca: /etc/cronstable/cluster-ca.pem # trust anchor for peer certificates
cert: /etc/cronstable/this-node.pem # this node's certificate
key: /etc/cronstable/this-node.key
peers:
- host: cronstable-b.internal:8443
- host: cronstable-c.internal:8443
nodeName: cronstable-a # optional; defaults to the system hostname
interval: 30 # optional; seconds per round (default 30)
connectTimeout: 10 # optional; per-peer connect timeout (default 10)
driftAfter: 3 # optional; rounds before "drifted" (default 3)
electLeader: true # observe-only if false (the default)
Each node independently elects, as leader, the lowest nodeName among the
members it currently sees agreeing on the job-set id, but only if that set is a
quorum (a strict majority) of the cluster, so under a clean partition at
most one side leads. This is best-effort (the default gossip backend keeps no
shared state); for a fenced, exactly-once guarantee set
cluster.backend: kubernetes or cluster.backend: etcd
to elect through a coordination.k8s.io/v1Lease or a lease-bound etcd key
instead.
Each job can override the cluster-wide default with a per-job clusterPolicy,
picking its own point on the liveness-vs-duplication trade-off:
clusterPolicy
healthy (quorate)
partitioned / sub-quorum
use for
Leader(default)
leader runs once
nobody runs (skips)
non-idempotent jobs where a duplicate is harmful and an occasional skip is OK (billing, outbound email)
PreferLeader
lowest node runs once
each side's lowest node runs (may double-run)
important and idempotent jobs that should never skip
EveryNode
every node runs
every reachable node runs
genuinely per-node work (local log rotation), or fully idempotent jobs
The current view (members, elected leader, quorum, and any conflicts) is
available at GET /cluster and shown as a panel in the dashboard. This is a
teaser: the full trust model, per-peer status table, quorum math, sizing
guidance, distribution: spread load-balancing, and the fenced lease backends
are all covered in depth in the
Clustering and Leader Election
guide in the wiki. To watch it live, see Try it below.
Includes
You may have a use case where it's convenient to have multiple config files,
and choose at runtime which one to use. In that case, it might be useful if
you can put common definitions (such as defaults for reporting, shell, etc.)
in a separate file, that is included by the other files.
To support this use case, it is possible to ask one config file to include
another one, via the include directive. It takes a list of file names:
those files will be parsed as configuration and merged in with this file.
Example, your main config file could be:
include:
- _inc.yaml
jobs:
- name: my job
...
And your included _inc.yaml file could contain some useful defaults:
Any string value in the config can pull from cronstable's environment with
${VAR}, or ${VAR:-default} for a fallback, so one config file serves many
environments without a wrapper script templating it. Write $$ for a literal
$. Interpolation runs after the file is validated, so it reaches any
string-typed field (a listen address, a state path, a timezone, a webhook URL),
and a ${VAR} that is unset and has no default is a hard configuration error
that names the variable, caught by cronstable --validate-config.
web:
listen:
- "0.0.0.0:${WEB_PORT:-8080}" # port from the environment, default 8080
state:
path: ${STATE_DIR} # required: unset fails --validate-config
jobs:
- name: rollup-${REGION}
command: run-rollup # ${VAR} in a command is left for the shell
schedule:
minute: "0"
timezone: ${TZ:-UTC}
A job's (and reporter's) command and shell are deliberately left untouched,
so their ${VAR} is expanded by the runtime shell against the job's own
environment, not the daemon's; the logging section is likewise left for
Python's logging.config. See
Environment-Variable Interpolation
for the full rules, including how it affects the job-set id.
Custom logging
It's possible to provide a custom logging configuration, via the logging
configuration section. For example, the following configuration displays log lines with
an embedded timestamp for each message.
logging:
# In the format of:
# https://docs.python.org/3/library/logging.config.html#dictionary-schema-details
version: 1
disable_existing_loggers: false
formatters:
simple:
format: '%(asctime)s [%(processName)s/%(threadName)s] %(levelname)s (%(name)s): %(message)s'
datefmt: '%Y-%m-%d %H:%M:%S'
handlers:
console:
class: logging.StreamHandler
level: DEBUG
formatter: simple
stream: ext://sys.stdout
root:
level: INFO
handlers:
- console
Obscure configuration options
enabled: true|false (default true)
It is possible to disable a specific cron job by adding a enabled: false option. Jobs
with enabled: false will simply be skipped, as if they aren't there, apart from
validating the configuration.
jobs:
- name: test-01
enabled: false # this cron job will not run until you change this to `true`
command: echo "foobar"
shell: /bin/bash
schedule: "* * * * *"
Performance
cronstable is built to run on small and old machines, and CI holds it to
that: every commit runs an exhaustive benchmark suite (startup time, schedule
computation for 100,000 jobs, config parsing, DAG planning, durable-state
I/O, memory footprint, about 37 metrics in all) paired against the latest
release on the same runner. A release that regresses a metric past its
declared limit does not ship, and every release page carries a chart and a
full table of the change against the previous release.
Run the suite yourself with python benchmarks/bench.py --quick; see
Performance Benchmarks
for how the comparison and the gate work.
Bug reports, feature ideas, and pull requests are welcome; see
CONTRIBUTING.md for the development setup, how to sign off
your commits (DCO), and
Contributing and Releasing
for how releases work. cronstable is MIT-licensed; see
LICENSING.md for how the repository's licensing is organized.
Trademarks. The MIT License covers the code, not the brand. cronstable™ and
the cronstable logo are trademarks of the cronstable authors; see
TRADEMARKS.md.
cronstable is a fork of yacron (by Gustavo Carneiro), continuing development from version 0.19.