Cron Job Best Practices for Production Systems

A cron expression is the easy 5% of running a scheduled job. The hard 95% is making sure the job actually runs, doesn't run twice, tells you when it fails, and doesn't quietly corrupt data at 3 AM while everyone is asleep. These are the practices we'd expect in any production cron setup, whether it's plain Linux cron, a Node.js worker, or a managed scheduler like AWS EventBridge.

1. Make every job idempotent

Idempotent means running the job twice with the same inputs produces the same end state, with no duplicate side effects. This matters because cron jobs get retried after crashes, sometimes double-fire due to DST changes or scheduler bugs, and get re-run manually during incident response. Design for it up front:

• Use INSERT ... ON CONFLICT DO NOTHING / UPSERT instead of plain INSERT for database writes.
• Include a unique idempotency key (e.g. date + job name) so a duplicate run is detected and skipped.
• For file exports, write to a temp file and atomically rename, rather than appending.
• For "send email" style jobs, record a sent-flag before sending, or use a provider-side dedupe key.

2. Lock the job so it can't overlap itself

If a job scheduled every 5 minutes occasionally takes 8 minutes (slow query, network hiccup, large batch), the next scheduled run will start while the first is still going. Two instances writing to the same resource is a classic source of subtle bugs. On Linux, wrap the job with flock so a second invocation exits instantly instead of running concurrently:

# crontab entry using flock to prevent overlap
*/5 * * * * /usr/bin/flock -n /tmp/sync-orders.lock /opt/scripts/sync-orders.sh

# -n = non-blocking: if the lock is already held, exit immediately
# instead of waiting for the previous run to finish

In Node.js or Python schedulers without a system-level lock, use a database advisory lock or a Redis key with a TTL slightly longer than the expected job duration.

3. Log with enough context to debug after the fact

By default, cron discards a job's output unless you redirect it. At minimum, capture stdout and stderr to a file with a timestamp, and rotate logs so they don't fill the disk:

0 2 * * * /opt/scripts/backup.sh >> /var/log/backup-`date +\%Y\%m\%d`.log 2>&1

# Better: log structured JSON lines to stdout and ship them to a central
# log system (CloudWatch, Datadog, ELK) so you can search and alert on them.

Include a start log line, an end log line with duration and row/record counts, and the job's exit code. This alone answers "did it run, and did it work" without SSH-ing into a box.

4. Alert on failure, not just log it

A log file nobody reads is not monitoring. Choose one of these patterns based on how critical the job is:

• Exit-code alerting — wrap the command so a non-zero exit triggers a Slack/email/PagerDuty webhook.
• Dead man's switch / healthcheck ping — the job pings a monitoring URL (Healthchecks.io, Cronitor, BetterUptime) on success; if the ping doesn't arrive within the expected window, you get alerted even if the job never ran at all.
• Threshold alerting — for data jobs, alert if the processed row count is unexpectedly zero or wildly different from the historical average.

5. Set explicit timeouts and clean up on failure

A hung job holds its lock forever and silently blocks every future run. Always run long jobs under a timeout, and make sure the lock/temp-file cleanup happens in a finally-equivalent block, not just on the happy path:

# kill the job if it runs longer than 10 minutes
*/15 * * * * /usr/bin/timeout 600 /opt/scripts/reconcile.sh || \
  curl -fsS -m 10 https://hc-ping.com/your-uuid/fail

6. Keep secrets and environment out of the crontab

Cron runs with a minimal environment — no shell profile, often no PATH beyond /usr/bin:/bin. Don't hardcode secrets in the crontab where they end up in crontab -l output and shell history. Instead, source a restricted-permission env file at the top of the script, or read secrets from a vault / secrets manager at runtime.

Frequently Asked Questions