Which of your automated jobs are quietly broken before you leave for a week?

Hypothesis

A fleet of scheduled jobs looks healthy right up until the day nobody is watching. Before an unattended week with only phone reach, an agent should be able to read every timer, cron, container, and background daemon across the whole estate and tell you exactly which ones will fail silently while you are gone.

What we built

A read-only audit lane that walked the entire scheduled fleet in one pass: every custom systemd timer and long-running daemon on the server, the server-side agent cron jobs, the Docker containers, and the laptop's launchd agents. For each job it recorded the schedule, the last run, the last result, and the one thing that matters most before an unattended week, how a failure would actually reach a phone. Nothing was changed during the read; the output was a single ranked list of what to fix.

Learnings

• The estate was in better shape than feared, but the audit found two jobs that would have bitten during the week. One weekly scan was misconfigured so its runtime environment variable was silently truncated, killing the process on every run, and it was set to run twice more during the unattended stretch. One web service was running now but marked disabled, so any maintenance reboot would have taken it down for the whole week with no alert.
• The sharpest question to ask of any automated job is not "is it running" but "if it dies, who finds out". A surprising number of daemons had no failure path to Telegram at all, which means a crash-loop to their restart limit would be completely invisible.
• Machine jobs on the laptop only run while it is awake and logged in. Nothing can page you about a job that never starts, so an unattended week means either keep the machine awake or accept the gap and know it is there.
• Both critical fixes were applied and verified live: the truncated environment variable was quoted and a manual run came back clean, and the disabled service was enabled with a failure alert wired in. The remaining lower-severity findings, mostly missing failure alerts, are queued for a supervised deploy day rather than touched unattended.

Log

• 2026-07-15 — Full fleet audit run across server timers, daemons, agent crons, containers, and laptop launchd. Two critical issues found and fixed live; the rest ranked and queued for a supervised deploy day.