Scheduled Tasks That Fail Loudly

An unattended job that fails quietly is worse than no job, because it leaves a belief behind.

Prerequisites

Anything that runs unattended: a nightly sync, a weekly draft, a monitor. A channel where its output lands and a human occasionally looks.

Steps

1. Decide what the task may do, before scheduling it. An unattended run has no operator to ask, so its authority must be settled in advance. The working rule: scheduled runs may read and may write reversibly; anything irreversible produces a proposal, not an action.

2. Make success visible, not just failure. The instinct is to alert only on error, which is exactly backwards for unattended work: a task that has stopped running produces no errors either. Every run reports:

sync ok — 412 rows, 3 sources, 0 conflicts, 2026-08-14 03:10

Now absence of a report is itself the alarm, and it is an alarm that fires for the failure mode that actually happens.

3. Report what was examined, not only what was found. A run that found nothing must say what it looked at. 0 problems and 0 problems across 3 sources, 412 rows are different claims, and only the second can be evaluated.

4. Never swallow errors. A failed step is reported with its message, not summarised into a status. An unattended task that catches an exception and continues produces output that looks identical to a clean run.

5. Write to a durable location, not only to a notification. Notifications are missed and expire. A log file that accumulates lets someone reconstruct when a failure started, which is usually the first question asked.

6. Give it a kill condition. A task that has failed on 3 consecutive runs stops trying and escalates. Infinite retry against a broken dependency produces noise that trains everyone to ignore the channel.

7. Separate a run that found nothing from a run that could not look. These are the two outcomes that look identical in almost every logging setup, and the difference is the whole value of the report. checked 3 sources, 0 findings and could not reach 2 of 3 sources are opposite states that both summarise to nothing to report.

8. Put the schedule where the code is. A task defined in one place and scheduled in another drifts: the code gets updated, the schedule keeps calling an old entry point, and nothing errors because the old one still runs. Keeping the schedule definition beside the task makes the coupling visible when either changes.

Verify

Break it on purpose. Remove a permission, rename an input, and confirm the failure is visible within one cycle without anyone looking for it.

Then check the harder case: make the task not run at all — disable the schedule — and see how long before that is noticed. If the answer is never, step 2 is not implemented, whatever the code says.

Troubleshooting

The channel is full of noise and nobody reads it. Success reports are too verbose. One line per run, structured, with detail in the log — not the reverse.

A task ran for weeks doing nothing useful. It reported success against a check that could not see the failure mode that occurred. Make it state its scope in its own output.

The task exists but has never been scheduled. More common than it sounds, and the reason a capability with no caller is not a capability. Confirm the schedule exists by observing a run, not by reading the config.

[Scheduled Tasks That Fail Loudly](https://stillvalid.dev/playbooks/scheduled-tasks-that-fail-loudly) — stillvalid, SV-7140 (playbook, verified 2026-08-14)