fix: persist and retry run alerts and honor notification modes

This commit is contained in:
2026-10-20 15:42:21 +01:00
parent e4f48ecbb1
commit 016fcb552a
20 changed files with 697 additions and 41 deletions
+15
View File
@@ -574,6 +574,21 @@ Everything else is free-form and preserved under the step's `markdown` output.
the run. Artifacts are served from `/artifacts/<run>/<step>/<name>`; shared files
appear with a `public/` prefix.
### Run notification modes and receipts
`--notify auto|on|off` records `notify_mode` on the run (also returned in
summaries and `run_event` frames). Auto retains stage policies and run-level
waiting/failure defaults. On also alerts terminal success without requiring a
stage hook. Off suppresses stage, run-level system and browser tray alerts.
`inspect --json` / `get_run` includes top-level `notifications` receipts with
`sequence`, `event`, `channel`, `status`, `attempts`, and optional `diagnostic`
and `message_id`. Status is pending, sending, sent, failed, or covered (already
published by the stage). Run-level system alerts persist and retry at most three
times, recovering pending work after restart. A publisher acknowledgement is not
proof of reception; a crash before saving it can duplicate a message. See
[`notes/audit-notifications.md`](notes/audit-notifications.md) for limits.
## Investigating runs with a configured harness
On the runs list or inspector, press `D` or use `:debug`; CLI equivalent:
+36
View File
@@ -0,0 +1,36 @@
# Run notification audit fixes
`notify_mode=auto` retains existing behavior: stage policies plus run-level
waiting/failure defaults. `on` also alerts terminal success, even with no stage
notification hook. `off` suppresses stage, run-level system and browser tray
notifications. Needs you remains a status/action surface, not a delivery channel.
Summaries and pushed events carry the mode; the browser seeds and folds it.
`run_notifications` is an additive, idempotent SQLite migration. Each
run/sequence/event/channel has one durable outbox row. Its status is `pending`,
`sending`, `sent`, `failed`, or `covered` (a stage already published this event).
The worker polls once per second, uses an atomic 60-second sending lease and
allows three total attempts with 5/10-second retry delays. Pending and expired
leases survive restart. A final expired sending lease becomes failed with an
unknown-acknowledgement diagnostic. Existing retention deletes associated rows.
`get_run` exposes run-level `notifications` receipts (sequence, event, channel,
status, attempts, diagnostic, message_id), separate from step receipts.
Successful stage publications cover the corresponding waiting/failure alert;
a successful stage targeting `#success` also covers final success. Failed stage
publication does not swallow the run-level fallback/retry. Stage artifact
publications themselves are not made retryable by this change.
Publisher acknowledgement is **not device reception**. A crash after publish
but before the acknowledgement is saved can duplicate a notification. The
transition and enqueue are separate writes: a crash between them can miss an
alert; this is bounded retry of enqueued alerts, not guaranteed delivery.
Browser delivery remains page-session best effort and requires permission plus
HTTPS/localhost; successful runs do not become Needs you entries. The older L5
parity description is attention-only for auto; on now adds success.
Tests use temporary databases and fake HTTP publishers only, covering modes,
no-hook success, stage dedupe, failed publisher retry/exhaustion, restart/lease
recovery, audit persistence, and browser off/on. No live notifications, deployed
database/configuration, service restart, binary installation or OS reception
verification was part of this work. No new UI captures were taken.