Files
pipelines/docs/ntfy.md
T
alan 890d267c71 run attention: publish to ntfy and surface what needs the operator
Nothing told the operator that a run was waiting for a person or had failed; they
found out by going looking. A run that needs attention now reaches the ntfy they
already run, and the app answers "what needs me now" in one live, ordered surface.

Server (smallest change, reusing the existing publisher):
- internal/server/notify.go adds notifyRun, called from finishRunState (failed)
  and when a run parks on a human (runner.go runLoop). It publishes to the same
  already-configured `system` destination through the unchanged stage.PublishNtfy,
  so there is no second publisher, topic or service and the token handling is the
  stage's own. runAttentionEvent maps only waiting and failed; queued, running,
  succeeded, canceled and interrupted publish nothing.
- No storms: runner.attentionSent records sequence:event per run before publishing,
  so a run that stays waiting is never republished on a re-request or restart, and
  a stage notify policy that already covers the same event marks it first so the
  run-level default is suppressed. The set is dropped when the run is terminal. A
  delivery failure is logged, never fails the run, and is not retried.
- The payload names the run and pipeline and carries a click action
  pipelines://run/<id> (validated: absolute URL, no userinfo, no fragment, no
  credentials). PublishNtfy gained the optional click parameter; existing callers
  pass "".

App:
- AndroidManifest/MainActivity declare the pipelines://run/<id> deep link
  (singleTop); runIdFromDeepLink parses it and PipelinesApp opens that run once,
  clearing the link. It opens a run; it starts nothing.
- ui/Attention.kt is the pure model (status -> Decision/Failed/Finished or
  nothing, ordered by kind then newest-first) and ui/components/AttentionSurface.kt
  is the floating indicator plus bottom sheet. It is derived from the same run
  events the server already pushes, so an entry appears with no refresh and clears
  when resolved; a finished run clears once looked at, and any entry can be
  dismissed, persisted in SettingsRepository. The existing destinations, pill and
  bottom strip are untouched and no competing navigation surface is added.

Checked before committing:
- Packaged authoring skill references: internal/author/skill/references/
  notifications.md documents the automatic run-attention notifications and the
  app surface. The only packaged "template" carrying notify is that reference; no
  pipeline definition template exists to update and install.go's managed file list
  is unchanged (notifications.md already existed).
- Repository guidelines: docs/ntfy.md gains an "Automatic run-attention
  notifications" section and no longer claims "No block means no notifications";
  the top-level README phone-notifications note now says a waiting/failed run
  notifies automatically; android/README.md gains the dated feature section and
  supersedes the previous section's note that the attention surface was absent.
- docker.md, install docs and the CLI/TUI docs describe mechanisms this change
  does not alter; left as they are.

Tests: go test ./internal/server/... ./internal/stage/... ./internal/author/...
all ok; notify_test.go covers the event mapping and the real HTTP delivery
(destination, topic, click, no duplicate). Android :app:testDebugUnitTest ->
BUILD SUCCESSFUL, 58 suites / 531 tests / 0 failures (new AttentionTest, 13 cases,
covering ordering, appearing from a push, clearing on resolution, dismissal, the
empty case, persistence and the deep link). Device screenshots and the phone's own
notification are visual review's per the brief's device policy.

Scope: android/ plus the smallest server change; no new dependency, no run started
from the app, no installer or Docker change.
2026-10-17 16:11:57 +01:00

216 lines
11 KiB
Markdown

# Self-hosted ntfy delivery
Every stage can have an optional `notify` property block. Notifications go to
your own ntfy instance and can attach Markdown and output files. It does not ask
an LLM to handle credentials or run curl. The full Markdown remains in the run
audit as well as the ntfy attachment. A separate HTML report viewer is not included.
## Per-stage notifications (recommended)
### One terminal-path alert without adding stages
For a sequential workflow such as `research → report`, where intermediate
blocked/error routes go straight to `#failure`, add `on=failure` notification
hooks to intermediate stages and `on=success,failure` to the final stage.
Keep the graph and output contracts unchanged; omit `artifacts` for event-only
alerts. This avoids intermediate success alerts while covering early stage
errors and the final success/error result. Putting a hook only on the last stage
misses earlier failures; putting success hooks on every stage sends progress alerts.
These are stage hooks, not a universal run-finalizer: they run before the final
status is saved, and do not guarantee notifications for cancellation, crashes,
persistence failures or runtime step/visit limits. Recovery routes and loops can
notify before termination or more than once. An explicit native publisher stage
is not necessary for ordinary completion alerts and cannot reliably consume the
required `title`/`summary`/`markdown` after an upstream execution error.
The installed authoring skill includes this recipe and native-delivery details in
`references/notifications.md`; authoring does not require a checkout of this project.
````markdown
# stage compose
```notify
on=success,failure
artifacts=markdown,files
```
...stage output, prompt, tools, edges, and configs...
````
- `on` is `success`, `failure`, or `success,failure`. A failure is an execution
error, the `error` outcome, or a route to `#failure`; any other completed stage
is a success. This reports the stage's result, not the final run's result.
- All notification hooks use one system-wide server-to-phone connection. Stages
do not choose topics or addresses. Configure `PIPELINES_NTFY_URL` (publishing
topic URL), `PIPELINES_NTFY_PUBLIC_URL` (phone-facing server base URL), and
`PIPELINES_NTFY_TOKEN` or `PIPELINES_NTFY_TOKEN_FILE` once on the server.
The topic is `pipelines` in the provided setup. No block means no per-stage
notifications; a run that needs a person is still announced automatically (see
below). Legacy `destination` lines remain parseable for immutable-history compatibility
but no longer select a connection for notification hooks. Explicit native
`type=ntfy` stages retain their separate destination configuration.
- `artifacts=markdown` sends the string in `output.markdown` as `digest.md`.
- `artifacts=files` sends paths listed in `output.artifacts`, e.g.
`["reports/result.txt", "screenshots/chart.png"]`. Model stages with this policy
are instructed to list files they actually created/modified. No workspace scan
or file discovery occurs; an invalid model response may leave no artifact list.
- Both artifact kinds can be selected. Each file gets a separate notification
(ntfy supports one attachment per notification). If none were produced, an
event-only notification is still sent, including on failure.
- Files must be workspace-relative regular files, at most 1 MiB each and 10 files
per stage. Traversal, absolute paths, symlinks, hidden paths, known database/key
paths, `secrets/`, and `node_modules/` are rejected. These guards are not a
content scanner/DLP system: only list files you intend to share.
- Notifications have a 30-second total budget, with no automatic retries. Their
receipts/failures appear in the selected run stage's **Metadata**, separately
from output. They do not change routing or convert a successful stage into a
failed run. A crash can leave a `sending` receipt with uncertain delivery;
notifications are not replayed during recovery. Tool access follows the stage's
declared tools; notification settings do not grant additional tools.
## Automatic run-attention notifications
Independently of any `notify` block, the server publishes one run-level
notification to the `system` destination when a run needs a person: when it enters
`waiting` (a human stage asked for a decision) and when it fails. Nothing else
notifies — queued, running, succeeded, canceled and interrupted runs are silent.
The payload names the run and its pipeline and carries a `click` action
`pipelines://run/<id>` that opens that run in the Android app.
A genuine event notifies once: a run that stays `waiting` is not republished on a
read or restart, and a stage `notify` policy that already covers the same event
(`on=waiting`, or a failure hook) suppresses the run-level default, so one event
means one notification. A failed delivery is logged, never fails the run, and is
not retried. The Android app shows the same information in one attention surface
ordered by importance — a run waiting for a decision, then a failed run, then a
run that finished since the operator last looked — derived from the run events the
server already pushes, so it needs no refresh and starts nothing.
## Explicit delivery stage (optional)
The preceding stage must output three nonempty UTF-8 strings: `title`, `summary`,
and `markdown`. Title/summary must be single-line (256/2048 byte limits); Markdown
is limited to 1 MiB. Native delivery stages do not accept model prompts/tools or
inline credentials. They require `destination` and a `success` edge:
````markdown
# stage notify
```configs
type=ntfy
destination=system
```
```edges
success -> #success
error -> #failure
```
````
Configure the **pipeline server process** outside the pipeline file:
```sh
export PIPELINES_NTFY_URL='https://ntfy.your-domain.example/pipelines'
# Set PIPELINES_NTFY_TOKEN_FILE to the existing publishing token file.
go run ./cmd/pipelines-server
```
The URL includes one topic, without credentials, query parameters, or a fragment.
`destination=system` uses the same connection as hooks. Other destinations use
the named pattern: `destination=reports` reads
`PIPELINES_NTFY_REPORTS_URL` and `PIPELINES_NTFY_REPORTS_TOKEN` (or `_TOKEN_FILE`
pointing to a secret file). Set only one token source. Destination names
use lowercase letters, digits, and underscores. A token is optional only when
the configured server permits anonymous publishing; private deployments need it.
Never put tokens in a prompt, committed file, or chat. Environment credentials
are inherited by subprocesses: do not grant untrusted stages shell/code-execution
access. Prefer a dedicated publishing user with write-only access to this topic.
## Local server
The root `compose.yaml` runs **both the pipeline server and ntfy** with persistent
volumes, internal service networking, and a private topic. See the
[Docker + Android setup](docker.md) for bootstrap, credentials, and timezone
detection. The previous ntfy-only Compose file has been replaced. Isolated real
Docker health and publishing checks have passed; phone reception and the user's
remote Tailscale connectivity remain **not tested** locally.
Use a TLS reverse proxy or a private VPN (e.g. Tailscale). A phone cannot reach
your computer's `127.0.0.1`; that address is only for local publishing/proxying.
To bind the container to a particular LAN/VPN interface, explicitly set
`NTFY_BIND_ADDRESS` to that interface IP. Do not expose an unauthenticated server,
or send access tokens over public plain HTTP. Set `NTFY_BASE_URL` to the same
address your phone uses so attachment URLs resolve correctly. If the pipeline
server is another container, localhost is that container, not the ntfy host.
## Phone
### Tailscale address times out while LAN works
Use `http://<server-Tailscale-IP>:8090`, not port 80; enable Tailscale on both the
server and phone. The default Compose mapping binds ntfy to `0.0.0.0:8090`.
Binding to all interfaces does not bypass tailnet grants/ACLs, host firewalls,
or Docker forwarding rules. A timeout alone cannot identify which is blocking.
Do not disable firewalls or publicly forward this port as a workaround.
On the server, these are read-only diagnostics (the remote checks are not tested
from this development machine):
```sh
tailscale status
docker compose port ntfy 80
curl --connect-timeout 5 http://127.0.0.1:8090/v1/health
curl --connect-timeout 5 "http://$(tailscale ip -4):8090/v1/health"
```
If the host health checks succeed but the phone times out, check that the phone
is in the intended tailnet and that tailnet access rules/firewalls allow this
phone to reach server TCP 8090. Once reachable, log in as `phone-reader` for the
Tailscale server entry, separately from the LAN entry. Set `NTFY_BASE_URL` to a
phone-reachable address if attachment URLs currently point to a LAN-only address;
that setting is not a substitute for fixing a TCP timeout.
Install the ntfy app, add your own server, authenticate with the reader account,
and subscribe to `anime`. Ensure notifications and required background operation
are allowed. Outside your home network, both topic subscriptions and attachment
downloads need access to your server through VPN or public HTTPS.
For **iOS instant notifications**, ntfy's current documentation requires
`NTFY_UPSTREAM_BASE_URL=https://ntfy.sh` (or an APNS-connected upstream with a
custom app). The upstream sends wake-up requests; message content is fetched
from your instance. This is not completely independent of third-party push
infrastructure. Without upstream forwarding, delivery can be substantially
delayed. Phone delivery/background behavior has **not been tested** here.
## Execution and failure behavior
- Publishing is bounded to 20 seconds and does not follow redirects.
- No automatic publish retry: timeouts or malformed acknowledgments may mean the
notification was delivered but its acknowledgment was lost. Inspect ntfy
before retrying an ambiguous failure.
- For the optional native `type=ntfy` stage, a confirmed response adds `delivery` metadata (destination name, message ID,
attachment URL, filename) to the existing output. Failures take the error route
and are recorded in normal run/step audits, without remote response bodies or
credentials in diagnostics.
- Repeating the same run request key does not publish again. A new run or a graph
loop visiting delivery again can send another notification; content-level
"only new items" deduplication is not yet implemented. For the initial digest,
keep notification last with no feedback edge.
- Markdown reports remain available in pipeline run history even after ntfy expires its
attachment. Configure ntfy retention for your needs; its attachment directory
must be dedicated to ntfy, not shared with other files.
File-attachment bytes are stored by ntfy, not permanently archived in SQLite;
run metadata stores the delivered filename/URL, and workspace files may later
change. Download attachments before your ntfy retention expires.
Bundled starters do not require notification or schedule setup. Add a `notify`
block to the stages of your own workflow using the example above; configure the
system phone connection first. Adding a schedule activates it on publication,
so review it explicitly rather than copying a scheduled niche example.
References: [publishing](https://docs.ntfy.sh/publish/),
[server configuration](https://docs.ntfy.sh/config/),
[iOS self-hosting](https://docs.ntfy.sh/config/#ios-instant-notifications).