6ca415c134
Chat hits sort by the chat's last use whatever field matched, and run, step, field, artifact and pipeline hits sort by their own date, before the result cap applies so the cap drops the oldest. Each kind's row scan also walks newest first. Score still reports the match tier. The app sorts both groups newest first as well.
1091 lines
67 KiB
Markdown
1091 lines
67 KiB
Markdown
# CLI and agent authoring
|
||
|
||
In the TUI, publishing is labelled **Create pipeline** for a new definition and
|
||
**Update pipeline** for a new immutable version. The CLI `submit` command and
|
||
`:submit` shortcut remain compatible. Templates offer description/graph panes:
|
||
Tab expands the focused pane, `n/p` selects a template, and `hjkl` pans the focused
|
||
graph. Create opens one confirmation with an editable, prefilled template name;
|
||
Enter creates the pipeline without a separate naming or editor step. Duplicate
|
||
names are rejected rather than updating an existing pipeline.
|
||
|
||
`pipelines help` lists commands; `pipelines help edit` (or `edit --help`) shows
|
||
command-specific flags and examples. Installed commands work from any directory.
|
||
The installed command is a compiled Go binary. Optional autostart uses the OS user
|
||
service manager directly. Legacy Python-managed installations require an explicit
|
||
`PIPELINES_PREFIX`; no configuration is inferred from the executable's location.
|
||
For source development, use `go run ./cmd/pipelines help`; start the native server
|
||
separately. Set `PIPELINES_URL` or pass `--url` to select another server.
|
||
|
||
## Dashboard and status
|
||
|
||
`pipelines --url ws://HOST:PORT/connect ping --json` returns `pong` with the
|
||
running server's compiled `build.version`, full Git `build.commit`, and
|
||
`build.dirty`. Uncommitted builds have a `-dirty` suffix. Missing metadata is
|
||
explicit (`commit: "unknown"`, `dirty: null`); older servers omit `build`.
|
||
This identifies the server binary, not the CLI or files mounted into the container.
|
||
Plain `ping` also prints the server version/commit and, beside the local client
|
||
build, the client/server relationship. `pipelines version --json`
|
||
(or `pipelines --version`) identifies the local client without connecting to a
|
||
server. Include both identities when reporting a problem; they can differ.
|
||
|
||
`ping`, `connection`, `status` and every server command now check that relationship.
|
||
The server stamps its `build` and integer `protocol` onto each response (the
|
||
unsolicited welcome envelope too). The client compares this build against the
|
||
server: a same-protocol build mismatch prints exactly one `warning:` on stderr
|
||
(stdout JSON stays parseable) and a server speaking a newer protocol fails the
|
||
command non-zero before it acts on the frame. If a response carries a top-level
|
||
field this client does not model — for example an `artifacts` field from a newer
|
||
server — the client prints a one-time `server sent unrecognized top-level field(s)
|
||
...` notice instead of silently appearing to have no data. Rebuild or reinstall the
|
||
CLI so it matches the server; the installer also warns when a different `pipelines`
|
||
resolves earlier on PATH than the freshly installed binary.
|
||
|
||
|
||
Home's `r` opens **all** runs, not only failures or the newest 50. Inside a pipeline,
|
||
`r` and `:runs` show only that pipeline's runs across its versions. In any run list,
|
||
press `/` for fuzzy search, type to filter, and Enter to return to the results.
|
||
Search includes pipeline name, version, status, outcome, stage and diagnostic;
|
||
`#123` searches run-ID prefixes. Esc while searching clears the filter without
|
||
closing the list. The filter survives inspection/back and live list refreshes.
|
||
Home's `/` instead opens the omni popup, which searches pipelines, versions, runs,
|
||
steps/visits, artifact names, and the text of stage inputs, outputs and evidence.
|
||
Each row carries a short type badge (`pipeline`, `run`, `step`, `artifact` or
|
||
`match`); a text match also shows a snippet with the matched part bracketed so it
|
||
is legible without colour. A bare status word (`queued`, `running`, `succeeded`,
|
||
`failed`, `canceled`, `interrupted`) or a date (`today`, `yesterday`, `this week`,
|
||
or `YYYY-MM-DD` in local time) selects runs. Enter opens the matched pipeline,
|
||
run, stage visit, artifact or field. The same query is available non-interactively
|
||
as `pipelines search QUERY [--type KIND] [--limit N] [--json]`; `--type` is the
|
||
only kind-narrowing mechanism.
|
||
Wide terminals
|
||
show runs and pipelines side by side, with stacked panels on narrow terminals.
|
||
Home run cards prioritize reported results, status, completed visits and duration;
|
||
active cards also show the current stage. Start/finish times remain in run details.
|
||
The usage summary is aggregated over all stored history, including retry attempts:
|
||
total/active runs, outcomes, success rate, average completed duration, pipeline
|
||
usage and a zero-filled 14-day daily activity chart (UTC). Success rate is
|
||
succeeded / (succeeded + failed); active, canceled and interrupted runs are excluded.
|
||
The average includes all runs with a finish timestamp. It also totals the reported
|
||
model usage across history: `tokens_total`, an in/out split only when every
|
||
token-reporting stage reported one, and the summed cost, plus counts of stages
|
||
that did or did not report a cost. A stage whose provider reported no cost is
|
||
counted as cost-unknown and never contributes a fabricated or zero cost, and no
|
||
cost/token count is inferred from output length or transcript text. The totals are
|
||
the sum of the per-run totals `inspect` shows. `pipelines stats --json` exposes
|
||
these aggregates.
|
||
|
||
Use `:status` on any TUI page to open server and ntfy information. Scroll with
|
||
arrows or `j/k`, PgUp/PgDn; `r` refreshes and Esc closes without changing the page.
|
||
`pipelines status [--json]` returns the same server data, including hostname,
|
||
start/uptime, the connected server endpoint/port, active interface IPs, ntfy
|
||
publishing health, public phone URL/topic/port and candidate URLs. It does not
|
||
start services or send a notification. The ntfy health probe has a two-second
|
||
deadline; it checks the publishing server, not phone reachability.
|
||
|
||
IPs belong to the **server computer or container**, not necessarily the client.
|
||
Candidate IP/port combinations are not reachability checks: loopback bindings,
|
||
firewalls, container port mapping and TLS can prevent access. ntfy candidates
|
||
are only derived when a configured endpoint identifies this computer; remote
|
||
ntfy hosts are not misrepresented as local. Publishing URLs and credentials are
|
||
never returned. Missing public phone configuration is labeled explicitly.
|
||
`pipelines service status` remains the separate OS autostart status command.
|
||
|
||
## Search
|
||
|
||
`pipelines search QUERY [--type KIND] [--limit N] [--json]` runs one read-only
|
||
query across the whole system: pipeline names and their versions, run identity,
|
||
status and dates, stage and visit identifiers, artifact names and types, and the
|
||
text of stage inputs, outputs, evidence and transcripts. It never reads
|
||
artifact contents, never makes a per-result follow-up request, and asks for no
|
||
confirmation.
|
||
|
||
- Free text matches case-insensitively. A bare status word (`queued`, `running`,
|
||
`succeeded`, `failed`, `canceled`, `interrupted`) selects those runs, and a lone
|
||
date expression (`today`, `yesterday`, `this week` — Monday 00:00 local — or a
|
||
literal `YYYY-MM-DD`) selects the runs created in that window; any other text,
|
||
including a status word inside a sentence, stays free text.
|
||
- `--type` is the only narrowing mechanism and takes one kind: `pipeline`,
|
||
`version`, `run`, `step`, `field` or `artifact`. An unknown kind is an error,
|
||
never a silently ignored filter.
|
||
- Order: results are newest first by `recency` (pipeline/version creation,
|
||
run creation, step start), then numeric id and target, before the cap applies,
|
||
so the cap drops the oldest hits; each kind's row scan also walks newest
|
||
first. `score` still records the match tier (exact name 3, name substring 2,
|
||
body text 1). Every result carries `score`, `matched_field` and `recency` so a
|
||
client can re-rank.
|
||
- Bounds: the result cap defaults to 50 and clamps to 200 (`0` = default);
|
||
every kind scans at most 2000 rows, and field-text scanning stops at 2000 steps
|
||
or 1 MiB decoded. `truncated` plus `scanned_steps`, `scanned_bytes`, `cap` and
|
||
`scan_bound_bytes` report when a bound applied.
|
||
- Targets are stable and navigable: `pipeline:<id>`, `pipeline:<id>:v<version>`,
|
||
`run:<id>`, `run:<id>:<sequence>`, `run:<id>:<sequence>:<field>`, or an
|
||
artifact URL served by the artifact endpoint.
|
||
- Chats: a `search` request with `kind: "chat"` (protocol only; the all-kind
|
||
search never includes chats, so older clients see no change) matches the
|
||
operator's chat sessions. Every listed chat's title and directory are
|
||
matched from the registry; for up to 24 of the most recently used chats not
|
||
matched there, the newest 100 messages' text is read through the chat broker,
|
||
four at a time, within a one-second budget. A transcript still unread when the
|
||
budget expires is left out and the reply says `truncated`; `scanned_chats`
|
||
reports how many transcripts were read. Each chat yields at most one hit
|
||
(`target` `chat:<session id>`, `matched_field` `title`, `directory` or
|
||
`message`) whose `chat` payload carries the title, directory, harness and,
|
||
for a message match, the message id, role and a snippet. Hits are ordered by
|
||
the chat's last use, newest first, whatever field matched, before the cap
|
||
applies. The server answers it
|
||
off the connection's read loop, so it never delays other replies; a newer chat
|
||
search on the same connection cancels the previous one. The app sends it after
|
||
its run search and shows the hits as a Chats group.
|
||
|
||
```sh
|
||
pipelines search "release notes" --json
|
||
pipelines search failed --json
|
||
pipelines search 2026-10-16 --type run --json
|
||
pipelines search "acceptance check" --type field --limit 10
|
||
```
|
||
|
||
## Draft-first harness workflow
|
||
|
||
```sh
|
||
pipelines new # the configured harness asks what to build and names it
|
||
pipelines edit # fuzzy pipeline selection
|
||
pipelines edit "bug fix" # ask the harness what to change
|
||
pipelines edit "bug fix" "Add a review step"
|
||
pipelines new "release notes" "Summarize changes and notify my phone"
|
||
pipelines new "Add a feature" --template task
|
||
```
|
||
|
||
`new` opens the configured harness (Pi or OpenCode; host by default, or
|
||
Docker-linked when a checkout is linked) immediately when its name/request is
|
||
omitted. The skill asks only for missing
|
||
decisions and records the chosen name in draft metadata. `edit` accepts an exact
|
||
name or ID, with names taking precedence over numeric IDs. Both retain private
|
||
drafts and launch the harness once with a short `@pipelines` reference and
|
||
context-file path. There is no preliminary harness API call, temporary service,
|
||
session creation, or instruction preload. The prompt's CLI prefix is
|
||
Docker-aware: a Docker-linked profile embeds the in-container `pipelines`
|
||
executable and, for this Compose service's own loopback endpoint, scoped
|
||
transport aliases. Loopback is recognized only for parsed IP literals and the
|
||
exact `localhost` name (case-insensitive), so a remote DNS name such as
|
||
`127.remote.example` stays remote with no Docker probe. The single bounded
|
||
`docker compose port` probe must return a complete `host:port` binding that
|
||
covers the requested loopback address and port (wildcard or matching family);
|
||
malformed or non-matching bindings fail clearly, and a loopback `wss` endpoint
|
||
is rejected as having no supported local TLS mapping rather than downgraded to
|
||
plaintext. nvim remains a client-side editor.
|
||
`./install` installs/updates the global skill and format reference. Plain
|
||
`go install` users run `pipelines skill install` separately. Shared login/config
|
||
and user-modified skills are preserved; harness permissions remain in effect.
|
||
|
||
The skill encourages editing only the draft, running server validation, summarizing
|
||
the changes, and leaving submission to the user. This is an instruction, not a
|
||
security sandbox: an agent with CLI/shell access can submit when explicitly asked.
|
||
Every submission, including agent-issued `submit`/`rpc`, still uses server validation.
|
||
|
||
## Drafts in the TUI
|
||
|
||
The server runs one active run per resolved workspace, including overlapping
|
||
parent/descendant directories (and symlink aliases). Separate sibling worktrees
|
||
remain independent. A new request key in a
|
||
different workspace is admitted immediately and executes concurrently; a second
|
||
run in a workspace an active run is already changing is refused, naming the
|
||
conflicting run and advising you to wait or use a different workspace (for
|
||
example another checkout of the repository). Reusing the original request key
|
||
reattaches idempotently.
|
||
|
||
`cancel ID` / `cancel_run` records `cancel_requested: true` before signaling the
|
||
executor. The reply can still have `status: running` (or `queued`) and no
|
||
`finished_at`: this means **stopping**, not stopped. The workspace stays reserved
|
||
until execution returns and owned-session cleanup is confirmed; late success
|
||
cannot override cancellation. A parked human run has no executor and cancels
|
||
immediately. Repeating Stop while stopping is safe.
|
||
|
||
If cleanup cannot be confirmed, the run stays nonterminal with
|
||
`cleanup_unconfirmed: true`, retains partial step evidence, and reserves its
|
||
workspace. The web client labels this explicitly and hides duplicate Stop.
|
||
Successful startup recovery interrupts and confirms recorded sessions before
|
||
settling the run; without a recoverable session pointer, uncertainty and the
|
||
reservation survive restart rather than inventing stop confirmation. Recovery
|
||
does not resume a canceled run. A database-sidecar advisory server lock is held
|
||
through runner shutdown, so a second server on that database cannot recover or
|
||
interrupt the first one's live runs. Keep the sidecar file in place; its existence
|
||
is not ownership (the kernel lock is). This is a local-filesystem lock, not a
|
||
distributed lease; hard-link database aliases and filesystems without reliable
|
||
advisory locking are unsupported.
|
||
|
||
Use `:templates` for four simple, manual-only starters: task, review, research,
|
||
and plan/implement/review. **Submit template** (`s` or Enter) asks for a
|
||
name without opening an editor. Enter saves a retained draft, then a separate
|
||
confirmation publishes it as a new pipeline. Esc before naming creates nothing;
|
||
Esc at publication keeps the draft. **Customize** (`e`) opens the editor picker
|
||
instead. Both paths inherit the configured default model and preserve existing
|
||
pipelines/history. The research starter requires configured web search support.
|
||
Starter stages use goal-appropriate tools: the task worker and the three-stage
|
||
implementer edit and run checks, the task, planner and review stages may
|
||
search/fetch official sources only when external knowledge is genuinely needed,
|
||
the three-stage implementer has no web tools and follows the planner's research,
|
||
the standalone review stage never edits or runs commands and says so explicitly,
|
||
and the three-stage reviewer independently inspects the real changes and receives
|
||
the planner-owned acceptance checks rather than the implementer's summary,
|
||
approving only when the applicable acceptance checks were independently observed
|
||
passing and no unresolved substantive issue remains. The task and three-stage
|
||
templates end in a committer stage that stages the runner's captured changed paths
|
||
and commits locally without ever pushing; a genuine operator gate is a `type=human`
|
||
decision stage, never a model stage pretending to approve. Editing a bundled
|
||
template source changes no installed or published definition; a starter becomes an
|
||
immutable version only after explicit authoring and publication.
|
||
|
||
Home's `n` / `:new` and **Customize** (`e`) on a template first ask how to author the new
|
||
pipeline: **nvim** opens a retained Markdown draft file, or a configured harness
|
||
(Pi/OpenCode) opens the collaborative session with `@pipelines`. Tab/Shift+Tab or
|
||
arrows select; typing fuzzy-searches the choices; Enter launches and Esc cancels
|
||
without creating a draft. The choice overrides `VISUAL`/`EDITOR`. New drafts preserve
|
||
the create-only publication guard and still require explicit submission unless
|
||
you already published them from the harness. The configured harness can ask for a
|
||
missing name; for an unnamed nvim draft, `pipelines submit <draft.md> --name NAME`
|
||
names and publishes it. CLI `pipelines new` retains its existing direct-harness
|
||
behavior; `--no-open` creates the file without launching an editor.
|
||
|
||
Press `d` on the dashboard or use `:drafts` to browse the same retained drafts as
|
||
`pipelines drafts`. Use `n/p` to select, `j/k` to scroll source, `e`/Enter to reopen
|
||
the existing draft, `v` for server validation, and `s` for submission confirmation.
|
||
`r` refreshes the list. Drafts can be validated, edited, and submitted to whichever
|
||
server you are connected to. Same-server submissions preserve create-only/version
|
||
guards. A different destination imports the draft as a new pipeline, ignoring
|
||
the origin's version number and refusing an existing destination name; the
|
||
confirmation shows the destination, and opening, validating or submitting a draft
|
||
whose recorded server differs from the configured one prints a notice naming both
|
||
URLs so it is never silently retargeted. Retained source and origin metadata stay
|
||
unchanged. For an edit draft, CLI `submit --expected-version N` can explicitly
|
||
approve an update against version N on the destination (or `--force` can explicitly
|
||
skip that version check). For unnamed drafts, submission asks you to
|
||
type a pipeline name directly in the TUI. Enter saves the name in retained metadata
|
||
and opens publication confirmation; it does not publish yet. Esc keeps the draft
|
||
without saving unfinished name input. Draft selection/editor completion never publishes it.
|
||
On editor exit, the connected TUI checks the authoritative server before asking
|
||
to submit. If the draft source is already published (including a newer version
|
||
after the draft's base that was subsequently superseded), it skips confirmation
|
||
and reloads the latest pipeline/page and Home list. Local-only changes still need
|
||
explicit submission. A failed check retains the draft and says publication could
|
||
not be verified; it never assumes publication from an editor's exit code. Original
|
||
draft conflict guards and retained draft files remain unchanged.
|
||
New/edit drafts open the configured default harness; `--harness opencode|pi|nvim`
|
||
selects another configured harness or nvim. TUI pickers show only configured
|
||
harnesses, default first, plus nvim. See [harnesses and runtime control](harnesses-and-control.md)
|
||
for fuzzy model setup, separate defaults, shared memory and runner-owned limits.
|
||
|
||
For agents or scripts that should not open a terminal:
|
||
|
||
```sh
|
||
pipelines new "release notes" --no-open
|
||
pipelines edit "bug fix" --json
|
||
pipelines drafts --json
|
||
pipelines validate /path/to/pipeline-123.md --json
|
||
pipelines submit /path/to/pipeline-123.md --json
|
||
```
|
||
|
||
`--json` never opens an editor or picker; provide a name explicitly. It returns
|
||
draft metadata including `path`, `name`, `server`, `create_only`, and `base_version`.
|
||
`--no-open` prints the path and review commands. `$PIPELINES_DRAFT_DIR` overrides
|
||
the shared cache directory. CLI and TUI drafts use private Markdown plus metadata
|
||
sidecars; `submit FILE` can publish either. External files require `--name NAME`.
|
||
|
||
New drafts use create-only submissions and cannot overwrite existing names.
|
||
Edits carry a latest-version guard checked atomically by the server. On conflict,
|
||
inspect/export the latest version and reconcile the draft. `--force` explicitly
|
||
bypasses the version guard only, **never** syntax, graph, model, tool, schedule,
|
||
or notification validation. Original versions/sources remain immutable.
|
||
Submitting activates included schedules, so review them before publishing.
|
||
|
||
## CLI/TUI capability map
|
||
|
||
| TUI action | Agent-capable CLI |
|
||
| --- | --- |
|
||
| Browse/search pipelines | `list --json`, `show NAME --json` |
|
||
| Search everything | `search QUERY --json` (see [Search](#search)) |
|
||
| Original source and history | `export NAME --version N`, `versions NAME --json` |
|
||
| Browse templates/source | `templates --json`, `template KEY --source` |
|
||
| Create or edit a definition | `new`, `edit`, `drafts` |
|
||
| Validate and publish | `validate FILE`, `submit FILE` |
|
||
| Start a run | `run NAME PROMPT [--title TITLE] [--model P/M] [--stage-model S=P/M] --yes --json` |
|
||
| Run history/inspection | `runs --json`, `inspect ID --json` |
|
||
| Rename a run | `title ID "TITLE" --json` |
|
||
| Decide a waiting human stage | `decide ID [STAGE] --option KEY [--feedback TEXT] --json` |
|
||
| Cancel | `cancel ID --json` |
|
||
| Connection and phone settings | `connection --json`, `ping --json` |
|
||
| Server/ntfy status popup | `status --json` |
|
||
| Usage statistics/charts | `stats --json` |
|
||
| Run-history retention | `retention report --keep N --json`, `retention apply --keep N --yes` |
|
||
| Legacy operational diagnostics | managed `logs`, `doctor` |
|
||
|
||
The TUI does not spawn CLI processes for each request. Both front ends share
|
||
`internal/client`'s command transport and `internal/draft`'s persisted drafts;
|
||
the server owns all validation, persistence, execution, and schedules. `rpc`
|
||
exposes the full protocol as an escape hatch, so no server command is TUI-only:
|
||
|
||
```sh
|
||
printf '%s' '{"type":"get_pipeline","pipeline_id":1,"version":1}' | pipelines rpc -
|
||
```
|
||
|
||
Existing protocol commands: `ping`, `list_pipelines`, `get_pipeline`,
|
||
`list_templates`, `validate`, `submit`, `run_pipeline`, `list_runs`, `get_run`,
|
||
`cancel_run`, `retry_run`, `decide_run`, `set_run_title`, `get_run_evidence`,
|
||
`get_status`, `search`. `search` takes a `search` request object (`query`, with an
|
||
optional `kind` and `limit`) and returns a `search_results` response whose ranked
|
||
`results` each carry a stable `target` the client can open plus the typed payload
|
||
and matched field. `list_runs` also returns all-history `usage`
|
||
aggregates. `attention_acks` reads and `ack_attention` adds to the operator's
|
||
shared in-app acknowledgements (dismissed Needs-you entries and chat read
|
||
watermarks; see [ntfy.md](ntfy.md#clearing-notifications-across-devices)); other
|
||
connections receive an `attention_acks_changed` push. `subscribe_notifications`
|
||
(optional `after` cursor) streams the same notifications the server publishes to
|
||
ntfy as `notification` pushes, `clear_notifications` (`ids`) clears them on every
|
||
device, and `notifications_cleared` reports clears; `ntfy_enabled` in the
|
||
notification settings turns the ntfy publish off without stopping the stream (see
|
||
[ntfy.md](ntfy.md#app-notification-stream-no-ntfy-app-needed)).
|
||
|
||
Every run summary (`list_runs` rows, `get_run`, subscribe snapshots and terminal
|
||
`run_event` frames) carries `size_bytes`: bytes on disk for the run (artifact tree
|
||
with turn documents and session exports, server-created Pi session directories,
|
||
plus an estimate of its database rows). It is cached in `pipeline_runs.size_bytes`,
|
||
invalidated by triggers when the run changes, measured on completion, and
|
||
backfilled in the background; 0 means not measured yet. `delete_runs` removes
|
||
finished runs to free space:
|
||
|
||
```json
|
||
{"type":"delete_runs","id":"1","run_ids":[12,13,99]}
|
||
{"type":"runs_deleted","id":"1","runs_deleted":{"deleted":[12],"refused":[{"run_id":13,"reason":"run is running; cancel it or wait for it to finish"},{"run_id":99,"reason":"run not found"}],"freed_bytes":48213}}
|
||
{"type":"runs_removed","run_ids":[12]}
|
||
```
|
||
|
||
Only terminal runs (succeeded, failed, canceled, interrupted) are deleted; active
|
||
runs, unconfirmed cleanup, a running session export, a run an unfinished retry
|
||
depends on, and unknown ids are refused individually. Deletion removes every
|
||
row scoped to the run in one transaction plus `<artifacts>/<run>` and owned Pi
|
||
session directories; user workspaces are never touched. Subscribed connections
|
||
whose scope covers the runs receive the `runs_removed` push. `pipelines runs rm
|
||
<id...>` is the CLI form. See
|
||
`internal/protocol/message.go` for JSON envelopes. Mutating RPCs
|
||
have real effects; they are not previews. TUI-only animation, scrolling, and
|
||
transient client logs are presentation, not separate server capabilities.
|
||
|
||
## Saved run schedules
|
||
|
||
Schedules are independent pipeline/version + run-information + timer records,
|
||
not workflow Markdown. See [saved schedules](schedules.md) for the web flow,
|
||
JSON examples, clock semantics and legacy migration.
|
||
|
||
| Command | Payload | Reply |
|
||
| --- | --- | --- |
|
||
| `list_schedules` | optional `pipeline_id` | `schedules` array |
|
||
| `save_schedule` | full `schedule` record, optional creation `request_key` | `schedule` |
|
||
| `delete_schedule` | `schedule_id`, `expected_revision` | `schedule_deleted` |
|
||
|
||
The record contains `name`, `pipeline_id`, positive pinned `version`, `enabled`,
|
||
`input` (nonempty string `prompt`), existing absolute server `workspace`, `rule`,
|
||
and optional `model`, `stage_models`, `notify_mode`, `title`, `output_format`,
|
||
`write_paths` and legacy `allow_tools`. Updates require the current `revision`.
|
||
`next_due`, creation and update timestamps are server-owned. These are execution
|
||
controls and do not enable workflow publication on an execution-only server.
|
||
|
||
Pipeline summaries carry `scheduled`, `schedule_count`, `enabled_schedule_count`.
|
||
Run summaries and live frames carry actual timer provenance: `scheduled`,
|
||
optional `schedule_id` and `schedule_name`. A manual/API run of a scheduled
|
||
pipeline is not automatically scheduled. Historical provenance survives deletion.
|
||
|
||
## Chat sessions
|
||
|
||
Structured tools use `chat_answer` with
|
||
`chat_answer: {session_id, form_id, answers, action?}`. Optional `action` is
|
||
`accept`, `decline` or `cancel` only when advertised by the pending form's
|
||
`actions`; ordinary question replies can omit it. Answers retain JSON types.
|
||
OpenCode cancellation calls the existing form DELETE API. Codex MCP elicitation
|
||
returns the corresponding action/content; other Codex questions accept answers
|
||
only. Permission frames may provide `always_label: "Allow for session"` to avoid
|
||
misrepresenting a session-scoped grant as a permanent policy change.
|
||
|
||
Chat streams may send `chat_event.kind="resync"` to request an authoritative
|
||
history refresh without inventing text deltas or ending a queued prompt. Native
|
||
step markers may include `step.turn_id`; a terminal marker for an older turn
|
||
must not clear a newer turn's Stop control or release its queue slot.
|
||
|
||
Native context compaction is reported as `chat_event.kind="compaction"`, with
|
||
`compaction: {status: "running" | "completed" | "failed", id?: string}`.
|
||
Optional `native_turn_id` scopes the activity to its real harness turn; ignore
|
||
activity for an older turn when a newer native turn is known.
|
||
This is session activity and may have no `message_id`: clients must not invent
|
||
an assistant message or tool call for it. Show **Compacting context…** while
|
||
running; completion clears that phase but does not itself end the model turn.
|
||
A new subscriber receives any retained active compaction. Clear stale activity
|
||
before reconnecting, then preserve fresh replayed activity while history loads.
|
||
This reports native observations only; it does not request or trigger compaction.
|
||
|
||
Usage frames (`chat_event.kind="usage"`) may carry full native snapshots with
|
||
`usage.scope="session"`, `total_tokens`, `context_tokens`, `context_window`,
|
||
`cost_usd`, and `output_tokens_per_second`. Optional fields are unknown when
|
||
absent, not zero; reported zero values are retained explicitly. A new snapshot
|
||
replaces the previous one—never add repeated/replayed totals together.
|
||
`rate_basis="generation"` uses a native generation interval;
|
||
`rate_basis="turn"` denotes an end-to-end average that may include tools/waits.
|
||
Throughput must not be inferred from output text length.
|
||
|
||
Total tokens count processed usage across model calls; they are not the current
|
||
context size. Context usage/window and USD costs are shown only when available
|
||
from the harness. In particular, a subscription without reported USD cost is not
|
||
free, and the browser does not estimate a bill from a price table. Usage frames
|
||
are session metadata: no assistant/tool message is created and no queued prompt
|
||
is admitted or completed by a usage update. Subscriptions replay retained usage;
|
||
native readback refreshes it where supported.
|
||
|
||
`usage.account` is the account's subscription quota as the harness reported it,
|
||
absent when unknown:
|
||
`{"harness","source","plan","as_of","windows":[{"id","label","used_percent","resets_at"}],"unsupported"}`.
|
||
`used_percent` is 0–100 (it can exceed 100); `as_of` and `resets_at` are epoch
|
||
milliseconds. Claude Code fills it from its `rate_limit_event` stream lines
|
||
(5-hour and weekly windows; plan from the initialize `account`), shared by every
|
||
chat on the same account; API-key logins never report one. Codex reads
|
||
`account/rateLimits/read` once per connection (again when a chat opens after
|
||
five minutes) and merges `account/rateLimits/updated` notifications.
|
||
OpenCode has no quota API, so it sends `{"harness":"OpenCode","unsupported":true}`.
|
||
A usage frame with only `account` updates the quota and leaves token figures as
|
||
they were. No quota is estimated and no model call is made to obtain one.
|
||
|
||
The brokered OpenCode chat exposes the operator's own sessions, separate from the
|
||
sessions pipeline runs create. The protocol types are:
|
||
|
||
- `chat_sessions` lists the operator's own sessions only, ordered
|
||
most-recently-used first. Each carries its `directory`, its `tag` and its
|
||
recency. A session with
|
||
no recorded tag is never listed, and no pipeline stage session is ever listed:
|
||
stage sessions are still recorded in `run_steps`/`run_session_owners`
|
||
for the run that owns them. The list is fail-closed: with no registry and no
|
||
store it returns an empty list, never the endpoint's whole session list. A
|
||
session is never marked `busy`: any number of clients may watch and write one.
|
||
- `chat_subscribe` attaches a connection to one session's shared live stream (body
|
||
`session_id`) and replays the session's still-pending approval requests, so every
|
||
client that watches a session sees every frame any client's turn produces — the
|
||
other devices' prompts and the turn's streaming output — and can answer a shared
|
||
approval. Attaching is idempotent per connection; the subscription is released
|
||
when the connection ends. The reply carries `chat_queue` (`session_id`,
|
||
`prompts`): the prompts the server still holds for the session, each with
|
||
`operation_id`, `message_id`, `native_message_id`, `text`, `client`,
|
||
`attachments` and `queued_at`, so a reloaded client draws them as queued.
|
||
- `chat_prompt` may come from any client. A prompt whose session already has a turn
|
||
in flight is handed to that turn at once when the harness accepts mid-turn input
|
||
(Claude Code reads it from stdin at its next step; Codex takes it with
|
||
`turn/steer` pinned to the active turn) and is acknowledged as sent. Otherwise
|
||
(OpenCode, or a turn that ended meanwhile) it is written to the session's durable
|
||
server-side queue (bounded, currently 8) and acknowledged with
|
||
`chat_queued: true`; the agent never receives two interleaved turns. Queued
|
||
prompts belong to the session, not the connection: they are delivered in order
|
||
when the turn ends even with no client attached, and after a server restart once
|
||
the harness is idle. A retry with the same `operation_id` is acknowledged again,
|
||
never queued or delivered twice. The prompt is echoed to the other watching
|
||
clients as a `chat_event` of kind `user`, carried with the sender's `client`
|
||
label (a device name), while the sender's own transcript keeps its local copy;
|
||
a queued prompt's user event is published again as it is delivered.
|
||
- `chat_queue` is pushed to every watching client whenever a session's queue
|
||
changes, with the current `prompts` and the change: `delivered`, `removed`, or
|
||
`failed` with `error` (the harness refused it; it is not retried).
|
||
- `chat_unqueue` (body `chat_unqueue`: `session_id`, `operation_id`) removes one
|
||
prompt still waiting in the queue and replies with the updated `chat_queue`. A
|
||
prompt already handed to the harness is refused as already sent. Closing a chat
|
||
drops its queue.
|
||
- `chat_reply` is exactly-once: the first answer to a pending request wins, is
|
||
recorded with the answering client's label and is pushed back to every watching
|
||
client as the same `chat_permission` frame with `resolved: true`, `decision` and
|
||
`resolved_by`, so each card clears and shows the outcome. A later answer is
|
||
refused as already answered (naming the client that won), never silently ignored
|
||
and never delivered to the agent twice.
|
||
- `chat_create` takes a `title` (default "New chat") and a `directory`
|
||
(default `~`, the operator's home; `~`, an omitted or empty value and a `~/...`
|
||
value all resolve to the home directory). A relative or missing directory is
|
||
rejected before any OpenCode call. The created session is recorded with the
|
||
stored tag `operator` and returned as the response's `chat_session`.
|
||
An optional `ephemeral: true` creates the session for automated verification:
|
||
it is created, bootstrapped and usable exactly like any other chat and can still
|
||
be closed and reopened, but its registry row is hidden, so `chat_sessions` never
|
||
lists it and a check that drives the real UI cannot leave a stray "New chat" in
|
||
the operator's list. The flag only removes a chat from the shared list, so it
|
||
cannot reveal or alter another session.
|
||
- `chat_directories` returns the operator's home first, then the directories of
|
||
previous operator sessions most-recently-used first, deduplicated and bounded
|
||
(currently 8). It is a dedicated request so the create dialog can load the
|
||
choices without first listing sessions; `chat_sessions` stays a pure session
|
||
list.
|
||
- `list_directory` backs the app's "Choose folder" picker. The request body is
|
||
`list_directory: {"path": "..."}`; `path` is optional, absolute or starting
|
||
with `~` (`~` or `~/sub`), and defaults to the server user's home. The reply
|
||
is `directory_listing: {"path", "home", "parent", "entries", "truncated"}`:
|
||
`path` is the absolute, cleaned directory (expressed under `home`), `parent` is
|
||
`null` at the home root, and each entry is
|
||
`{"name", "path", "is_dir", "is_git_repo", "hidden"}` — directories first,
|
||
then files, each group sorted case-insensitively. Files are names only (never
|
||
contents) and are not selectable; `is_git_repo` marks a directory containing
|
||
`.git`; `hidden` marks a dot-name. Listing is confined to the home subtree:
|
||
the path is cleaned and its symlinks resolved first, so `..`, an absolute path
|
||
outside home, or a symlink that escapes home is refused with an `error` frame
|
||
("... is outside the home directory ..."); a symlink entry that escapes home is
|
||
listed only as a plain name. Kernel pseudo filesystems (proc, sysfs, devfs and
|
||
similar) are refused, permission problems are reported as
|
||
`permission denied: <path>`, and a reply carries at most 2000 entries with
|
||
`truncated: true` when more exist.
|
||
- `chat_adopt` takes a `session_id` that already exists on the configured
|
||
endpoint and records it in the operator session registry, using the endpoint's
|
||
own directory and title (never a guess or the server's working directory). The
|
||
response echoes the adopted session as `chat_session` and, like `chat_create`,
|
||
its registry row in `chat_sessions`, so `chat_sessions` then lists it and
|
||
`chat_prompt` can continue it. A nil payload or an invalid/blank id is refused
|
||
before any endpoint call; an id the endpoint does not report is refused naming
|
||
the id and records nothing; an endpoint record with an empty directory is
|
||
refused rather than substituting the home or working directory. The CLI
|
||
equivalent is `pipelines chat-adopt <session-id>` (`--json` for the full
|
||
response; otherwise a short `Adopted <id> · <directory>` line).
|
||
|
||
### Choosing the endpoint
|
||
|
||
By default chat brokers from the single `opencode serve` instance the server
|
||
spawns for itself, unchanged. To broker from the operator's own OpenCode service
|
||
instead, set the opt-in `-opencode-endpoint` server flag or the
|
||
`PIPELINES_OPENCODE_ENDPOINT` environment variable. Precedence is explicit flag,
|
||
then explicit environment, then the endpoint recorded on a previous run; with
|
||
none set the spawned default is used. The reference must be an http(s) URL with a
|
||
host and no credentials, query, fragment or path. It is validated once at
|
||
startup: the backend is detected from the service's own `/api/info` (V2) or
|
||
`/global/health` (native) contract, and a streamable `text/event-stream` event
|
||
path is required. A malformed reference, an unreachable host, a reachable
|
||
non-OpenCode service, or a healthy service with no SSE each disable chat with an
|
||
error naming the endpoint — never a silent fallback to another instance. An
|
||
endpoint that authenticates accepts an optional password through the private file
|
||
`PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE` (a regular 0600 file; never an
|
||
inline secret), and `PIPELINES_OPENCODE_ENDPOINT_USERNAME` overrides the detected
|
||
default user. The validated choice is recorded so a restart with no flag or
|
||
environment keeps brokering from the same place; an unreachable recorded endpoint
|
||
disables chat with a clear message and no fallback. The endpoint choice is
|
||
preserved by `pipelines service enable` autostart (the variables, with the
|
||
password file as a reference).
|
||
|
||
The stored tag is exactly `operator`; the display label `Operator` is derived from
|
||
it, so there is no second tag vocabulary. Chat uses the server's one OpenCode
|
||
transport. A session's stream is fanned out to every client watching it, prompts
|
||
are serialised per session (queued behind the in-flight turn rather than
|
||
rejected), and approvals are shared and exactly-once; the retained-state bounds,
|
||
the resync-on-reconnect path, the exclusion of a run's own session and
|
||
per-connection write serialisation are unchanged.
|
||
|
||
The operator-session registry, filter, directory defaulting and the list bound are
|
||
covered by the `pipeline`, `opencode` and `server` unit tests. The live end-to-end
|
||
check is opt-in (`PIPELINES_TEST_OPENCODE_CHAT_LIVE=1`); it runs against a scratch
|
||
store and never touches the deployed server. Android rendering of the tag is out
|
||
of scope for the server change; the wire carries `tag: "operator"`.
|
||
|
||
## Scripted runs
|
||
|
||
Every run records a short, single-line `title` saying what it was for. `pipelines
|
||
run --title "..."` sets it explicitly; otherwise the server generates it from the
|
||
prompt with the configured default harness. It is never empty: when generation is
|
||
unavailable or times out, the server derives the prompt's first line, and
|
||
`title_source` (`model`, `fallback`, or `explicit` for an operator-set title) plus
|
||
`title_model` record how it was produced so a diagnosis can tell them apart.
|
||
`pipelines runs` shows the title beside the pipeline name and version, and
|
||
`pipelines inspect` returns the title, name and version, so two runs of one
|
||
pipeline are distinguishable. Rename after the fact with `pipelines title
|
||
<run-id> "<title>"`; the run's input, steps and execution are unchanged. The
|
||
stored prompt is unchanged, and listing/renaming never calls a model.
|
||
|
||
### Model overrides at launch
|
||
|
||
A pipeline version pins one model per stage. Without republishing, a run can
|
||
choose different models at launch:
|
||
|
||
```sh
|
||
pipelines run "bug fix" "Fix the layout" --json --yes \
|
||
--model openai/gpt-5 \
|
||
--stage-model review=openai/o4-mini
|
||
```
|
||
|
||
- `--model PROVIDER/MODEL` overrides the model for **every** model stage of that
|
||
run.
|
||
- `--stage-model STAGE=PROVIDER/MODEL` overrides **one** stage and is repeatable,
|
||
so any number of stages can be overridden. A repeated key for the same stage is
|
||
last-wins.
|
||
- Precedence: a per-stage override beats the whole-run override, which beats the
|
||
model pinned in the pipeline version.
|
||
|
||
Overrides apply to that run only. The pipeline definition and its immutable
|
||
version are never modified, `show <name> --version N` still returns the pinned
|
||
source and graph, and the run records the `model` and `stage_models` it was
|
||
launched with so it stays reproducible; `retry` reuses them. At launch every
|
||
value must be a `provider/model` pair of the configured form and every stage key
|
||
must exist in the pinned version — an unknown key fails naming the valid keys and
|
||
no run is created. A per-stage key that names a non-model stage is refused
|
||
because that stage never runs a model; `--model` on a pipeline with no model stage
|
||
is accepted and inert. The effective model reaches the stage and its
|
||
repair/formatter path from the same resolved definition, and each step's evidence
|
||
records the provider/model it actually ran with (`inspect <id> --json` exposes
|
||
per-step `provider`/`model`, and the run's graph shows the effective pair). Each
|
||
model step also records the usage the harness reported, carried from the same
|
||
accounting the stage already used for its budget: `usage.tokens_total`, the
|
||
`tokens_in`/`tokens_out` split when the harness reported it (`split_known`),
|
||
`cost_usd` when reported, and the explicit `tokens_known`/`cost_known`/
|
||
`split_known` markers. The run detail totals those stages as `usage`, so a
|
||
mixed-model run shows each stage's model, tokens and cost and the run total. A
|
||
stage whose provider reported no cost shows its tokens with `cost_known: false`
|
||
(never a fabricated or zero cost), and a harness that reports only a total (Pi)
|
||
shows `tokens_total` with the split marked unavailable rather than presenting the
|
||
total as input tokens. The
|
||
CLI is the deliverable; a TUI picker for choosing overrides is a documented
|
||
follow-up, not half-built.
|
||
|
||
|
||
On either the runs list or run inspector, press `P` (or use `:pipeline`) to open
|
||
the selected run's exact pipeline version. Navigation does not cancel the run.
|
||
|
||
Initial input includes `pwd`, the resolved run directory (normally the directory
|
||
you invoked Pipelines from), alongside the task and any custom input fields.
|
||
It is recorded by the server and shown in the run's Input section.
|
||
|
||
Press `r` or use `:retry` on a failed run to restart it from the first stage.
|
||
CLI equivalent: `pipelines retry <run-id>` (optional `--yes`, `--json`, `--wait`,
|
||
`--wait-timeout`, and `--request-key`). The new attempt retains the initial input,
|
||
directory, pinned pipeline version, and edit scope, and records `retry_of`. Original
|
||
history is unchanged. Legacy runs gain `pwd` from their recorded workspace on retry.
|
||
Retry replays the entire workflow; earlier edits or notifications are not undone
|
||
and may happen again. A missing original directory blocks retry rather than
|
||
silently using your current directory.
|
||
|
||
Retry is hidden and rejected while a run is executing or agent cleanup remains
|
||
active, even if a terminal status was already persisted. Run RPCs report
|
||
`executing` alongside status so clients can distinguish cleanup from completion.
|
||
|
||
A pipeline may contain a `type=human` decision stage (see the authoring skill's
|
||
`references/format.md`). Its step becomes `waiting`, the run reports that it is
|
||
waiting on a person, and no later stage runs. `runs`/`inspect --json` show the
|
||
`waiting` status, the stage and its recorded instructions and options. Answer it
|
||
with `pipelines decide <run-id> [stage] --option KEY [--feedback TEXT] [--json]`
|
||
or, in the run inspector, press `c` to open the option picker, choose with
|
||
`↑/↓`, type optional feedback with `Tab`, and confirm with Enter. The choice and
|
||
feedback are recorded as the step's evidence and the run resumes along
|
||
`edges[KEY]`. An unknown option is refused with the valid list, a decision to a
|
||
stage that is not waiting is refused, and a repeated decision for an
|
||
already-decided visit does not double-apply. The wait is persisted in the run
|
||
rows, so it survives a server restart and `decide` still works afterwards;
|
||
`cancel` and `retry` keep working while waiting and after a decision, and a
|
||
human stage is never killed by a stage timeout or `max_visits`.
|
||
|
||
The run inspector's upper half shows the entire graph from the run's immutable
|
||
pinned version using the shared renderer and viewport. Executed stages and taken
|
||
routes retain their colors; unexecuted stages, unchosen routes and unreached
|
||
terminals are dimmed. Queued runs show the full graph before the first visit.
|
||
Repeated visits remain individually inspectable in the stage sidebar, while the
|
||
graph highlights every audited route through a loop. On small terminals details
|
||
take precedence. Older servers without a graph fall back to a matching loaded
|
||
definition or the recorded execution path, never a different pipeline/version. Press
|
||
`g` to focus the graph, arrows to pan, and `j/k` or PgUp/PgDn to scroll. Press `g`
|
||
or Tab to return to details. When not focused, the graph follows the latest visit
|
||
or selected stage. Existing polling refreshes the graph once per second, and
|
||
running durations show elapsed time rather than “in progress.”
|
||
|
||
The server can also push run changes. A client sends `subscribe` scoped to a
|
||
workspace (and optionally to explicit run ids); the server first returns the
|
||
current state of the matching runs, then a compact `run_event` delta on each
|
||
transition — which run changed, its status, outcome, current stage, completed-step
|
||
count and timestamps, and, when a single step changed, that step's sequence, stage,
|
||
status, outcome, target and diagnostic. It is a delta, never the whole run, and
|
||
rapid changes are coalesced to at most a few frames per second per run.
|
||
`unsubscribe` or disconnecting stops the frames. This lets a client drop the
|
||
once-per-second poll: the Android app consumes these pushes instead, while the
|
||
TUI still polls in this release.
|
||
|
||
Run-list summaries and subscribe snapshots additionally carry compact
|
||
`graph_evidence: {complete: true, visits: [...]}`. Each visit contains only
|
||
`sequence`, `stage`, `status`, `outcome` and `target`: the last visit of each
|
||
stage and each distinct taken route, ordered by sequence. Repeated loops do not
|
||
grow the summary indefinitely, and private step inputs, outputs, transcripts
|
||
and diagnostics are excluded from this projection. Live `run_event.graph_visits`
|
||
preserves changed visits across coalescing (unlike the single latest `step`).
|
||
Clients merge these by sequence and retain last-stage/distinct-route evidence.
|
||
Missing evidence from an older peer means unknown, not that every stage shares
|
||
the overall run result. These fields are additive; older peers ignore them.
|
||
|
||
In the run inspector, `n`/`p` select the next/previous recorded execution visit,
|
||
including repeated visits in loops, from any pane. Selection stops at the last
|
||
visit or overview; it does not wrap. The selected stage has a full-row highlight;
|
||
the focused stage/details pane has a highlighted title and pointer. In details,
|
||
the section containing the top visible row is highlighted and named in the pane
|
||
title. `Tab` cycles stages → details → graph (Shift+Tab reverses); `g` switches
|
||
graph focus. Each graph stage title shows live progress from its latest execution
|
||
visit. New run pages open with the graph focused and using roughly three quarters
|
||
of the available height. Tab expands the focused pane and contracts the others;
|
||
the stage list also widens when focused, and shrinks to a sidebar for details.
|
||
Pending stages are dimmed. The graph is centered horizontally/vertically
|
||
when it fits; the box-and-arrow diagram is retained at every viewport size,
|
||
never replaced with a stage list to fit. Oversized diagrams remain
|
||
pannable with `h/j/k/l` or arrows while the graph is focused; `c` recenters the
|
||
selected/current stage. `P` opens the
|
||
pinned pipeline (lowercase `p` remains an alias only on the runs list).
|
||
|
||
The pipeline page also gives most height to its focused graph or stage-details
|
||
pane; Tab switches focus and resizes the two panes. Focused titles use the same
|
||
background highlight as the run page.
|
||
Graph boxes are sized to their contents rather than the longest label anywhere
|
||
in the graph. Fork columns share only their own required width, route tracks are
|
||
adjacent, and branch spacing follows stacked boxes rather than tall trunk/merge
|
||
nodes. All stages, outcome labels, ports, and terminal arrows remain visible in
|
||
the full pannable diagram. Incoming route IDs are shown on the left and outgoing
|
||
outcome labels on the right of shared rows, without separate input/output
|
||
sections or divider rows. This reduces height while allowing enough width to
|
||
keep both labels readable. Failure-bound error exits remain at the bottom.
|
||
Pipeline and run graph panes both center horizontally and vertically when the
|
||
diagram fits, and clamp scroll/pan offsets on resize or graph changes.
|
||
|
||
Press `a` to select the active/latest retained configured-harness agent session and
|
||
open its tool evidence. Session ID is available during execution; tool transcripts
|
||
appear when persisted (currently after generation). This does not attach a live
|
||
harness client or stream its messages. Supporting live attachment consistently
|
||
across the shared and privately owned harness backends is deliberately deferred.
|
||
|
||
## Stage turn files
|
||
|
||
Each model-stage visit receives a file-based turn under the run's artifact
|
||
directory. `pipelines inspect <run-id> --json` and the run inspector list these
|
||
files as that step's artifacts:
|
||
|
||
```
|
||
<artifacts>/<run>/stages/01-implementer/input.md
|
||
<artifacts>/<run>/stages/01-implementer/memory.md
|
||
<artifacts>/<run>/stages/01-implementer/output.md
|
||
<artifacts>/<run>/stages/01-implementer/private/
|
||
<artifacts>/<run>/public/
|
||
```
|
||
|
||
`input.md` is the run prompt on the first visit; later visits carry the run
|
||
prompt under `# Run task` followed by the previous visit's output body (without
|
||
its outcome section) under `# Previous turn output`. `memory.md` is the accumulated markdown of completed turns plus the
|
||
files each turn added to the shared `public/` directory. A text-only stage (an
|
||
explicit empty `tools` block) receives both files' contents in its prompt
|
||
instead, because it has no file-reading tool. The model returns the complete
|
||
`output.md` markdown; only its `# outcome` section is interpreted by the server.
|
||
Everything else is free-form and preserved under the step's `markdown` output.
|
||
`private/` belongs to its own visit, while `public/` is shared by every stage in
|
||
the run. Artifacts are served from `/artifacts/<run>/<step>/<name>`; shared files
|
||
appear with a `public/` prefix.
|
||
|
||
### Run notification modes and receipts
|
||
|
||
`--notify auto|on|off` records `notify_mode` on the run (also returned in
|
||
summaries and `run_event` frames). Auto retains stage policies and the
|
||
run-level waiting alert; its failure or success is silent unless the server's
|
||
`run_finished` setting ("Notify when any run ends", off by default) is on. On
|
||
also alerts when the run fails or succeeds, without requiring a stage hook. Off
|
||
suppresses stage, run-level system and browser tray alerts.
|
||
|
||
`inspect --json` / `get_run` includes top-level `notifications` receipts with
|
||
`sequence`, `event`, `channel`, `status`, `attempts`, and optional `diagnostic`
|
||
and `message_id`. Status is pending, sending, sent, failed, or covered (already
|
||
published by the stage). Run-level system alerts persist and retry at most three
|
||
times, recovering pending work after restart. A publisher acknowledgement is not
|
||
proof of reception; a crash before saving it can duplicate a message. See
|
||
[`notes/audit-notifications.md`](notes/audit-notifications.md) for limits.
|
||
|
||
## Investigating runs with a configured harness
|
||
|
||
On the runs list or inspector, press `D` or use `:debug`; CLI equivalent:
|
||
`pipelines debug <run-id>`. This opens a fresh conversation in the configured
|
||
harness (Pi or OpenCode), not the implementing session. It supplies exact
|
||
commands for `inspect <run-id> --json` (including stage history, exact outputs,
|
||
diagnostics, notifications, session IDs and bounded transcript previews),
|
||
`show <pipeline-id> --version N --json`, and `runs --pipeline <pipeline-id>
|
||
--json`. The commands reuse the same Docker-aware CLI prefix as authoring, so a
|
||
Docker-linked profile runs them in-container against the original logical
|
||
endpoint. All retrieval remains available through the CLI and can be repeated
|
||
while execution is active. No new model API or live attachment integration is
|
||
required. The injected server-side session evidence is the latest bounded,
|
||
credential-redacted evidence for up to the 3 most recent session-bearing steps;
|
||
when a preview is bounded or omitted, inspect the retained session itself rather
|
||
than assuming a complete tool history.
|
||
|
||
`debug --json` returns the context and prompt without launching the harness;
|
||
`--no-open` prints the prompt. `--harness opencode|pi|nvim` selects a configured
|
||
harness or nvim; `--opencode` selects the host executable. Debugging
|
||
works for active and terminal runs and does not automatically retry, cancel,
|
||
or resume them. If the recorded directory is unavailable, the harness launches
|
||
from your current directory and retrieves historical evidence through the CLI.
|
||
|
||
## Editor choice and draft cleanup
|
||
|
||
On the pipeline detail page, `e` / `:edit` opens an editor picker: **nvim** or
|
||
the configured harness (Pi/OpenCode, default first). Type to fuzzy-search;
|
||
Tab/Shift+Tab (or arrows) choose; Enter opens;
|
||
Esc cancels without creating a draft. Explicit choice overrides VISUAL/EDITOR.
|
||
Both choices edit a retained copy with the normal version guard, never the
|
||
published definition. Submission remains separate and explicit.
|
||
|
||
In the draft browser, `x` asks to delete the selected draft; `y` confirms and
|
||
Esc or `n` keeps it. The CLI equivalent, `pipelines draft-delete <draft.md>`, is
|
||
an explicit deletion command and needs no server connection; `--json` reports
|
||
the result. It asks for confirmation; `--yes` explicitly approves scripted deletion.
|
||
Only managed drafts inside the configured draft cache are accepted.
|
||
Source and metadata symlinks/directories and external Markdown files are
|
||
rejected. Published definitions and run history are not deleted. Cleanup can
|
||
also remove orphaned metadata when a draft source was already removed.
|
||
|
||
```sh
|
||
cd ~/repos/project
|
||
pipelines run "bug fix" "Fix the layout" --json
|
||
pipelines run "bug fix" "Fix the layout" --json --wait --wait-timeout 10m
|
||
pipelines run "bug fix" "Fix the layout" --json --request-key my-unique-task
|
||
```
|
||
|
||
Runs default to the client's current directory and latest accepted version.
|
||
The server resolves that directory without a repository allowlist. `--version` chooses
|
||
an immutable version in JSON mode. Starting a run enables its stage-declared tools
|
||
without a second approval. Editing supports all file types and new files within
|
||
the run directory; no file list is required. Shell commands use the repository's
|
||
actual tooling. `--write-paths` is an optional narrower scope, not a prerequisite.
|
||
`--yes` skips action confirmation for explicitly approved automation;
|
||
`--allow-tools` remains a compatibility flag and is not required.
|
||
`--wait` polls until terminal status, returns nonzero on failure/cancellation,
|
||
and does not cancel a run when waiting times out. `--wait-timeout` bounds only the
|
||
client wait after admission and is independent of the per-request `--timeout`; a
|
||
reached wait limit leaves the admitted run active and never cancels or replays it.
|
||
When it is reached, inspect read-only with `pipelines inspect <run-id> --json`, or
|
||
keep waiting by rerunning `--wait` and reusing the original `--request-key` rather
|
||
than starting a duplicate run. Reuse `--request-key` to prevent duplicate work.
|
||
A supplied pipeline/task opens an action-confirmation popup before
|
||
starting with a live inspector (unless `--yes` was supplied). When a task is omitted,
|
||
Enter inserts a newline and Ctrl+S opens the confirmation for the composed task.
|
||
There is no file-list question or typed RUN confirmation. Shell runs with your
|
||
user's authority: the run directory and tool policy are not an OS sandbox.
|
||
|
||
Run, retry, cancellation, CLI publication, and draft deletion require confirmation.
|
||
CLI prompts go to stderr and default to No; use `--yes` for non-interactive approval.
|
||
The TUI's run/retry/cancel popup accepts `y`; Enter does not implicitly approve,
|
||
and `n`/Esc cancels without sending the action. Retry restarts with new sessions
|
||
and original input/pwd, never the failed conversation; side effects may repeat.
|
||
Publication retains its explicit submission confirmation and version conflict guards.
|
||
Quit with `:quit`, not `q`; Ctrl+C remains an interrupt.
|
||
|
||
JSON results go to stdout, errors to stderr (JSON error envelopes when requested),
|
||
and failures exit nonzero. Export/template `--source` deliberately prints raw
|
||
Markdown. `--timeout` bounds each server request; `--wait-timeout` bounds the
|
||
optional client wait independently and, when reached, leaves admitted work active.
|
||
Neither changes the stage execution timeout.
|
||
|
||
### Stage limits
|
||
|
||
Model-stage `configs` support these optional settings:
|
||
|
||
```text
|
||
retries=2
|
||
timeout=1h
|
||
budget_usd=1.50
|
||
budget_tokens=50000
|
||
```
|
||
|
||
Retries default to **2** (three total attempts); timeout defaults to **one hour
|
||
per attempt**. Omitted budgets are unlimited; configured budgets must be positive.
|
||
Each budget covers all attempts, format-only repairs, and repeated visits to that
|
||
stage in one run. A new run starts new accounting. Tokens include input, output,
|
||
reasoning, and cache reads/writes. USD amounts use OpenCode's reported cost.
|
||
Missing or invalid required accounting stops the stage rather than counting zero.
|
||
Usage is polled during execution and read again afterward; an in-flight request
|
||
may exceed a limit before usage arrives. These are not provider-side hard caps.
|
||
|
||
Automatic retries cover retryable transport failures and per-attempt timeouts.
|
||
An explicit read-only stage (a tools block listing neither `edit` nor `shell`)
|
||
may also retry a post-admission deadline or transport failure and re-emit a
|
||
malformed final result up to `retries`; a stage whose tools include `edit`/`shell`
|
||
or that has no tools block never replays admitted tool work or invalid results.
|
||
Budget failures always stop. Cleanup must be confirmed before another attempt.
|
||
Existing explicit `retries=0` settings remain unchanged. The old 30-minute run deadline no longer truncates
|
||
stage timeouts; the 30-step and five-visits-per-stage limits still apply.
|
||
Native ntfy stages also accept `timeout` (default `1h`), but never automatically
|
||
retry potentially delivered notifications and do not accept model budgets.
|
||
|
||
Pinned definitions contain the configured limits. Run transcripts retain limits,
|
||
attempt counts and reported budget usage, also shown in the TUI's Evidence pane.
|
||
The same reported usage is recorded on the run as typed per-step evidence and
|
||
totalled for the run, so `inspect <id> --json` exposes tokens in/out, the total,
|
||
the cost when reported and the model each stage ran with. A stage revisited within
|
||
a run shares one budget, so each step records only that visit's increment of it
|
||
and the run total is not the cumulative budget counted again. Missing or unknown cost
|
||
or split is reported as unavailable (never zero or inferred), and a historical or
|
||
asked-for run recorded before this evidence existed simply omits `usage`.
|
||
Exact stage outputs are authoritative; a transcript over the 32 KiB retained
|
||
bound is stored as a valid, credential-redacted preview envelope that preserves
|
||
available stage limits and failure/format diagnostics and names the retained
|
||
session instead of slicing to malformed JSON.
|
||
|
||
### Result-first views
|
||
|
||
Home shows retained result previews and completed visits alongside each run and
|
||
pipeline. Pipeline pages show the latest result for the **selected version**,
|
||
not a result from a different published definition. Run opens focused on results,
|
||
with output before input and the graph below. Failed or ongoing runs preserve
|
||
partial output and completed stage reports without labeling them final success.
|
||
These previews quote recorded output; they do not independently verify model claims.
|
||
Tab and Shift+Tab toggle **Details ↔ Graph** only. The visit selector lives inside
|
||
Details; `n`/`p` still navigate recorded visits, including repeated stages, and `0`
|
||
returns to the overview. `j`/`k` scroll the report; the focused graph uses `hjkl`
|
||
for panning and `c` to center. Secondary progress/input/evidence/execution sections
|
||
are collapsed initially. Small inactive graph previews collapse instead of showing
|
||
severed boxes. See [UI/UX review](ui-ux-review.md) for screen-by-screen feedback.
|
||
|
||
## Run retention
|
||
|
||
Nothing prunes run history automatically. `pipelines retention report` and
|
||
`pipelines retention apply` are the only commands that remove run history or run
|
||
artifacts, they are local and operator-controlled, and they never run as a side
|
||
effect of any other command, server startup, `newRunner`, the scheduler,
|
||
`list_runs`, `stats`, `runs` or `inspect`. Pipeline definitions and their
|
||
immutable versions are never touched: retention is about run history and the
|
||
artifacts runs produced.
|
||
|
||
```sh
|
||
# Read-only: show exactly what a policy would remove.
|
||
pipelines retention report --db ~/.local/share/pipelines/data/pipelines.db --keep 200 --json
|
||
|
||
# Explicit removal of that same set (artifacts go with their run).
|
||
pipelines retention apply --db ~/.local/share/pipelines/data/pipelines.db --keep 200 --yes --json
|
||
```
|
||
|
||
Both commands accept `--older-than DURATION`, `--keep N`, `--db FILE`,
|
||
`--artifacts DIR` and `--json`; `apply` also accepts `--yes`. At least one policy
|
||
flag is required; an unknown subcommand, a negative count and a non-positive age
|
||
are refused. `--db` must name an existing database and is never created. With only
|
||
`--db`, the artifacts root defaults to `artifacts/` beside the database, matching
|
||
`cmd/pipelines-server/main.go`; `--artifacts` overrides it. The commands are
|
||
dispatched before server URL resolution, so they work without a running server
|
||
(for example `--url not-a-server`) and never require, restart or redeploy the
|
||
deployed server.
|
||
|
||
### Policy
|
||
|
||
- `--older-than` selects runs created **strictly before** `now - DURATION`. The
|
||
age is a rolling UTC duration, not a calendar-day boundary. It accepts Go
|
||
durations plus a count of days or weeks (`720h`, `30d`, `12w`).
|
||
- `--keep N` retains the newest `N` runs by run id and selects ranks `N+1` and
|
||
beyond. `--keep 0` selects every eligible run; a count larger than the history
|
||
selects none.
|
||
- Both together are the conservative **intersection**: a run is selected only
|
||
when it is both older than the age bound and outside the newest `N`.
|
||
|
||
### Protected runs
|
||
|
||
A policy-selected run is nonetheless never removed when it:
|
||
|
||
- is `queued` or `running`;
|
||
- is `waiting` on a person (a `waiting` run or a `waiting` step);
|
||
- has any step with `agent_active=1` (cleanup still active); or
|
||
- is referenced by a run that will remain through retry provenance
|
||
(`run_retries.original_run_id`), propagated transitively along a retry chain.
|
||
If both an original and its retry are selected, no retained link remains and
|
||
both may be removed.
|
||
|
||
Protected runs are reported with their reason and their artifacts are never
|
||
counted as reclaimable. Free-text run ids in prompts are not treated as
|
||
dependencies; retry provenance is the only structured cross-run dependency the
|
||
store records.
|
||
|
||
If a selected run becomes protected between the report and the delete — a new
|
||
retry reference, a waiting/active state or new agent activity — `apply` aborts
|
||
with the run ids and reasons, deletes no rows or artifacts, and un-stages
|
||
anything already renamed. A removal never half-completes.
|
||
|
||
### What apply removes and reclaims
|
||
|
||
`apply` stages only the would-remove run directories into one
|
||
`artifacts/.retention-staging-<timestamp>-<pid>/` directory (a same-filesystem
|
||
rename), deletes the rows for those runs in one immediate write transaction
|
||
(`step_notifications`, `step_file_changes`, `step_control`, `step_artifacts`,
|
||
`step_verdicts`, `step_session_exports`, `step_usage`, `run_session_owners`,
|
||
`run_steps`, `run_retries`, then
|
||
`pipeline_runs`, batched at 500 ids), commits, and removes the staging directory.
|
||
A symlink at `<root>/<id>` is removed as a link without touching its target; a
|
||
regular file at `<root>/<id>` is refused. A symlinked artifacts root is followed
|
||
consistently: the report sizes it and apply stages through the same directory, so
|
||
the artifacts really go. All regular files under a removed run directory are
|
||
counted, including files beyond the server's 200-file/8-MiB/64-MiB inventory
|
||
caps, which bound the recorded inventory, not disk ownership.
|
||
|
||
The database is then compacted with one `VACUUM` and one
|
||
`PRAGMA wal_checkpoint(TRUNCATE)`, retried at most three short times on
|
||
`SQLITE_BUSY`. `database_bytes_reclaimed` and `space_reclaimed_bytes` are
|
||
measured, not estimated. If `VACUUM` fails the command returns an error, sets
|
||
`vacuum_error` and claims no reclaimed database pages while still reporting the
|
||
artifact bytes it did remove; a later `apply` with nothing left to remove can
|
||
retry the `VACUUM`. If staging cleanup fails the report still gives the removed
|
||
run count and totals, names the staging path and the remaining bytes/files, and
|
||
does not count the leftovers as reclaimed.
|
||
|
||
### Measured example (this machine)
|
||
|
||
Reported before, applied, reported after, against the live database while its
|
||
server kept running (server PID 99492 unchanged throughout; the server was also
|
||
creating new runs concurrently, which is why the artifact `du` moved
|
||
independently):
|
||
|
||
```text
|
||
report --keep 200: total 207 · would remove 7 · protected 0 · retained 200
|
||
artifacts 418 files · 279668627 bytes (266.7 MiB)
|
||
database 27897856 bytes · free 0 · reclaimable 279668627
|
||
apply --keep 200: removed 7 runs · 418 files · 266.7 MiB
|
||
vacuumed · database reclaimed 1.5 MiB · now 25.1 MiB · free 0
|
||
total space reclaimed 268.2 MiB
|
||
report --keep 200: total 200 · would remove 0 · retained 200
|
||
artifacts 0 files · 0 bytes
|
||
database 26353664 bytes · free 0
|
||
pipelines 19 · pipeline_versions 45 (unchanged)
|
||
```
|
||
|
||
Independent observation: `du -sk` of the artifacts root fell from 2815316 to
|
||
2572456 KiB (the running server added artifacts during the same window), the
|
||
database file from 27897856 to 26353664 bytes, the WAL was truncated to zero and
|
||
the logs stayed at 18404 KiB. A retained run's artifact still served (HTTP 200,
|
||
`/artifacts/100/1/00_run94_artifacts.png`) while a removed run's URL returned 404
|
||
(`/artifacts/83/1/...`). The measured numbers above come from that
|
||
before/apply/after output, not an estimate.
|
||
|
||
## Verification scope
|
||
|
||
CLI integration tests exercise draft/validate/submit/history/conflict handling and
|
||
scripted run/inspection/cancellation through a real local WebSocket server (the
|
||
run test uses a controlled executor, not a model). The isolated installed CLI
|
||
also exercised the actual OpenCode API's session creation and skill activation,
|
||
while capturing interactive launch arguments to avoid a model turn. Full
|
||
model-driven authoring and Linux execution are **not tested**.
|
||
|
||
## Per-run report contract
|
||
|
||
`pipelines run NAME PROMPT --output-format html` requires a server-written
|
||
`public/report.html` before run success; `--output-format markdown` requires
|
||
`public/report.md`. Omit the option for historical behavior. The `run_pipeline`
|
||
wire request and returned run record carry `output_format` (`html` or `markdown`),
|
||
and retries inherit it. Request-key replays must retain the same format.
|
||
Reports use nonempty terminal text output and are bounded to 2 MiB; missing,
|
||
empty, unsafe or unverifiable files fail the run. This checks structure, not
|
||
factual correctness. See [audit notes](notes/audit-deliverables.md).
|
||
|
||
`pipelines repair-session-export RUN_ID SEQUENCE [--json]` asks the runtime to
|
||
read back that settled visit's exact recorded native session/backend. It never
|
||
executes a stage or changes a definition. It writes a versioned repair artifact
|
||
and appends evidence without replacing original files or outcomes. The protocol
|
||
command is `repair_session_export` with `run_id` and `export_sequence`; the reply
|
||
is a run detail containing each visit's `session_export_repairs`. CLI JSON emits
|
||
the repair record with explicit completeness `status`, and incomplete repair
|
||
exits nonzero. Recovery still depends on upstream session availability.
|
||
|
||
Incomplete native stage-session exports also recover automatically after the
|
||
run settles: up to three durable runtime attempts, with backoff and restart-safe
|
||
leases. `inspect RUN_ID --json` includes each visit's `session_export_recovery`
|
||
status/attempt count/diagnostic and preserved `session_export_repairs`. Exhausted
|
||
automatic recovery remains a warning; manual versioned repair is still available.
|
||
A run parked on an export handoff never waits beyond that: once its job is
|
||
exhausted (or the wait exceeds 15 minutes with the job pending or past its
|
||
lease), the visit keeps its partial/unavailable evidence and diagnostic, the run
|
||
continues to its next stage or terminal, and its workspace is released. The
|
||
server applies this at startup and every 30 seconds, so a run parked before an
|
||
upgrade recovers after the next restart. A native session larger than the 8 MiB
|
||
export bound (typically inline image payloads read by a tool) is always partial.
|