476ecc781c
Keeps this branch's advisory review edge, bounded blocked_retryable implementer edge and their tests alongside main's run-attention/ntfy notification work, the split of the Android chat conversation screen, and the retention/history changes. No conflicts: the two sides touch disjoint files.
419 lines
22 KiB
Markdown
419 lines
22 KiB
Markdown
# Pipelines
|
|
|
|
Configured OpenCode/Pi harnesses, separate model defaults, runner-enforced review
|
|
limits and selected run-local context: [guide](docs/harnesses-and-control.md).
|
|
|
|
Reusable, sequential workflows for your repositories, with a Go CLI/TUI,
|
|
OpenCode model stages, retained run history, and optional phone notifications.
|
|
|
|
**First-version preview:** a single-user tool for supervised work. Start with a
|
|
disposable repository, review edits and command evidence, and expect rough edges.
|
|
See the [tester walkthrough and known limits](docs/first-version-testing.md).
|
|
|
|
Stage results include [retained, highlighted workspace diffs](docs/stage-file-diffs.md)
|
|
for newly executed visits, alongside their reports and audit evidence. Stages can
|
|
also [record and serve their own artifacts](docs/stage-artifacts.md) - images,
|
|
screenshots, reports and logs - per step.
|
|
|
|
Use `:export` / `:import` to [select and transfer pipeline definitions](docs/pipeline-transfer.md)
|
|
between servers as portable ZIP bundles.
|
|
|
|
[Real-provider reference workflows](docs/reference-workflows.md) exercise structured
|
|
handoffs, file edits/verification, failure routing and reported-usage budgets with
|
|
independent acceptance checks and retained evidence. Live inference is opt-in.
|
|
|
|
## In action
|
|
|
|
Real terminal captures from a live Pipelines instance on 12 October 2026.
|
|
The execution views show retained runs from 11 October 2026.
|
|
|
|
**Workflow:** plan, implement, review, and UI review, with rejection routes back
|
|
to implementation.
|
|
|
|

|
|
|
|
**Execution:** a completed plan → implement → review run, with the review result
|
|
and the executed route highlighted.
|
|
|
|

|
|
|
|
**Harnesses:** OpenCode and Pi stages in the same successful Docker-backed run.
|
|
|
|

|
|
|
|
## Install
|
|
|
|
`./install` installs Pipelines **natively on the host by default** (macOS and
|
|
Linux); `./install --docker` installs the **Docker server + ntfy** explicitly.
|
|
Native setup detects (but never installs) Go, python3, git, curl and a C compiler
|
|
(`cc`/`clang`/`gcc`), plus an existing `opencode`/`pi`; it builds `pipelines-server`,
|
|
`pipelines` and ntfy from source into `~/.local/share/pipelines` (honouring
|
|
`PIPELINES_PREFIX`) and generates an ntfy configuration mirroring the Docker
|
|
settings. ntfy is compiled locally because upstream publishes no macOS server
|
|
binary, and the C compiler is required because its SQLite driver is cgo. Docker
|
|
setup needs Docker Compose and Python 3 and includes OpenCode/Pi in the persistent
|
|
server container.
|
|
For the host CLI/TUI walkthrough below, also install Go 1.27+; the Docker
|
|
installer installs/updates those clients when host Go is available.
|
|
|
|
```sh
|
|
git clone https://git.a-wan.net/alan/pipelines.git
|
|
cd pipelines
|
|
./install # native host install (Go, python3, git, curl, cc, opencode)
|
|
./install --docker # explicit Docker services + container provider login
|
|
```
|
|
|
|
Native setup reuses your existing host OpenCode login and never installs,
|
|
upgrades or downloads OpenCode/Pi. It writes host harness profiles (no Docker
|
|
link) for the harnesses it detects, so authoring works immediately, and clears a
|
|
stale Docker link when migrating a Docker install. After building, it warns if a
|
|
**different** `pipelines` resolves earlier on PATH than `<prefix>/bin/pipelines`,
|
|
naming that path and both versions, so a stale CLI is not left in front of a newer
|
|
server (it never modifies or upgrades that other binary). Start it with
|
|
`PIPELINES_PREFIX=~/.local/share/pipelines ~/.local/share/pipelines/bin/pipelines start`.
|
|
When Docker volumes are present it migrates the existing database and ntfy auth
|
|
store (never overwriting newer native data; use `--replace-data` to opt in). See
|
|
[installation](docs/install.md) for details.
|
|
|
|
The installer requires this Git checkout (including `.git`); a standalone source
|
|
archive cannot self-install. Without host Go you can still use the
|
|
Docker-contained TUI/CLI. Run Compose from the checkout with `--project-directory`
|
|
so it finds `compose.yaml`, and pass `--workdir "$PWD"` so the container keeps the
|
|
actual (disposable) workspace rather than its own image directory:
|
|
|
|
```sh
|
|
# TUI (allocate a TTY)
|
|
docker compose --project-directory /absolute/path/to/pipelines exec --workdir "$PWD" pipelines pipelines
|
|
# Plain CLI command (no TTY)
|
|
docker compose --project-directory /absolute/path/to/pipelines exec -T --workdir "$PWD" pipelines pipelines <command>
|
|
```
|
|
|
|
Point a host client at the mapped host port with
|
|
`--url ws://127.0.0.1:<actual-mapped-port>/connect`, reading the port from
|
|
`docker compose --project-directory /absolute/path/to/pipelines port pipelines 8080`.
|
|
The container listens on port **8080**; only the default host mapping is 8081.
|
|
Prefer the installer-saved configured endpoint when present; never replace it with
|
|
a guessed default.
|
|
|
|
Go uses `GOBIN` or its normal `GOPATH/bin` default; put that directory on PATH.
|
|
Running `pipelines` also compares its build/protocol with the server and warns (or
|
|
fails on a newer protocol) instead of dropping fields it does not model. The
|
|
Docker installer automatically detects/configures LAN networking for ntfy;
|
|
you do not need to look up your IP or supply a URL/listen address.
|
|
Both Docker services bind `0.0.0.0`; reinstall normalizes those bind settings
|
|
without replacing your URLs, timezone, ports or credentials.
|
|
The installer builds only CLI/TUI clients on the host when Go is available;
|
|
the server and stage harnesses run in Docker. It configures ntfy accounts/token,
|
|
checks publishing/health, and opens container OpenCode provider login on first setup;
|
|
existing settings, credentials and volumes are retained on reinstall.
|
|
Docker provider credentials live in its persistent container volume, separately
|
|
from your host OpenCode login; reinstalling or reconfiguring reuses the existing
|
|
container accounts and volumes rather than replacing them. Setup asks for ntfy
|
|
account passwords, then your model-provider login and models. It selects separate
|
|
saved defaults in order: first the interactive harness model, then the
|
|
new-pipeline model, in two fuzzy-searchable pickers (the saved value is
|
|
preselected and marked `(current)`; typing filters, arrow keys or Tab navigate,
|
|
Enter confirms, and Esc cancels without changing anything). Model selection
|
|
performs no inference; only the OpenCode path validates the new-pipeline model
|
|
with a small real generation (provider charges may apply). Fresh
|
|
`./install --docker --harness pi` skips OpenCode login and that generation check. Pi has no
|
|
standalone `auth login`: setup opens a fresh `/login` session (regular TUI mode)
|
|
where you choose an authentication method, reuse existing volume credentials by
|
|
exiting, or sign in with an API key. Each
|
|
setup resolves the current supported OpenCode V2 release from the single
|
|
documented complete `git/matching-refs/tags/v2` response (no pagination; never the
|
|
moving `latest` tag) and the current Pi release from npm, builds `FROM` the
|
|
OpenCode image's immutable `sha256` digest, and verifies the built image reports
|
|
exactly those executable versions (and a Node-compatible Pi engine). It stops with
|
|
a clear error instead of silently substituting a stale pin when discovery, build
|
|
or compatibility fails. Use `./install --opencode` or `./install --pi` to
|
|
reconfigure login and models inside the existing container without restarting
|
|
services: an image matching the release identity needs no rebuild, while a dirty
|
|
source always invokes the Docker build and relies on layer caching. The built
|
|
image includes python3 so the installer/verification tests can also run inside
|
|
the container.
|
|
Configuration and data remain per-user. The native default installer is described
|
|
above and in [installation](docs/install.md).
|
|
|
|
Docker setup starts the server and saves its client endpoint (the container port
|
|
is **8080**; only the default host mapping is **8081**). That saved endpoint is
|
|
authoritative: prefer it and never replace it with a guessed default. Do **not**
|
|
also start `pipelines serve` for this setup. Explicit `--url` and `PIPELINES_URL`
|
|
override the saved endpoint; restart existing TUIs after setup.
|
|
Docker mounts your home and installation checkout at their original absolute paths
|
|
and uses your UID/GID, so the checkout supplies `compose.yaml` while
|
|
`--workdir "$PWD"` preserves the actual disposable workspace. Other host
|
|
repositories need a bind mount at the same absolute path; repositories on another
|
|
client computer are not automatically transferred.
|
|
These unauthenticated server ports are for trusted networks only—do not expose
|
|
them to the public Internet.
|
|
|
|
### Developer/legacy local server (unsupported)
|
|
|
|
For developing Pipelines itself or running a previous Python-supervised legacy
|
|
host setup, an explicit foreground server remains available. This is not the
|
|
native installer and is not part of Docker setup; fresh databases contain no
|
|
pipelines:
|
|
|
|
```sh
|
|
pipelines serve # foreground developer/legacy server
|
|
# In another terminal:
|
|
pipelines
|
|
```
|
|
|
|
Optional developer/legacy autostart at login (starts the server now):
|
|
|
|
```sh
|
|
pipelines service enable
|
|
pipelines service status
|
|
```
|
|
|
|
Uses user-level systemd on Linux or launchd on macOS; no sudo. Runs use the
|
|
directory you invoke `pipelines run` from. Preview with `--print`. Stop any existing
|
|
foreground/legacy server first. Disable with `pipelines service disable` before
|
|
uninstalling. See [installation](docs/install.md) for environment and boot options.
|
|
|
|
## Your first pipeline and run
|
|
|
|
Start in the repository you want the workflow to operate on:
|
|
|
|
```sh
|
|
cd /path/to/your/repository
|
|
pipelines
|
|
```
|
|
|
|
1. **Create:** press **n** or use **:new**. Choose the configured **OpenCode** or
|
|
**Pi** harness to author a workflow together, or **nvim** to edit the Markdown
|
|
yourself. Harness authoring runs in the Docker server; only the **nvim** editor
|
|
is required on your client computer. Describe the stages, their inputs/outputs,
|
|
and the tools they need. Templates (**b** on Home) are optional; a fresh
|
|
installation has no published pipelines.
|
|
To skip authoring, use **:templates**, choose one of the four generic starters,
|
|
and press **s** or Enter on **Create pipeline**. Review the prefilled name
|
|
(edit it if needed), then press Enter to create—no separate naming or editor
|
|
step. **e Customize** edits it first. Templates include a smaller graph preview;
|
|
**Tab** switches focus and space between the description and graph. Use **hjkl**
|
|
to pan a focused graph and **n/p** to browse templates.
|
|
2. **Keep a draft:** finish/exit the editor. Nothing is published just because you
|
|
edited it. Press **d** / **:drafts** to find retained drafts; **e** edits and
|
|
**v** validates the selected draft.
|
|
3. **Create/update:** press **s** in Drafts. If it has no name, type one into the name
|
|
prompt and press Enter. Then review and explicitly confirm submission. The
|
|
action is **Create pipeline** for a new definition or **Update pipeline** for
|
|
a new version. The server validates it and retains immutable versions; the draft stays.
|
|
4. **Run:** select the published pipeline, press Enter to open it, then **:run** to
|
|
start a run. Supply your task and review the confirmation. The run uses the
|
|
directory from which you launched the client, not the installer checkout.
|
|
5. **Inspect:** watch the full graph and stage progress. **Tab** expands/focuses
|
|
panes; **n/p** select recorded stage visits; **hjkl** pan a focused graph.
|
|
On a failed run, **D** / **:debug** opens a fresh OpenCode debugging conversation;
|
|
retry requires confirmation and restarts the pinned workflow, so earlier side
|
|
effects may happen again.
|
|
|
|
Home shows only **/** search and **:** commands as action hints; **/** opens the omni
|
|
popup, which finds pipelines, versions, runs, stages/visits, artifact names and the
|
|
text of stage inputs, outputs and evidence, and Enter opens the chosen result. Use
|
|
the command palette for creation, drafts, templates, history, status, logs, and
|
|
quitting.
|
|
|
|
Quit with **:quit**; **q** does not exit. For a terminal-only run after publication:
|
|
|
|
```sh
|
|
pipelines run "your pipeline name" "Describe the task"
|
|
```
|
|
|
|
### Phone notifications
|
|
|
|
Open **:status** for the server and ntfy connection details. In the ntfy phone
|
|
app, add the displayed ntfy server, sign in as **phone-reader** with the password
|
|
you set during installation, and subscribe to the displayed topic. A run that
|
|
waits for a decision or fails notifies automatically; to notify on a stage's own
|
|
result, add a `notify` block. See [ntfy setup](docs/ntfy.md). Installation checks
|
|
publishing, not reception on your phone.
|
|
|
|
For Tailscale, both devices must be connected; use
|
|
`http://<server-Tailscale-IP>:8090`. Binding to `0.0.0.0` does not bypass firewalls:
|
|
UFW/Docker forwarding rules can allow LAN sources but drop Tailscale sources.
|
|
See [Tailscale troubleshooting](docs/ntfy.md#tailscale-address-times-out-while-lan-works).
|
|
|
|
## Uninstall
|
|
|
|
```sh
|
|
pipelines uninstall # remove Go binaries; keep history, drafts, and ntfy data
|
|
```
|
|
|
|
## A four-stage pipeline
|
|
|
|
**Plan → Implement → Review → Commit**. The first stage receives your task prompt.
|
|
Later stages receive the previous output or fields selected by an explicit
|
|
`memory` block. One outcome selects the next stage—no
|
|
parallel branches. Definitions use Markdown `output`, `prompt`, `tools`,
|
|
`configs`, and `edges` blocks. For example:
|
|
|
|
````markdown
|
|
# stage planner
|
|
|
|
```output
|
|
{ task, plan, design, reuse_targets, acceptance_checks, research }
|
|
```
|
|
|
|
```prompt
|
|
Turn input.prompt into a focused plan. Discover repository conventions and
|
|
reuse targets, and research official sources only when external knowledge is
|
|
actually required. State why when no research is needed.
|
|
```
|
|
|
|
```tools
|
|
read
|
|
glob
|
|
grep
|
|
websearch
|
|
webfetch
|
|
```
|
|
|
|
```configs
|
|
provider=opencode-go
|
|
model=deepseek-v4.1-flash
|
|
retries=2
|
|
timeout=1h
|
|
```
|
|
|
|
```edges
|
|
success -> implementer
|
|
blocked -> #failure
|
|
error -> #failure
|
|
```
|
|
````
|
|
|
|
The [complete three-stage example](examples/three-stage.md) adds the implementer,
|
|
reviewer and committer: the planner may search/fetch official sources, the
|
|
implementer has no web tools and edits and runs checks from the planner's research,
|
|
the reviewer independently inspects the real changes with its own shell access,
|
|
receiving the original task and the planner-owned acceptance checks rather than the
|
|
implementer's summary, and the committer stages the runner's captured changed paths
|
|
and commits locally without ever pushing. The reviewer approves only when the
|
|
applicable acceptance checks were independently observed passing and no unresolved
|
|
substantive issue remains, and otherwise rejects with concrete findings. Review and
|
|
explicitly submit it:
|
|
|
|
```sh
|
|
pipelines validate examples/three-stage.md
|
|
pipelines submit examples/three-stage.md --name "three stage" --create-only
|
|
pipelines run "three stage"
|
|
```
|
|
|
|
Every submission is validated and creates an immutable version. Examples are
|
|
optional; nothing is imported automatically. The four stages give a visible
|
|
plan → implement → review → commit path using your own accounts and repository; no
|
|
maintainer accounts or published definitions are assumed. Run from your
|
|
repository directory; stage tools are enabled automatically. Edits support any
|
|
language/file type and new files. No file list or typed RUN confirmation is
|
|
required.
|
|
|
|
## Useful commands
|
|
|
|
```sh
|
|
pipelines # dashboard; b browses templates
|
|
pipelines list
|
|
pipelines run # fuzzy selection + multiline prompt
|
|
pipelines run "bug fix" "Fix the layout" # run in the current repository
|
|
pipelines new "daily digest" "Summarize changes"
|
|
pipelines edit "bug fix" "Add a review step"
|
|
pipelines drafts --json
|
|
pipelines validate /path/to/draft.md
|
|
pipelines submit /path/to/draft.md # explicit publication
|
|
pipelines runs
|
|
pipelines inspect 1
|
|
pipelines serve # foreground server; Ctrl+C stops it
|
|
pipelines ping
|
|
pipelines version --json # local client identity (offline)
|
|
pipelines ping --json # running server identity
|
|
pipelines retention report --keep 200 # show what run-history retention would remove
|
|
pipelines retention apply --keep 200 --yes # remove it (artifacts go with their run)
|
|
pipelines help edit
|
|
```
|
|
|
|
Run-history retention is explicit and never automatic: nothing prunes runs or
|
|
their artifacts until you run `pipelines retention apply`, protected runs (active,
|
|
waiting on a person, or referenced by a retained retry) are kept, and pipeline
|
|
definitions and versions are never touched.
|
|
|
|
CLI `new`/`edit` open the configured harness (**OpenCode** or **Pi**, native host
|
|
by default; a profile is Docker-linked only when a checkout is explicitly linked)
|
|
directly with the Pipelines skill. The installed skill asks
|
|
missing questions; `pipelines new` needs no preliminary questionnaire. Changes stay
|
|
as drafts by default; CLI and TUI share validation and draft storage. Most
|
|
commands support `--json` for agents and scripts.
|
|
|
|
In the TUI, press **d** (or use **:drafts**) to browse retained drafts, including
|
|
CLI-created ones. **e** reopens a draft, **v** validates it, and **s** opens explicit
|
|
submission confirmation, asking for a name first if needed. Drafts remain retained
|
|
after submission. Drafts are portable: submit to the currently connected server.
|
|
When that differs from the origin, submission creates a new pipeline rather than
|
|
applying the origin's version number to an existing destination pipeline, and a
|
|
notice names both server URLs instead of silently retargeting the draft.
|
|
|
|
From a run, press **P** / **:pipeline** to open the pipeline version it used.
|
|
Initial run input also records `pwd`. Press **r** / **:retry** on a failed run
|
|
(or `pipelines retry <run-id>`) to start a new attempt with the same original
|
|
input, directory, and pipeline version. Completed side effects are not undone.
|
|
Retries restart at the entry stage with fresh OpenCode sessions, never the previous
|
|
conversation or intermediate stage output. Pipelines does not automatically delete
|
|
generation, agent, or formatting-repair sessions, including on failure/cancellation.
|
|
Primary session IDs are retained in run steps (`pipelines inspect <run-id> --json`).
|
|
Every run also records a short, single-line title: `pipelines run ... --title "…"`
|
|
sets one explicitly, otherwise it is generated from the prompt. `pipelines runs`
|
|
and `pipelines inspect` show it beside the pipeline name and version, and
|
|
`pipelines title <run-id> "…"` renames it afterwards.
|
|
A pipeline may include a `type=human` decision stage: the run pauses as
|
|
**waiting** with the stage's instructions and options visible in `inspect` and the
|
|
run inspector. Answer it with `pipelines decide <run-id> [stage] --option KEY
|
|
[--feedback "…"]` or by pressing **c** in the run inspector to pick an option and
|
|
type feedback. The choice and feedback are recorded as the step's evidence and the
|
|
run resumes along that edge. The wait is persisted, so it survives a server
|
|
restart; an unknown option is refused with the valid list, a stage that is not
|
|
waiting is refused, and a repeated decision does not double-apply. **cancel** and
|
|
**retry** keep working while waiting and after a decision.
|
|
On Home, **r** opens global run history; **/** opens the omni search popup over
|
|
pipelines, versions, runs, stages/visits, artifacts and stage input/output/evidence
|
|
text. Inside a pipeline, **r** / **:runs** opens only that pipeline's history.
|
|
Quit the TUI with **:quit**; **q** does not exit. **Ctrl+C** remains an interrupt.
|
|
Starting/retrying/canceling runs and deleting drafts require confirmation. CLI
|
|
publication also prompts; use `--yes` only for explicitly approved automation.
|
|
**n / :new** and creating from a template offer a Tab-navigable, fuzzy-searchable
|
|
picker: **nvim** opens the draft Markdown file; **OpenCode**/**Pi** author it with
|
|
you. Canceling the picker creates no draft and publishes nothing.
|
|
Wide terminals show runs and pipelines side by side; narrow terminals stack them.
|
|
The TUI uses soft light/dark-adaptive cards, muted metadata, subtle selections,
|
|
and local loading animations instead of heavy character borders and inverted bars.
|
|
The dashboard includes all-history usage, success rate, average completed duration,
|
|
per-pipeline usage, and a 14-day activity chart. CLI: `pipelines stats [--json]`.
|
|
Use **:status** for two panels: server addresses and ntfy phone connection details.
|
|
CLI: `pipelines status [--json]`. Candidate URLs are not
|
|
verified reachable; publishing endpoints and credentials are not disclosed.
|
|
Retry is offered only after the failed run and its agent have fully stopped.
|
|
Graphs use content-sized boxes/columns and tightly spaced branch lanes while
|
|
retaining every stage and labeled box-and-arrow route. Incoming route IDs (left)
|
|
and outgoing outcomes (right) share rows to reduce box height. Both pipeline and
|
|
run views center the graph horizontally and vertically when it fits.
|
|
The run inspector shows the entire pinned graph above the details, dimming stages
|
|
and routes not executed. **g** focuses the graph; **hjkl** or arrows pan it.
|
|
The path and elapsed duration update while running. **a** opens the active/latest
|
|
OpenCode session's recorded tool evidence; this does not attach a live OpenCode UI.
|
|
Press **D** / **:debug** (or `pipelines debug <run-id>`) for a fresh OpenCode
|
|
conversation about that run. It uses CLI queries rather than resuming or retrying
|
|
the implementing agent. Pipeline **Edit** offers a fuzzy-searchable **nvim /
|
|
OpenCode** picker; Tab and Shift+Tab navigate it.
|
|
In the draft browser, **x** then **y** deletes the selected retained draft and
|
|
metadata. CLI: `pipelines draft-delete <draft.md>`. Published history is untouched.
|
|
|
|
Model results use native JSON-schema output where supported, precise parsing
|
|
errors, and bounded format-only repair without replaying tools. Retained failures
|
|
distinguish final-result validation from harness transport. Optional stage
|
|
permissions narrow file/command access without imposing a language-specific
|
|
default. See [model result recovery and permissions](docs/model-results.md).
|
|
The run graph header shows start time and elapsed/final duration, plus finish
|
|
time for completed runs.
|
|
|
|
More: [CLI](docs/cli.md) · [installation](docs/install.md) · [pipeline format](internal/author/skill/references/format.md).
|