914345cb31
Native installs left Docker-era harness profiles in harnesses.json, so every authoring launch failed trying to reach a Docker service, and a retained draft bound to the old Docker server was silently projected onto the configured one. - harness: add Profile.Mode and VerifyDockerLink as the single mode/verification helpers. `harness setup --host` configures or converts a host profile and clears docker_directory; reconfiguring a Docker-linked profile re-verifies the recorded directory still owns compose.yaml and fails naming it instead of saving a link that cannot launch. Setup's summary and `harness list` (via the new List/ListView shape) report the resulting mode and, for Docker, the directory, without changing the stored Config schema. - setup-native.py: after build/install-skill it writes host (no docker_directory) profiles for the detected harnesses, defaulting to the detected OpenCode profile rather than a stale pi default, taking models from --model or the existing valid profile and never inventing one; it clears a stale Docker link and preserves unrelated keys/profiles. Detection-only: no harness install, upgrade or download. - draft: add Draft.Resolve, the shared mismatch helper, which projects a draft onto the configured server and returns a dual-URL notice. CLI validate/submit and the TUI draft browser use it so a foreign draft is never silently retargeted; the TUI pre-wraps the notice to the panel width. - Tests cover the Docker->host and host->host transitions, the missing-directory failure, installer write/migration (Python), and the CLI/TUI notice, using the real config file path in a temp dir. - Update help text, README, docs and the packaged authoring skill reference to describe host as the default and the Docker-linked case.
388 lines
20 KiB
Markdown
388 lines
20 KiB
Markdown
# Pipelines
|
|
|
|
Configured OpenCode/Pi harnesses, separate model defaults, runner-enforced review
|
|
limits and selected run-local context: [guide](docs/harnesses-and-control.md).
|
|
|
|
Reusable, sequential workflows for your repositories, with a Go CLI/TUI,
|
|
OpenCode model stages, retained run history, and optional phone notifications.
|
|
|
|
**First-version preview:** a single-user tool for supervised work. Start with a
|
|
disposable repository, review edits and command evidence, and expect rough edges.
|
|
See the [tester walkthrough and known limits](docs/first-version-testing.md).
|
|
|
|
Stage results include [retained, highlighted workspace diffs](docs/stage-file-diffs.md)
|
|
for newly executed visits, alongside their reports and audit evidence. Stages can
|
|
also [record and serve their own artifacts](docs/stage-artifacts.md) - images,
|
|
screenshots, reports and logs - per step.
|
|
|
|
Use `:export` / `:import` to [select and transfer pipeline definitions](docs/pipeline-transfer.md)
|
|
between servers as portable ZIP bundles.
|
|
|
|
[Real-provider reference workflows](docs/reference-workflows.md) exercise structured
|
|
handoffs, file edits/verification, failure routing and reported-usage budgets with
|
|
independent acceptance checks and retained evidence. Live inference is opt-in.
|
|
|
|
## In action
|
|
|
|
Real terminal captures from a live Pipelines instance on 12 October 2026.
|
|
The execution views show retained runs from 11 October 2026.
|
|
|
|
**Workflow:** plan, implement, review, and UI review, with rejection routes back
|
|
to implementation.
|
|
|
|

|
|
|
|
**Execution:** a completed plan → implement → review run, with the review result
|
|
and the executed route highlighted.
|
|
|
|

|
|
|
|
**Harnesses:** OpenCode and Pi stages in the same successful Docker-backed run.
|
|
|
|

|
|
|
|
## Install
|
|
|
|
`./install` installs Pipelines **natively on the host by default** (macOS and
|
|
Linux); `./install --docker` installs the **Docker server + ntfy** explicitly.
|
|
Native setup detects (but never installs) Go, python3, git, curl and a C compiler
|
|
(`cc`/`clang`/`gcc`), plus an existing `opencode`/`pi`; it builds `pipelines-server`,
|
|
`pipelines` and ntfy from source into `~/.local/share/pipelines` (honouring
|
|
`PIPELINES_PREFIX`) and generates an ntfy configuration mirroring the Docker
|
|
settings. ntfy is compiled locally because upstream publishes no macOS server
|
|
binary, and the C compiler is required because its SQLite driver is cgo. Docker
|
|
setup needs Docker Compose and Python 3 and includes OpenCode/Pi in the persistent
|
|
server container.
|
|
For the host CLI/TUI walkthrough below, also install Go 1.27+; the Docker
|
|
installer installs/updates those clients when host Go is available.
|
|
|
|
```sh
|
|
git clone https://git.a-wan.net/alan/pipelines.git
|
|
cd pipelines
|
|
./install # native host install (Go, python3, git, curl, cc, opencode)
|
|
./install --docker # explicit Docker services + container provider login
|
|
```
|
|
|
|
Native setup reuses your existing host OpenCode login and never installs,
|
|
upgrades or downloads OpenCode/Pi. It writes host harness profiles (no Docker
|
|
link) for the harnesses it detects, so authoring works immediately, and clears a
|
|
stale Docker link when migrating a Docker install. Start it with
|
|
`PIPELINES_PREFIX=~/.local/share/pipelines ~/.local/share/pipelines/bin/pipelines start`.
|
|
When Docker volumes are present it migrates the existing database and ntfy auth
|
|
store (never overwriting newer native data; use `--replace-data` to opt in). See
|
|
[installation](docs/install.md) for details.
|
|
|
|
The installer requires this Git checkout (including `.git`); a standalone source
|
|
archive cannot self-install. Without host Go you can still use the
|
|
Docker-contained TUI/CLI. Run Compose from the checkout with `--project-directory`
|
|
so it finds `compose.yaml`, and pass `--workdir "$PWD"` so the container keeps the
|
|
actual (disposable) workspace rather than its own image directory:
|
|
|
|
```sh
|
|
# TUI (allocate a TTY)
|
|
docker compose --project-directory /absolute/path/to/pipelines exec --workdir "$PWD" pipelines pipelines
|
|
# Plain CLI command (no TTY)
|
|
docker compose --project-directory /absolute/path/to/pipelines exec -T --workdir "$PWD" pipelines pipelines <command>
|
|
```
|
|
|
|
Point a host client at the mapped host port with
|
|
`--url ws://127.0.0.1:<actual-mapped-port>/connect`, reading the port from
|
|
`docker compose --project-directory /absolute/path/to/pipelines port pipelines 8080`.
|
|
The container listens on port **8080**; only the default host mapping is 8081.
|
|
Prefer the installer-saved configured endpoint when present; never replace it with
|
|
a guessed default.
|
|
|
|
Go uses `GOBIN` or its normal `GOPATH/bin` default; put that directory on PATH.
|
|
The Docker installer automatically detects/configures LAN networking for ntfy;
|
|
you do not need to look up your IP or supply a URL/listen address.
|
|
Both Docker services bind `0.0.0.0`; reinstall normalizes those bind settings
|
|
without replacing your URLs, timezone, ports or credentials.
|
|
The installer builds only CLI/TUI clients on the host when Go is available;
|
|
the server and stage harnesses run in Docker. It configures ntfy accounts/token,
|
|
checks publishing/health, and opens container OpenCode provider login on first setup;
|
|
existing settings, credentials and volumes are retained on reinstall.
|
|
Docker provider credentials live in its persistent container volume, separately
|
|
from your host OpenCode login; reinstalling or reconfiguring reuses the existing
|
|
container accounts and volumes rather than replacing them. Setup asks for ntfy
|
|
account passwords, then your model-provider login and models. It selects separate
|
|
saved defaults in order: first the interactive harness model, then the
|
|
new-pipeline model, in two fuzzy-searchable pickers (the saved value is
|
|
preselected and marked `(current)`; typing filters, arrow keys or Tab navigate,
|
|
Enter confirms, and Esc cancels without changing anything). Model selection
|
|
performs no inference; only the OpenCode path validates the new-pipeline model
|
|
with a small real generation (provider charges may apply). Fresh
|
|
`./install --docker --harness pi` skips OpenCode login and that generation check. Pi has no
|
|
standalone `auth login`: setup opens a fresh `/login` session (regular TUI mode)
|
|
where you choose an authentication method, reuse existing volume credentials by
|
|
exiting, or sign in with an API key. Each
|
|
setup resolves the current supported OpenCode V2 release from the single
|
|
documented complete `git/matching-refs/tags/v2` response (no pagination; never the
|
|
moving `latest` tag) and the current Pi release from npm, builds `FROM` the
|
|
OpenCode image's immutable `sha256` digest, and verifies the built image reports
|
|
exactly those executable versions (and a Node-compatible Pi engine). It stops with
|
|
a clear error instead of silently substituting a stale pin when discovery, build
|
|
or compatibility fails. Use `./install --opencode` or `./install --pi` to
|
|
reconfigure login and models inside the existing container without restarting
|
|
services: an image matching the release identity needs no rebuild, while a dirty
|
|
source always invokes the Docker build and relies on layer caching. The built
|
|
image includes python3 so the installer/verification tests can also run inside
|
|
the container.
|
|
Configuration and data remain per-user. The native default installer is described
|
|
above and in [installation](docs/install.md).
|
|
|
|
Docker setup starts the server and saves its client endpoint (the container port
|
|
is **8080**; only the default host mapping is **8081**). That saved endpoint is
|
|
authoritative: prefer it and never replace it with a guessed default. Do **not**
|
|
also start `pipelines serve` for this setup. Explicit `--url` and `PIPELINES_URL`
|
|
override the saved endpoint; restart existing TUIs after setup.
|
|
Docker mounts your home and installation checkout at their original absolute paths
|
|
and uses your UID/GID, so the checkout supplies `compose.yaml` while
|
|
`--workdir "$PWD"` preserves the actual disposable workspace. Other host
|
|
repositories need a bind mount at the same absolute path; repositories on another
|
|
client computer are not automatically transferred.
|
|
These unauthenticated server ports are for trusted networks only—do not expose
|
|
them to the public Internet.
|
|
|
|
### Developer/legacy local server (unsupported)
|
|
|
|
For developing Pipelines itself or running a previous Python-supervised legacy
|
|
host setup, an explicit foreground server remains available. This is not the
|
|
native installer and is not part of Docker setup; fresh databases contain no
|
|
pipelines:
|
|
|
|
```sh
|
|
pipelines serve # foreground developer/legacy server
|
|
# In another terminal:
|
|
pipelines
|
|
```
|
|
|
|
Optional developer/legacy autostart at login (starts the server now):
|
|
|
|
```sh
|
|
pipelines service enable
|
|
pipelines service status
|
|
```
|
|
|
|
Uses user-level systemd on Linux or launchd on macOS; no sudo. Runs use the
|
|
directory you invoke `pipelines run` from. Preview with `--print`. Stop any existing
|
|
foreground/legacy server first. Disable with `pipelines service disable` before
|
|
uninstalling. See [installation](docs/install.md) for environment and boot options.
|
|
|
|
## Your first pipeline and run
|
|
|
|
Start in the repository you want the workflow to operate on:
|
|
|
|
```sh
|
|
cd /path/to/your/repository
|
|
pipelines
|
|
```
|
|
|
|
1. **Create:** press **n** or use **:new**. Choose the configured **OpenCode** or
|
|
**Pi** harness to author a workflow together, or **nvim** to edit the Markdown
|
|
yourself. Harness authoring runs in the Docker server; only the **nvim** editor
|
|
is required on your client computer. Describe the stages, their inputs/outputs,
|
|
and the tools they need. Templates (**b** on Home) are optional; a fresh
|
|
installation has no published pipelines.
|
|
To skip authoring, use **:templates**, choose one of the five generic starters,
|
|
and press **s** or Enter on **Create pipeline**. Review the prefilled name
|
|
(edit it if needed), then press Enter to create—no separate naming or editor
|
|
step. **e Customize** edits it first. Templates include a smaller graph preview;
|
|
**Tab** switches focus and space between the description and graph. Use **hjkl**
|
|
to pan a focused graph and **n/p** to browse templates.
|
|
2. **Keep a draft:** finish/exit the editor. Nothing is published just because you
|
|
edited it. Press **d** / **:drafts** to find retained drafts; **e** edits and
|
|
**v** validates the selected draft.
|
|
3. **Create/update:** press **s** in Drafts. If it has no name, type one into the name
|
|
prompt and press Enter. Then review and explicitly confirm submission. The
|
|
action is **Create pipeline** for a new definition or **Update pipeline** for
|
|
a new version. The server validates it and retains immutable versions; the draft stays.
|
|
4. **Run:** select the published pipeline, press Enter to open it, then **:run** to
|
|
start a run. Supply your task and review the confirmation. The run uses the
|
|
directory from which you launched the client, not the installer checkout.
|
|
5. **Inspect:** watch the full graph and stage progress. **Tab** expands/focuses
|
|
panes; **n/p** select recorded stage visits; **hjkl** pan a focused graph.
|
|
On a failed run, **D** / **:debug** opens a fresh OpenCode debugging conversation;
|
|
retry requires confirmation and restarts the pinned workflow, so earlier side
|
|
effects may happen again.
|
|
|
|
Home shows only **/** search and **:** commands as action hints; use the command
|
|
palette for creation, drafts, templates, history, status, logs, and quitting.
|
|
|
|
Quit with **:quit**; **q** does not exit. For a terminal-only run after publication:
|
|
|
|
```sh
|
|
pipelines run "your pipeline name" "Describe the task"
|
|
```
|
|
|
|
### Phone notifications
|
|
|
|
Open **:status** for the server and ntfy connection details. In the ntfy phone
|
|
app, add the displayed ntfy server, sign in as **phone-reader** with the password
|
|
you set during installation, and subscribe to the displayed topic. Stages must
|
|
include a `notify` block to send notifications; see [ntfy setup](docs/ntfy.md).
|
|
Installation checks publishing, not reception on your phone.
|
|
|
|
For Tailscale, both devices must be connected; use
|
|
`http://<server-Tailscale-IP>:8090`. Binding to `0.0.0.0` does not bypass firewalls:
|
|
UFW/Docker forwarding rules can allow LAN sources but drop Tailscale sources.
|
|
See [Tailscale troubleshooting](docs/ntfy.md#tailscale-address-times-out-while-lan-works).
|
|
|
|
## Uninstall
|
|
|
|
```sh
|
|
pipelines uninstall # remove Go binaries; keep history, drafts, and ntfy data
|
|
```
|
|
|
|
## A three-stage pipeline
|
|
|
|
**Plan → Implement → Review**. The first stage receives your task prompt.
|
|
Later stages receive the previous output or fields selected by an explicit
|
|
`memory` block. One outcome selects the next stage—no
|
|
parallel branches. Definitions use Markdown `output`, `prompt`, `tools`,
|
|
`configs`, and `edges` blocks. For example:
|
|
|
|
````markdown
|
|
# stage planner
|
|
|
|
```output
|
|
{ task, plan, design, reuse_targets, acceptance_checks, research }
|
|
```
|
|
|
|
```prompt
|
|
Turn input.prompt into a focused plan. Discover repository conventions and
|
|
reuse targets, and research official sources only when external knowledge is
|
|
actually required. State why when no research is needed.
|
|
```
|
|
|
|
```tools
|
|
read
|
|
glob
|
|
grep
|
|
websearch
|
|
webfetch
|
|
```
|
|
|
|
```configs
|
|
provider=opencode-go
|
|
model=deepseek-v4.1-flash
|
|
retries=2
|
|
timeout=1h
|
|
```
|
|
|
|
```edges
|
|
success -> implementer
|
|
blocked -> #failure
|
|
error -> #failure
|
|
```
|
|
````
|
|
|
|
The [complete three-stage example](examples/three-stage.md) adds the implementer
|
|
and reviewer: the planner may search/fetch official sources, the implementer has no
|
|
web tools and edits and runs checks from the planner's research, and the reviewer
|
|
independently inspects the real changes with its own shell access, receiving the
|
|
original task and the planner-owned acceptance checks rather than the
|
|
implementer's summary. The reviewer approves only when the applicable acceptance
|
|
checks were independently observed passing and no unresolved substantive issue
|
|
remains, and otherwise rejects with concrete findings. Review and explicitly
|
|
submit it:
|
|
|
|
```sh
|
|
pipelines validate examples/three-stage.md
|
|
pipelines submit examples/three-stage.md --name "three stage" --create-only
|
|
pipelines run "three stage"
|
|
```
|
|
|
|
Every submission is validated and creates an immutable version. Examples are
|
|
optional; nothing is imported automatically. The three stages give a visible
|
|
plan → implement → review path using your own accounts and repository; no
|
|
maintainer accounts or published definitions are assumed. Run from your
|
|
repository directory; stage tools are enabled automatically. Edits support any
|
|
language/file type and new files. No file list or typed RUN confirmation is
|
|
required.
|
|
|
|
## Useful commands
|
|
|
|
```sh
|
|
pipelines # dashboard; b browses templates
|
|
pipelines list
|
|
pipelines run # fuzzy selection + multiline prompt
|
|
pipelines run "bug fix" "Fix the layout" # run in the current repository
|
|
pipelines new "daily digest" "Summarize changes"
|
|
pipelines edit "bug fix" "Add a review step"
|
|
pipelines drafts --json
|
|
pipelines validate /path/to/draft.md
|
|
pipelines submit /path/to/draft.md # explicit publication
|
|
pipelines runs
|
|
pipelines inspect 1
|
|
pipelines serve # foreground server; Ctrl+C stops it
|
|
pipelines ping
|
|
pipelines version --json # local client identity (offline)
|
|
pipelines ping --json # running server identity
|
|
pipelines help edit
|
|
```
|
|
|
|
CLI `new`/`edit` open the configured harness (**OpenCode** or **Pi**, native host
|
|
by default; a profile is Docker-linked only when a checkout is explicitly linked)
|
|
directly with the Pipelines skill. The installed skill asks
|
|
missing questions; `pipelines new` needs no preliminary questionnaire. Changes stay
|
|
as drafts by default; CLI and TUI share validation and draft storage. Most
|
|
commands support `--json` for agents and scripts.
|
|
|
|
In the TUI, press **d** (or use **:drafts**) to browse retained drafts, including
|
|
CLI-created ones. **e** reopens a draft, **v** validates it, and **s** opens explicit
|
|
submission confirmation, asking for a name first if needed. Drafts remain retained
|
|
after submission. Drafts are portable: submit to the currently connected server.
|
|
When that differs from the origin, submission creates a new pipeline rather than
|
|
applying the origin's version number to an existing destination pipeline, and a
|
|
notice names both server URLs instead of silently retargeting the draft.
|
|
|
|
From a run, press **P** / **:pipeline** to open the pipeline version it used.
|
|
Initial run input also records `pwd`. Press **r** / **:retry** on a failed run
|
|
(or `pipelines retry <run-id>`) to start a new attempt with the same original
|
|
input, directory, and pipeline version. Completed side effects are not undone.
|
|
Retries restart at the entry stage with fresh OpenCode sessions, never the previous
|
|
conversation or intermediate stage output. Pipelines does not automatically delete
|
|
generation, agent, or formatting-repair sessions, including on failure/cancellation.
|
|
Primary session IDs are retained in run steps (`pipelines inspect <run-id> --json`).
|
|
On Home, **r** opens global run history; **/** fuzzy-searches it. Inside a pipeline,
|
|
**r** / **:runs** opens only that pipeline's history.
|
|
Quit the TUI with **:quit**; **q** does not exit. **Ctrl+C** remains an interrupt.
|
|
Starting/retrying/canceling runs and deleting drafts require confirmation. CLI
|
|
publication also prompts; use `--yes` only for explicitly approved automation.
|
|
**n / :new** and creating from a template offer a Tab-navigable, fuzzy-searchable
|
|
picker: **nvim** opens the draft Markdown file; **OpenCode**/**Pi** author it with
|
|
you. Canceling the picker creates no draft and publishes nothing.
|
|
Wide terminals show runs and pipelines side by side; narrow terminals stack them.
|
|
The TUI uses soft light/dark-adaptive cards, muted metadata, subtle selections,
|
|
and local loading animations instead of heavy character borders and inverted bars.
|
|
The dashboard includes all-history usage, success rate, average completed duration,
|
|
per-pipeline usage, and a 14-day activity chart. CLI: `pipelines stats [--json]`.
|
|
Use **:status** for two panels: server addresses and ntfy phone connection details.
|
|
CLI: `pipelines status [--json]`. Candidate URLs are not
|
|
verified reachable; publishing endpoints and credentials are not disclosed.
|
|
Retry is offered only after the failed run and its agent have fully stopped.
|
|
Graphs use content-sized boxes/columns and tightly spaced branch lanes while
|
|
retaining every stage and labeled box-and-arrow route. Incoming route IDs (left)
|
|
and outgoing outcomes (right) share rows to reduce box height. Both pipeline and
|
|
run views center the graph horizontally and vertically when it fits.
|
|
The run inspector shows the entire pinned graph above the details, dimming stages
|
|
and routes not executed. **g** focuses the graph; **hjkl** or arrows pan it.
|
|
The path and elapsed duration update while running. **a** opens the active/latest
|
|
OpenCode session's recorded tool evidence; this does not attach a live OpenCode UI.
|
|
Press **D** / **:debug** (or `pipelines debug <run-id>`) for a fresh OpenCode
|
|
conversation about that run. It uses CLI queries rather than resuming or retrying
|
|
the implementing agent. Pipeline **Edit** offers a fuzzy-searchable **nvim /
|
|
OpenCode** picker; Tab and Shift+Tab navigate it.
|
|
In the draft browser, **x** then **y** deletes the selected retained draft and
|
|
metadata. CLI: `pipelines draft-delete <draft.md>`. Published history is untouched.
|
|
|
|
Model results use native JSON-schema output where supported, precise parsing
|
|
errors, and bounded format-only repair without replaying tools. Retained failures
|
|
distinguish final-result validation from harness transport. Optional stage
|
|
permissions narrow file/command access without imposing a language-specific
|
|
default. See [model result recovery and permissions](docs/model-results.md).
|
|
The run graph header shows start time and elapsed/final duration, plus finish
|
|
time for completed runs.
|
|
|
|
More: [CLI](docs/cli.md) · [installation](docs/install.md) · [pipeline format](internal/author/skill/references/format.md).
|