Document the current install, harnesses and notifications
Rewrite the README around the server, web/Android app and CLI, and replace docs/install.md with the current native install (all-or-nothing Docker web build, flags from setup-native.py, lifecycle commands, data locations, password, upgrading). Add docs/harnesses.md and docs/notifications.md; point ntfy.md at the new overview and fix its topic name.
This commit is contained in:
@@ -58,9 +58,10 @@ a new execution can repeat side effects. Never delete data/volumes to fix a run.
|
||||
## Notifications
|
||||
|
||||
Browser alerts require the page to be open; closed-app web push is not supported.
|
||||
ntfy is a separate optional channel and can notify your phone without the
|
||||
The Android app delivers notifications in the background over its own
|
||||
connection. ntfy is an optional fallback channel that works without the
|
||||
Pipelines Android app. Installation verifies publishing, not reception. Check
|
||||
your subscription and device reception before relying on it. See [ntfy](ntfy.md).
|
||||
device reception before relying on it. See [Notifications](notifications.md).
|
||||
|
||||
## Known limits
|
||||
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
# Harnesses
|
||||
|
||||
A harness is the agent program Pipelines drives. Pipelines never installs,
|
||||
upgrades or signs in a harness for you; it uses the copy and login you already
|
||||
have on the server machine.
|
||||
|
||||
| Harness | Pipeline stages | Chat | Mid-turn messages | Authoring/debug sessions |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| OpenCode | yes (default) | yes (default) | queued until the turn ends | yes |
|
||||
| Pi | yes (`harness=pi`) | no | n/a | yes |
|
||||
| Claude Code | no | yes | yes, delivered into the running turn | no |
|
||||
| Codex | no | yes | yes, via `turn/steer` | no |
|
||||
|
||||
Stage configs accept only `harness=opencode` or `harness=pi`.
|
||||
|
||||
## Choosing models
|
||||
|
||||
**Pipeline stages.** Each model stage pins its own model in its `configs`
|
||||
block:
|
||||
|
||||
````markdown
|
||||
```configs
|
||||
provider=PROVIDER
|
||||
model=MODEL
|
||||
harness=opencode
|
||||
reasoning_effort=high
|
||||
```
|
||||
````
|
||||
|
||||
`harness` defaults to `opencode`. `reasoning_effort` is optional; see
|
||||
[harnesses and control](harnesses-and-control.md). Use the exact IDs your
|
||||
harness lists (`opencode models`, `pi --list-models`).
|
||||
|
||||
**Per run**, without changing the definition:
|
||||
|
||||
```sh
|
||||
pipelines run "my pipeline" "task" --model PROVIDER/MODEL
|
||||
pipelines run "my pipeline" "task" --stage-model review=PROVIDER/MODEL
|
||||
```
|
||||
|
||||
Precedence: `--stage-model` > `--model` > the model pinned in the stage.
|
||||
Unknown stage keys and malformed models are refused before the run starts.
|
||||
|
||||
**Defaults for new pipelines and interactive sessions** come from harness
|
||||
profiles in `~/.config/pipelines/harnesses.json` (override with
|
||||
`PIPELINES_HARNESS_CONFIG`). Each profile has an `executable`, a `model`
|
||||
(interactive authoring/debug sessions) and a `pipeline_model` (stages in new
|
||||
drafts and starters). Published versions are never rewritten.
|
||||
|
||||
```sh
|
||||
pipelines harness list # profiles, default, launch mode
|
||||
pipelines harness setup opencode --host # fuzzy-pick both models
|
||||
pipelines harness setup pi --host --login # sign in to Pi, then pick models
|
||||
pipelines harness setup opencode --host --model P/M --pipeline-model P/M
|
||||
pipelines harness default pi # default harness for new/edit/debug
|
||||
```
|
||||
|
||||
`--host` writes a native profile and clears any Docker link. `./install`
|
||||
writes the OpenCode host profile itself (from `--model`, the saved profile, or
|
||||
OpenCode's own default) and keeps an existing valid Pi profile; it never adds
|
||||
a Pi profile on its own. `./install --opencode` and `./install --pi` (which run
|
||||
`scripts/setup-harness.py`) apply only to a Docker installation.
|
||||
|
||||
**Chat** models are chosen in the chat's model picker; each harness supplies
|
||||
its own catalog.
|
||||
|
||||
## Verifying
|
||||
|
||||
```sh
|
||||
pipelines doctor # health + OpenCode API discovery, no inference
|
||||
pipelines provider-check --model P/M # one tiny tool-free generation (may cost)
|
||||
pipelines provider-check --harness pi --model P/M
|
||||
python3 scripts/check-pi-rpc.py # Pi RPC and policy, no inference
|
||||
```
|
||||
|
||||
`pipelines restart` and `pipelines doctor` print the server's startup report.
|
||||
Its **Chat** row lists the chat harnesses the server found (for example
|
||||
`claude, codex, opencode`), and the **Harnesses** rows list the profiles. Then
|
||||
start a chat with each harness in the app.
|
||||
|
||||
## OpenCode
|
||||
|
||||
**Used for:** default stage harness, default chat harness, and `pipelines
|
||||
new`, `edit` and `debug` sessions.
|
||||
|
||||
**Found by:** `PIPELINES_OPENCODE`, then `PATH`, then `/opt/homebrew/bin`,
|
||||
`/usr/local/bin`, `/usr/bin`, `~/.local/bin`. `./install` requires `opencode`
|
||||
on `PATH` and records its absolute path.
|
||||
|
||||
**Launched as:** the server starts its own loopback `opencode serve` for chat
|
||||
and for stage sessions, using your normal OpenCode configuration and login.
|
||||
A server started with `-opencode-endpoint URL` (or
|
||||
`PIPELINES_OPENCODE_ENDPOINT`) attaches chat to that service instead; the
|
||||
native installer does not set this.
|
||||
|
||||
**Install and sign in:** install OpenCode yourself, then sign in to your
|
||||
providers with OpenCode (`opencode auth login`). Pipelines reads the default
|
||||
model with `opencode api get /api/model/default`.
|
||||
|
||||
**Mid-turn messages:** not supported by the harness. A message sent while a
|
||||
turn runs is stored in the server's durable queue (up to 8 per chat) and sent
|
||||
in order when the turn ends, also after a server restart.
|
||||
|
||||
## Pi
|
||||
|
||||
**Used for:** pipeline stages with `harness=pi`, and `new`/`edit`/`debug`
|
||||
with `--harness pi`. Not a chat harness.
|
||||
|
||||
**Found by:** the `executable` in the `pi` profile. A run with a Pi stage is
|
||||
refused unless that profile exists on the server.
|
||||
|
||||
**Launched as:** `pi --mode rpc --provider P --model M --session-dir <run dir>
|
||||
--no-approve --no-extensions --no-skills --no-context-files
|
||||
--no-prompt-templates --no-themes --extension policy.js`. The policy extension
|
||||
enforces the stage's tool and edit permissions. `reasoning_effort` maps to Pi's
|
||||
thinking level and is verified before the task is sent.
|
||||
|
||||
**Install and sign in:**
|
||||
|
||||
```sh
|
||||
# install Pi yourself so `pi` is on PATH, then:
|
||||
pipelines harness setup pi --host --login
|
||||
```
|
||||
|
||||
`--login` opens a bare Pi session; run `/login` there, finish the provider
|
||||
sign-in, and exit with Ctrl+C or Ctrl+D to return to model selection.
|
||||
|
||||
## Claude Code
|
||||
|
||||
**Used for:** chat only.
|
||||
|
||||
**Found by:** `PIPELINES_CLAUDE`, then `PATH`, then `~/.local/bin/claude`,
|
||||
`/opt/homebrew/bin/claude`, `/usr/local/bin/claude`. The server checks again
|
||||
every 10 seconds, so installing it later needs no restart.
|
||||
|
||||
**Launched as:** one `claude -p` process per active chat with stream-json input
|
||||
and output and `--permission-prompt-tool=stdio` (at most 8 live at once). Idle
|
||||
chats are resumed with `--resume <id>`. Approvals and `AskUserQuestion` appear
|
||||
as cards in the app.
|
||||
|
||||
**Sign in:** use your existing Claude Code login. A background service can see
|
||||
a stale keychain entry; when the default login check fails, the server retries
|
||||
with the login file in `~/.claude`. For a login that does not depend on either,
|
||||
create a long-lived token and store it privately:
|
||||
|
||||
```sh
|
||||
claude setup-token
|
||||
# save the printed token:
|
||||
install -m 600 /dev/null ~/.config/pipelines/claude-token
|
||||
$EDITOR ~/.config/pipelines/claude-token
|
||||
```
|
||||
|
||||
`PIPELINES_CLAUDE_TOKEN_FILE` names another file. The file must be a regular,
|
||||
owner-only file. The startup report shows `Claude service token present` when
|
||||
it is read. An `ANTHROPIC_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN` or
|
||||
`CLAUDE_CONFIG_DIR` in the server's environment takes precedence.
|
||||
|
||||
**Models:** the catalog comes from the CLI; switching applies live and is kept
|
||||
for the next turn.
|
||||
|
||||
**Mid-turn messages:** yes. The message is written to the running CLI's stdin
|
||||
and read at its next step. Model changes apply from the next turn.
|
||||
|
||||
## Codex
|
||||
|
||||
**Used for:** chat only.
|
||||
|
||||
**Found by:** the Codex App Server control socket at
|
||||
`$CODEX_HOME/app-server-control/app-server-control.sock` (default
|
||||
`~/.codex/...`). The Codex app provides it while it runs. The server checks for
|
||||
it every 10 seconds and attaches when it appears. Pipelines never starts a
|
||||
Codex daemon or changes Codex settings.
|
||||
|
||||
**Sign in:** your existing Codex login. Threads created from Pipelines are
|
||||
ordinary Codex threads and also show in the Codex app.
|
||||
|
||||
**Behavior:** new threads use the `workspace-write` sandbox and Codex's own
|
||||
approval policy; command, file-change and permission requests appear as
|
||||
approval cards.
|
||||
|
||||
**Mid-turn messages:** yes, with `turn/steer` pinned to the running turn. If
|
||||
the turn already ended, the message is queued for the next turn instead.
|
||||
|
||||
## Adopting existing sessions
|
||||
|
||||
```sh
|
||||
pipelines chat-adopt <session-id> [--harness opencode|claude|codex] [--title TEXT]
|
||||
```
|
||||
|
||||
Without `--harness`, each available harness is asked. Stop using the session
|
||||
elsewhere once you continue it in Pipelines.
|
||||
|
||||
## Limits
|
||||
|
||||
- Stage session exports: the native session read is capped at 8 MiB and the
|
||||
rendered `session.html` at 8 MiB. A session over the read cap cannot be
|
||||
exported by retrying; the stage reports it.
|
||||
- Claude Code chat history: a transcript over 64 MiB is reported as exceeding
|
||||
the history limit instead of being truncated.
|
||||
- Codex tool detail in history is truncated at 64 KiB and marked as such.
|
||||
- Losing the Codex App Server connection drops pending approvals; they are
|
||||
reported as an error, not shown as still actionable.
|
||||
- Queued chat messages: at most 8 per chat while a turn runs.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Harness works in a terminal but not from the service.** The login service
|
||||
does not read your shell profile. It uses the `PATH` recorded when you ran
|
||||
`./install` (prefixed with `<prefix>/bin`). Rerun `./install` from a shell
|
||||
where the harness is on `PATH`, or set an absolute `executable` with
|
||||
`pipelines harness setup <name> --host --executable /path/to/bin`.
|
||||
- **Claude Code not listed under Chat.** Check `claude` resolves in one of the
|
||||
locations above and `claude auth status` reports logged in.
|
||||
- **Codex not listed under Chat.** Open the Codex app and confirm the socket
|
||||
exists under `~/.codex/app-server-control/`. The installer does not record
|
||||
a custom `CODEX_HOME` for the service, so the default `~/.codex` is used.
|
||||
- **"Pi is not configured on this execution server".** Run
|
||||
`pipelines harness setup pi --host` on the server machine.
|
||||
- **"invalid provider/model identifier".** Use the exact IDs from the harness's
|
||||
model list; placeholders such as `default/default` are refused.
|
||||
+199
-508
@@ -1,564 +1,255 @@
|
||||
# Installation
|
||||
|
||||
`./install` installs **natively on the host by default** (macOS and Linux);
|
||||
`./install --docker` runs the existing Docker server + ntfy installation.
|
||||
To reconfigure login and separate interactive/new-pipeline models inside an
|
||||
existing Docker installation without a service reinstall: `./install --opencode`
|
||||
or `./install --pi`.
|
||||
See [configured harnesses](harnesses-and-control.md) for scope and verification limits.
|
||||
`./install` installs Pipelines natively for your user on macOS or Linux. It
|
||||
builds the server, the CLI and the web app together, installs them under
|
||||
`~/.local/share/pipelines`, and starts them under a login service. No sudo.
|
||||
|
||||
```sh
|
||||
git clone https://git.a-wan.net/alan/pipelines.git
|
||||
cd pipelines
|
||||
./install # native: server, authenticated web, CLI, ntfy + login autostart
|
||||
./install --docker # explicit Docker server + ntfy
|
||||
./install --web # rebuild the web app (Docker) after app changes
|
||||
./install --apk # optional: build the Android app (Docker, slower)
|
||||
pipelines update # later: pull, reinstall the same way, restart
|
||||
./install
|
||||
```
|
||||
|
||||
## Updating
|
||||
## Prerequisites
|
||||
|
||||
`pipelines update` brings an installation to the newest source. It waits while
|
||||
runs are executing, fast-forwards the checkout it was installed from
|
||||
(`git pull --ff-only`), re-runs `./install` with the options recorded at install
|
||||
time, restarts the server, web proxy and ntfy, and prints the startup report.
|
||||
Data, the web password, ntfy accounts and the Android signing key are kept; the
|
||||
web app and APK are rebuilt only when their sources changed. Phones pick up a
|
||||
new APK (after `./install --apk`) from Settings → App updates.
|
||||
The installer only detects tools. It never installs, upgrades or downloads
|
||||
them, and it stops naming whatever is missing.
|
||||
|
||||
`./install --fresh` starts with empty data even when volumes from an earlier
|
||||
Docker installation exist (by default they are migrated).
|
||||
| Tool | Why |
|
||||
| --- | --- |
|
||||
| Go (version in `go.mod`, currently 1.27) | builds `pipelines-server`, `pipelines` and ntfy |
|
||||
| Node.js 18+ | runs the authenticated web proxy (`web/serve.mjs`) |
|
||||
| Python 3.9+ | installer and the lifecycle manager (`manage.py`) |
|
||||
| git, curl | source checkout, updates, release lookup |
|
||||
| C compiler (`cc`, `clang` or `gcc`) | ntfy's SQLite driver is cgo (Xcode Command Line Tools on macOS) |
|
||||
| **Docker, running** | builds the Flutter web app and the APK |
|
||||
| OpenCode on `PATH` | required; default stage and chat harness |
|
||||
| Pi, Claude Code, Codex | optional; see [Harnesses](harnesses.md) |
|
||||
| tailscale | optional; tailnet HTTPS access |
|
||||
|
||||
## Native installation (default)
|
||||
The install is **all or nothing**: the web app and the server are built before
|
||||
either is installed. If Docker is not running, `./install` stops with
|
||||
"Docker is not running, so the web app cannot be built" and leaves any existing
|
||||
installation running unchanged.
|
||||
|
||||
`./install`, `./install --native` and `./install native` run
|
||||
`scripts/setup-native.py`. It only detects the tools it needs and never installs
|
||||
or upgrades them: **Go (version required by `go.mod`), Python 3.9+, Node.js 18+,
|
||||
git, curl and a C compiler (`cc`/`clang`/`gcc`)**
|
||||
must already be on PATH, and **Docker** must be running to build the app. The C compiler is required because ntfy stores its cache
|
||||
and auth database in SQLite through the cgo driver `mattn/go-sqlite3`. It also
|
||||
detects an existing `opencode` (required) and `pi` (optional) on PATH and records
|
||||
their absolute paths; it never downloads, installs or upgrades OpenCode or Pi
|
||||
(harness installation stays Docker-only).
|
||||
Run as your normal logged-in user. macOS needs `launchctl` in a GUI login;
|
||||
Linux needs `systemctl --user`. Other platforms are refused.
|
||||
|
||||
Run as your normal logged-in user, not root. macOS requires `launchctl` and an
|
||||
active GUI login; Linux requires `systemctl` and a working `systemd --user`
|
||||
session. Unsupported platforms and missing tools fail instead of silently
|
||||
installing an unsupervised server. Go downloads dependencies and the pinned ntfy
|
||||
source during a fresh build; the installer does not install system prerequisites.
|
||||
## What `./install` does
|
||||
|
||||
**`./install` is the only setup command.** It builds the Flutter web app (`app/`)
|
||||
in a Docker container pinned to `app/.flutter-version`, installs it at
|
||||
`<prefix>/lib/app-web`, configures the authenticated Node proxy (`web/serve.mjs`,
|
||||
copied to `<prefix>/lib/web`) and starts all three processes under one login
|
||||
service. Builds are cached in `~/.cache/pipelines/app-builds` by a digest of the
|
||||
app sources: an unchanged app is reused, a changed one rebuilt. Without Docker
|
||||
the install still completes and the web page says to run `./install --web`.
|
||||
1. Checks tools, port settings, the login service manager and the OpenCode
|
||||
executable.
|
||||
2. Builds the Flutter web app in Docker (pinned to `app/.flutter-version`).
|
||||
Builds are cached in `~/.cache/pipelines/app-builds` by a digest of the app
|
||||
sources, so an unchanged app is reused.
|
||||
3. Builds `pipelines-server` and `pipelines` with `go build -trimpath`.
|
||||
4. Installs into the prefix: binaries in `bin/`, web app in `lib/app-web`, web
|
||||
proxy in `lib/web`, `lib/manage.py`, and an empty `lib/app-updates` for APKs.
|
||||
5. Runs `pipelines skill install` (the authoring skill for OpenCode and Claude
|
||||
Code skill roots).
|
||||
6. Writes host harness profiles to `~/.config/pipelines/harnesses.json`. If no
|
||||
usable OpenCode default model is found, it opens the model picker in the
|
||||
terminal (or fails without a terminal; pass `--model`).
|
||||
7. Builds ntfy from its pinned Go module, writes its config and creates the
|
||||
`pipeline-publisher` and `phone-reader` accounts.
|
||||
8. Migrates data from an earlier Docker installation, if its volumes exist.
|
||||
9. Registers the login service, writes the manifest
|
||||
`<prefix>/pipelines-install.json`, saves `ws://127.0.0.1:<port>/connect` as
|
||||
the CLI default in `~/.config/pipelines/client.json`, and starts everything.
|
||||
10. Retires the legacy standalone web agent `net.awan.pipelines.web` on macOS
|
||||
(stopped, plist renamed to `.disabled`), so only one web app is served.
|
||||
11. If `PIPELINES_WEB_PUBLIC_URL` is an `https://…ts.net` address, runs
|
||||
`tailscale serve --bg --https=<port> http://127.0.0.1:<web-port>`.
|
||||
12. Prints the browser URLs.
|
||||
|
||||
The Android app is optional because its build needs the Android SDK and takes
|
||||
longer (on Apple Silicon the x86-64 build image runs under emulation).
|
||||
`./install --apk` builds it in Docker for this installation's web address and
|
||||
offers it at `/updates/` on that address, behind the sign-in, so phones install
|
||||
it from Settings → App updates → Check for updates. Every APK is signed with one
|
||||
key kept in `~/.config/pipelines/android/` (seeded from `~/.android/debug.keystore`
|
||||
when present) so each new build installs over the last.
|
||||
It also warns when a different `pipelines` earlier on `PATH` shadows the new
|
||||
one; it does not touch that binary.
|
||||
|
||||
`pipelines start`, `restart` and `status` print the server's own startup report
|
||||
(version, API, database, OpenCode, chat backends and harnesses) and the web,
|
||||
web-app and phone-app status. The default browser URL is
|
||||
`http://localhost:8082/` on this computer, or `http://HOST:8082/` on a trusted
|
||||
LAN/VPN (`--web-port` / `--web-host` override it). Setup prints the actual URL.
|
||||
Re-running `./install` preserves the saved API, web and ntfy ports and bind
|
||||
addresses. Explicit flags override those settings, for example
|
||||
`./install --web-port 8096`. An already-running supervisor retains its current
|
||||
settings until restarted; restart only when no work is active.
|
||||
Native installation and web/APK builds finish by printing the browser URLs,
|
||||
including the saved web port, LAN address and configured public origin.
|
||||
The unauthenticated Go API defaults to **127.0.0.1:8080**; expose only the web
|
||||
proxy. Use TLS termination or an encrypted VPN for remote access; plain HTTP
|
||||
does not encrypt the sign-in password.
|
||||
### Addresses
|
||||
|
||||
Existing web credentials in `PIPELINES_WEB_AUTH_FILE`, or
|
||||
`~/.config/pipelines/web-auth.json`, are reused unchanged. On a new computer
|
||||
setup asks for the sign-in password (8+ characters, typed twice; set
|
||||
`PIPELINES_WEB_PASSWORD` to answer without a terminal) and writes a private 0600
|
||||
file there with it and a generated cookie-signing `secret`. An invalid or
|
||||
non-private existing file stops setup instead of resetting credentials. The
|
||||
service never inherits the test-only `PIPELINES_WEB_AUTH=disabled` bypass.
|
||||
| Service | Default | Flag |
|
||||
| --- | --- | --- |
|
||||
| Web proxy (sign-in, web app, `/updates/`) | `0.0.0.0:8082` | `--web-port`, `--web-host` |
|
||||
| Server API (no auth) | `127.0.0.1:8080` | `--port`, `--bind-address` |
|
||||
| ntfy | `:8090` | `--ntfy-port`, `--ntfy-url` |
|
||||
|
||||
A **fresh** native installation records `allow_authoring: true` in its manifest
|
||||
and explicitly starts Go with `-allow-authoring`, so the single-user web workflow
|
||||
can create and publish pipelines. `./install --no-authoring` instead selects an
|
||||
execution-only role. Reinstalls preserve the recorded role; an older manifest
|
||||
without this field stays execution-only. To deliberately enable authoring for
|
||||
such an installation, use `./install --allow-authoring`. The installer prints the
|
||||
selected role. This is a server permission, not a browser authentication bypass;
|
||||
the authenticated proxy and loopback API defaults still apply. Docker's role and
|
||||
startup flags are unchanged.
|
||||
Role changes apply on the supervisor's next start. A reinstall does not forcibly
|
||||
restart an already-running supervisor; if reconfiguring an active installation,
|
||||
use its `pipelines restart` only when no work is active.
|
||||
Expose only the web proxy. A reinstall keeps the saved ports and bind
|
||||
addresses unless you pass the flag again, so an installation set up with
|
||||
`--web-port 8083` stays on 8083.
|
||||
|
||||
Setup resolves `PIPELINES_PREFIX` or defaults to `~/.local/share/pipelines`,
|
||||
creates `bin/`, `lib/`, `data/`, `logs/` and `secrets/` (`secrets` and `data` at
|
||||
0700), copies `scripts/manage.py` to `<prefix>/lib/manage.py`, builds exactly
|
||||
`./cmd/pipelines-server` and `./cmd/pipelines` with `go build -trimpath` into
|
||||
`<prefix>/bin`, and runs `<prefix>/bin/pipelines skill install` so the packaged
|
||||
authoring skill lands in your OpenCode skills directory. No `pipelines-tui` is
|
||||
built. It writes `<prefix>/pipelines-install.json` atomically at 0600; that
|
||||
manifest is what `scripts/manage.py` and the Go CLI load. Provider/model defaults
|
||||
are never invented: valid saved Pipelines defaults are preserved, and an explicit
|
||||
`--model PROVIDER/MODEL` fills missing OpenCode profile fields and is recorded as
|
||||
the server's default for new starters. Otherwise setup reads OpenCode's resolved
|
||||
host default using `opencode api get /api/model/default` from your home directory,
|
||||
not the installer checkout. OpenCode handles its own JSONC and credentials; the
|
||||
installer does not parse configuration files or generate a model response.
|
||||
`PIPELINES_WEB_PUBLIC_URL` is the address phones and notification links use.
|
||||
Set it once; reinstalls keep it:
|
||||
|
||||
After the build, setup checks whether a **different** `pipelines` resolves earlier
|
||||
on PATH than `<prefix>/bin/pipelines` (the common cause of a stale CLI talking to a
|
||||
newer server). When it does, setup prints a `WARNING` naming that absolute path and
|
||||
both versions (`version --json` on each binary, read-only) and tells you to put
|
||||
`<prefix>/bin` first on PATH. It never modifies, moves, deletes or upgrades that
|
||||
other binary, and it installs no unrelated tool; the manifest keeps
|
||||
`<prefix>/bin` first on the recorded PATH.
|
||||
```sh
|
||||
PIPELINES_WEB_PUBLIC_URL=https://mymac.tailnet-name.ts.net ./install
|
||||
```
|
||||
|
||||
The native install also writes **host** harness profiles for the harnesses it
|
||||
detects into XDG config home's `pipelines/harnesses.json` (no
|
||||
`docker_directory`), preserving a valid host default (or choosing OpenCode instead
|
||||
of a stale Docker default) and taking models from the saved profile, `--model`,
|
||||
or OpenCode's actual configured default. The selected stage default is also
|
||||
recorded as `PIPELINES_DEFAULT_MODEL` in the private manifest's server environment,
|
||||
so web starters receive a real provider/model rather than `default/default`.
|
||||
When migrating a Docker install it clears the recorded Docker link instead of
|
||||
leaving the config Docker-linked. It never installs, upgrades or downloads a
|
||||
harness. If no usable default exists, **`./install` itself opens the existing model
|
||||
picker in a terminal**, choosing separate interactive and new-pipeline defaults
|
||||
without logging in or performing inference. No separate configuration command is
|
||||
needed. Noninteractive setup without a usable model fails before claiming ready
|
||||
or starting services; pass `./install --model PROVIDER/MODEL` with an actual model
|
||||
you have configured, or rerun `./install` in a terminal. Existing valid defaults,
|
||||
credentials and unrelated profile fields are retained.
|
||||
Discovery and selection do not verify an actual model generation or incur an
|
||||
inference charge; provider execution remains a runtime check.
|
||||
Without it the installer records `http://<LAN IP>:<web-port>`. Use HTTPS or a
|
||||
VPN for remote access; plain HTTP does not protect the password.
|
||||
|
||||
ntfy is built from source with no Homebrew or package manager. Setup resolves the
|
||||
exact stable `binwiederhier/ntfy` tag (never the moving `latest` tag and never an
|
||||
older fallback), then compiles the server from the pinned `heckel.io/ntfy/v2` Go
|
||||
module at that tag into `<prefix>/bin/ntfy`. A source build is required because
|
||||
upstream publishes **no macOS server binary** (its `darwin_all` asset is built
|
||||
with `-tags noserver` and only provides the client commands), so both macOS and
|
||||
Linux take the same path; the Go toolchain verifies the module against its public
|
||||
checksum database. The build uses
|
||||
`-tags sqlite_omit_load_extension,osusergo,netgo` and creates the same empty web
|
||||
placeholders the upstream Makefile uses for local builds, so the bundled ntfy web
|
||||
UI is not populated (notifications and the API are unaffected). A binary for the
|
||||
same resolved version is reused instead of rebuilt. Setup generates
|
||||
`<prefix>/data/ntfy/server.yml` mirroring `compose.yaml` (`listen-http`,
|
||||
`base-url`, `cache-file`, `auth-file`, `auth-default-access: deny-all`,
|
||||
`enable-login`, attachment cache directory, expiry and size limits), creates the
|
||||
`pipeline-publisher` (write-only on `pipelines`) and `phone-reader` (read-only)
|
||||
accounts with ntfy's own CLI (`--ignore-exists`, so existing passwords are kept;
|
||||
the account subcommands read the generated config through `NTFY_CONFIG_FILE`),
|
||||
and writes the publisher token only to `<prefix>/secrets/publisher-token` at 0600.
|
||||
## Flags
|
||||
|
||||
When Docker volumes `pipelines_pipelines-data` or `pipelines_ntfy-data` exist,
|
||||
setup migrates the pipeline database to `<prefix>/data/pipelines.db` and the ntfy
|
||||
auth store to `<prefix>/data/ntfy/auth.db` using the local Pipelines image
|
||||
(never an implicit pull). Existing native data is never overwritten unless you
|
||||
pass `--replace-data`; a requested migration that fails aborts the install
|
||||
instead of starting empty. Absent volumes are a no-op. If the Docker CLI is
|
||||
present but its volumes cannot be inspected (for example a stopped daemon), setup
|
||||
fails clearly instead of silently skipping migration, so an existing Docker
|
||||
database is never shadowed by an empty native start.
|
||||
From `arguments()` in `scripts/setup-native.py`:
|
||||
|
||||
Setup already starts it and enables login autostart, and prints a copyable
|
||||
`<prefix>/bin/pipelines service enable` command. That installed executable safely
|
||||
selects its own native manifest; an unrelated CLI never guesses a default prefix.
|
||||
Optional terminal use needs only the installed binary on PATH:
|
||||
| Flag | Effect |
|
||||
| --- | --- |
|
||||
| `--prefix PATH` | install location (default `~/.local/share/pipelines`, or `PIPELINES_PREFIX`) |
|
||||
| `--port N` | server API port (default 8080) |
|
||||
| `--bind-address IP` | server API bind (default `127.0.0.1`) |
|
||||
| `--web-port N` | web proxy port (default 8082) |
|
||||
| `--web-host IP` | web proxy bind (default `0.0.0.0`) |
|
||||
| `--ntfy-port N` | ntfy port (default 8090) |
|
||||
| `--ntfy-url URL` | ntfy address given to phones (default `http://<LAN IP>:<ntfy-port>`) |
|
||||
| `--timezone TZ` | override the detected timezone (fallback UTC) |
|
||||
| `--model PROVIDER/MODEL` | default model for new starters and missing OpenCode profile fields |
|
||||
| `--allow-authoring` | allow creating and publishing pipelines (fresh installs default to this) |
|
||||
| `--no-authoring` | execution-only server |
|
||||
| `--fresh` | start empty even if earlier Docker volumes exist |
|
||||
| `--replace-data` | let Docker migration overwrite existing native data |
|
||||
|
||||
Reinstalls keep the recorded authoring role unless you pass one of the two
|
||||
role flags.
|
||||
|
||||
## Other install modes
|
||||
|
||||
```sh
|
||||
./install --web # rebuild and install only the web app (Docker), then reload the page
|
||||
./install --apk # build the Android APK (Docker, slower) and offer it to phones
|
||||
./install --docker # server and ntfy in Docker Compose instead of native
|
||||
```
|
||||
|
||||
`./install --apk` builds a release APK for this installation's web address,
|
||||
copies it to `<prefix>/lib/app-updates/pipelines.apk` with a `version.json`,
|
||||
and serves it at `<web>/updates/` behind the sign-in. On the phone:
|
||||
**Settings → App updates → Check for updates**, then **Download and install**.
|
||||
The first install can use `<web>/updates/pipelines.apk` directly. Every APK is
|
||||
signed with the key in `~/.config/pipelines/android/` (seeded from
|
||||
`~/.android/debug.keystore` if present), so each build installs over the last.
|
||||
Options: `--server-url ws(s)://host:port/connect`, `--notes TEXT`, `--prefix`.
|
||||
|
||||
`./install --docker` is the optional Compose installation with its own
|
||||
container harness credentials; see [Docker](docker.md). Do not run it
|
||||
alongside a native install on the same ports. `./install --opencode` and
|
||||
`./install --pi` reconfigure a **Docker** installation's harness and refuse on
|
||||
a native install; native installs use `pipelines harness setup` (see
|
||||
[Harnesses](harnesses.md)).
|
||||
|
||||
## Lifecycle commands
|
||||
|
||||
Put the installed CLI on `PATH`:
|
||||
|
||||
```sh
|
||||
export PATH="$HOME/.local/share/pipelines/bin:$PATH"
|
||||
pipelines service enable # idempotently enable/start this native install
|
||||
pipelines service enable --print # read-only existing definition preview
|
||||
pipelines service status # login registration state; no credentials
|
||||
pipelines service disable # disable login autostart AND stop its supervisor
|
||||
pipelines doctor
|
||||
# pipelines stop / start / restart / logs
|
||||
```
|
||||
|
||||
`start`, `stop`, `restart`, `doctor`, `logs` and `uninstall` reach
|
||||
`<prefix>/lib/manage.py`. Ports default to 8080 (server) and 8090 (ntfy) and can
|
||||
be changed with `--port`/`--ntfy-port`; setup starts both services itself, so
|
||||
those ports and the web port (8082 by default) must be free (a running Docker installation holds them — stop it
|
||||
first; the migration still needs its volumes and image). ntfy runs from
|
||||
`<prefix>/bin/ntfy` with the generated config; per-user launchd (macOS) or
|
||||
`systemd --user` (Linux) starts `<python> <prefix>/lib/manage.py supervise --prefix
|
||||
<prefix>`, which owns server, ntfy and web lifecycles. macOS writes
|
||||
`~/Library/LaunchAgents/net.a-wan.pipelines.plist` and uses `launchctl enable`,
|
||||
`bootstrap`, and `kickstart` in `gui/UID`. Linux writes
|
||||
`$XDG_CONFIG_HOME/systemd/user/net.a-wan.pipelines.service` (default `~/.config`)
|
||||
and runs `systemctl --user daemon-reload` then `enable --now
|
||||
net.a-wan.pipelines.service`. This is login autostart, not boot-without-login;
|
||||
Linux lingering is an explicit administrator decision. `pipelines service` uses
|
||||
this existing supervisor and database, never a second developer server. An
|
||||
explicit `PIPELINES_PREFIX` can also select it from another CLI. Native service
|
||||
commands reject standalone `--db`/`--addr` options; use installer flags to change
|
||||
the manifest. Definitions belonging to a developer service or another native
|
||||
prefix are never overwritten by these commands. Logs are in
|
||||
`<prefix>/logs/{server,ntfy,web,supervisor}.log` (Linux supervisor output also goes
|
||||
to `journalctl --user -u net.a-wan.pipelines.service`).
|
||||
The installed `pipelines` finds its own manifest. `PIPELINES_PREFIX` selects
|
||||
another installation.
|
||||
|
||||
Dated evidence (2026-10-13, macOS arm64): a scratch-prefix run of
|
||||
`scripts/setup-native.py` resolved ntfy v2.30.0, compiled it from source (`ntfy
|
||||
--version` = v2.30.0 with the `Server commands` present), built the server and
|
||||
CLI, installed the skill, migrated 8 pipeline definitions and 28 runs plus both
|
||||
ntfy accounts from the live Docker volumes, and wrote the manifest.
|
||||
`pipelines start` then brought up ntfy and the server through the supervisor
|
||||
(`{"healthy":true}` from both `/v1/health` and `/healthz` on spare ports) and
|
||||
`pipelines stop` released both ports. launchd/systemd registration and real phone
|
||||
delivery are **not tested**.
|
||||
| Command | Effect |
|
||||
| --- | --- |
|
||||
| `pipelines start` | start server, web proxy and ntfy; print the startup report |
|
||||
| `pipelines stop` | stop all three (active runs are interrupted; edits are not rolled back) |
|
||||
| `pipelines restart` | stop then start, keeping data |
|
||||
| `pipelines doctor` | startup report, health of server and ntfy, OpenCode API check (no inference) |
|
||||
| `pipelines status` | server, host and ntfy publishing health, from the running server |
|
||||
| `pipelines logs` | follow `logs/server.log` |
|
||||
| `pipelines update` | pull and reinstall the same way, then restart (see below) |
|
||||
| `pipelines uninstall [--purge] [--yes]` | remove the service and binaries; `--purge` also deletes data |
|
||||
| `pipelines service enable [--print]` | enable and start login autostart (`--print` previews the definition) |
|
||||
| `pipelines service disable` | disable autostart and stop the supervisor |
|
||||
| `pipelines service status` | autostart and supervisor state |
|
||||
|
||||
The newer integrated web/autostart path is covered by scratch-prefix installer
|
||||
mocks, rendered launchd/systemd definitions and mocked lifecycle/auth-preservation
|
||||
checks. It has **not** been live-installed or tested through a real login on
|
||||
either OS; Linux coverage here is mocked, not native Linux verification.
|
||||
The login service is `~/Library/LaunchAgents/net.a-wan.pipelines.plist` on
|
||||
macOS and `~/.config/systemd/user/net.a-wan.pipelines.service` on Linux. It
|
||||
runs `manage.py supervise`, which owns all three processes. This is login
|
||||
autostart, not boot-without-login.
|
||||
|
||||
## Docker installation (explicit)
|
||||
## Where things live
|
||||
|
||||
`./install --docker` (or `./install docker`) runs the unchanged
|
||||
`scripts/setup-docker.py`. It requires Docker Compose and Python 3 and includes
|
||||
OpenCode/Pi in the persistent server container. Host Go builds the CLI/TUI
|
||||
clients, not a native server.
|
||||
| Path | Contents |
|
||||
| --- | --- |
|
||||
| `<prefix>/pipelines-install.json` | manifest (ports, paths, install args, environment) |
|
||||
| `<prefix>/bin/` | `pipelines`, `pipelines-server`, `ntfy` |
|
||||
| `<prefix>/data/pipelines.db` | SQLite database (definitions, runs, chats, settings) |
|
||||
| `<prefix>/data/artifacts/` | run artifacts and session exports |
|
||||
| `<prefix>/data/ntfy/` | ntfy config, cache and accounts |
|
||||
| `<prefix>/secrets/publisher-token` | ntfy publishing token |
|
||||
| `<prefix>/logs/` | `server.log`, `web.log`, `ntfy.log`, `supervisor.log` |
|
||||
| `~/.config/pipelines/web-auth.json` | web password and cookie secret (`PIPELINES_WEB_AUTH_FILE`) |
|
||||
| `~/.config/pipelines/harnesses.json` | harness profiles (`PIPELINES_HARNESS_CONFIG`) |
|
||||
| `~/.config/pipelines/client.json` | default server URL for the CLI/TUI |
|
||||
| `~/.config/pipelines/claude-token` | optional Claude Code service token |
|
||||
| `~/.config/pipelines/android/` | APK signing key |
|
||||
| `~/.cache/pipelines/app-builds/` | cached web and APK builds |
|
||||
|
||||
The installer builds host `pipelines` and `pipelines-tui` clients when Go is
|
||||
available, using `GOBIN` or `GOPATH/bin`; put that directory on PATH. It does not
|
||||
install a host server. Reconfiguration uses `./install --opencode` or `./install --pi`
|
||||
inside the existing Docker installation and never falls back to host execution. It
|
||||
resolves and verifies the same current releases and refuses to configure against an
|
||||
image that does not match them (rebuild with `./install --docker` first); it does not restart
|
||||
or redeploy services.
|
||||
`<prefix>` is `~/.local/share/pipelines` unless changed. On Linux the
|
||||
supervisor's output also reaches `journalctl --user -u net.a-wan.pipelines.service`.
|
||||
|
||||
Native Go builds embed VCS identity; Docker setup passes the checkout revision and
|
||||
dirty state as compile-time build arguments (without copying `.git` into the image).
|
||||
For a build-only refresh use `python3 scripts/build_metadata.py`; it resolves the
|
||||
current OpenCode V2/Pi releases and runs `docker compose build` with the identity
|
||||
arguments plus the resolved version/digest arguments, without restarting services.
|
||||
A plain `docker compose build` cannot resolve the OpenCode image digest, so it must
|
||||
be given `OPENCODE_VERSION`, `OPENCODE_DIGEST` and `PI_VERSION` through Docker
|
||||
`--build-arg` flags (and `PIPELINES_VERSION`, `PIPELINES_COMMIT` and
|
||||
`PIPELINES_DIRTY` outside a Git checkout); `./install --docker` supplies all of them. After deployment,
|
||||
`pipelines --url ws://HOST:PORT/connect ping --json` identifies the running server.
|
||||
Docker setup checks that the started server's pong matches the source identity
|
||||
passed to the build and fails setup if metadata is missing or mismatched.
|
||||
## Web sign-in password
|
||||
|
||||
Each setup resolves the current releases before touching settings or services: the
|
||||
newest supported **OpenCode V2** release from the single documented complete
|
||||
`GET .../git/matching-refs/tags/v2` response (a bounded complete array with no
|
||||
pagination parameters; never the moving `latest` tag or the registry's insertion
|
||||
order) and the current Pi release from npm. The image is built `FROM` the OpenCode
|
||||
image's immutable `sha256` digest, resolved from GHCR, and labeled with the
|
||||
resolved OpenCode/Pi versions, digest and source identity; the moving `latest` tag
|
||||
and version-only references are never used. Setup verifies the built image reports
|
||||
exactly those executable versions and a Node-compatible Pi engine, and stops with
|
||||
a clear error (keeping the last verified Dockerfile fallback pins) when discovery,
|
||||
resolution, build or compatibility fails rather than guessing or silently
|
||||
installing stale or unchecked copies. A full install skips a rebuild only when the
|
||||
release receipt **and** the actual built image's release and source identity labels
|
||||
already match; a dirty source always invokes the Docker build and relies on layer
|
||||
caching. A model/login-only reconfiguration compares only the verified release
|
||||
image identity, so a changed source alone does not rebuild on reconfigure.
|
||||
Setup writes that release receipt to the machine-local, generated root file
|
||||
`.pipelines-releases.json` (with atomic `.pipelines-releases-*` temporary files)
|
||||
and both are git-ignored, so a fresh clean checkout stays clean after setup and
|
||||
reinstall still reuses the verified release/image identity. The actual built
|
||||
image's labels stay authoritative, clean-checkout identities/reinstall reuse are
|
||||
unchanged, and a dirty source still rebuilds.
|
||||
On a new computer `./install` asks for the password (8+ characters, typed
|
||||
twice). Without a terminal, set `PIPELINES_WEB_PASSWORD`. It is stored in
|
||||
`~/.config/pipelines/web-auth.json` (mode 0600) with a generated cookie secret,
|
||||
and reused unchanged on every reinstall. An invalid or non-private file stops
|
||||
the installer rather than being reset.
|
||||
|
||||
Docker mode detects the computer's LAN IPv4 address and timezone automatically,
|
||||
configures `http://LAN_IP:8090`, and binds both Docker services to `0.0.0.0` without asking
|
||||
for networking settings (timezone falls back to UTC). On reinstall it updates only
|
||||
the bind settings to `0.0.0.0`, refreshes the detected host LAN IP for Status,
|
||||
and retains existing URLs/timezone/ports/credentials. It bootstraps
|
||||
the publishing/phone accounts and publishing token, builds the server, installs
|
||||
the skill inside the container, and opens **container OpenCode provider login**
|
||||
on first setup. Choose any supported provider; signup is handled by that provider.
|
||||
Provider credentials remain in the `opencode-home` volume. Reinstalls skip login
|
||||
after completed setup; `./install --relogin` opens it again without
|
||||
deleting accounts. Setup then chooses **separate saved defaults** in order: first the
|
||||
interactive OpenCode harness model, then the new-pipeline model, in two
|
||||
fuzzy-searchable pickers: type to filter, arrow keys or Tab navigate, Enter confirms,
|
||||
the saved value is preselected and marked `(current)`, and Esc cancels with nothing
|
||||
changed. `--model PROVIDER/MODEL` and `--harness-model
|
||||
PROVIDER/MODEL` set them explicitly; reinstall retests the saved selection. Container
|
||||
credentials and accounts stay separate from host credentials and are reused on
|
||||
reinstall/reconfigure. Picker
|
||||
selection performs no inference; the OpenCode path then requires a real, small
|
||||
**tool-free model response** for the new-pipeline model before marking setup complete
|
||||
(provider charges may apply). Login, discovery and validation prime
|
||||
OpenCode's integration catalog in their own container/service. Discovery retries empty model
|
||||
reads in the same container, avoiding a cold-start empty-catalog response.
|
||||
An empty catalog is not proof of an invalid key; saved credentials are retained.
|
||||
New drafts/templates inherit the new-pipeline selection;
|
||||
published definitions and retry versions stay unchanged. Failed checks print
|
||||
bounded, credential-redacted evidence and retain their OpenCode session.
|
||||
The installer then starts/recreates the Compose services,
|
||||
checks ntfy health through Pipelines, and sends an authenticated test notification.
|
||||
It does not register an OS service, delete volumes, or seed pipeline definitions.
|
||||
Phone reception is not verified by installation.
|
||||
The same password is given to the ntfy `phone-reader` account when that account
|
||||
is first created; existing ntfy accounts keep their passwords.
|
||||
|
||||
Docker setup mounts the host user's home and the installation checkout at their
|
||||
original absolute paths, and runs Pipelines/OpenCode using that host user's UID
|
||||
and GID. This makes a client invocation directory such as
|
||||
`/home/tank/repos/pipelines` refer to the actual host checkout rather than an
|
||||
unavailable container path or the unrelated image checkout at `/workspace`.
|
||||
Stage permissions/edit scopes still apply; this is a filesystem mount, not a
|
||||
workspace allowlist or automatic file transfer from another client computer.
|
||||
For repositories outside those mounts, add a Compose bind at the same absolute
|
||||
path. Reinstallation stops the old Pipelines service before changing ownership
|
||||
of its managed data/auth/workspace volumes. It never recursively chowns the host
|
||||
home/repository, deletes credentials, or removes volumes. Existing data/auth
|
||||
volumes use `nocopy` so Docker cannot reset a freshly prepared empty volume's
|
||||
ownership to the image UID. Setup verifies that the server can enter the host
|
||||
checkout before reporting completion.
|
||||
To change the web password, edit `password` in `web-auth.json` (keep it 0600)
|
||||
and run `pipelines restart`.
|
||||
|
||||
For fresh Docker configuration you can supply `--ntfy-url https://ntfy.example.com
|
||||
--bind-address 127.0.0.1 --timezone Europe/London --port 8081` to `./install --docker`.
|
||||
The defaults are ready for direct phone access on your trusted LAN. Only use
|
||||
plain HTTP on a trusted network. Other existing `.env` settings are not replaced.
|
||||
These Docker flags apply to `./install --docker`; the explicit Docker mode fails
|
||||
rather than silently falling back to native binaries if the LAN address cannot be
|
||||
detected. Use an interactive terminal for first Docker/provider setup.
|
||||
The Docker Pipelines API is unauthenticated: keep its port on a trusted network
|
||||
and do not forward it to the Internet. For a manual loopback-only deployment,
|
||||
set `PIPELINES_BIND_ADDRESS=127.0.0.1` after installation and use Compose directly.
|
||||
Ctrl+C cancels cleanly without deleting configuration, credentials or volumes.
|
||||
If Go is available, Docker setup also updates standard host CLI/TUI binaries and
|
||||
the skill. Restart an already-running TUI to load the changes. Without Go, the
|
||||
installer explicitly warns that the host client was not updated; Docker still runs
|
||||
the server, and you can build the client later with host Go. See [Docker setup](docker.md).
|
||||
|
||||
After ntfy verification, setup also checks the published Pipelines endpoint from
|
||||
the host (when Go or a host CLI is available) and saves it in
|
||||
`$XDG_CONFIG_HOME/pipelines/client.json`, or `~/.config/pipelines/client.json`.
|
||||
Only `server_url` is updated; unrelated settings are preserved. CLI/TUI precedence
|
||||
is `--url`, then `PIPELINES_URL`, then the saved default, then localhost:8080.
|
||||
Docker's default host mapping is port 8081 (the container listens on 8080), so an
|
||||
old native server on 8080 must not accidentally become the client target. Setup warns if an environment override still selects
|
||||
another address; it does not stop or delete that other server.
|
||||
|
||||
If Status says ntfy is **not configured**, check which Pipelines server the client
|
||||
is reaching before reinstalling. On the Docker server computer:
|
||||
## Upgrading
|
||||
|
||||
```sh
|
||||
docker compose port pipelines 8080
|
||||
pipelines --url ws://127.0.0.1:8081/connect status --json
|
||||
pipelines --url ws://127.0.0.1:8081/connect tui
|
||||
pipelines update
|
||||
```
|
||||
|
||||
Use the actual mapped port instead of 8081 if customized. On a different computer,
|
||||
use the Docker host's LAN address instead of 127.0.0.1. The saved default applies
|
||||
only on the computer/user that ran the installer. Phone reception remains untested
|
||||
by these checks.
|
||||
`update` refuses while runs are executing (finish or cancel them first). It
|
||||
then runs `git pull --ff-only` in the checkout it was installed from (skipped
|
||||
when the branch has no upstream), re-runs `./install` with the recorded
|
||||
arguments, restarts, and prints the startup report. Data, passwords, ntfy
|
||||
accounts and the signing key are kept. If the installer fails, the previous
|
||||
version keeps running.
|
||||
|
||||
## Failure evidence and confirmations
|
||||
|
||||
TUI run/retry/cancel requests cannot be sent before their confirmation modal.
|
||||
Draft creation/editing and opening a debugging session also require approval.
|
||||
CLI mutations prompt with a default of No; `--yes` is explicit noninteractive
|
||||
approval. The standalone TUI always keeps its modals, including with `-yes`.
|
||||
|
||||
`pipelines debug RUN_ID --no-open` retrieves the retained run plus read-only
|
||||
OpenCode session evidence from the **server**, not the client's unrelated service.
|
||||
Debug includes the exact pinned pipeline definition and full run snapshot directly.
|
||||
Large context packets are retained in a private client cache file which OpenCode
|
||||
is instructed to read, rather than discarding evidence or exceeding argument limits.
|
||||
New failed operations retain operation, provider/model, executable, directory,
|
||||
timing, exit/HTTP status and bounded stdout/stderr under `opencode_failures` in
|
||||
the step transcript. Credentials are redacted, but project/provider messages may
|
||||
still contain private context: review before sharing. Older failures cannot gain
|
||||
discarded subprocess output retroactively; Debug attempts retained session reads.
|
||||
|
||||
Native OpenCode schema support is checked with a retained, deny-all, no-inference
|
||||
POST/readback probe. OpenCode 1.18.35 accepts explicit formats but rejects their
|
||||
stored messages on readback; affected backends omit `format` and require the same
|
||||
strict local `{outcome, output}` validation. Compatible backends retain schema
|
||||
mode with no automatic tool-replaying schema retries. The compatibility decision
|
||||
and probe session are recorded, without rewriting published definitions/history.
|
||||
For old format-bearing sessions whose HTTP readback fails, Debug uses read-only
|
||||
`opencode export` with an exact session-ID check to recover original messages.
|
||||
`StructuredOutputError` is surfaced directly; DSML/tool-call markup and flattened
|
||||
results are rejected and retained, never silently interpreted as successful tool
|
||||
execution or wrapped into a guessed result. Native HTTP evidence distinguishes
|
||||
adapter/HTTP failures from actual subprocess exit codes.
|
||||
|
||||
To validate a model on the execution computer, run:
|
||||
|
||||
```sh
|
||||
pipelines provider-check --model PROVIDER/MODEL
|
||||
# Docker credentials are separate from host credentials (plain CLI: no TTY):
|
||||
docker compose exec -T pipelines pipelines provider-check --model PROVIDER/MODEL
|
||||
```
|
||||
|
||||
For a developer or legacy host build, a private repository uses your normal Git
|
||||
credentials and `GOPRIVATE` to compile the client/host binary (this is not a
|
||||
supported server installation):
|
||||
|
||||
```sh
|
||||
GOPRIVATE=git.a-wan.net go install git.a-wan.net/alan/pipelines/cmd/...@latest
|
||||
pipelines skill install
|
||||
```
|
||||
|
||||
For a system-wide command, build as your normal user and install the three binaries
|
||||
into an administrator-selected bin directory using your normal system permissions.
|
||||
Services and OpenCode accounts still run as the logged-in user, never as root.
|
||||
The installer does not hardcode a system destination or run sudo.
|
||||
|
||||
## Developer/legacy host server (unsupported)
|
||||
|
||||
This explicit foreground server is for developing Pipelines itself or running a
|
||||
previous Python-supervised legacy host setup. It is **not** the native installer;
|
||||
a fresh database starts empty.
|
||||
|
||||
```sh
|
||||
pipelines serve # foreground developer/legacy server; Ctrl+C stops it
|
||||
# In another terminal:
|
||||
pipelines
|
||||
```
|
||||
|
||||
The native installer binds the Go server to loopback by default and exposes the
|
||||
authenticated web proxy instead. An explicit `--bind-address` changes the manifest.
|
||||
The Pipelines API
|
||||
has no authentication, so keep that port on a trusted LAN/VPN. Change into your
|
||||
repository and run a pipeline there; the client sends its current directory to the
|
||||
server. The server's own working directory does not restrict which repositories
|
||||
you can use. Model tool permissions and approved edit paths remain separate,
|
||||
per-run/stage controls.
|
||||
|
||||
The SQLite database uses the standard user data location, honoring `XDG_DATA_HOME`
|
||||
when set. On macOS it uses the user Application Support directory. `-db FILE`
|
||||
selects another database, including an existing one; nothing is imported or moved
|
||||
automatically. `pipelines serve --help` prints the actual defaults on your machine.
|
||||
Drafts use the OS user cache; `PIPELINES_DRAFT_DIR` overrides that location.
|
||||
|
||||
## Developer/legacy autostart (unsupported)
|
||||
|
||||
For a standalone developer binary (not the CLI inside a native install), stop any
|
||||
existing foreground or legacy server first, then explicitly configure the OS user service:
|
||||
|
||||
```sh
|
||||
pipelines service enable
|
||||
pipelines service status
|
||||
```
|
||||
|
||||
This starts the server now and at subsequent logins. Linux uses
|
||||
`systemctl --user`; macOS uses a LaunchAgent in your user `Library/LaunchAgents`.
|
||||
There is no sudo, Docker dependency, or second supervisor. `--db`
|
||||
and `--addr` select the database/listen address, just like foreground startup.
|
||||
Re-run enable to update the definition and restart with the new settings.
|
||||
|
||||
```sh
|
||||
pipelines service enable --print # preview only
|
||||
pipelines service disable # stop and remove
|
||||
```
|
||||
|
||||
Setup records the current PATH, HOME, XDG directory overrides, selected OpenCode
|
||||
executable, and supported ntfy URL/token-file settings in a private service file.
|
||||
It does not copy provider credentials or the rest of the shell environment.
|
||||
Use `PIPELINES_NTFY_TOKEN_FILE`, not an inline token. Re-run enable after changing
|
||||
these settings or moving binaries. Relative database/token-file paths are resolved
|
||||
at configuration time. Services do not read your shell startup files.
|
||||
|
||||
On Linux this is **login autostart**, not automatically boot-without-login.
|
||||
If needed, explicitly enable lingering using your system's normal administration
|
||||
policy (`loginctl enable-linger "$USER"`). Pipelines does not change that policy.
|
||||
macOS LaunchAgents likewise require a user login. Windows autostart is unsupported.
|
||||
Use `journalctl --user -u pipelines.service` for Linux logs. On macOS inspect
|
||||
`pipelines service status`; run `pipelines serve` in the foreground for diagnostics.
|
||||
|
||||
## OpenCode runtime (developer/legacy host)
|
||||
|
||||
For a developer or legacy host server, use your existing `opencode` on PATH, or set
|
||||
`PIPELINES_OPENCODE` to an explicit executable. No OpenCode version is required just
|
||||
to install binaries. At runtime:
|
||||
|
||||
- The standard/native API release uses an owned, authenticated loopback
|
||||
`opencode serve` process, scoped to the requested repository and using the
|
||||
existing user's configuration/login. The API, not a version number, is checked.
|
||||
- A CLI advertising `api` uses the V2 command interface and its normal discovery.
|
||||
- Both adapters select the pipeline's configured model explicitly and preserve
|
||||
stage tool permissions. Native `shell` permissions map to OpenCode's `bash`.
|
||||
- Audited native sessions record their backend and a private service reference
|
||||
so recovery addresses the same service, rather than a new unrelated instance.
|
||||
- `new`/`edit` open the normal interface directly with `@pipelines` and a draft
|
||||
context-file reference. The skill handles questions and draft validation inside
|
||||
OpenCode; no preliminary session/API calls or instruction preload are performed.
|
||||
|
||||
The global skill is installed in `$XDG_CONFIG_HOME/opencode/skills/pipelines`,
|
||||
or the normal `~/.config/opencode/skills/pipelines` default. Only unchanged files
|
||||
previously installed by Pipelines are updated. A user-managed/modified skill is
|
||||
never overwritten; preserve or rename it before running `pipelines skill install`.
|
||||
Installation does not require OpenCode itself. Uninstall retains the skill and drafts.
|
||||
The packaged format, notification, operations, transfer and verification references
|
||||
are self-contained; agents read only those relevant to the task. Authoring
|
||||
does not require Pipelines source in the current repository. Without a task prompt,
|
||||
OpenCode is instructed to ask what to create/change before investigating files or
|
||||
running CLI commands. See [authoring session review](authoring-session-review.md)
|
||||
for the reasoning and verification limits.
|
||||
New features and confirmed limitations are maintained in the packaged skill alongside
|
||||
the topic docs. Skill tests check that every embedded reference is installed and
|
||||
its links resolve. Refresh the global managed skill from current source with
|
||||
`go run ./cmd/pipelines skill install`; this does not deploy server/runtime fixes.
|
||||
|
||||
The bundled examples retain `opencode-go/deepseek-v4.1-flash`. Availability and
|
||||
authentication for that model are runtime requirements, not installation checks.
|
||||
No login or provider credentials are copied or exported by Pipelines.
|
||||
|
||||
## Optional notifications
|
||||
|
||||
Installation configures this automatically via `./install`.
|
||||
ntfy is independent of Android: it is an optional notification channel for any
|
||||
compatible subscriber, including a separate ntfy app. A web-only user does not
|
||||
need an Android client, and observing that the browser is the only client is not
|
||||
a reason to disable an existing ntfy configuration. Browser-closed web push is
|
||||
**not implemented**; installing/configuring ntfy does not add that capability.
|
||||
For a developer/legacy host server, use an existing ntfy server or the Docker setup in [docs/ntfy.md](ntfy.md).
|
||||
Configure `PIPELINES_NTFY_URL`, `PIPELINES_NTFY_PUBLIC_URL`, and a publishing token
|
||||
via `PIPELINES_NTFY_TOKEN_FILE` in the server's environment. Keep secrets out of
|
||||
pipeline definitions. Use HTTPS or an encrypted VPN for phone access; do not
|
||||
expose an unauthenticated pipeline server or HTTP credentials to untrusted networks.
|
||||
Running `./install` again from the checkout does the same without the pull.
|
||||
Run `./install --apk` again to offer a new APK to phones.
|
||||
|
||||
## Uninstall
|
||||
|
||||
For the native installer, select the manifest and run `pipelines uninstall`:
|
||||
|
||||
```sh
|
||||
PIPELINES_PREFIX="$HOME/.local/share/pipelines" "$HOME/.local/share/pipelines/bin/pipelines" uninstall
|
||||
pipelines uninstall # keeps data and credentials in <prefix>
|
||||
pipelines uninstall --purge # also deletes <prefix>, including the database
|
||||
```
|
||||
|
||||
This stops its supervisor and removes its login definition and managed binaries;
|
||||
data and credentials are retained. The following applies only to standalone
|
||||
developer binaries: disable `pipelines service disable` if configured, then:
|
||||
Neither touches OpenCode, Claude Code, Codex, Pi or files in
|
||||
`~/.config/pipelines`.
|
||||
|
||||
## Developer server
|
||||
|
||||
For working on Pipelines itself, run a server in the foreground on spare
|
||||
ports. This is not the supported installation and does not serve the web app:
|
||||
|
||||
```sh
|
||||
pipelines uninstall
|
||||
# or pipelines uninstall --yes
|
||||
go run ./cmd/pipelines-server -db /tmp/dev.db -addr 127.0.0.1:8081
|
||||
go run ./cmd/pipelines --url ws://127.0.0.1:8081/connect tui
|
||||
```
|
||||
|
||||
Only Go binaries whose embedded build metadata matches this repository are
|
||||
removed from the current command's bin directory. History, drafts, OpenCode,
|
||||
and optional ntfy data remain untouched. Use the permissions appropriate to your
|
||||
chosen bin directory. Custom database paths cannot be inferred, so uninstallation
|
||||
does not purge data.
|
||||
`go run ./cmd/pipelines-server -help` lists the server flags (`-addr`, `-db`,
|
||||
`-allow-authoring`, `-opencode-endpoint`). Without `-db` it uses the standard
|
||||
per-user data directory (`XDG_DATA_HOME`, or Application Support on macOS).
|
||||
|
||||
## Existing legacy managed installations
|
||||
## Troubleshooting
|
||||
|
||||
The previous Python-supervised setup is retained only for explicit legacy use.
|
||||
Set `PIPELINES_PREFIX` to its existing directory for its lifecycle commands;
|
||||
legacy manifests are never inferred from an executable location. The old
|
||||
`--system-command`, provisioning, and autostart installer flags have been removed.
|
||||
|
||||
To migrate, stop the old server first, install normal Go binaries, and start with
|
||||
`pipelines serve -db EXISTING_DATABASE`. Keep the old
|
||||
data and credentials until you have verified the new setup. There is no automatic
|
||||
purge, database move, login migration, or shared OpenCode service modification.
|
||||
- **"Docker is not running"**: start Docker Desktop (or the daemon) and rerun.
|
||||
- **Ports busy**: a Docker installation or an old server may hold 8080, 8082 or
|
||||
8090. Stop it, or choose other ports with the flags above.
|
||||
- **"No usable configured harness model"**: rerun `./install` in a terminal to
|
||||
pick models, or pass `--model PROVIDER/MODEL`.
|
||||
- **CLI talks to the wrong server**: `PIPELINES_URL` overrides the saved
|
||||
default; unset it. Check `pipelines connection`.
|
||||
- **Web app missing**: the server refuses to start without it. Run
|
||||
`./install --web`.
|
||||
- **A harness works in the terminal but not in the service**: the service uses
|
||||
the `PATH` recorded at install time. Rerun `./install` from a shell where the
|
||||
harness is on `PATH`. See [Harnesses](harnesses.md#troubleshooting).
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
# Notifications
|
||||
|
||||
The server records every notification once and delivers it two ways:
|
||||
|
||||
1. **In the Pipelines app** (Android, and the web page while it is open), over
|
||||
the app's own connection. No ntfy app is needed.
|
||||
2. **ntfy** (optional fallback), for devices that still use the ntfy app.
|
||||
|
||||
The same settings decide what is sent on both paths.
|
||||
|
||||
## What notifies
|
||||
|
||||
| Event | When |
|
||||
| --- | --- |
|
||||
| Run needs your decision | a run reaches a human decision stage (runs not set to Silent) |
|
||||
| Run failed / succeeded | runs started with **Notify me** (`--notify on`), or every run when **Notify when any run ends** is on |
|
||||
| Stage hook | a stage's own `notify` block fires ([ntfy reference](ntfy.md)) |
|
||||
| Chat needs you | a chat asks a question or requests approval |
|
||||
| Chat replied | an assistant turn ends (off by default) |
|
||||
|
||||
Each event notifies once. Opening or dismissing it on one device clears it on
|
||||
the others.
|
||||
|
||||
Chat events are seen while some client has the chat open or the server still
|
||||
holds queued messages for it. A turn that finishes with no client attached and
|
||||
nothing queued does not produce a "replied" notification.
|
||||
|
||||
## Settings
|
||||
|
||||
In the app, **Settings**:
|
||||
|
||||
**Notifications** asks for the platform's notification permission.
|
||||
|
||||
**Notifications in Pipelines** (Android only):
|
||||
|
||||
- **Receive notifications in Pipelines**: starts a background service that
|
||||
keeps its own connection to the server and posts notifications. Android shows
|
||||
a quiet "Pipelines is connected" notification while it runs. It restarts
|
||||
after a reboot or an app update.
|
||||
- **Status**: the service's connection state.
|
||||
- **Allow notifications**: shown when Android is blocking Pipelines
|
||||
notifications.
|
||||
- **Allow background delivery**: asks Android to exempt Pipelines from battery
|
||||
optimisation so delivery stays instant while the phone is idle. Grant it.
|
||||
|
||||
**Phone notifications** (stored on the server, shared by every device):
|
||||
|
||||
| Setting | Default |
|
||||
| --- | --- |
|
||||
| Notify when any run ends (otherwise only runs started with **Notify me**) | off |
|
||||
| Chat needs your answer or approval | on |
|
||||
| Chat reply finished | off |
|
||||
| Also send to ntfy (for devices that still use the ntfy app) | on |
|
||||
|
||||
Turning **Also send to ntfy** off stops publishing to ntfy; the app still
|
||||
receives everything.
|
||||
|
||||
Per run, choose **Notify me**, the default (auto), or **Silent** when starting
|
||||
it. From the CLI: `pipelines run … --notify on|auto|off`.
|
||||
|
||||
## The web app
|
||||
|
||||
The web page shows notifications only while it is open and the browser allows
|
||||
them. Closed-browser web push is not implemented.
|
||||
|
||||
## ntfy (optional)
|
||||
|
||||
`./install` builds and runs ntfy on port 8090 with a private `pipelines` topic.
|
||||
The installer prints the subscription details at the end, and
|
||||
`pipelines doctor` repeats them (`Phone server … · topic: pipelines ·
|
||||
username: phone-reader`).
|
||||
|
||||
To subscribe from the ntfy app:
|
||||
|
||||
1. Add a server: the ntfy address (default `http://<LAN IP>:8090`, or the
|
||||
`--ntfy-url` you installed with). Off your home network, use a VPN such as
|
||||
Tailscale.
|
||||
2. Sign in as `phone-reader`. A freshly created account uses the web sign-in
|
||||
password; an account kept from an earlier install keeps its old password.
|
||||
3. Subscribe to the topic `pipelines`.
|
||||
|
||||
A tap opens `pipelines://run/<id>` or `pipelines://chat/<id>` in the Android
|
||||
app; the **Open in browser** action opens the same page on
|
||||
`PIPELINES_WEB_PUBLIC_URL`.
|
||||
|
||||
Notifications already shown by the ntfy app cannot be retracted by the server;
|
||||
swipe them away there.
|
||||
|
||||
See [ntfy reference](ntfy.md) for stage `notify` hooks, explicit ntfy stages,
|
||||
the app notification protocol and network troubleshooting.
|
||||
+6
-2
@@ -1,5 +1,9 @@
|
||||
# Self-hosted ntfy delivery
|
||||
|
||||
For the user-facing overview (in-app Android delivery, settings, subscribing)
|
||||
see [Notifications](notifications.md). This page is the reference for stage
|
||||
hooks, the app notification protocol and ntfy itself.
|
||||
|
||||
Every stage can have an optional `notify` property block. Notifications go to
|
||||
your own ntfy instance and can attach Markdown and output files. It does not ask
|
||||
an LLM to handle credentials or run curl. The full Markdown remains in the run
|
||||
@@ -179,7 +183,7 @@ explicit `type=ntfy` stages are not affected.
|
||||
| `run_finished` — Notify when any run ends | off | every **auto**-mode run that fails or succeeds notifies; otherwise only runs started with **Notify me** (`on`) announce their end |
|
||||
| `chat_needs_you` — Chat needs your answer or approval | on | chat question/form and approval requests notify |
|
||||
| `chat_reply_finished` — Chat reply finished | off | each finished assistant turn in a chat notifies |
|
||||
| `ntfy_enabled` — Publish to ntfy | on | when off, nothing is published to ntfy; the app notification stream (below) still delivers every notification |
|
||||
| `ntfy_enabled` — Also send to ntfy | on | when off, nothing is published to ntfy; the app notification stream (below) still delivers every notification |
|
||||
|
||||
Change them in the app under **Settings → Phone notifications** (checkboxes; a
|
||||
refused change reverts and shows the server's reason). Over the websocket
|
||||
@@ -342,7 +346,7 @@ phone-reachable address if attachment URLs currently point to a LAN-only address
|
||||
that setting is not a substitute for fixing a TCP timeout.
|
||||
|
||||
Install the ntfy app, add your own server, authenticate with the reader account,
|
||||
and subscribe to `anime`. Ensure notifications and required background operation
|
||||
and subscribe to `pipelines`. Ensure notifications and required background operation
|
||||
are allowed. Outside your home network, both topic subscriptions and attachment
|
||||
downloads need access to your server through VPN or public HTTPS.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user