Files
pipelines/docs/install.md
T

16 KiB

Installation

./install installs Pipelines natively for your user on macOS or Linux. It builds the server, the CLI and the web app together, installs them under ~/.local/share/pipelines, and starts them under a login service. No sudo.

git clone https://git.a-wan.net/alan/pipelines.git
cd pipelines
./install

Prerequisites

The installer only detects tools. It never installs, upgrades or downloads them, and it stops naming whatever is missing.

Tool Why
Go (version in go.mod, currently 1.27) builds pipelines-server, pipelines and ntfy
Node.js 18+ runs the authenticated web proxy (web/serve.mjs)
Python 3.9+ installer and the lifecycle manager (manage.py)
git, curl source checkout, updates, release lookup
C compiler (cc, clang or gcc) ntfy's SQLite driver is cgo (Xcode Command Line Tools on macOS)
Docker, running builds the Flutter web app and the APK
OpenCode on PATH required; default stage and chat harness (curl -fsSL https://opencode.ai/install | bash)
Pi, Claude Code, Codex optional; set up after install with pipelines harness setup <name>
tailscale optional; tailnet HTTPS access

The install is all or nothing: the web app and the server are built before either is installed. If Docker is not running, ./install stops with "Docker is not running, so the web app cannot be built" and leaves any existing installation running unchanged.

Run as your normal logged-in user. macOS needs launchctl in a GUI login; Linux needs systemctl --user. Other platforms are refused.

What ./install does

  1. Checks tools, port settings, the login service manager and the OpenCode executable.
  2. Builds the Flutter web app in Docker (pinned to app/.flutter-version). Builds are cached in ~/.cache/pipelines/app-builds by a digest of the app sources, so an unchanged app is reused.
  3. Builds pipelines-server, pipelines, pipelines-claude-host and pipelines-codex-host (retain Claude processes and Codex interaction callbacks across server restarts) with go build -trimpath.
  4. Installs into the prefix: binaries in bin/, web app in lib/app-web, web proxy in lib/web, lib/manage.py, and an empty lib/app-updates for APKs.
  5. Runs pipelines skill install (the authoring skill for OpenCode and Claude Code skill roots).
  6. Writes host harness profiles to ~/.config/pipelines/harnesses.json. If no usable OpenCode default model is found, it opens the model picker in the terminal (or fails without a terminal; pass --model).
  7. Builds ntfy from its pinned Go module, writes its config and creates the pipeline-publisher and phone-reader accounts.
  8. Migrates data from an earlier Docker installation, if its volumes exist.
  9. Registers the login service, writes the manifest <prefix>/pipelines-install.json, saves ws://127.0.0.1:<port>/connect as the CLI default in ~/.config/pipelines/client.json, and starts everything.
  10. Retires the legacy standalone web agent net.awan.pipelines.web on macOS (stopped, plist renamed to .disabled), so only one web app is served.
  11. If PIPELINES_WEB_PUBLIC_URL is an https://…ts.net address, runs tailscale serve --bg --https=<port> http://127.0.0.1:<web-port>.
  12. Prints the browser URLs and the next step, setting up harnesses.

Set up harnesses

One command per harness, the same on native and Docker installations:

pipelines harness setup opencode   # stages + chat (required; ./install already configured it)
pipelines harness setup pi         # stages
pipelines harness setup claude     # chat
pipelines harness setup codex      # chat
pipelines harness list             # status of all four

Each run checks the CLI is installed where the server finds it (printing the install command if not), signs in with the harness's own login when needed, picks OpenCode/Pi default models, and verifies with the server's check. No restart is needed. See Harnesses.

It also warns when a different pipelines earlier on PATH shadows the new one; it does not touch that binary.

Addresses

Service Default Flag
Web proxy (sign-in, web app, /updates/) 0.0.0.0:8082 --web-port, --web-host
Server API (no auth) 127.0.0.1:8080 --port, --bind-address
ntfy :8090 --ntfy-port, --ntfy-url

Expose only the web proxy. A reinstall keeps the saved ports and bind addresses unless you pass the flag again, so an installation set up with --web-port 8083 stays on 8083.

The server advertises itself on the LAN with mDNS/DNS-SD as _pipelines._tcp, so the phone app's "Find servers on this network" lists it. The advertised port is the web proxy's (when it listens beyond loopback; the supervisor passes -mdns-port), or the server's own -addr port when that is a LAN address. TXT records: v (version), name (host), path=/connect, tls=0, auth=password|none. Set PIPELINES_MDNS=off (or -mdns=false) to disable it; a failure to advertise is logged and never stops the server.

PIPELINES_WEB_PUBLIC_URL is the address phones and notification links use. Set it once; reinstalls keep it:

PIPELINES_WEB_PUBLIC_URL=https://mymac.tailnet-name.ts.net ./install

Without it the installer records http://<LAN IP>:<web-port>. Use HTTPS or a VPN for remote access; plain HTTP does not protect the password.

Flags

From arguments() in scripts/setup-native.py:

Flag Effect
--prefix PATH install location (default ~/.local/share/pipelines, or PIPELINES_PREFIX)
--port N server API port (default 8080)
--bind-address IP server API bind (default 127.0.0.1)
--web-port N web proxy port (default 8082)
--web-host IP web proxy bind (default 0.0.0.0)
--ntfy-port N ntfy port (default 8090)
--ntfy-url URL ntfy address given to phones (default http://<LAN IP>:<ntfy-port>)
--timezone TZ override the detected timezone (fallback UTC)
--model PROVIDER/MODEL default model for new starters and missing OpenCode profile fields
--allow-authoring allow creating and publishing pipelines (fresh installs default to this)
--no-authoring execution-only server
--fresh start empty even if earlier Docker volumes exist
--replace-data let Docker migration overwrite existing native data

Reinstalls keep the recorded authoring role unless you pass one of the two role flags.

Other install modes

./install --web        # rebuild and install only the web app (Docker), then reload the page
./install --apk        # build the Android APK (Docker, slower) and offer it to phones
./install --docker     # server and ntfy in Docker Compose instead of native

./install --apk builds a release APK for this installation's web address, copies it to <prefix>/lib/app-updates/pipelines.apk with a version.json, and serves it at <web>/updates/ behind the sign-in. On the phone: Settings → App updates → Check for updates, then Download and install. The first install can use <web>/updates/pipelines.apk directly. Every APK is signed with the key in ~/.config/pipelines/android/ (seeded from ~/.android/debug.keystore if present), so each build installs over the last. Options: --server-url ws(s)://host:port/connect, --notes TEXT, --prefix.

./install --docker is the optional Compose installation with its own container harness credentials; see Docker. Do not run it alongside a native install on the same ports. Its harnesses are set up with the same pipelines harness setup opencode|pi (run from the checkout), which configures them inside the container. The old ./install --opencode and ./install --pi only print that command and exit.

Lifecycle commands

Put the installed CLI on PATH:

export PATH="$HOME/.local/share/pipelines/bin:$PATH"

The installed pipelines finds its own manifest. PIPELINES_PREFIX selects another installation.

Command Effect
pipelines start start server, web proxy and ntfy; print the startup report
pipelines stop stop all three; harness sessions keep running and the next start reattaches them (see below)
pipelines restart stop then start, keeping data; chat turns and stage sessions continue
pipelines doctor startup report, health of server and ntfy, OpenCode API check (no inference)
pipelines status server, host and ntfy publishing health, from the running server
pipelines logs follow logs/server.log
pipelines update pull and reinstall the same way, then restart (see below)
pipelines uninstall [--purge] [--yes] remove the service and binaries; --purge also deletes data
pipelines service enable [--print] enable and start login autostart (--print previews the definition)
pipelines service disable disable autostart and stop the supervisor
pipelines service status autostart and supervisor state

Restarts keep sessions running

A server restart, pipelines stop and pipelines update stop the Pipelines processes, not the work running on harnesses. On SIGTERM the server leaves every OpenCode and Codex session running, records what it needs to reattach, and exits. It does not interrupt or abort anything. On the next start:

  • Runs: a stage whose OpenCode V2 session is still alive is re-adopted. The runner waits for that same session (no new prompt), or reads the result it reached while the server was down, and then continues the graph. A stage that had not yet handed work to a harness starts again. A run between stages continues with its next stage. The stage timeout restarts when the stage is reattached. Its file-change list is marked partial, because the snapshot taken before the restart is gone.
  • Chats: turns, pending questions/approvals, background tasks and queued prompts are picked up again, and a reply that arrived during the downtime still notifies. See Chat backends.

If the harness is gone, the old behaviour applies and the run is interrupted with the reason in its diagnostic, for example "Server restarted; the OpenCode service running stage … was gone". This happens when the OpenCode service process died, or the session belongs to a Pi or OpenCode V1 process that ended with the server.

Only explicit operator actions stop sessions:

  • pipelines cancel <run> (or Cancel in the app) interrupts a run's stage.
  • Closing a chat, or Stop on its turn, interrupts that chat.

To stop everything: cancel the active runs and close or stop the busy chats, then run pipelines stop. The owned OpenCode stage service stops itself at shutdown when idle. The owned chat service, used only without an attached endpoint, keeps running and is reused. Stop it by killing the pid in ~/Library/Caches/pipelines/private/opencode-chat-service.json (~/.cache/... on Linux). To make every shutdown interrupt runs and close chat clients as before, add "PIPELINES_SHUTDOWN_SESSIONS": "stop" to opencode_env in <prefix>/pipelines-install.json and restart.

The OpenCode services the server starts (stage service, owned chat service) run in their own process session, so macOS launchd leaves them running. The systemd unit uses KillMode=process, so Linux stops only the supervisor, which stops its own children; the detached services survive. An attached OpenCode endpoint and the Codex App Server are separate processes anyway.

The login service is ~/Library/LaunchAgents/net.a-wan.pipelines.plist on macOS and ~/.config/systemd/user/net.a-wan.pipelines.service on Linux. It runs manage.py supervise, which owns all three processes. This is login autostart, not boot-without-login.

Where things live

Path Contents
<prefix>/pipelines-install.json manifest (ports, paths, install args, environment)
<prefix>/bin/ pipelines, pipelines-server, pipelines-claude-host, pipelines-codex-host, ntfy
<prefix>/data/pipelines.db SQLite database (definitions, runs, chats, settings)
<prefix>/data/artifacts/ run artifacts and session exports
<prefix>/data/ntfy/ ntfy config, cache and accounts
<prefix>/secrets/publisher-token ntfy publishing token
<prefix>/logs/ server.log, web.log, ntfy.log, supervisor.log
~/.config/pipelines/web-auth.json web password and cookie secret (PIPELINES_WEB_AUTH_FILE)
~/.config/pipelines/harnesses.json harness profiles (PIPELINES_HARNESS_CONFIG)
~/.config/pipelines/client.json default server URL for the CLI/TUI
~/.config/pipelines/claude-token optional Claude Code service token
~/.config/pipelines/android/ APK signing key
~/.cache/pipelines/app-builds/ cached web and APK builds

<prefix> is ~/.local/share/pipelines unless changed. On Linux the supervisor's output also reaches journalctl --user -u net.a-wan.pipelines.service.

Web sign-in password

On a new computer ./install asks for the password (8+ characters, typed twice). Without a terminal, set PIPELINES_WEB_PASSWORD. It is stored in ~/.config/pipelines/web-auth.json (mode 0600) with a generated cookie secret, and reused unchanged on every reinstall. An invalid or non-private file stops the installer rather than being reset.

The same password is given to the ntfy phone-reader account when that account is first created; existing ntfy accounts keep their passwords.

To change the web password, edit password in web-auth.json (keep it 0600) and run pipelines restart.

Upgrading

pipelines update

update refuses while runs are executing (finish or cancel them first). It then runs git pull --ff-only in the checkout it was installed from (skipped when the branch has no upstream), re-runs ./install with the recorded arguments, restarts, and prints the startup report. Data, passwords, ntfy accounts and the signing key are kept. If the installer fails, the previous version keeps running.

Running ./install again from the checkout does the same without the pull. Run ./install --apk again to offer a new APK to phones.

Uninstall

pipelines uninstall          # keeps data and credentials in <prefix>
pipelines uninstall --purge  # also deletes <prefix>, including the database

Neither touches OpenCode, Claude Code, Codex, Pi or files in ~/.config/pipelines.

Developer server

For working on Pipelines itself, run a server in the foreground on spare ports. This is not the supported installation and does not serve the web app:

go run ./cmd/pipelines-server -db /tmp/dev.db -addr 127.0.0.1:8081
go run ./cmd/pipelines --url ws://127.0.0.1:8081/connect tui

go run ./cmd/pipelines-server -help lists the server flags (-addr, -db, -allow-authoring, -opencode-endpoint, -mdns, -mdns-port). Without -db it uses the standard per-user data directory (XDG_DATA_HOME, or Application Support on macOS).

Troubleshooting

  • "Docker is not running": start Docker Desktop (or the daemon) and rerun.
  • Ports busy: a Docker installation or an old server may hold 8080, 8082 or 8090. Stop it, or choose other ports with the flags above.
  • "No usable configured harness model": rerun ./install in a terminal to pick models, or pass --model PROVIDER/MODEL.
  • CLI talks to the wrong server: PIPELINES_URL overrides the saved default; unset it. Check pipelines connection.
  • Web app missing: the server refuses to start without it. Run ./install --web.
  • A harness works in the terminal but not in the service: the service uses the PATH recorded at install time; pipelines harness setup <name> says so. Rerun ./install from a shell where the harness is on PATH. See Harnesses.