Files
pipelines/docs/docker.md
T
alan a48e4a5b6a Replace starters with four graded examples seeded into new databases
The template catalog is now Example 1-4: one stage, two stages (handoff),
feedback loop (runner-owned limits, retries, timeout) and human decision
(type=human approve/revise/abort with notify hooks). All read-only or leave
changes uncommitted; none commits, pushes or deploys.

A brand-new database (schema created by this Open) gets the examples once at
server start, bound to the configured default model and harness. A
server_settings marker records the decision, so removed examples never come
back and existing installs are marked without any pipeline being added or
rewritten.
2026-11-03 19:35:47 +00:00

11 KiB

Pipeline server + ntfy in one Compose project (Android)

Run ./install --docker from an interactive terminal (Docker is the explicit --docker/docker mode; ./install.sh remains a compatibility alias, and a bare ./install now performs the native host installation). It handles Compose configuration, ntfy users/permissions/token, image build, container skill install, first-time container harness provider login (OpenCode by default; ./install --docker --harness pi configures Pi and skips OpenCode login), startup, and an authenticated test notification. Choose your own provider in the configured harness; signup/login credentials stay in the Docker volume, not your host account. Re-run to update while preserving .env, tokens, histories and volumes. Use --relogin to connect another provider. No fixed OpenCode Go login is required. Health/publishing checks do not prove model inference or phone reception. The phone-facing ntfy URL uses the detected LAN IP; both services bind 0.0.0.0 by default. There are no networking questions. Advanced URL/bind/timezone flags remain optional.

The steps below are for manual setup or a developer/legacy local Pipelines server with Docker ntfy; they are not the supported installation and are not needed when you use ./install.

Manual developer/legacy: local pipeline server, Docker ntfy

To reuse your host OpenCode login without container authentication (manual, developer/legacy use—not a supported installation):

docker compose up -d ntfy
# Bootstrap ntfy users/token as described below, then:
sh scripts/run-local-server.sh
go run ./cmd/pipelines-tui -url ws://127.0.0.1:8080/connect

Do not run docker compose up -d for this mode: that starts the pipeline container too. The local launcher uses the existing host database and OpenCode account, publishes via localhost:8090, and advertises .env's NTFY_BASE_URL to the TUI. The top bar shows ntfy, the external server URL, and topic beside the pipeline server address on every page. No credentials are included in the public setup data. Stop any existing local pipeline server before launching another against the same database/port.

compose.yaml runs two services: pipelines (Go server + OpenCode V2) and ntfy. The pipeline server publishes internally to http://ntfy/pipelines; Android connects to your host's LAN/VPN URL. No host OpenCode service or credential database is mounted. Separate volumes persist pipeline history, workspace files, OpenCode credentials/sessions, and ntfy data. Do not use docker compose down -v unless you intend to delete those volumes.

Verification: docker compose build pipelines completed, and docker compose up -d ntfy started ntfy. A local authenticated Markdown upload was accepted with a phone-facing attachment URL. The local server's WebSocket advertised the public URL/topic and accepted the notification test pipeline. Go race tests and vet passed. Pipeline-container startup/model authentication and Android delivery are still not tested. Existing host history is preserved; the local server was restarted with notification configuration. The image now also includes python3 so the installer regression suite can run inside the container; a real 40x16/80x24/120x36 terminal UI review remains an independent step.

1. Local settings

Start Docker Desktop/Colima/your Docker daemon first. Choose the host address your phone can reach. Example (replace the LAN IP):

python3 scripts/configure-compose.py \
  --ntfy-url http://192.168.1.20:8090 \
  --bind-address 0.0.0.0
docker compose config --quiet

The helper detects the host's timezone and writes it to .env, so the Linux containers do not silently schedule in UTC. It refuses to overwrite .env or an existing token. If detection is unavailable, use --timezone Europe/London (or your actual IANA timezone). Rerun/edit settings if you move/change timezones.

The helper creates an empty, protected token file for bootstrap; no notifications are sent until it is filled. .env, secrets/, and data/ are git-ignored and excluded from image builds. The container publishes port 8080; the default host mapping is 8081, bound to host 0.0.0.0, avoiding the existing native server on 8080. Query the actual mapped port with docker compose port pipelines 8080. The unauthenticated pipeline WebSocket API shares that port, so expose it only on a trusted LAN/VPN and never to the public internet.

Use HTTP only on a trusted LAN or encrypted VPN. For public access, use HTTPS with a reverse proxy/VPN; configure the external URL in NTFY_BASE_URL. The container's attachment URLs use that external URL, not the internal service name. Both host port mappings default to 0.0.0.0; firewall them appropriately. The Pipelines API has no authentication and must not be exposed to untrusted networks.

2. Bootstrap ntfy users and token

docker compose up -d ntfy
docker compose exec ntfy ntfy user add pipeline-publisher
docker compose exec ntfy ntfy access pipeline-publisher pipelines write-only
docker compose exec ntfy ntfy user add phone-reader
docker compose exec ntfy ntfy access phone-reader pipelines read-only
docker compose exec -T ntfy ntfy token add pipeline-publisher | python3 scripts/save-ntfy-token.py

User creation prompts for passwords. Keep secrets/ mode 0700 on the host. The helper extracts only the raw token (the ntfy CLI includes prose in its response) and makes the final file readable inside the non-root pipeline container: local Compose file secrets are bind mounts and do not reliably remap uid/mode. Its private parent directory protects the source file from other host users. Do not commit tokens or paste them into a chat. Recreating/replacing the secret file requires restarting the pipeline container. The application accepts _TOKEN_FILE or _TOKEN, not both, and never puts the token in model prompts, outputs, or diagnostics.

3. Authenticate the harness, then start the server

python3 scripts/build_metadata.py  # resolve releases and stamp revision/dirty identity
docker compose stop --timeout 60 pipelines
# Default managed volumes only; never substitute a host bind mount for these paths.
docker compose run --rm --no-deps --user 0:0 --entrypoint sh pipelines -c \
  'chown -R "$PIPELINES_UID:$PIPELINES_GID" /data /home/pipelines /workspace'
docker compose run --rm --no-deps --entrypoint opencode pipelines auth login
# Pi instead (no standalone auth login): docker compose run --rm --no-deps --entrypoint pi pipelines --tui-mode regular --no-extensions --no-skills --no-context-files --no-prompt-templates --no-tools
docker compose up -d
docker compose logs -f pipelines

Select your preferred provider and follow its authentication flow. For Pi, type /login and choose an authentication method ("Sign in with an API key" for provider API credentials; account and Radius remain your choice), then exit Pi with ctrl+c/ctrl+d. Existing credentials stay in the volume and are reused by exiting when no new sign-in is needed; completing a sign-in can update the credentials stored in the volume. Credentials are stored in the dedicated opencode-home volume, separate from your host account. These interactive login is not tested in this regression. The Docker workspace/UID checks were executed with the actual image and a mocked stage. The image includes Go/git; installer-configured servers use the host user's UID/GID. The home and installation checkout are bound at their original paths so runs edit the real checkout, not the separate persistent image workspace at /workspace.

Connect the native TUI:

go run ./cmd/pipelines-tui -url ws://127.0.0.1:8081/connect
go run ./cmd/pipelines --url ws://127.0.0.1:8081/connect ping --json

If the TUI is on another machine, use an SSH/VPN tunnel instead of exposing the unauthenticated pipeline port publicly. A brand-new database starts with the four bundled example pipelines, seeded once; an existing database is never changed. Open :templates and use Submit template to name/review/publish another copy, or Customize to edit it first. All bundled examples are manual-only.

4. Android

Install ntfy, add the custom server http://192.168.1.20:8090 (or your HTTPS/VPN URL), sign in as phone-reader, and subscribe to pipelines. This is the single system-wide notification topic, shared by every pipeline. Grant notifications and background operation as required by your Android app/device. Outside your LAN, enable the VPN or use your HTTPS address. localhost on Android is the phone, not your server. No iOS/APNS upstream is needed for this Android setup.

Use the app to confirm a test notification and download an attachment before relying on unattended schedules. This phone/background-delivery check has not been performed. Attachments/messages expire after seven days in this config.

Existing data and rebuilds

The containers initially use a fresh SQLite database; the host's pipelines.db is left untouched. To migrate, first stop the host pipeline server (preventing two schedulers), make a consistent SQLite backup, and import it into the stopped container's /data/pipelines.db. Copying only a live .db without its WAL is unsafe. Do not import over an existing container database without backing it up. The database import and OpenCode session-recovery implications are not tested.

Rebuild executable changes with ./install; for a source-only refresh that still resolves the current releases, use python3 scripts/build_metadata.py. A plain docker compose build lacks the resolved OpenCode digest and cannot build now. The workspace volume does not automatically replace its existing source tree on image rebuild; back it up and update it explicitly if you need new source for Go verification. Do not delete it casually: it may contain user-created artifacts. compose.yaml still uses ntfy's latest tag; pin a tested version/digest before production. OpenCode and Pi are not fixed version pins: ./install resolves the current supported OpenCode V2 release from the single documented complete GET /repos/{owner}/{repo}/git/matching-refs/tags/v2 response — a complete bounded array with no page/per_page query and no Link pagination (never the moving latest tag or the registry's insertion order) — and the current Pi release from npm, then builds FROM the OpenCode image's immutable sha256 digest and labels the built image with the resolved versions, digest and source identity. It fails closed (keeping the last verified Dockerfile fallback pins) rather than guessing or silently substituting a stale pin when discovery, digest resolution, build or compatibility verification fails. A full ./install --docker always invokes the Docker build for a dirty source and relies on layer caching; a model/login-only reconfiguration rebuilds only when the verified release image identity (versions/digest) no longer matches the resolved releases, so a source change alone does not rebuild on reconfigure.