Files
alan a48e4a5b6a Replace starters with four graded examples seeded into new databases
The template catalog is now Example 1-4: one stage, two stages (handoff),
feedback loop (runner-owned limits, retries, timeout) and human decision
(type=human approve/revise/abort with notify hooks). All read-only or leave
changes uncommitted; none commits, pushes or deploys.

A brand-new database (schema created by this Open) gets the examples once at
server start, bound to the configured default model and harness. A
server_settings marker records the decision, so removed examples never come
back and existing installs are marked without any pipeline being added or
rewritten.
2026-11-03 19:35:47 +00:00

197 lines
11 KiB
Markdown

# Pipeline server + ntfy in one Compose project (Android)
## Complete installer (recommended for both services in Docker)
Run `./install --docker` from an interactive terminal (Docker is the explicit
`--docker`/`docker` mode; `./install.sh` remains a compatibility alias, and a bare
`./install` now performs the native host installation). It handles Compose
configuration, ntfy users/permissions/token, image build, container skill install,
first-time container harness provider login (OpenCode by default; `./install
--docker --harness pi` configures Pi and skips OpenCode login), startup, and an
authenticated test notification. Choose your own provider in the configured
harness; signup/login credentials stay in the Docker volume, not your host
account. Re-run to update while
preserving `.env`, tokens, histories and volumes. Use `--relogin` to connect another
provider. No fixed OpenCode Go login is required. Health/publishing checks do not
prove model inference or phone reception.
The phone-facing ntfy URL uses the detected LAN IP; both services bind `0.0.0.0` by default.
There are no networking questions. Advanced URL/bind/timezone flags remain optional.
The steps below are for manual setup or a developer/legacy local Pipelines server
with Docker ntfy; they are **not** the supported installation and are not needed when
you use `./install`.
## Manual developer/legacy: local pipeline server, Docker ntfy
To reuse your host OpenCode login without container authentication (manual,
developer/legacy use—not a supported installation):
```sh
docker compose up -d ntfy
# Bootstrap ntfy users/token as described below, then:
sh scripts/run-local-server.sh
go run ./cmd/pipelines-tui -url ws://127.0.0.1:8080/connect
```
Do not run `docker compose up -d` for this mode: that starts the pipeline container
too. The local launcher uses the existing host database and OpenCode account,
publishes via localhost:8090, and advertises `.env`'s `NTFY_BASE_URL` to the TUI.
The top bar shows **ntfy**, the external server URL, and topic beside the pipeline
server address on every page.
No credentials are included in the public setup data. Stop any existing local
pipeline server before launching another against the same database/port.
`compose.yaml` runs two services: `pipelines` (Go server + OpenCode V2) and `ntfy`.
The pipeline server publishes internally to `http://ntfy/pipelines`; Android connects
to your host's LAN/VPN URL. No host OpenCode service or credential database is
mounted. Separate volumes persist pipeline history, workspace files, OpenCode
credentials/sessions, and ntfy data. Do not use `docker compose down -v` unless
you intend to delete those volumes.
**Verification:** `docker compose build pipelines` completed, and
`docker compose up -d ntfy` started ntfy. A local authenticated Markdown upload
was accepted with a phone-facing attachment URL. The local server's WebSocket
advertised the public URL/topic and accepted the notification test pipeline.
Go race tests and vet passed. Pipeline-container startup/model authentication
and Android delivery are still **not tested**. Existing host history is preserved;
the local server was restarted with notification configuration. The image now
also includes python3 so the installer regression suite can run inside the
container; a real 40x16/80x24/120x36 terminal UI review remains an independent
step.
## 1. Local settings
Start Docker Desktop/Colima/your Docker daemon first. Choose the host address your
phone can reach. Example (replace the LAN IP):
```sh
python3 scripts/configure-compose.py \
--ntfy-url http://192.168.1.20:8090 \
--bind-address 0.0.0.0
docker compose config --quiet
```
The helper detects the **host's timezone** and writes it to `.env`, so the Linux
containers do not silently schedule in UTC. It refuses to overwrite `.env` or an
existing token. If detection is unavailable, use `--timezone Europe/London` (or
your actual IANA timezone). Rerun/edit settings if you move/change timezones.
The helper creates an empty, protected token file for bootstrap; no notifications
are sent until it is filled. `.env`, `secrets/`, and `data/` are git-ignored and
excluded from image builds. The container publishes port **8080**; the default
host mapping is **8081**, bound to host `0.0.0.0`, avoiding the existing native
server on 8080. Query the actual mapped port with
`docker compose port pipelines 8080`. The unauthenticated pipeline WebSocket API
shares that port, so expose it only on a trusted LAN/VPN and never to the public
internet.
Use HTTP only on a trusted LAN or encrypted VPN. For public access, use HTTPS
with a reverse proxy/VPN; configure the external URL in `NTFY_BASE_URL`. The
container's attachment URLs use that external URL, not the internal service name.
Both host port mappings default to `0.0.0.0`; firewall them appropriately. The
Pipelines API has no authentication and must not be exposed to untrusted networks.
## 2. Bootstrap ntfy users and token
```sh
docker compose up -d ntfy
docker compose exec ntfy ntfy user add pipeline-publisher
docker compose exec ntfy ntfy access pipeline-publisher pipelines write-only
docker compose exec ntfy ntfy user add phone-reader
docker compose exec ntfy ntfy access phone-reader pipelines read-only
docker compose exec -T ntfy ntfy token add pipeline-publisher | python3 scripts/save-ntfy-token.py
```
User creation prompts for passwords. Keep `secrets/` mode **0700** on the host.
The helper extracts only the raw token (the ntfy CLI includes prose in its response)
and makes the final file readable inside the non-root pipeline container: local Compose
file secrets are bind mounts and do not reliably remap uid/mode. Its private parent
directory protects the source file from other host users. Do not commit tokens
or paste them into a chat. Recreating/replacing the secret file requires restarting
the pipeline container. The application accepts `_TOKEN_FILE` or `_TOKEN`, not
both, and never puts the token in model prompts, outputs, or diagnostics.
## 3. Authenticate the harness, then start the server
```sh
python3 scripts/build_metadata.py # resolve releases and stamp revision/dirty identity
docker compose stop --timeout 60 pipelines
# Default managed volumes only; never substitute a host bind mount for these paths.
docker compose run --rm --no-deps --user 0:0 --entrypoint sh pipelines -c \
'chown -R "$PIPELINES_UID:$PIPELINES_GID" /data /home/pipelines /workspace'
docker compose run --rm --no-deps --entrypoint opencode pipelines auth login
# Pi instead (no standalone auth login): docker compose run --rm --no-deps --entrypoint pi pipelines --tui-mode regular --no-extensions --no-skills --no-context-files --no-prompt-templates --no-tools
docker compose up -d
docker compose logs -f pipelines
```
Select your preferred provider and follow its authentication flow. For Pi, type
`/login` and choose an authentication method ("Sign in with an API key" for
provider API credentials; account and Radius remain your choice), then exit Pi
with `ctrl+c`/`ctrl+d`. Existing credentials stay in the volume and are reused by
exiting when no new sign-in is needed; completing a sign-in can update the
credentials stored in the volume. Credentials are stored
in the dedicated `opencode-home` volume, separate from your host account. These
interactive login is **not tested** in this regression. The Docker workspace/UID
checks were executed with the actual image and a mocked stage. The image includes
Go/git; installer-configured servers use the host user's UID/GID. The home and
installation checkout are bound at their original paths so runs edit the real
checkout, not the separate persistent image workspace at `/workspace`.
Connect the native TUI:
```sh
go run ./cmd/pipelines-tui -url ws://127.0.0.1:8081/connect
go run ./cmd/pipelines --url ws://127.0.0.1:8081/connect ping --json
```
If the TUI is on another machine, use an SSH/VPN tunnel instead of exposing the
unauthenticated pipeline port publicly. A brand-new database starts with the four
bundled example pipelines, seeded once; an existing database is never changed.
Open `:templates` and use **Submit template** to name/review/publish another copy,
or **Customize** to edit it first. All bundled examples are manual-only.
## 4. Android
Install ntfy, add the custom server `http://192.168.1.20:8090` (or your HTTPS/VPN
URL), sign in as `phone-reader`, and subscribe to **pipelines**. This is the single
system-wide notification topic, shared by every pipeline. Grant notifications
and background operation as required by your Android app/device. Outside your
LAN, enable the VPN or use your HTTPS address. `localhost` on Android is the
phone, not your server. No iOS/APNS upstream is needed for this Android setup.
Use the app to confirm a test notification and download an attachment before
relying on unattended schedules. This phone/background-delivery check has **not
been performed**. Attachments/messages expire after seven days in this config.
## Existing data and rebuilds
The containers initially use a fresh SQLite database; the host's `pipelines.db`
is left untouched. To migrate, first stop the host pipeline server (preventing
two schedulers), make a consistent SQLite backup, and import it into the stopped
container's `/data/pipelines.db`. Copying only a live `.db` without its WAL is
unsafe. Do not import over an existing container database without backing it up.
The database import and OpenCode session-recovery implications are **not tested**.
Rebuild executable changes with `./install`; for a source-only refresh that still
resolves the current releases, use `python3 scripts/build_metadata.py`. A plain
`docker compose build` lacks the resolved OpenCode digest and cannot build now.
The workspace volume does not automatically replace its existing source tree
on image rebuild; back it up and update it explicitly if you need new source for
Go verification. Do not delete it casually: it may contain user-created artifacts.
`compose.yaml` still uses ntfy's `latest` tag; pin a tested version/digest before
production. OpenCode and Pi are not fixed version pins: `./install` resolves the
current supported OpenCode V2 release from the single documented complete
`GET /repos/{owner}/{repo}/git/matching-refs/tags/v2` response — a complete bounded
array with no page/per_page query and no `Link` pagination (never the moving
`latest` tag or the registry's insertion order) — and the current Pi release from
npm, then builds `FROM` the OpenCode image's immutable `sha256` digest and labels
the built image with the resolved versions, digest and source identity. It fails
closed (keeping the last verified Dockerfile fallback pins) rather than guessing or
silently substituting a stale pin when discovery, digest resolution, build or
compatibility verification fails. A full `./install --docker` always invokes the
Docker build for a dirty source and relies on layer caching; a model/login-only
reconfiguration rebuilds only when the verified release image identity
(versions/digest) no longer matches the resolved releases, so a source change
alone does not rebuild on reconfigure.