05ae530442
Install current verified harness releases inside Docker, preserve persistent accounts and settings, and provide separate interactive and pipeline model defaults with terminal onboarding and fuzzy pickers. Add Docker-aware authoring transport, Pi stage execution, bounded workflow controls, reliable final-result readback and usage settlement, and finite short-lived CLI pipe draining. Keep dated installation evidence and packaged authoring guidance consistent. Verified with a real Docker installation, both harness stages and interactive authors, the built-in plan/implement/review starter, independent host race/vet checks, installer tests, and packaged skill tests. Generated release receipts stay ignored in clean checkouts.
194 lines
11 KiB
Markdown
194 lines
11 KiB
Markdown
# Pipeline server + ntfy in one Compose project (Android)
|
|
|
|
## Complete installer (recommended for both services in Docker)
|
|
|
|
Run `./install` from an interactive terminal (Docker is the default;
|
|
`./install.sh` remains a compatibility alias). It handles Compose
|
|
configuration, ntfy users/permissions/token, image build, container skill install,
|
|
first-time container harness provider login (OpenCode by default; `./install
|
|
--harness pi` configures Pi and skips OpenCode login), startup, and an
|
|
authenticated test notification. Choose your own provider in the configured
|
|
harness; signup/login credentials stay in the Docker volume, not your host
|
|
account. Re-run to update while
|
|
preserving `.env`, tokens, histories and volumes. Use `--relogin` to connect another
|
|
provider. No fixed OpenCode Go login is required. Health/publishing checks do not
|
|
prove model inference or phone reception.
|
|
The phone-facing ntfy URL uses the detected LAN IP; both services bind `0.0.0.0` by default.
|
|
There are no networking questions. Advanced URL/bind/timezone flags remain optional.
|
|
|
|
The steps below are for manual setup or a developer/legacy local Pipelines server
|
|
with Docker ntfy; they are **not** the supported installation and are not needed when
|
|
you use `./install`.
|
|
|
|
## Manual developer/legacy: local pipeline server, Docker ntfy
|
|
|
|
To reuse your host OpenCode login without container authentication (manual,
|
|
developer/legacy use—not a supported installation):
|
|
|
|
```sh
|
|
docker compose up -d ntfy
|
|
# Bootstrap ntfy users/token as described below, then:
|
|
sh scripts/run-local-server.sh
|
|
go run ./cmd/pipelines-tui -url ws://127.0.0.1:8080/connect
|
|
```
|
|
|
|
Do not run `docker compose up -d` for this mode: that starts the pipeline container
|
|
too. The local launcher uses the existing host database and OpenCode account,
|
|
publishes via localhost:8090, and advertises `.env`'s `NTFY_BASE_URL` to the TUI.
|
|
The top bar shows **ntfy**, the external server URL, and topic beside the pipeline
|
|
server address on every page.
|
|
No credentials are included in the public setup data. Stop any existing local
|
|
pipeline server before launching another against the same database/port.
|
|
|
|
`compose.yaml` runs two services: `pipelines` (Go server + OpenCode V2) and `ntfy`.
|
|
The pipeline server publishes internally to `http://ntfy/pipelines`; Android connects
|
|
to your host's LAN/VPN URL. No host OpenCode service or credential database is
|
|
mounted. Separate volumes persist pipeline history, workspace files, OpenCode
|
|
credentials/sessions, and ntfy data. Do not use `docker compose down -v` unless
|
|
you intend to delete those volumes.
|
|
|
|
**Verification:** `docker compose build pipelines` completed, and
|
|
`docker compose up -d ntfy` started ntfy. A local authenticated Markdown upload
|
|
was accepted with a phone-facing attachment URL. The local server's WebSocket
|
|
advertised the public URL/topic and accepted the notification test pipeline.
|
|
Go race tests and vet passed. Pipeline-container startup/model authentication
|
|
and Android delivery are still **not tested**. Existing host history is preserved;
|
|
the local server was restarted with notification configuration. The image now
|
|
also includes python3 so the installer regression suite can run inside the
|
|
container; a real 40x16/80x24/120x36 terminal UI review remains an independent
|
|
step.
|
|
|
|
## 1. Local settings
|
|
|
|
Start Docker Desktop/Colima/your Docker daemon first. Choose the host address your
|
|
phone can reach. Example (replace the LAN IP):
|
|
|
|
```sh
|
|
python3 scripts/configure-compose.py \
|
|
--ntfy-url http://192.168.1.20:8090 \
|
|
--bind-address 0.0.0.0
|
|
docker compose config --quiet
|
|
```
|
|
|
|
The helper detects the **host's timezone** and writes it to `.env`, so the Linux
|
|
containers do not silently schedule in UTC. It refuses to overwrite `.env` or an
|
|
existing token. If detection is unavailable, use `--timezone Europe/London` (or
|
|
your actual IANA timezone). Rerun/edit settings if you move/change timezones.
|
|
|
|
The helper creates an empty, protected token file for bootstrap; no notifications
|
|
are sent until it is filled. `.env`, `secrets/`, and `data/` are git-ignored and
|
|
excluded from image builds. The container publishes port **8080**; the default
|
|
host mapping is **8081**, bound to host `0.0.0.0`, avoiding the existing native
|
|
server on 8080. Query the actual mapped port with
|
|
`docker compose port pipelines 8080`. Android does not need access to the
|
|
unauthenticated pipeline API.
|
|
|
|
Use HTTP only on a trusted LAN or encrypted VPN. For public access, use HTTPS
|
|
with a reverse proxy/VPN; configure the external URL in `NTFY_BASE_URL`. The
|
|
container's attachment URLs use that external URL, not the internal service name.
|
|
Both host port mappings default to `0.0.0.0`; firewall them appropriately. The
|
|
Pipelines API has no authentication and must not be exposed to untrusted networks.
|
|
|
|
## 2. Bootstrap ntfy users and token
|
|
|
|
```sh
|
|
docker compose up -d ntfy
|
|
docker compose exec ntfy ntfy user add pipeline-publisher
|
|
docker compose exec ntfy ntfy access pipeline-publisher pipelines write-only
|
|
docker compose exec ntfy ntfy user add phone-reader
|
|
docker compose exec ntfy ntfy access phone-reader pipelines read-only
|
|
docker compose exec -T ntfy ntfy token add pipeline-publisher | python3 scripts/save-ntfy-token.py
|
|
```
|
|
|
|
User creation prompts for passwords. Keep `secrets/` mode **0700** on the host.
|
|
The helper extracts only the raw token (the ntfy CLI includes prose in its response)
|
|
and makes the final file readable inside the non-root pipeline container: local Compose
|
|
file secrets are bind mounts and do not reliably remap uid/mode. Its private parent
|
|
directory protects the source file from other host users. Do not commit tokens
|
|
or paste them into a chat. Recreating/replacing the secret file requires restarting
|
|
the pipeline container. The application accepts `_TOKEN_FILE` or `_TOKEN`, not
|
|
both, and never puts the token in model prompts, outputs, or diagnostics.
|
|
|
|
## 3. Authenticate the harness, then start the server
|
|
|
|
```sh
|
|
python3 scripts/build_metadata.py # resolve releases and stamp revision/dirty identity
|
|
docker compose stop --timeout 60 pipelines
|
|
# Default managed volumes only; never substitute a host bind mount for these paths.
|
|
docker compose run --rm --no-deps --user 0:0 --entrypoint sh pipelines -c \
|
|
'chown -R "$PIPELINES_UID:$PIPELINES_GID" /data /home/pipelines /workspace'
|
|
docker compose run --rm --no-deps --entrypoint opencode pipelines auth login
|
|
# Pi instead (no standalone auth login): docker compose run --rm --no-deps --entrypoint pi pipelines --tui-mode regular --no-extensions --no-skills --no-context-files --no-prompt-templates --no-tools
|
|
docker compose up -d
|
|
docker compose logs -f pipelines
|
|
```
|
|
|
|
Select your preferred provider and follow its authentication flow. For Pi, type
|
|
`/login` and choose an authentication method ("Sign in with an API key" for
|
|
provider API credentials; account and Radius remain your choice), then exit Pi
|
|
with `ctrl+c`/`ctrl+d`. Existing credentials stay in the volume and are reused by
|
|
exiting when no new sign-in is needed; completing a sign-in can update the
|
|
credentials stored in the volume. Credentials are stored
|
|
in the dedicated `opencode-home` volume, separate from your host account. These
|
|
interactive login is **not tested** in this regression. The Docker workspace/UID
|
|
checks were executed with the actual image and a mocked stage. The image includes
|
|
Go/git; installer-configured servers use the host user's UID/GID. The home and
|
|
installation checkout are bound at their original paths so runs edit the real
|
|
checkout, not the separate persistent image workspace at `/workspace`.
|
|
|
|
Connect the native TUI:
|
|
|
|
```sh
|
|
go run ./cmd/pipelines-tui -url ws://127.0.0.1:8081/connect
|
|
go run ./cmd/pipelines --url ws://127.0.0.1:8081/connect ping --json
|
|
```
|
|
|
|
If the TUI is on another machine, use an SSH/VPN tunnel instead of exposing the
|
|
unauthenticated pipeline port publicly. The bundled examples are not automatically
|
|
submitted. Open `:templates` and use **Submit template** to name/review/publish a
|
|
simple starter, or **Customize** to edit it first. All bundled starters are manual-only.
|
|
|
|
## 4. Android
|
|
|
|
Install ntfy, add the custom server `http://192.168.1.20:8090` (or your HTTPS/VPN
|
|
URL), sign in as `phone-reader`, and subscribe to **pipelines**. This is the single
|
|
system-wide notification topic, shared by every pipeline. Grant notifications
|
|
and background operation as required by your Android app/device. Outside your
|
|
LAN, enable the VPN or use your HTTPS address. `localhost` on Android is the
|
|
phone, not your server. No iOS/APNS upstream is needed for this Android setup.
|
|
|
|
Use the app to confirm a test notification and download an attachment before
|
|
relying on unattended schedules. This phone/background-delivery check has **not
|
|
been performed**. Attachments/messages expire after seven days in this config.
|
|
|
|
## Existing data and rebuilds
|
|
|
|
The containers initially use a fresh SQLite database; the host's `pipelines.db`
|
|
is left untouched. To migrate, first stop the host pipeline server (preventing
|
|
two schedulers), make a consistent SQLite backup, and import it into the stopped
|
|
container's `/data/pipelines.db`. Copying only a live `.db` without its WAL is
|
|
unsafe. Do not import over an existing container database without backing it up.
|
|
The database import and OpenCode session-recovery implications are **not tested**.
|
|
|
|
Rebuild executable changes with `./install`; for a source-only refresh that still
|
|
resolves the current releases, use `python3 scripts/build_metadata.py`. A plain
|
|
`docker compose build` lacks the resolved OpenCode digest and cannot build now.
|
|
The workspace volume does not automatically replace its existing source tree
|
|
on image rebuild; back it up and update it explicitly if you need new source for
|
|
Go verification. Do not delete it casually: it may contain user-created artifacts.
|
|
`compose.yaml` still uses ntfy's `latest` tag; pin a tested version/digest before
|
|
production. OpenCode and Pi are not fixed version pins: `./install` resolves the
|
|
current supported OpenCode V2 release from the single documented complete
|
|
`GET /repos/{owner}/{repo}/git/matching-refs/tags/v2` response — a complete bounded
|
|
array with no page/per_page query and no `Link` pagination (never the moving
|
|
`latest` tag or the registry's insertion order) — and the current Pi release from
|
|
npm, then builds `FROM` the OpenCode image's immutable `sha256` digest and labels
|
|
the built image with the resolved versions, digest and source identity. It fails
|
|
closed (keeping the last verified Dockerfile fallback pins) rather than guessing or
|
|
silently substituting a stale pin when discovery, digest resolution, build or
|
|
compatibility verification fails. A full `./install` always invokes the Docker build
|
|
for a dirty source and relies on layer caching; a model/login-only reconfiguration
|
|
rebuilds only when the verified release image identity (versions/digest) no longer
|
|
matches the resolved releases, so a source change alone does not rebuild on
|
|
reconfigure.
|