Install Pipelines natively by default on macOS and Linux
Add scripts/setup-native.py and make ./install (and --native) build the server, the CLI and ntfy from source into ~/.local/share/pipelines, detect an existing host OpenCode and Pi instead of installing them, install the packaged authoring skill, migrate an existing Docker database and ntfy auth store, save the host client default and start both services. ./install --docker keeps the previous Docker server + ntfy installation unchanged. ntfy is compiled from its pinned Go module on every host because upstream publishes no macOS server binary (its darwin asset is built with -tags noserver); its cgo SQLite driver is why a C compiler is now required natively. The ntfy credential bootstrap and the client default writer are shared by both installers, installation.Load resolves symlinked prefixes, the Docker migration copies the SQLite write-ahead log so the newest definitions and runs are not lost, and the account subcommands read the generated configuration through NTFY_CONFIG_FILE. Verified on macOS: 115 Python tests, go build/vet/test, and scratch installs that migrate 9 pipelines and 31 runs, build ntfy v2.30.0 and start both services healthy on spare ports before stopping them.
This commit is contained in:
+96
-14
@@ -1,21 +1,103 @@
|
||||
# Installation
|
||||
|
||||
To reconfigure login and separate interactive/new-pipeline models without a
|
||||
service reinstall: `./install --opencode` or `./install --pi`.
|
||||
`./install` installs **natively on the host by default** (macOS and Linux);
|
||||
`./install --docker` runs the existing Docker server + ntfy installation.
|
||||
To reconfigure login and separate interactive/new-pipeline models inside an
|
||||
existing Docker installation without a service reinstall: `./install --opencode`
|
||||
or `./install --pi`.
|
||||
See [configured harnesses](harnesses-and-control.md) for scope and verification limits.
|
||||
|
||||
`./install` sets up **Docker server + ntfy**, the only supported runtime installation.
|
||||
It requires Docker Compose and Python 3 and includes OpenCode/Pi in the persistent
|
||||
server container. Host Go builds the CLI/TUI clients, not a native server.
|
||||
|
||||
```sh
|
||||
git clone https://git.a-wan.net/alan/pipelines.git
|
||||
cd pipelines
|
||||
./install
|
||||
# Optional: choose Pi rather than OpenCode for setup:
|
||||
./install --harness pi
|
||||
./install # native: build server, CLI and ntfy on this host
|
||||
./install --docker # explicit Docker server + ntfy
|
||||
```
|
||||
|
||||
## Native installation (default)
|
||||
|
||||
`./install`, `./install --native` and `./install native` run
|
||||
`scripts/setup-native.py`. It only detects the tools it needs and never installs
|
||||
or upgrades them: **Go, python3, git, curl and a C compiler (`cc`/`clang`/`gcc`)**
|
||||
must already be on PATH. The C compiler is required because ntfy stores its cache
|
||||
and auth database in SQLite through the cgo driver `mattn/go-sqlite3`. It also
|
||||
detects an existing `opencode` (required) and `pi` (optional) on PATH and records
|
||||
their absolute paths; it never downloads, installs or upgrades OpenCode or Pi
|
||||
(harness installation stays Docker-only).
|
||||
|
||||
Setup resolves `PIPELINES_PREFIX` or defaults to `~/.local/share/pipelines`,
|
||||
creates `bin/`, `lib/`, `data/`, `logs/` and `secrets/` (`secrets` and `data` at
|
||||
0700), copies `scripts/manage.py` to `<prefix>/lib/manage.py`, builds exactly
|
||||
`./cmd/pipelines-server` and `./cmd/pipelines` with `go build -trimpath` into
|
||||
`<prefix>/bin`, and runs `<prefix>/bin/pipelines skill install` so the packaged
|
||||
authoring skill lands in your OpenCode skills directory. No `pipelines-tui` is
|
||||
built. It writes `<prefix>/pipelines-install.json` atomically at 0600; that
|
||||
manifest is what `scripts/manage.py` and the Go CLI load. Provider/model defaults
|
||||
are never invented: your existing host OpenCode configuration is reused (an
|
||||
explicit `--model` is recorded if supplied), and setup never starts a picker.
|
||||
|
||||
ntfy is built from source with no Homebrew or package manager. Setup resolves the
|
||||
exact stable `binwiederhier/ntfy` tag (never the moving `latest` tag and never an
|
||||
older fallback), then compiles the server from the pinned `heckel.io/ntfy/v2` Go
|
||||
module at that tag into `<prefix>/bin/ntfy`. A source build is required because
|
||||
upstream publishes **no macOS server binary** (its `darwin_all` asset is built
|
||||
with `-tags noserver` and only provides the client commands), so both macOS and
|
||||
Linux take the same path; the Go toolchain verifies the module against its public
|
||||
checksum database. The build uses
|
||||
`-tags sqlite_omit_load_extension,osusergo,netgo` and creates the same empty web
|
||||
placeholders the upstream Makefile uses for local builds, so the bundled ntfy web
|
||||
UI is not populated (notifications and the API are unaffected). A binary for the
|
||||
same resolved version is reused instead of rebuilt. Setup generates
|
||||
`<prefix>/data/ntfy/server.yml` mirroring `compose.yaml` (`listen-http`,
|
||||
`base-url`, `cache-file`, `auth-file`, `auth-default-access: deny-all`,
|
||||
`enable-login`, attachment cache directory, expiry and size limits), creates the
|
||||
`pipeline-publisher` (write-only on `pipelines`) and `phone-reader` (read-only)
|
||||
accounts with ntfy's own CLI (`--ignore-exists`, so existing passwords are kept;
|
||||
the account subcommands read the generated config through `NTFY_CONFIG_FILE`),
|
||||
and writes the publisher token only to `<prefix>/secrets/publisher-token` at 0600.
|
||||
|
||||
When Docker volumes `pipelines_pipelines-data` or `pipelines_ntfy-data` exist,
|
||||
setup migrates the pipeline database to `<prefix>/data/pipelines.db` and the ntfy
|
||||
auth store to `<prefix>/data/ntfy/auth.db` using the local Pipelines image
|
||||
(never an implicit pull). Existing native data is never overwritten unless you
|
||||
pass `--replace-data`; a requested migration that fails aborts the install
|
||||
instead of starting empty. Absent volumes are a no-op. If the Docker CLI is
|
||||
present but its volumes cannot be inspected (for example a stopped daemon), setup
|
||||
fails clearly instead of silently skipping migration, so an existing Docker
|
||||
database is never shadowed by an empty native start.
|
||||
|
||||
Start and manage it with the manifest's prefix on PATH:
|
||||
|
||||
```sh
|
||||
PIPELINES_PREFIX=~/.local/share/pipelines ~/.local/share/pipelines/bin/pipelines start
|
||||
PIPELINES_PREFIX=~/.local/share/pipelines ~/.local/share/pipelines/bin/pipelines status
|
||||
```
|
||||
|
||||
`start`, `stop`, `restart`, `doctor`, `logs` and `uninstall` reach
|
||||
`<prefix>/lib/manage.py`. Ports default to 8080 (server) and 8090 (ntfy) and can
|
||||
be changed with `--port`/`--ntfy-port`; setup starts both services itself, so
|
||||
those ports must be free (a running Docker installation holds them — stop it
|
||||
first; the migration still needs its volumes and image). ntfy runs from
|
||||
`<prefix>/bin/ntfy` with the generated config; per-user launchd (macOS) or
|
||||
`systemd --user` (Linux) registration and the supervised fallback both remain.
|
||||
|
||||
Dated evidence (2026-10-13, macOS arm64): a scratch-prefix run of
|
||||
`scripts/setup-native.py` resolved ntfy v2.30.0, compiled it from source (`ntfy
|
||||
--version` = v2.30.0 with the `Server commands` present), built the server and
|
||||
CLI, installed the skill, migrated 8 pipeline definitions and 28 runs plus both
|
||||
ntfy accounts from the live Docker volumes, and wrote the manifest.
|
||||
`pipelines start` then brought up ntfy and the server through the supervisor
|
||||
(`{"healthy":true}` from both `/v1/health` and `/healthz` on spare ports) and
|
||||
`pipelines stop` released both ports. launchd/systemd registration and real phone
|
||||
delivery are **not tested**.
|
||||
|
||||
## Docker installation (explicit)
|
||||
|
||||
`./install --docker` (or `./install docker`) runs the unchanged
|
||||
`scripts/setup-docker.py`. It requires Docker Compose and Python 3 and includes
|
||||
OpenCode/Pi in the persistent server container. Host Go builds the CLI/TUI
|
||||
clients, not a native server.
|
||||
|
||||
The installer builds host `pipelines` and `pipelines-tui` clients when Go is
|
||||
available, using `GOBIN` or `GOPATH/bin`; put that directory on PATH. It does not
|
||||
install a host server. Reconfiguration uses `./install --opencode` or `./install --pi`
|
||||
@@ -111,9 +193,9 @@ For fresh configuration you can supply `--ntfy-url https://ntfy.example.com
|
||||
--bind-address 127.0.0.1 --timezone Europe/London --port 8081` directly to `./install`.
|
||||
The defaults are ready for direct phone access on your trusted LAN. Only use
|
||||
plain HTTP on a trusted network. Other existing `.env` settings are not replaced.
|
||||
Docker is also the default without a terminal; setup fails rather than
|
||||
silently falling back to native binaries if the LAN address cannot be detected.
|
||||
Use an interactive terminal for first Docker/provider setup.
|
||||
These Docker flags apply to `./install --docker`; the explicit Docker mode fails
|
||||
rather than silently falling back to native binaries if the LAN address cannot be
|
||||
detected. Use an interactive terminal for first Docker/provider setup.
|
||||
The Docker Pipelines API is unauthenticated: keep its port on a trusted network
|
||||
and do not forward it to the Internet. For a manual loopback-only deployment,
|
||||
set `PIPELINES_BIND_ADDRESS=127.0.0.1` after installation and use Compose directly.
|
||||
@@ -202,8 +284,8 @@ The installer does not hardcode a system destination or run sudo.
|
||||
## Developer/legacy host server (unsupported)
|
||||
|
||||
This explicit foreground server is for developing Pipelines itself or running a
|
||||
previous native setup. It is **not** a supported installation; a fresh database
|
||||
starts empty.
|
||||
previous Python-supervised legacy host setup. It is **not** the native installer;
|
||||
a fresh database starts empty.
|
||||
|
||||
```sh
|
||||
pipelines serve # foreground developer/legacy server; Ctrl+C stops it
|
||||
|
||||
Reference in New Issue
Block a user