Install authenticated native web proxy with user autostart

Register the existing native supervisor with launchd/systemd, include the dependency-free authenticated web proxy, detect Node, and keep the API loopback by default. Preserve existing web credentials and configured host harness defaults. Clarify single-step setup, lifecycle selection, and optional ntfy limits.

Checks: 44 mocked native installer/lifecycle tests; 8 entrypoint tests; 42 Docker installer tests; GOPROXY=off GOTOOLCHAIN=local go test ./internal/installation ./cmd/pipelines; sh -n install install.sh; git diff --check. No live install, deployment, restart, downloads, real autostart/login verification or native Linux verification.
This commit is contained in:
2026-10-20 21:30:07 +01:00
parent b56d6af180
commit ae6ba6ae10
6 changed files with 397 additions and 46 deletions
+74 -19
View File
@@ -10,7 +10,7 @@ See [configured harnesses](harnesses-and-control.md) for scope and verification
```sh
git clone https://git.a-wan.net/alan/pipelines.git
cd pipelines
./install # native: build server, CLI and ntfy on this host
./install # native: server, authenticated web, CLI, ntfy + login autostart
./install --docker # explicit Docker server + ntfy
```
@@ -18,13 +18,37 @@ cd pipelines
`./install`, `./install --native` and `./install native` run
`scripts/setup-native.py`. It only detects the tools it needs and never installs
or upgrades them: **Go, python3, git, curl and a C compiler (`cc`/`clang`/`gcc`)**
or upgrades them: **Go (version required by `go.mod`), Python 3.9+, Node.js 18+,
git, curl and a C compiler (`cc`/`clang`/`gcc`)**
must already be on PATH. The C compiler is required because ntfy stores its cache
and auth database in SQLite through the cgo driver `mattn/go-sqlite3`. It also
detects an existing `opencode` (required) and `pi` (optional) on PATH and records
their absolute paths; it never downloads, installs or upgrades OpenCode or Pi
(harness installation stays Docker-only).
Run as your normal logged-in user, not root. macOS requires `launchctl` and an
active GUI login; Linux requires `systemctl` and a working `systemd --user`
session. Unsupported platforms and missing tools fail instead of silently
installing an unsupervised server. Go downloads dependencies and the pinned ntfy
source during a fresh build; the installer does not install system prerequisites.
**`./install` is the only setup command.** It also copies the dependency-free web
client into `<prefix>/lib/web`, configures its authenticated Node proxy and starts
all three processes under one login service. No npm install or separate manual
`node serve.mjs` step is needed. The default browser URL is
`http://localhost:8082/` on this computer, or `http://HOST:8082/` on a trusted
LAN/VPN (`--web-port` / `--web-host` override it). Setup prints the actual URL.
The unauthenticated Go API defaults to **127.0.0.1:8080**; expose only the web
proxy. Use TLS termination or an encrypted VPN for remote access; plain HTTP
does not encrypt the sign-in password.
Existing web credentials in `PIPELINES_WEB_AUTH_FILE`, or
`~/.config/pipelines/web-auth.json`, are reused unchanged. A fresh setup generates
a private 0600 file there with `password` and a cookie-signing `secret`; read its
`password` field locally to sign in (never share the whole file). An invalid or
non-private existing file stops setup instead of resetting credentials. The
service never inherits the test-only `PIPELINES_WEB_AUTH=disabled` bypass.
Setup resolves `PIPELINES_PREFIX` or defaults to `~/.local/share/pipelines`,
creates `bin/`, `lib/`, `data/`, `logs/` and `secrets/` (`secrets` and `data` at
0700), copies `scripts/manage.py` to `<prefix>/lib/manage.py`, builds exactly
@@ -46,8 +70,8 @@ other binary, and it installs no unrelated tool; the manifest keeps
The native install also writes **host** harness profiles for the harnesses it
detects into XDG config home's `pipelines/harnesses.json` (no
`docker_directory`), defaulting to the detected OpenCode profile rather than a
stale `pi` default and taking models from `--model` or the existing valid profile.
`docker_directory`), preserving a valid host default (or choosing OpenCode instead
of a stale Docker default) and taking models from `--model` or the existing valid profile.
When migrating a Docker install it clears the recorded Docker link instead of
leaving the config Docker-linked. It never installs, upgrades or downloads a
harness. If no valid model can be resolved it writes nothing and points you at
@@ -83,20 +107,34 @@ present but its volumes cannot be inspected (for example a stopped daemon), setu
fails clearly instead of silently skipping migration, so an existing Docker
database is never shadowed by an empty native start.
Start and manage it with the manifest's prefix on PATH:
Setup already starts it and enables login autostart. Optional CLI management
requires the prefix selector (the executable path alone does not select it):
```sh
PIPELINES_PREFIX=~/.local/share/pipelines ~/.local/share/pipelines/bin/pipelines start
PIPELINES_PREFIX=~/.local/share/pipelines ~/.local/share/pipelines/bin/pipelines status
export PIPELINES_PREFIX="$HOME/.local/share/pipelines"
export PATH="$PIPELINES_PREFIX/bin:$PATH"
pipelines doctor
# pipelines stop / start / restart / logs
```
`start`, `stop`, `restart`, `doctor`, `logs` and `uninstall` reach
`<prefix>/lib/manage.py`. Ports default to 8080 (server) and 8090 (ntfy) and can
be changed with `--port`/`--ntfy-port`; setup starts both services itself, so
those ports must be free (a running Docker installation holds them — stop it
those ports and the web port (8082 by default) must be free (a running Docker installation holds them — stop it
first; the migration still needs its volumes and image). ntfy runs from
`<prefix>/bin/ntfy` with the generated config; per-user launchd (macOS) or
`systemd --user` (Linux) registration and the supervised fallback both remain.
`systemd --user` (Linux) starts `<python> <prefix>/lib/manage.py supervise --prefix
<prefix>`, which owns server, ntfy and web lifecycles. macOS writes
`~/Library/LaunchAgents/net.a-wan.pipelines.plist` and uses `launchctl enable`,
`bootstrap`, and `kickstart` in `gui/UID`. Linux writes
`$XDG_CONFIG_HOME/systemd/user/net.a-wan.pipelines.service` (default `~/.config`)
and runs `systemctl --user daemon-reload` then `enable --now
net.a-wan.pipelines.service`. This is login autostart, not boot-without-login;
Linux lingering is an explicit administrator decision. Do **not** run the separate
`pipelines service enable` for this installation: it configures a standalone
developer server, not the manifest database/ntfy/web supervisor. Logs are in
`<prefix>/logs/{server,ntfy,web,supervisor}.log` (Linux supervisor output also goes
to `journalctl --user -u net.a-wan.pipelines.service`).
Dated evidence (2026-10-13, macOS arm64): a scratch-prefix run of
`scripts/setup-native.py` resolved ntfy v2.30.0, compiled it from source (`ntfy
@@ -108,6 +146,11 @@ ntfy accounts from the live Docker volumes, and wrote the manifest.
`pipelines stop` released both ports. launchd/systemd registration and real phone
delivery are **not tested**.
The newer integrated web/autostart path is covered by scratch-prefix installer
mocks, rendered launchd/systemd definitions and mocked lifecycle/auth-preservation
checks. It has **not** been live-installed or tested through a real login on
either OS; Linux coverage here is mocked, not native Linux verification.
## Docker installation (explicit)
`./install --docker` (or `./install docker`) runs the unchanged
@@ -120,7 +163,7 @@ available, using `GOBIN` or `GOPATH/bin`; put that directory on PATH. It does no
install a host server. Reconfiguration uses `./install --opencode` or `./install --pi`
inside the existing Docker installation and never falls back to host execution. It
resolves and verifies the same current releases and refuses to configure against an
image that does not match them (rebuild with `./install` first); it does not restart
image that does not match them (rebuild with `./install --docker` first); it does not restart
or redeploy services.
Native Go builds embed VCS identity; Docker setup passes the checkout revision and
@@ -131,7 +174,7 @@ arguments plus the resolved version/digest arguments, without restarting service
A plain `docker compose build` cannot resolve the OpenCode image digest, so it must
be given `OPENCODE_VERSION`, `OPENCODE_DIGEST` and `PI_VERSION` through Docker
`--build-arg` flags (and `PIPELINES_VERSION`, `PIPELINES_COMMIT` and
`PIPELINES_DIRTY` outside a Git checkout); `./install` supplies all of them. After deployment,
`PIPELINES_DIRTY` outside a Git checkout); `./install --docker` supplies all of them. After deployment,
`pipelines --url ws://HOST:PORT/connect ping --json` identifies the running server.
Docker setup checks that the started server's pong matches the source identity
passed to the build and fails setup if metadata is missing or mismatched.
@@ -206,8 +249,8 @@ volumes use `nocopy` so Docker cannot reset a freshly prepared empty volume's
ownership to the image UID. Setup verifies that the server can enter the host
checkout before reporting completion.
For fresh configuration you can supply `--ntfy-url https://ntfy.example.com
--bind-address 127.0.0.1 --timezone Europe/London --port 8081` directly to `./install`.
For fresh Docker configuration you can supply `--ntfy-url https://ntfy.example.com
--bind-address 127.0.0.1 --timezone Europe/London --port 8081` to `./install --docker`.
The defaults are ready for direct phone access on your trusted LAN. Only use
plain HTTP on a trusted network. Other existing `.env` settings are not replaced.
These Docker flags apply to `./install --docker`; the explicit Docker mode fails
@@ -310,9 +353,9 @@ pipelines serve # foreground developer/legacy server; Ctrl+C stops it
pipelines
```
The server binds to `0.0.0.0` by default (the LAN, so the phone client can reach
it), matching the Docker install; record `127.0.0.1` on the manifest with
`./install --bind-address 127.0.0.1` for a loopback-only server. The Pipelines API
The native installer binds the Go server to loopback by default and exposes the
authenticated web proxy instead. An explicit `--bind-address` changes the manifest.
The Pipelines API
has no authentication, so keep that port on a trusted LAN/VPN. Change into your
repository and run a pipeline there; the client sends its current directory to the
server. The server's own working directory does not restrict which repositories
@@ -401,6 +444,11 @@ No login or provider credentials are copied or exported by Pipelines.
## Optional notifications
Installation configures this automatically via `./install`.
ntfy is independent of Android: it is an optional notification channel for any
compatible subscriber, including a separate ntfy app. A web-only user does not
need an Android client, and observing that the browser is the only client is not
a reason to disable an existing ntfy configuration. Browser-closed web push is
**not implemented**; installing/configuring ntfy does not add that capability.
For a developer/legacy host server, use an existing ntfy server or the Docker setup in [docs/ntfy.md](ntfy.md).
Configure `PIPELINES_NTFY_URL`, `PIPELINES_NTFY_PUBLIC_URL`, and a publishing token
via `PIPELINES_NTFY_TOKEN_FILE` in the server's environment. Keep secrets out of
@@ -409,8 +457,15 @@ expose an unauthenticated pipeline server or HTTP credentials to untrusted netwo
## Uninstall
Disable autostart with `pipelines service disable` if configured, or stop the
foreground server first, then:
For the native installer, select the manifest and run `pipelines uninstall`:
```sh
PIPELINES_PREFIX="$HOME/.local/share/pipelines" "$HOME/.local/share/pipelines/bin/pipelines" uninstall
```
This stops its supervisor and removes its login definition and managed binaries;
data and credentials are retained. The following applies only to standalone
developer binaries: disable `pipelines service disable` if configured, then:
```sh
pipelines uninstall
@@ -423,7 +478,7 @@ and optional ntfy data remain untouched. Use the permissions appropriate to your
chosen bin directory. Custom database paths cannot be inferred, so uninstallation
does not purge data.
## Existing managed installations
## Existing legacy managed installations
The previous Python-supervised setup is retained only for explicit legacy use.
Set `PIPELINES_PREFIX` to its existing directory for its lifecycle commands;