Simplify Go installation and support standard OpenCode releases

This commit is contained in:
2026-10-08 23:17:38 +01:00
parent 0f30a001c2
commit accbe63e39
101 changed files with 1331 additions and 682 deletions
+2 -2
View File
@@ -8,7 +8,7 @@ To reuse your host OpenCode login without container authentication:
docker compose up -d ntfy
# Bootstrap ntfy users/token as described below, then:
sh scripts/run-local-server.sh
go run ./cmd/tui -url ws://127.0.0.1:8080/connect
go run ./cmd/pipelines-tui -url ws://127.0.0.1:8080/connect
```
Do not run `docker compose up -d` for this mode: that starts the pipeline container
@@ -100,7 +100,7 @@ The Docker workspace is persistent and separate from your host checkout.
Connect the native TUI:
```sh
go run ./cmd/tui -url ws://127.0.0.1:8081/connect
go run ./cmd/pipelines-tui -url ws://127.0.0.1:8081/connect
```
If the TUI is on another machine, use an SSH/VPN tunnel instead of exposing the
+44
View File
@@ -0,0 +1,44 @@
# Installation review
Two independent reviewers examined installation conventions and stable OpenCode
compatibility. Both agreed that removing the V2 gate alone would hide real API
incompatibility rather than fix it.
## Findings addressed
- Installation mixed binary building with Docker provisioning, provider login,
private npm installation, service registration, and startup. It is now a thin
standard `go install ./cmd/...` operation, with no runtime checks or side effects.
- Go's repository import path did not match `go.mod`. The module now matches the
repository, enabling normal module installation (with `GOPRIVATE` for private access).
- Generic `server`/`tui` commands required renaming and a custom layout. They are
now `pipelines-server` and `pipelines-tui`, installed directly by Go.
- Runtime configuration was guessed relative to the binary, and a saved PATH
overrode the caller. Standard locations/explicit flags now select data; legacy
setup requires an explicit `PIPELINES_PREFIX`, and the caller's PATH is preserved.
- Native stable OpenCode has no equivalent to the V2 CLI/API contract. It now has
a separate HTTP adapter, using the existing executable/account, strict JSON
contracts, explicit models, translated scoped permissions, and an owned loopback
service. The existing V2 adapter is retained rather than pretending APIs match.
- Crash ownership must include the backend and native service. Additive metadata
records those references without rewriting immutable definitions or histories.
- The skill name did not match its directory. It is now `pipelines`. Native
authoring preloads the skill/reference text without inference; V2 activates it.
- Ownership persistence failure and authoring `SESSION` write failure could leak
created sessions. Those failure paths now clean up only their owned sessions.
## Deliberately removed from default installation
Private dependency installs, login prompts, mandatory notifications, automatic
LAN HTTP exposure, generated system bin symlinks, private binary prefixes, and a
second background supervisor. Configure optional services separately using normal
OS tools. The old managed installation helper remains only for explicit legacy
data/lifecycle access; it is not the standard installer.
## Verification limitations
The reviewer checked official stable OpenCode 1.18.35 in isolation and found V2
bridge URLs that return HTTP 200 **HTML**. The adapter rejects non-JSON responses.
No live inference or real model tool-permission enforcement was run. See
[verification scope](installation-verification.md); do not confuse successful
installation or stored permission rules with proven model execution.
+78 -75
View File
@@ -1,101 +1,104 @@
# Install and test
# Installation
Run as your normal user, **not** with sudo. Requires Python 3, a running Docker
daemon with Compose, and OpenCode V2 connected to OpenCode Go. Source installs
also require Go 1.27+. Platform archives contain the server, CLI, and TUI binaries. If OpenCode
is absent and npm is available, the installer offers a private pinned CLI install;
Docker itself is not installed automatically.
Installation requires **Go only**. OpenCode is needed when authoring or executing
model stages. Docker/ntfy is optional and is configured separately.
```sh
./install.sh
# Optional: expose the command in /usr/local/bin (sudo for the symlink only):
./install.sh --system-command
git clone https://git.a-wan.net/alan/pipelines.git
cd pipelines
go install ./cmd/...
# ./install.sh is a thin wrapper around the same command.
```
Setup prompts for phone username, password, and global notification topic. Blank
password generates one. `--yes` accepts defaults and generates a password;
`--password-file PATH` supplies one without putting it in shell arguments.
Phone connection details are saved privately in
`~/.local/share/pipelines/secrets/phone-login.txt`. Configure the ntfy phone app
with those credentials and subscribe to that topic. Phone delivery still needs
an actual device test. Prefer Tailscale or HTTPS; HTTP exposes credentials on an
untrusted network.
This installs `pipelines`, `pipelines-server`, and `pipelines-tui`. Go chooses the
destination using `GOBIN`, or the usual `GOPATH/bin` default. Put that directory
on PATH. No privileged operations, private dependency installation, login prompts,
Docker provisioning, service registration, or background processes are involved.
Re-running installation updates the binaries without changing data or configuration.
The native server and Docker ntfy are managed together:
For a private repository, use your normal Git credentials and `GOPRIVATE`:
```sh
pipelines # dashboard; b opens rendered templates
pipelines status
pipelines doctor
pipelines stop
pipelines start
pipelines restart
pipelines logs
GOPRIVATE=git.a-wan.net go install git.a-wan.net/alan/pipelines/cmd/...@latest
```
Default command path is `~/.local/bin/pipelines`; add `~/.local/bin` to PATH if
needed. It points directly to a compiled Go binary, not a shell launcher.
Source installation builds all binaries with `go install`. Service registration uses
per-user launchd on macOS or systemd on Linux.
`--no-autostart` uses background supervision instead. Services retain the current
user's OpenCode config/login, without exporting credentials into Docker.
`--system-command` is a system PATH shortcut to this user's install, **not** a
shared multi-user service. Other users should install their own instance.
For a system-wide command, build as your normal user and install the three binaries
into an administrator-selected bin directory using your normal system permissions.
Services and OpenCode accounts still run as the logged-in user, never as root.
The installer does not hardcode a system destination or run sudo.
## Run in a repository
## Start explicitly
```sh
cd ~/repos/my-project
pipelines run # fuzzy pipeline picker, then prompt
pipelines run "bug fix" # prompt composer
pipelines run "bug fix" "Fix the layout" # explicit execution confirmation
pipelines run "bug fix" "Fix the layout" --yes
pipelines serve -workspace-root "$HOME" # foreground server; Ctrl+C stops it
# In another terminal:
pipelines
```
The composer supports multiline typing/paste: **Enter** adds a newline,
**Ctrl+S** or **Alt+Enter** continues. The current directory is recorded as the
run workspace. After confirmation, the live run inspector opens.
`--yes` requires a supplied name and nonempty prompt and does not approve tools.
Use `--allow-tools` with `--yes` to explicitly approve declared tools;
`--write-paths "internal/example.go"` grants exact existing Go-file edits.
Current edit tooling is Go-specific, not arbitrary repository editing.
The server listens on loopback. Limit `-workspace-root` to trusted repositories;
repeat the flag for multiple roots. Omission permits only the server's working
directory. Runtime repository requests cannot escape these roots via symlinks.
Installer workspace roots default to the user's home directory. Restrict them
using repeatable `--workspace-root ~/repos` on initial setup. Runs outside those
roots (including symlink escapes) are rejected by the server. Raw native servers
default to their working directory; add `-workspace-root` to permit other repos.
Do not expose the unauthenticated pipeline server to untrusted users.
The SQLite database uses the standard user data location, honoring `XDG_DATA_HOME`
when set. On macOS it uses the user Application Support directory. `-db FILE`
selects another database, including an existing one; nothing is imported or moved
automatically. `pipelines serve --help` prints the actual defaults on your machine.
Drafts use the OS user cache; `PIPELINES_DRAFT_DIR` overrides that location.
## Templates and data
Use your OS's normal service manager if you want autostart. Pipelines does not
install a second supervisor. Native foreground startup is the default.
New databases are empty. Press **b** on Home or use `:templates` to browse optional
examples, with rendered Markdown previews and **v** for source. Select a template,
choose a unique name, edit it in `$VISUAL`/`$EDITOR` (or an available fallback),
then explicitly submit. Existing names cannot be overwritten by this flow.
Submitting a scheduled template activates its schedule; previewing never does.
## OpenCode
Uninstall retains pipeline history, credentials, and the isolated ntfy volume:
Use your existing `opencode` on PATH, or set `PIPELINES_OPENCODE` to an explicit
executable. No OpenCode version is required just to install binaries. At runtime:
- The standard/native API release uses an owned, authenticated loopback
`opencode serve` process, scoped to the requested repository and using the
existing user's configuration/login. The API, not a version number, is checked.
- A CLI advertising `api` uses the V2 command interface and its normal discovery.
- Both adapters select the pipeline's configured model explicitly and preserve
stage tool permissions. Native `shell` permissions map to OpenCode's `bash`.
- Audited native sessions record their backend and a private service reference
so recovery addresses the same service, rather than a new unrelated instance.
- `new`/`edit` preload authoring instructions before opening the normal interface.
Native preloading uses `noReply=true`; V2 uses explicit skill activation.
The bundled examples retain `opencode-go/deepseek-v4.1-flash`. Availability and
authentication for that model are runtime requirements, not installation checks.
No login or provider credentials are copied or exported by Pipelines.
## Optional notifications
Use an existing ntfy server or the optional Docker setup in [docs/ntfy.md](ntfy.md).
Configure `PIPELINES_NTFY_URL`, `PIPELINES_NTFY_PUBLIC_URL`, and a publishing token
via `PIPELINES_NTFY_TOKEN_FILE` in the server's environment. Keep secrets out of
pipeline definitions. Use HTTPS or an encrypted VPN for phone access; do not
expose an unauthenticated pipeline server or HTTP credentials to untrusted networks.
## Uninstall
Stop the foreground server first, then:
```sh
./uninstall.sh
# Explicitly delete this installation's retained data too:
./uninstall.sh --purge
pipelines uninstall
# or pipelines uninstall --yes
```
Reinstall at the same prefix to reuse retained data. Host OpenCode remains
separate. Partial installations require inspection or explicit purge before retry.
Use `--prefix PATH --bin-dir PATH --server-port PORT --ntfy-port PORT` for isolated
testing; each prefix has its own Compose project and volume.
Only Go binaries whose embedded build metadata matches this repository are
removed from the current command's bin directory. History, drafts, OpenCode,
and optional ntfy data remain untouched. Use the permissions appropriate to your
chosen bin directory. Custom database paths cannot be inferred, so uninstallation
does not purge data.
## Tester archives
## Existing managed installations
```sh
python3 scripts/package.py --platform darwin-arm64 --version 0.1.0-beta.1
# Also: darwin-amd64, linux-amd64, linux-arm64, source
```
The previous Python-supervised setup is retained only for explicit legacy use.
Set `PIPELINES_PREFIX` to its existing directory for its lifecycle commands;
no command infers that directory from an executable location. The old
`--system-command`, provisioning, and autostart installer flags have been removed.
Outputs go to `dist/` with a SHA-256 checksum. Archives include only allowlisted
source, examples, docs, installation scripts, and optional release binaries—no
databases, runtime drafts, `.env`, logs, or secrets. Extract, then run `./install.sh`.
Cross-built Linux binaries, Linux systemd, system PATH installation, and automatic
npm installation require separate execution tests; build success is not a runtime test.
To migrate, stop the old server first, install normal Go binaries, and start with
`pipelines serve -db EXISTING_DATABASE -workspace-root TRUSTED_ROOT`. Keep the old
data and credentials until you have verified the new setup. There is no automatic
purge, database move, login migration, or shared OpenCode service modification.
+20 -46
View File
@@ -1,52 +1,26 @@
# Installation verification
# Verification scope
Executed on macOS (arm64), using an isolated prefix and ports 18080/18090;
the development database and services were not targets of these checks.
Regression commands:
- `go test -race -count=1 ./...`: passed, including fresh empty database,
template catalog/create-only protection, workspace allowlisting and symlink
rejection, multiline prompt, and run-state regressions.
- `go vet ./...`: passed.
- `python3 -m unittest discover -s scripts -p 'test_*.py'`: 14 passed (including
unchanged argument forwarding to the shared CLI).
- `./install.sh --yes --no-autostart --prefix <isolated> --bin-dir <isolated>`:
installed native binaries plus isolated Docker ntfy, with private credentials.
- Isolated `pipelines doctor`: both health endpoints and host OpenCode API passed.
- Actual pseudo-terminal check (`check-install-cli.py` in the session's temporary
storage): rendered template browser, editor return without automatic submission,
explicit submission, fuzzy CLI selection, multiline composer, and supplied
name/prompt confirmation passed.
- Isolated `pipelines restart`, retained-data uninstall, reinstall, and a second
terminal check: passed; the submitted pipeline survived reinstall.
- Authenticated HTTP publication and phone-reader polling: passed. Actual Android
app reception is **not tested**.
```sh
go test -race -count=1 ./...
go vet ./...
python3 -m unittest discover -s scripts -p 'test_*.py'
```
Full model-driven repository changes, launchd autostart, Linux systemd, privileged
`--system-command`, and automatic npm CLI installation are **not tested**.
Existing Go-file edit-scope restrictions still apply.
Optional no-inference stable OpenCode checks:
## CLI authoring checks
```sh
PIPELINES_TEST_OPENCODE_V1=/path/to/opencode \
go test ./internal/opencode ./internal/author -run TestStable -count=1 -v
```
- `check-author-cli.py` (session temporary storage): isolated installed help,
JSON draft/validate/submit/export/history, and real OpenCode session/skill
activation passed. Interactive launch arguments were captured without a model turn.
- `check-author-picker.py`: actual terminal fuzzy `edit` selection, draft creation,
and real skill preload passed, again without a model turn.
- CLI/server regression tests cover stale-version rejection, invalid submission
rejection even with `--force`, create-only collisions, headless run idempotency,
inspection/cancellation, and bounded waiting that leaves an active run intact
on timeout. Execution is controlled in these tests, not a live model.
The stable checks use isolated home/config/data/cache directories; they never
copy login credentials or invoke a model. They exercise session creation, explicit
model selection, stored scoped permissions, abort on the same owned service,
session deletion, and authoring instruction preload with `noReply=true`.
Full interactive model-driven pipeline authoring is **not tested**.
## Compiled Go command
- `GOBIN=<isolated> go install ./cmd/pipelines ./cmd/server ./cmd/tui`: passed.
- Source installer now uses `go install`; isolated installation and retained-data
reinstall passed. The command symlink resolved to a Mach-O Go binary, not a script.
- Compiled command `help`, `list --json`, README three-stage example validation /
submission, `doctor`, and `restart`: passed against isolated services.
- A separately Go-installed CLI selected the same installation using
`PIPELINES_PREFIX` and read the retained pipeline successfully.
Privileged `/usr/local/bin` linking, launchd, and Linux remain **not tested**.
Native generation/assistant completion parsing is regression-tested against a
controlled HTTP server. Full live inference, permission enforcement by real model
tool calls, cancellation during inference, optional Docker deployment, privileged
system-wide copying, and OS service-manager registration are **not tested**.
+1 -1
View File
@@ -74,7 +74,7 @@ Configure the **pipeline server process** outside the pipeline file:
```sh
export PIPELINES_NTFY_ANIME_URL='https://ntfy.your-domain.example/anime'
# Set PIPELINES_NTFY_ANIME_TOKEN securely to a publishing user's access token.
go run ./cmd/server
go run ./cmd/pipelines-server
```
The URL includes one topic, without credentials, query parameters, or a fragment.