Files
pipelines/internal/author/skill/references/harnesses.md
T
alan a70e3aef4b Install Pipelines natively by default on macOS and Linux
Add scripts/setup-native.py and make ./install (and --native) build the server, the CLI
and ntfy from source into ~/.local/share/pipelines, detect an existing host OpenCode
and Pi instead of installing them, install the packaged authoring skill, migrate an
existing Docker database and ntfy auth store, save the host client default and start
both services. ./install --docker keeps the previous Docker server + ntfy
installation unchanged.

ntfy is compiled from its pinned Go module on every host because upstream publishes
no macOS server binary (its darwin asset is built with -tags noserver); its cgo
SQLite driver is why a C compiler is now required natively. The ntfy credential
bootstrap and the client default writer are shared by both installers,
installation.Load resolves symlinked prefixes, the Docker migration copies the
SQLite write-ahead log so the newest definitions and runs are not lost, and the
account subcommands read the generated configuration through NTFY_CONFIG_FILE.

Verified on macOS: 115 Python tests, go build/vet/test, and scratch installs that
migrate 9 pipelines and 31 runs, build ntfy v2.30.0 and start both services healthy
on spare ports before stopping them.
2026-10-13 17:20:14 +01:00

12 KiB

Configured harnesses

Current source supports OpenCode and Pi interactive authoring, debugging and model stages. nvim remains a file editor/read-only debug packet viewer, not a stage backend. Only configured harnesses appear in TUI pickers, default first. CLI new/edit/debug --harness opencode|pi|nvim overrides the default. pipelines harness launches the configured default in the invocation directory.

CLI new/edit/debug and TUI author/debug build one shared, Docker-aware CLI prefix. A Docker-linked profile embeds the in-container /usr/local/bin/pipelines (never the host binary: Linux can see but cannot execute a mounted macOS host executable). Loopback is recognized only via net.ParseIP(...).IsLoopback() plus an exact case-insensitive localhost, so a remote DNS name such as 127.remote.example stays remote and runs no probe. For a loopback logical --url, a single bounded docker compose port probe parses each returned line as a complete host:port binding (IPv4/IPv6/wildcard) and the prefix maps only when a binding covers the requested loopback address and port; malformed or non-matching bindings fail closed. A loopback wss endpoint is rejected as having no supported local TLS mapping rather than downgraded to plaintext. On a match it embeds scoped PIPELINES_TRANSPORT_SOURCE/PIPELINES_TRANSPORT_TARGET aliases in the prefix string (not cmd.Env, so the pre-existing harness server's tools inherit them). The client rewrites an endpoint only on an exact source match; remote endpoints stay remote. Draft metadata (server, base version, create-only guard) is preserved unchanged. provider-check with a Docker-linked profile runs inside that profile's existing Compose service with -T, the original cwd/model/timeout and exact stdout/exit propagation; container profiles have no DockerDirectory and cannot recurse, and explicit native profiles keep the host path. The picker uses a true x/term TTY check, so /dev/null and pipes fail fast instead of hanging.

pipelines harness setup opencode|pi --login --default runs a styled onboarding wizard (bold title, verified Docker harness versions) and selects separate harness and new-pipeline models using type-to-filter fuzzy pickers, not numbered prompts. Each picker itself shows its numbered step with the title and purpose (no duplicate numbered prose is printed outside it); the interactive harness model is Step 1 of 2 and the new-pipeline model is Step 2 of 2, matching fresh Docker setup. Typing filters, arrow keys or Tab navigate, Enter confirms, and Esc cancels. The saved value is preselected and marked (current); the window follows the active row on narrow terminals and the heading, selected row and no-match state stay within the real terminal height. Automation: --model PROVIDER/MODEL --pipeline-model PROVIDER/MODEL. harness default [opencode|pi] switches defaults; harness list prints settings. Setup reads catalogs/login state without inference. Changing defaults never rewrites published definitions or retained drafts. Canceling or failing either picker leaves saved preferences byte-identical. Non-terminal automation with a missing model fails clearly instead of starting a picker.

./install --opencode / ./install --pi reopen login and selection. With an existing Docker setup, they configure inside its container and link the host launcher without restarting/deploying services. They resolve and verify the current OpenCode V2 release from the single documented complete GET .../git/matching-refs/tags/v2 response (a bounded complete array with no pagination parameters; never the moving latest tag or the registry's insertion order) and the current Pi release from npm, resolve the selected tag's immutable GHCR sha256 digest, and require a rebuild only when the verified release image identity no longer matches. A full install always invokes the Docker build for a dirty source (relying on layer caching); a model/login-only reconfiguration does not rebuild. Setup writes its generated release receipt to the machine-local root file .pipelines-releases.json (with atomic .pipelines-releases-* temporary files); both are git-ignored, so a fresh clean checkout stays clean and reinstall still reuses the verified release/image identity. The actual built image's labels stay authoritative and a dirty source still rebuilds. If discovery, digest resolution or completeness cannot be established they stop with an error (keeping the last verified Dockerfile fallback pins); they never guess, fall back to a host runtime or install a stale pin. Without a Docker installation, ./install --opencode/--pi stop and require an existing Docker setup; the native installation reuses the host OpenCode configuration. Container and host credentials and accounts remain independent and are reused on reinstall/reconfigure. ./install installs natively by default (--native is equivalent); explicit ./install --docker selects the Docker server + ntfy installation. Fresh setup accepts ./install --docker --harness pi to skip OpenCode login. Existing setup can configure either harness independently. Pi setup does not run a generation check; provider-check --harness pi --model PROVIDER/MODEL is a separate confirmed, potentially chargeable check. Profiles live in XDG config home's pipelines/harnesses.json, store preferences only, and preserve unrelated configuration. Harness CLI setup/default commands always configure the local execution user, not a remote --url server. OpenCode interactive launch uses mini --model: verified V2 full CLI lacks a model flag. Pi uses its CLI with the packaged skill explicitly loaded. Pi has no standalone auth login command (verified pi auth --help: only credential check/print): setup prints a title-plus-three-lines Pi sign-in handoff and opens a fresh, no-context, extension-free, no-prompt-templates, no-tools session with Pi's documented --tui-mode regular flag, so the header and handoff stay visible above Pi's interface at 80x30 instead of the default fullscreen clear. The user types /login, chooses an authentication method ("Sign in with an API key" for provider API credentials; account and Radius remain the user's choice), finishes provider sign-in, then exits with ctrl+c/ctrl+d to return to Pipelines model configuration. Existing sign-ins are reused by exiting when no new sign-in is needed; completing a sign-in can update the credentials stored in the Docker volume. No coding/task prompt or scripted keystrokes are injected. OpenCode V2 model stages settle long sessions by polling GET /api/session/active about once per second under the stage context, instead of the .../wait request whose CLI-side request timeout failed long stages after roughly six minutes (HTTP 499, dated 2026-10-10). V1 native settlement is unchanged. Final results and debug session evidence use the same single bounded latest-assistant readback (order=desc&limit=1&type=assistant), so a small final result is visible behind large prior tool content; retained transcripts over 32 KiB are stored as valid bounded preview envelopes that preserve stage limits and diagnostics, and exact outputs remain authoritative. Pi evidence is JSONL and is not parsed as one JSON object. Short-lived context-controlled CLI calls (--help detection, API calls and the native export fallback) share one finite inherited-pipe drain grace (os/exec.Cmd.WaitDelay ~250ms) so a canceled context or a descendant that still holds stdout/stderr cannot block for the child's natural lifetime; the context.Canceled/DeadlineExceeded and exec.ErrWaitDelay identities are preserved and partial stdout after the grace is never accepted as success.

Stages set harness=pi or harness=opencode in configs; omission preserves historical OpenCode behavior. Pi must be configured on the execution server. The RPC adapter starts new retained sessions per visit with explicit tools and a Pipelines permission extension. Ambient extensions, skills, context files, prompt templates, automatic retries/compaction are disabled. Tool gates are not an OS sandbox: approved shell can have broad side effects. Usage accumulates across attempts/visits; missing required accounting fails closed. Strict envelope validation and bounded syntax-only repair never replay tools. A model-driven repair, when needed, uses a separate deny-all session with retained ownership and the same cumulative budget. Its output is checked locally for value changes.

Session/RPC audit lives in the execution user's cache under pipelines/pi-sessions. Ownership persists before prompts. Cleanup waits for settlement; expired leases request abort through the extension. Unconfirmed cleanup blocks continuation/retry.

2026-10-10 verification: scripts/check-pi-rpc.py checked installed Pi 0.87.1 readiness, metadata/config RPC and idle lease cleanup without inference. Go RPC fixtures cover result/budget/missing-accounting/timeout/format-repair paths; Node tests cover permission checks. PIPELINES_PI_LOCAL=1 go test ./internal/stage -run TestPiInstalledLocalIntegration also exercised the installed Pi against scripted loopback responses: actual allowed/denied writes, budget exhaustion, and active-shell cancellation (including no delayed write after cleanup). The installed-Pi loopback suite also covers isolated deny-all syntax repair with retained ownership and cumulative usage. The compiled fuzzy picker passed PTY selection/cancellation; installed Pi catalog discovery/configuration passed with an unreachable local endpoint and no prompt. This is integration evidence, not real-provider/model evidence. Active lease-expiry recovery, web search delivery and fresh interactive Docker Pi setup remain not tested. Pi catalogs exclude ambient provider extensions for parity with audited stages; compatible custom endpoints can use Pi's private models.json. An isolated built Docker image also passed non-root, network-disabled policy/RPC readiness and idle lease cleanup, without credentials/mounts or inference. Codex/Claude adapters and AI-assisted server guidelines validation remain future work.

2026-10-11 source-level coverage (source-level regressions; the separate dated 2026-10-11 final install is live evidence): the shared Docker-aware CLI prefix, exact-match client transport rewriting, Docker-linked provider-check dispatch and the x/term TTY check have source-level regression tests (internal/harness, internal/client, internal/cli). The prefix tests cover the remote-DNS negative (127.remote.example stays unmapped with no probe), IPv4/IPv6/localhost positives, complete host:port binding matching (a different service on the same port, or a family mismatch, does not match), malformed/empty output failing clearly, multiple bindings under a single probe, and loopback wss rejection. Terminal Pick regression tests use a short deadline so a TTY regression fails fast rather than hanging. Docker dispatch tests assert --json is passed only in JSON mode while plain text stays readable. Python fixtures cover arbitrary-digit OpenCode patches (a parsed 2.0.260 is still rejected against an expected 2.0.26 by exact equality; 260/100 are illustrative valid/future examples, not verified releases) and reject both raw and canonical reserved homes. Real in-container provider-check/ping confirmation, a compiled PTY run and a real Docker reconfigure remain not tested in this change. These source-level regressions are not a claim about the deployed image; the separate dated 2026-10-11 FINAL INSTALLED verification recorded that the runtime was actually installed and used (OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1, image sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136, with both real stages executed in run17 and passed in Root's full host race/vet runs). Active lease-expiry recovery, web search delivery and fresh interactive Docker Pi sign-in remain not tested.