Files
pipelines/docs/operator-reference.md
alan 5090f1cf4e Pipelines skill: outcome-first output, aligned review and loop guidance, operator material moved to docs
Rewrite format.md around turn files, the output.md order (# outcome, edge word, # handoff, detail), checks, configs, human stages and the land stage delivery gate (sh -c in a fresh detached worktree, clean checkouts incl. untracked, 64 KB output cap, opencode only). Align review guidance across practices, verification, orchestration and the feedback-loop example. Fix loop-limit semantics in control.md against runner.go, budget_tokens sizing, read for large inputs, blocked-else-error and one stage naming set. Move dated verification logs to docs/verification-history.md and brokered chat, TUI keys and harness internals to docs/operator-reference.md; label operator-only references in SKILL.md.
2026-11-04 17:27:01 +00:00

19 KiB

Operator reference

Operator and developer material moved out of the packaged Pipelines skill (internal/author/skill/references/operations.md) because authoring agents never need it.

TUI keys

In the TUI, Home advertises / omni search and : commands. Home / opens one popup over pipelines, versions, runs, steps/visits, artifact names and stage input/output/evidence text; each row shows a short type badge and a text match shows a bracketed snippet, and a bare status word or date selects runs. The same query is available as pipelines search QUERY [--type KIND], where --type narrows the kind. Run Tab switches only Details ↔ Graph. n/p select recorded visits, including repeated stages; d focuses/toggles file changes, A opens the run's grouped artifacts view (name, content type, size and URL per entry, PgUp/PgDn scroll so a trailing visit that recorded none stays reachable, Enter/o to open the selected URL and y to copy it), a selects the active/latest retained configured-harness agent session's bounded, redacted tool evidence, D/:debug opens fresh investigation, and P opens the pinned pipeline version. :quit exits; q does not. Results are retained reports, not independently confirmed achievements. Distinguish partial output from full success.

Brokered OpenCode chat

The protocol declares the brokered chat_* surface (chat_sessions, chat_create, chat_directories, chat_prompt, chat_messages, chat_interrupt, chat_permission, chat_reply). The server dispatches all of them, and pushes chat_event frames derived from OpenCode's global SSE firehose (/event on the native serve, /api/event on the V2 API), with retained per-session/per-message state so a reconnecting client re-syncs instead of replaying deltas.

At startup cmd/pipelines-server calls opencode.OpenChat, which detects the installed backend and opens exactly one owned opencode serve endpoint for it: the native wire for v1, the V2 API for the installed V2 CLI. Chat is enabled only when that connection carries a URL-bearing, SSE-capable event endpoint; otherwise chat stays disabled and every chat_* request is answered with a protocol error naming the missing configuration rather than hanging. Stage generation still reaches V2 through the opencode api CLI, so the owned serve process is chat's single transport and starts no process for stages. The adapter skips firehose events it does not surface before decoding their data, so an unrelated event whose delta has a different shape (session.instructions.updated carries an object) never fails the stream.

The operator can instead broker chat from their own OpenCode service with the opt-in -opencode-endpoint flag or PIPELINES_OPENCODE_ENDPOINT, with precedence flag > environment > the endpoint recorded on a previous run and the spawned instance as the default. opencode.OpenEndpoint validates the reference once (the established http(s)/host/no-credentials/no-query/no-path idiom), detects the backend from the service's own /api/info (V2) or /global/health (native) contract and requires a streamable text/event-stream event path; it returns a connection with no owned process and never calls startServe. A malformed reference, an unreachable host, a reachable non-OpenCode service or a health-ok-but-no-SSE service each disable chat with an error naming the endpoint and never fall back to the spawned instance. An authenticating endpoint accepts an optional password only through the private file PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE (regular, 0600, single-owner), and PIPELINES_OPENCODE_ENDPOINT_USERNAME overrides the detected default user. The validated choice is recorded in the additive server_settings table so a restart with no flag/environment reuses it; an unreachable recorded endpoint disables chat with a clear message and no fallback. pipelines service enable preserves the endpoint and the password-file reference in autostart.

chat_sessions lists the operator's own chat sessions, not the sessions pipeline runs create. Operator sessions are recorded in the dedicated operator_sessions table (session_id primary key, plus directory, title, tag, created_at, last_used_at), separate from run_steps/run_session_owners, so an operator chat is a first-class thing and a pipeline stage session — recorded as run ownership, never in the operator registry — is never listed. The handler reads the registry once (bounded to 128 rows, most-recently-used first), calls OpenCode's session list once, and keeps only registry rows the endpoint still reports, so a session with no recorded tag is never listed and a deleted upstream session does not linger. It fills each session's directory, tag and created_at/ updated_at from the registry and marks busy when another client currently holds the session (an in-server claim, released when the stream ends or the client disconnects); a second client's chat_prompt for a held session is rejected before any OpenCode call. It is fail-closed: with no registry configured it returns an empty list, never the endpoint's whole session list. The OpenCode session list is read unscoped by directory (the V2 GET /api/session otherwise defaults to the caller's directory), because an operator session may live in any directory.

chat_create creates one operator session and records it in the same registry with the stored tag operator (the display label Operator is derived from it, so there is no second tag vocabulary). It takes a title (default "New chat") and a directory (default ~): an omitted or empty value, ~, and any ~/... value all resolve to os.UserHomeDir, never the server's working directory; a relative path or a missing directory is rejected before any OpenCode call, and a session created but not recordable is reported as an error naming its id. Create posts the verified V2 shape POST /api/session {"title":...,"location":{"directory":...}}. chat_directories returns the operator's home first, then the directories of previous operator sessions most-recently-used first, deduplicated and bounded (8); it is a dedicated request so the create dialog can load the choices without first listing sessions and chat_sessions stays a pure, fail-closed session list. A successful chat_prompt touches the session's last_used_at in one UPDATE, so it moves to the front of the list; a non-operator session id touches no row.

chat_adopt records a session that already exists on the configured endpoint in the same operator registry, using the endpoint's own directory and title verbatim (never a guess, the server's working directory or the home directory). It reads the endpoint once through the shared Chat.Session (which reuses Sessions/decodeChatSessions), records through the existing RecordOperatorSession with the operator tag, and echoes the adopted session plus its registry row like chat_create, so chat_sessions lists it and chat_prompt continues it. A nil payload, a blank or foreign-shaped id, a nil broker or a nil registry is refused before any endpoint call; an id the endpoint does not report is refused naming the id and records nothing; an endpoint record with an empty directory is refused rather than substituting a directory. The CLI equivalent is pipelines chat-adopt <session-id> (--json prints the full response, otherwise a short Adopted <id> · <directory> line).

While a chat_prompt subscription is open the server also polls GET /api/permission/request on a fixed small interval (first poll immediately) and pushes one chat_permission frame per newly pending approval request, deduped by request id so a repeated poll never re-pushes the same request. The app answers with chat_reply, which the server forwards as POST /api/session/{id}/permission/{requestID}/reply with {"decision":"once"|"always"|"reject"}. A missing session/request id or an invalid decision is rejected before any OpenCode call, and the poller stops when the stream ends or the client disconnects.

The permission list and reply are implemented against the V2 HTTP API only: their field names match the service's Permission.Request/Permission.Reply schemas (verified against the installed service's /openapi.json). The native v1 serve translation rejects them explicitly instead of guessing an endpoint, so on v1 the permission poller stops quietly rather than reporting the documented limitation as a per-prompt error.

Verification evidence (2026-10-17)

Verified against the installed OpenCode 2.0.20 on 2026-10-17:

  • opencode --version reports 2.0.20; opencode --help advertises api and serve. GET /openapi.json on an owned opencode serve lists GET /api/event (SSE) and the session/prompt/interrupt/permission endpoints.
  • A live turn streamed token-level session.text.delta and session.reasoning.delta events (each with assistantMessageID and ordinal), plus session.step.* and session.usage.updated; the V2 API server authenticates the fixed opencode basic-auth user.
  • The opt-in live end-to-end test (PIPELINES_TEST_OPENCODE_CHAT_LIVE=1 go test ./internal/server -run TestChatLiveEndToEnd -v) opened an owned V2 endpoint, created an operator session through chat_create in a chosen directory, listed exactly that session through chat_sessions (with its directory and tag operator), recorded a genuine pipeline stage session in run_steps/run_session_owners and showed it never appeared in the list, and streamed a completed turn as [step:started,text,step:streamed,step:ended] (first delta "pong"). The operator-session registry, the filter, directory defaulting and the recents bound are also covered by the pipeline, opencode and server unit tests, which run without the live gate.
  • Not verified live: a genuine pending-approval round trip and any backend other than the installed 2.0.20. The permission wire shapes are unit-tested over an httptest fixture, not against a live pending request.
  • Operator-supplied endpoint selection and chat_adopt (added 2026-10-17): opencode.OpenEndpoint and the flag/env/recorded precedence, the URL-shape validation failures (scheme, empty host, userinfo, query, fragment, non-empty path), the unreachable/non-OpenCode/no-SSE/auth-mismatch failures, the no-owned-process property (conn.process == nil) and the password-file rule are unit-tested over httptest fixtures in internal/opencode/endpoint_test.go. chat_adopt (endpoint directory/title recorded with the operator tag, echo, missing id refused naming the id and recording nothing, nil payload/blank id/nil broker/nil registry refused before any endpoint call, empty endpoint directory refused) is unit-tested over the server fake broker in internal/server/chat_test.go; the recorded-endpoint round-trip across a store Close/Open is in internal/pipeline/settings_test.go; the protocol round-trip and the CLI dispatch (pipelines chat-adopt) are covered in internal/protocol/chat_test.go and internal/cli/chat_adopt_test.go. The live scratch-server check against the operator's real endpoint (adopt, list, prompt) was not run in this change; only the unit fixtures above were executed.

Deployment state: not deployed. This is reviewed and committed locally only; deploying it (and restarting the server) is the operator's step. Do not describe brokered chat as already deployed.

Decide a waiting human stage in the TUI: open the run, press c, pick an option, optionally type feedback, Enter.

Docker release resolution

Docker setup resolves the current supported OpenCode V2 release from the single documented complete GET .../git/matching-refs/tags/v2 response (a bounded complete array with no pagination parameters; never the moving latest tag or the registry's insertion order) and the current Pi release from npm, resolves the selected tag's immutable GHCR sha256 digest, builds FROM that digest, and verifies the built image reports exactly those executable versions plus a Node-compatible Pi engine. It fails closed, keeping the last verified Dockerfile fallback pins, when discovery, digest resolution, completeness, build or compatibility cannot be established: never a hardcoded fallback, never a silent stale-pin substitution. A full install skips a rebuild only when the release receipt and the actual built image's release and source identity labels already match; a dirty source always invokes the Docker build and relies on layer caching. A model/login-only reconfiguration compares only the verified release image identity, so a changed source alone does not rebuild. pipelines harness setup opencode|pi reconfigures inside the existing container and refuses an image that does not match; it does not rebuild or restart services (an explicit ordinary ./install does when the user asks).

Harness internals

CLI new/edit/debug and TUI author/debug build one shared, Docker-aware CLI prefix. A Docker-linked profile embeds the in-container /usr/local/bin/pipelines (never the host binary: Linux can see but cannot execute a mounted macOS host executable). Loopback is recognized only via net.ParseIP(...).IsLoopback() plus an exact case-insensitive localhost, so a remote DNS name such as 127.remote.example stays remote and runs no probe. For a loopback logical --url, a single bounded docker compose port probe parses each returned line as a complete host:port binding (IPv4/IPv6/wildcard) and the prefix maps only when a binding covers the requested loopback address and port; malformed or non-matching bindings fail closed. A loopback wss endpoint is rejected as having no supported local TLS mapping rather than downgraded to plaintext. On a match it embeds scoped PIPELINES_TRANSPORT_SOURCE/PIPELINES_TRANSPORT_TARGET aliases in the prefix string (not cmd.Env, so the pre-existing harness server's tools inherit them). The client rewrites an endpoint only on an exact source match; remote endpoints stay remote. Draft metadata (server, base version, create-only guard) is preserved unchanged. provider-check with a Docker-linked profile runs inside that profile's existing Compose service with -T, the original cwd/model/timeout and exact stdout/exit propagation; container profiles have no DockerDirectory and cannot recurse, and explicit native profiles keep the host path. The picker uses a true x/term TTY check, so /dev/null and pipes fail fast instead of hanging.

On an existing Docker setup, pipelines harness setup opencode|pi configures inside its container and link the host launcher without restarting/deploying services. They resolve and verify the current OpenCode V2 release from the single documented complete GET .../git/matching-refs/tags/v2 response (a bounded complete array with no pagination parameters; never the moving latest tag or the registry's insertion order) and the current Pi release from npm, resolve the selected tag's immutable GHCR sha256 digest, and require a rebuild only when the verified release image identity no longer matches. A full install always invokes the Docker build for a dirty source (relying on layer caching); a model/login-only reconfiguration does not rebuild. Setup writes its generated release receipt to the machine-local root file .pipelines-releases.json (with atomic .pipelines-releases-* temporary files); both are git-ignored, so a fresh clean checkout stays clean and reinstall still reuses the verified release/image identity. The actual built image's labels stay authoritative and a dirty source still rebuilds. If discovery, digest resolution or completeness cannot be established they stop with an error (keeping the last verified Dockerfile fallback pins); they never guess, fall back to a host runtime or install a stale pin. (./install --opencode/--pi were removed and only point to that command.) The native installation reuses the host OpenCode configuration, writes host (non-Docker) profiles for the harnesses it detects into XDG config home's pipelines/harnesses.json, and clears a stale Docker link so a Docker-era profile is not left Docker-linked. It never installs, upgrades or downloads a harness. Container and host credentials and accounts remain independent and are reused on reinstall/reconfigure. ./install installs natively by default (--native is equivalent); explicit ./install --docker selects the Docker server + ntfy installation. Fresh setup accepts ./install --docker --harness pi to skip OpenCode login. Existing setup can configure either harness independently. Pi setup does not run a generation check; provider-check --harness pi --model PROVIDER/MODEL is a separate confirmed, potentially chargeable check. Profiles live in XDG config home's pipelines/harnesses.json, store preferences only, and preserve unrelated configuration. A profile launches a native host harness unless it records a Docker checkout directory, in which case it launches through that installation's Compose service. harness setup <opencode|pi> --host configures or converts a host profile and clears any Docker link; without --host, reconfiguring a Docker-linked profile re-verifies that the recorded directory still has compose.yaml and fails naming it rather than saving a link that cannot launch. Setup's summary and harness list report the resulting mode (and, for Docker, the directory). Harness CLI setup/default commands always configure the local execution user, not a remote --url server. OpenCode interactive launch uses mini --model: verified V2 full CLI lacks a model flag. Pi uses its CLI with the packaged skill explicitly loaded. Pi has no standalone auth login command (verified pi auth --help: only credential check/print): setup prints a title-plus-three-lines Pi sign-in handoff and opens a fresh, no-context, extension-free, no-prompt-templates, no-tools session with Pi's documented --tui-mode regular flag, so the header and handoff stay visible above Pi's interface at 80x30 instead of the default fullscreen clear. The user types /login, chooses an authentication method ("Sign in with an API key" for provider API credentials; account and Radius remain the user's choice), finishes provider sign-in, then exits with ctrl+c/ctrl+d to return to Pipelines model configuration. Existing sign-ins are reused by exiting when no new sign-in is needed; completing a sign-in can update the credentials stored in the Docker volume. No coding/task prompt or scripted keystrokes are injected. OpenCode V2 model stages settle long sessions by polling GET /api/session/active about once per second under the stage context, instead of the .../wait request whose CLI-side request timeout failed long stages after roughly six minutes (HTTP 499, dated 2026-10-10). V1 native settlement is unchanged. Final results and debug session evidence use the same single bounded latest-assistant readback (order=desc&limit=1&type=assistant), so a small final result is visible behind large prior tool content; retained transcripts over 32 KiB are stored as valid bounded preview envelopes that preserve stage limits and diagnostics, and exact outputs remain authoritative. Pi evidence is JSONL and is not parsed as one JSON object. Short-lived context-controlled CLI calls (--help detection, API calls and the native export fallback) share one finite inherited-pipe drain grace (os/exec.Cmd.WaitDelay ~250ms) so a canceled context or a descendant that still holds stdout/stderr cannot block for the child's natural lifetime; the context.Canceled/DeadlineExceeded and exec.ErrWaitDelay identities are preserved and partial stdout after the grace is never accepted as success.