Pipelines skill: outcome-first output, aligned review and loop guidance, operator material moved to docs
Rewrite format.md around turn files, the output.md order (# outcome, edge word, # handoff, detail), checks, configs, human stages and the land stage delivery gate (sh -c in a fresh detached worktree, clean checkouts incl. untracked, 64 KB output cap, opencode only). Align review guidance across practices, verification, orchestration and the feedback-loop example. Fix loop-limit semantics in control.md against runner.go, budget_tokens sizing, read for large inputs, blocked-else-error and one stage naming set. Move dated verification logs to docs/verification-history.md and brokered chat, TUI keys and harness internals to docs/operator-reference.md; label operator-only references in SKILL.md.
This commit is contained in:
@@ -0,0 +1,291 @@
|
||||
# Operator reference
|
||||
|
||||
Operator and developer material moved out of the packaged Pipelines skill
|
||||
(`internal/author/skill/references/operations.md`) because authoring agents
|
||||
never need it.
|
||||
|
||||
## TUI keys
|
||||
|
||||
In the TUI, Home advertises `/` omni search and `:` commands. Home `/` opens one
|
||||
popup over pipelines, versions, runs, steps/visits, artifact names and stage
|
||||
input/output/evidence text; each row shows a short type badge and a text match
|
||||
shows a bracketed snippet, and a bare status word or date selects runs. The same
|
||||
query is available as `pipelines search QUERY [--type KIND]`, where `--type`
|
||||
narrows the kind. Run Tab switches only
|
||||
Details ↔ Graph. `n/p` select recorded visits, including repeated stages; `d`
|
||||
focuses/toggles file changes, `A` opens the run's grouped artifacts view (name,
|
||||
content type, size and URL per entry, `PgUp`/`PgDn` scroll so a trailing visit
|
||||
that recorded none stays reachable, `Enter`/`o` to open the selected URL and
|
||||
`y` to copy it), `a` selects the active/latest retained
|
||||
configured-harness agent session's bounded, redacted tool evidence, `D`/`:debug`
|
||||
opens fresh investigation, and `P` opens the pinned pipeline version. `:quit`
|
||||
exits; `q` does not. Results are retained reports, not independently confirmed
|
||||
achievements. Distinguish partial output from full success.
|
||||
|
||||
## Brokered OpenCode chat
|
||||
|
||||
The protocol declares the brokered `chat_*` surface (`chat_sessions`,
|
||||
`chat_create`, `chat_directories`, `chat_prompt`, `chat_messages`,
|
||||
`chat_interrupt`, `chat_permission`, `chat_reply`). The server dispatches all of
|
||||
them, and pushes `chat_event` frames derived from OpenCode's global SSE firehose
|
||||
(`/event` on the native serve, `/api/event` on the V2 API), with retained
|
||||
per-session/per-message state so a reconnecting client re-syncs instead of
|
||||
replaying deltas.
|
||||
|
||||
At startup `cmd/pipelines-server` calls `opencode.OpenChat`, which detects the
|
||||
installed backend and opens exactly one owned `opencode serve` endpoint for it:
|
||||
the native wire for v1, the V2 API for the installed V2 CLI. Chat is enabled only
|
||||
when that connection carries a URL-bearing, SSE-capable event endpoint; otherwise
|
||||
chat stays disabled and every `chat_*` request is answered with a protocol error
|
||||
naming the missing configuration rather than hanging. Stage generation still
|
||||
reaches V2 through the `opencode api` CLI, so the owned serve process is chat's
|
||||
single transport and starts no process for stages. The adapter skips firehose
|
||||
events it does not surface before decoding their data, so an unrelated event
|
||||
whose `delta` has a different shape (`session.instructions.updated` carries an
|
||||
object) never fails the stream.
|
||||
|
||||
The operator can instead broker chat from their own OpenCode service with the
|
||||
opt-in `-opencode-endpoint` flag or `PIPELINES_OPENCODE_ENDPOINT`, with precedence
|
||||
flag > environment > the endpoint recorded on a previous run and the spawned
|
||||
instance as the default. `opencode.OpenEndpoint` validates the reference once
|
||||
(the established http(s)/host/no-credentials/no-query/no-path idiom), detects the
|
||||
backend from the service's own `/api/info` (V2) or `/global/health` (native)
|
||||
contract and requires a streamable `text/event-stream` event path; it returns a
|
||||
connection with **no owned process** and never calls `startServe`. A malformed
|
||||
reference, an unreachable host, a reachable non-OpenCode service or a
|
||||
health-ok-but-no-SSE service each disable chat with an error naming the endpoint
|
||||
and never fall back to the spawned instance. An authenticating endpoint accepts an
|
||||
optional password only through the private file
|
||||
`PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE` (regular, 0600, single-owner), and
|
||||
`PIPELINES_OPENCODE_ENDPOINT_USERNAME` overrides the detected default user. The
|
||||
validated choice is recorded in the additive `server_settings` table so a restart
|
||||
with no flag/environment reuses it; an unreachable recorded endpoint disables chat
|
||||
with a clear message and no fallback. `pipelines service enable` preserves the
|
||||
endpoint and the password-file reference in autostart.
|
||||
|
||||
`chat_sessions` lists the operator's own chat sessions, not the sessions pipeline
|
||||
runs create. Operator sessions are recorded in the dedicated `operator_sessions`
|
||||
table (`session_id` primary key, plus `directory`, `title`, `tag`, `created_at`,
|
||||
`last_used_at`), separate from `run_steps`/`run_session_owners`, so an operator
|
||||
chat is a first-class thing and a pipeline stage session — recorded as run
|
||||
ownership, never in the operator registry — is never listed. The handler reads the
|
||||
registry once (bounded to 128 rows, most-recently-used first), calls OpenCode's
|
||||
session list once, and keeps only registry rows the endpoint still reports, so a
|
||||
session with no recorded tag is never listed and a deleted upstream session does
|
||||
not linger. It fills each session's `directory`, `tag` and `created_at`/
|
||||
`updated_at` from the registry and marks `busy` when another client currently
|
||||
holds the session (an in-server claim, released when the stream ends or the client
|
||||
disconnects); a second client's `chat_prompt` for a held session is rejected
|
||||
before any OpenCode call. It is fail-closed: with no registry configured it
|
||||
returns an empty list, never the endpoint's whole session list. The OpenCode
|
||||
session list is read unscoped by directory (the V2 `GET /api/session` otherwise
|
||||
defaults to the caller's directory), because an operator session may live in any
|
||||
directory.
|
||||
|
||||
`chat_create` creates one operator session and records it in the same registry
|
||||
with the stored tag `operator` (the display label `Operator` is derived from it,
|
||||
so there is no second tag vocabulary). It takes a `title` (default "New chat") and
|
||||
a `directory` (default `~`): an omitted or empty value, `~`, and any `~/...` value
|
||||
all resolve to `os.UserHomeDir`, never the server's working directory; a relative
|
||||
path or a missing directory is rejected before any OpenCode call, and a session
|
||||
created but not recordable is reported as an error naming its id. Create posts the
|
||||
verified V2 shape `POST /api/session` `{"title":...,"location":{"directory":...}}`.
|
||||
`chat_directories` returns the operator's home first, then the directories of
|
||||
previous operator sessions most-recently-used first, deduplicated and bounded (8);
|
||||
it is a dedicated request so the create dialog can load the choices without first
|
||||
listing sessions and `chat_sessions` stays a pure, fail-closed session list. A
|
||||
successful `chat_prompt` touches the session's `last_used_at` in one UPDATE, so it
|
||||
moves to the front of the list; a non-operator session id touches no row.
|
||||
|
||||
`chat_adopt` records a session that already exists on the configured endpoint in
|
||||
the same operator registry, using the endpoint's own directory and title
|
||||
verbatim (never a guess, the server's working directory or the home directory).
|
||||
It reads the endpoint once through the shared `Chat.Session` (which reuses
|
||||
`Sessions`/`decodeChatSessions`), records through the existing
|
||||
`RecordOperatorSession` with the `operator` tag, and echoes the adopted session
|
||||
plus its registry row like `chat_create`, so `chat_sessions` lists it and
|
||||
`chat_prompt` continues it. A nil payload, a blank or foreign-shaped id, a nil
|
||||
broker or a nil registry is refused before any endpoint call; an id the endpoint
|
||||
does not report is refused naming the id and records nothing; an endpoint record
|
||||
with an empty directory is refused rather than substituting a directory. The CLI
|
||||
equivalent is `pipelines chat-adopt <session-id>` (`--json` prints the full
|
||||
response, otherwise a short `Adopted <id> · <directory>` line).
|
||||
|
||||
While a `chat_prompt` subscription is open the server also polls
|
||||
`GET /api/permission/request` on a fixed small interval (first poll immediately) and
|
||||
pushes one `chat_permission` frame per newly pending approval request, deduped by
|
||||
request id so a repeated poll never re-pushes the same request. The app answers with
|
||||
`chat_reply`, which the server forwards as
|
||||
`POST /api/session/{id}/permission/{requestID}/reply` with
|
||||
`{"decision":"once"|"always"|"reject"}`. A missing session/request id or an invalid
|
||||
decision is rejected before any OpenCode call, and the poller stops when the stream
|
||||
ends or the client disconnects.
|
||||
|
||||
The permission list and reply are implemented against the V2 HTTP API only: their
|
||||
field names match the service's `Permission.Request`/`Permission.Reply` schemas
|
||||
(verified against the installed service's `/openapi.json`). The native v1 serve
|
||||
translation rejects them explicitly instead of guessing an endpoint, so on v1 the
|
||||
permission poller stops quietly rather than reporting the documented limitation as
|
||||
a per-prompt error.
|
||||
|
||||
### Verification evidence (2026-10-17)
|
||||
|
||||
Verified against the installed OpenCode 2.0.20 on 2026-10-17:
|
||||
|
||||
- `opencode --version` reports `2.0.20`; `opencode --help` advertises `api` and
|
||||
`serve`. `GET /openapi.json` on an owned `opencode serve` lists `GET /api/event`
|
||||
(SSE) and the session/prompt/interrupt/permission endpoints.
|
||||
- A live turn streamed token-level `session.text.delta` and
|
||||
`session.reasoning.delta` events (each with `assistantMessageID` and `ordinal`),
|
||||
plus `session.step.*` and `session.usage.updated`; the V2 API server
|
||||
authenticates the fixed `opencode` basic-auth user.
|
||||
- The opt-in live end-to-end test (`PIPELINES_TEST_OPENCODE_CHAT_LIVE=1 go test
|
||||
./internal/server -run TestChatLiveEndToEnd -v`) opened an owned V2 endpoint,
|
||||
created an operator session through `chat_create` in a chosen directory, listed
|
||||
exactly that session through `chat_sessions` (with its directory and tag
|
||||
`operator`), recorded a genuine pipeline stage session in
|
||||
`run_steps`/`run_session_owners` and showed it never appeared in the list, and
|
||||
streamed a completed turn as `[step:started,text,step:streamed,step:ended]`
|
||||
(first delta `"pong"`). The operator-session registry, the filter, directory
|
||||
defaulting and the recents bound are also covered by the `pipeline`, `opencode`
|
||||
and `server` unit tests, which run without the live gate.
|
||||
- Not verified live: a genuine pending-approval round trip and any backend other
|
||||
than the installed 2.0.20. The permission wire shapes are unit-tested over an
|
||||
httptest fixture, not against a live pending request.
|
||||
- Operator-supplied endpoint selection and `chat_adopt` (added 2026-10-17):
|
||||
`opencode.OpenEndpoint` and the flag/env/recorded precedence, the URL-shape
|
||||
validation failures (scheme, empty host, userinfo, query, fragment, non-empty
|
||||
path), the unreachable/non-OpenCode/no-SSE/auth-mismatch failures, the
|
||||
no-owned-process property (`conn.process == nil`) and the password-file rule
|
||||
are unit-tested over httptest fixtures in `internal/opencode/endpoint_test.go`.
|
||||
`chat_adopt` (endpoint directory/title recorded with the `operator` tag, echo,
|
||||
missing id refused naming the id and recording nothing, nil payload/blank id/nil
|
||||
broker/nil registry refused before any endpoint call, empty endpoint directory
|
||||
refused) is unit-tested over the server fake broker in
|
||||
`internal/server/chat_test.go`; the recorded-endpoint round-trip across a store
|
||||
Close/Open is in `internal/pipeline/settings_test.go`; the protocol round-trip
|
||||
and the CLI dispatch (`pipelines chat-adopt`) are covered in
|
||||
`internal/protocol/chat_test.go` and `internal/cli/chat_adopt_test.go`. The
|
||||
live scratch-server check against the operator's real endpoint (adopt, list,
|
||||
prompt) was **not** run in this change; only the unit fixtures above were
|
||||
executed.
|
||||
|
||||
Deployment state: not deployed. This is reviewed and committed locally only;
|
||||
deploying it (and restarting the server) is the operator's step. Do not describe
|
||||
brokered chat as already deployed.
|
||||
|
||||
Decide a waiting human stage in the TUI: open the run, press `c`, pick an option,
|
||||
optionally type feedback, Enter.
|
||||
|
||||
## Docker release resolution
|
||||
|
||||
Docker setup resolves the current supported OpenCode V2 release from the single
|
||||
documented complete `GET .../git/matching-refs/tags/v2` response (a bounded
|
||||
complete array with no pagination parameters; never the moving `latest` tag or
|
||||
the registry's insertion order) and the current Pi release from npm, resolves the
|
||||
selected tag's immutable GHCR `sha256` digest, builds `FROM` that digest, and
|
||||
verifies the built image reports exactly those executable versions plus a
|
||||
Node-compatible Pi engine.
|
||||
It fails closed, keeping the last verified Dockerfile fallback pins, when discovery,
|
||||
digest resolution, completeness, build or compatibility cannot be established: never
|
||||
a hardcoded fallback, never a silent stale-pin substitution. A full install skips a
|
||||
rebuild only when the release receipt **and** the actual built image's release and
|
||||
source identity labels already match; a dirty source always invokes the Docker
|
||||
build and relies on layer caching. A model/login-only reconfiguration compares only
|
||||
the verified release image identity, so a changed source alone does not rebuild.
|
||||
`pipelines harness setup opencode|pi` reconfigures inside the existing container and
|
||||
refuses an image that does not match; it does not rebuild or restart services (an
|
||||
explicit ordinary `./install` does when the user asks).
|
||||
|
||||
## Harness internals
|
||||
|
||||
CLI `new/edit/debug` and TUI author/debug build one shared, Docker-aware CLI
|
||||
prefix. A Docker-linked profile embeds the in-container `/usr/local/bin/pipelines`
|
||||
(never the host binary: Linux can see but cannot execute a mounted macOS host
|
||||
executable). Loopback is recognized only via `net.ParseIP(...).IsLoopback()` plus
|
||||
an exact case-insensitive `localhost`, so a remote DNS name such as
|
||||
`127.remote.example` stays remote and runs no probe. For a loopback logical
|
||||
`--url`, a single bounded `docker compose port` probe parses each returned line
|
||||
as a complete `host:port` binding (IPv4/IPv6/wildcard) and the prefix maps only
|
||||
when a binding covers the requested loopback address and port; malformed or
|
||||
non-matching bindings fail closed. A loopback `wss` endpoint is rejected as
|
||||
having no supported local TLS mapping rather than downgraded to plaintext. On a
|
||||
match it embeds scoped `PIPELINES_TRANSPORT_SOURCE`/`PIPELINES_TRANSPORT_TARGET`
|
||||
aliases in the prefix string (not `cmd.Env`, so the pre-existing harness
|
||||
server's tools inherit them). The client rewrites an endpoint only on an exact
|
||||
source match; remote endpoints stay remote. Draft metadata (server, base
|
||||
version, create-only guard) is preserved unchanged.
|
||||
`provider-check` with a Docker-linked profile runs inside that profile's existing
|
||||
Compose service with `-T`, the original cwd/model/timeout and exact stdout/exit
|
||||
propagation; container profiles have no `DockerDirectory` and cannot recurse, and
|
||||
explicit native profiles keep the host path. The picker uses a true `x/term` TTY
|
||||
check, so `/dev/null` and pipes fail fast instead of hanging.
|
||||
|
||||
On an existing Docker setup, `pipelines harness setup opencode|pi` configures inside its container and link the host
|
||||
launcher without restarting/deploying services. They resolve and verify the current
|
||||
OpenCode V2 release from the single documented complete
|
||||
`GET .../git/matching-refs/tags/v2` response (a bounded complete array with no
|
||||
pagination parameters; never the moving `latest` tag or the registry's insertion
|
||||
order) and the current Pi release from npm, resolve the selected tag's immutable
|
||||
GHCR `sha256` digest, and require a rebuild only when the verified release image
|
||||
identity no longer matches. A full install always invokes the Docker build for a
|
||||
dirty source (relying on layer caching); a model/login-only reconfiguration does
|
||||
not rebuild. Setup writes its generated release receipt to the machine-local root
|
||||
file `.pipelines-releases.json` (with atomic `.pipelines-releases-*` temporary
|
||||
files); both are git-ignored, so a fresh clean checkout stays clean and reinstall
|
||||
still reuses the verified release/image identity. The actual built image's labels
|
||||
stay authoritative and a dirty source still rebuilds. If discovery, digest
|
||||
resolution or completeness cannot be established they stop with an error (keeping
|
||||
the last verified Dockerfile fallback pins); they never guess, fall back to a host
|
||||
runtime or install a stale pin.
|
||||
(`./install --opencode`/`--pi` were removed and only point to that command.) The native installation reuses the host OpenCode
|
||||
configuration, writes **host** (non-Docker) profiles for the harnesses it detects
|
||||
into XDG config home's `pipelines/harnesses.json`, and clears a stale Docker link
|
||||
so a Docker-era profile is not left Docker-linked. It never installs, upgrades or
|
||||
downloads a harness. Container and host credentials and accounts remain independent and
|
||||
are reused on reinstall/reconfigure. `./install` installs natively by default
|
||||
(`--native` is equivalent); explicit `./install --docker` selects the Docker
|
||||
server + ntfy installation.
|
||||
Fresh setup accepts `./install --docker --harness pi` to skip OpenCode
|
||||
login. Existing setup can configure either harness independently. Pi setup does
|
||||
not run a generation check; `provider-check --harness pi --model PROVIDER/MODEL`
|
||||
is a separate confirmed, potentially chargeable check.
|
||||
Profiles live in XDG config home's `pipelines/harnesses.json`, store preferences
|
||||
only, and preserve unrelated configuration. A profile launches a native host
|
||||
harness unless it records a Docker checkout directory, in which case it launches
|
||||
through that installation's Compose service. `harness setup <opencode|pi> --host`
|
||||
configures or converts a host profile and clears any Docker link; without
|
||||
`--host`, reconfiguring a Docker-linked profile re-verifies that the recorded
|
||||
directory still has `compose.yaml` and fails naming it rather than saving a link
|
||||
that cannot launch. Setup's summary and `harness list` report the resulting mode
|
||||
(and, for Docker, the directory). Harness CLI setup/default commands
|
||||
always configure the local execution user, not a remote `--url` server.
|
||||
OpenCode interactive launch uses
|
||||
`mini --model`: verified V2 full CLI lacks a model flag. Pi uses its CLI with the
|
||||
packaged skill explicitly loaded. Pi has no standalone `auth login` command
|
||||
(verified `pi auth --help`: only credential check/print): setup prints a
|
||||
title-plus-three-lines Pi sign-in handoff and opens a fresh, no-context,
|
||||
extension-free, no-prompt-templates, no-tools session with Pi's documented
|
||||
`--tui-mode regular` flag, so the header and handoff stay visible above Pi's
|
||||
interface at 80x30 instead of the default fullscreen clear. The user types
|
||||
`/login`, chooses an authentication method ("Sign in with an API key" for
|
||||
provider API credentials; account and Radius remain the user's choice), finishes
|
||||
provider sign-in, then exits with `ctrl+c`/`ctrl+d` to return to Pipelines model
|
||||
configuration. Existing sign-ins are reused by exiting when no new sign-in is
|
||||
needed; completing a sign-in can update the credentials stored in the Docker
|
||||
volume. No coding/task prompt or scripted keystrokes are injected.
|
||||
OpenCode V2 model stages settle long sessions by polling `GET /api/session/active`
|
||||
about once per second under the stage context, instead of the `.../wait` request
|
||||
whose CLI-side request timeout failed long stages after roughly six minutes
|
||||
(HTTP 499, dated 2026-10-10). V1 native settlement is unchanged. Final results
|
||||
and debug session evidence use the same single bounded latest-assistant readback
|
||||
(`order=desc&limit=1&type=assistant`), so a small final result is visible behind
|
||||
large prior tool content; retained transcripts over 32 KiB are stored as valid
|
||||
bounded preview envelopes that preserve stage limits and diagnostics, and exact
|
||||
outputs remain authoritative. Pi evidence is JSONL and is not parsed as one JSON
|
||||
object. Short-lived context-controlled CLI calls (`--help` detection, API calls
|
||||
and the native export fallback) share one finite inherited-pipe drain grace
|
||||
(`os/exec.Cmd.WaitDelay` ~250ms) so a canceled context or a descendant that still
|
||||
holds stdout/stderr cannot block for the child's natural lifetime; the
|
||||
`context.Canceled`/`DeadlineExceeded` and `exec.ErrWaitDelay` identities are
|
||||
preserved and partial stdout after the grace is never accepted as success.
|
||||
@@ -0,0 +1,612 @@
|
||||
# Verification history
|
||||
|
||||
Dated verification logs and backend caveats moved out of the packaged
|
||||
Pipelines skill (`internal/author/skill/references/verification.md` and
|
||||
`harnesses.md`) so authoring agents do not load them. They are a historical
|
||||
record: the JSON/line-format result envelope and the format-repair ladder they
|
||||
describe are retired. Model stages now write `output.md` and only its
|
||||
`# outcome` section is parsed (`internal/author/skill/references/format.md`,
|
||||
`docs/model-results.md`).
|
||||
|
||||
## From the skill verification reference
|
||||
|
||||
### Budgets and backend caveats
|
||||
|
||||
Defaults remain two retries and one hour per stage attempt; a run-level wall-clock
|
||||
deadline is not added. Budgets are optional, positive and independent USD/token
|
||||
reported-usage limits shared across attempts and repeat visits.
|
||||
Accounting failures stop budget-limited stages. In-flight calls can overshoot;
|
||||
these are **not provider-side hard caps or a guaranteed bill ceiling**. Reported
|
||||
USD may validly be zero; do not fabricate prices/accounting from output length.
|
||||
|
||||
Per-run usage is recorded as typed step evidence from the usage the stage already
|
||||
measured for its budget (no second harness read and no transcript parse), and
|
||||
totalled for the run and over all history. A stage revisited within a run shares
|
||||
one budget, so each step records only that visit's increment of it and the run
|
||||
total is not the cumulative stage budget counted again; a revisit that reports
|
||||
nothing new (a failure or timeout before any usage poll, or a Pi visit that never
|
||||
reaches a usage stat) records no usage for that visit rather than a known zero.
|
||||
`CLI_PREFIX inspect RUN_ID --json`
|
||||
exposes each model step's `tokens_total`, `tokens_in`/`tokens_out` (only when
|
||||
`split_known`), `cost_usd` (only when `cost_known`) and the model it ran with,
|
||||
plus the totalled `usage`; `CLI_PREFIX stats --json` adds the history-wide totals,
|
||||
which equal the sum of the per-run totals. A stage that reported no cost exposes
|
||||
its tokens and marks the cost unavailable rather than showing zero; a harness that
|
||||
reports only a total (Pi) exposes the total and marks the in/out split
|
||||
unavailable rather than presenting the total as input tokens. An upgraded or
|
||||
historical database gains an empty usage table with no backfill, so old runs omit
|
||||
`usage` instead of showing zero. This is current source, not deployment evidence.
|
||||
|
||||
Dated current-source evidence (2026-10-17, this machine). A throwaway read-only
|
||||
reader (`internal` store API) over a scratch copy of
|
||||
`~/.local/share/pipelines/data/pipelines.db`, taken with
|
||||
`sqlite3 "file:$HOME/.local/share/pipelines/data/pipelines.db?mode=ro" ".backup <scratch>"`
|
||||
so the live DB is never mutated, opened 228 runs and found **no** `step_usage`
|
||||
rows: every stored run predates this change, so per-run figures are unavailable
|
||||
rather than zero. Run 314 (this item's own run) and run 239 (whose implementer
|
||||
step ran `opencode-go/kimi-k2.7-code`, a stage model different from the pinned
|
||||
`opencode-go/deepseek-v4.1-flash`) both report `usage=(none recorded)`, and
|
||||
`stats` omits the history-wide totals. The recorded figures themselves are
|
||||
covered by the passing current-source round trips — `internal/pipeline/usage_test.go`,
|
||||
`internal/server/usage_test.go` (including `inspect --json` over a websocket) and
|
||||
the pre-upgrade/read-bounded/stats-sum cases — not by a live run, because the
|
||||
deployed server predates this change and was not restarted. No live model call,
|
||||
deploy or rebuild produced this evidence.
|
||||
|
||||
A live V2 trial found `/generate` returned text while its retained session had no
|
||||
messages and zero counters. Current source uses tracked `/prompt` → bounded
|
||||
active-session polling → message readback for **budgeted V2 text-only calls, with
|
||||
all tools denied**. Completed token-budgeted responses reporting zero tokens fail
|
||||
closed. No new definition flags or agent tools are required for this fix.
|
||||
Unbudgeted tool-free V2 generation can still have empty backend message history;
|
||||
pipeline inputs, results and command audit remain retained. An unbudgeted V2
|
||||
audited agent stage makes exactly one extra bounded final session-aggregate read
|
||||
to record any reported USD/token usage best-effort: missing or invalid usage stays
|
||||
unknown and never turns a valid run into a failure, cumulative sessions/retries
|
||||
are keyed per session so they cannot double count, and totals are never inferred
|
||||
from final-assistant-only or native-partial data. Budgeted settlement and native
|
||||
V1 enforcement are unchanged. Native V1 uses its existing path.
|
||||
|
||||
The V2 `.../experimental/session/{id}/wait` request is **no longer used**: a dated
|
||||
Docker log shows it timing out at the CLI's own request timeout after about six
|
||||
minutes (run started 2026-10-10T23:07:16.547Z, `TimeoutError` logged at
|
||||
2026-10-10T23:13:17.262Z, server `http.status=499`), which aborted a long
|
||||
implementation stage. Instead current source polls `GET /api/session/active`
|
||||
about once per second with a context-aware delay and no busy loop, until the
|
||||
admitted session is absent (inactive). Missing/null/non-object `data`, or a state
|
||||
other than the documented `running` type, fails closed and is never treated as
|
||||
inactive. This is **mocked regression coverage** in `internal/stage` (fake CLI),
|
||||
not live-provider evidence; V1 native settlement is unchanged.
|
||||
|
||||
Current source reads the V2 final result with a single bounded descending
|
||||
final-assistant readback (`order=desc&limit=1&type=assistant`) after the settled
|
||||
poll, for both audited agent stages and budgeted V2 text-only calls, so large
|
||||
retained history (including prior tool-read contents) is no longer fetched just
|
||||
to select the completed result. A truncated/incomplete readback, an empty data
|
||||
list, or a latest message that is neither completed `stop` nor a completed
|
||||
structured tool-call fails closed with a sanitized diagnostic that omits
|
||||
provider content; retained audit previews are **not** executable final results.
|
||||
Session/debug evidence now reuses the same single latest-assistant readback
|
||||
instead of a recent-20 all-message fetch, and returns each payload as a labelled
|
||||
bounded, credential-redacted envelope with explicit preview/truncation/missing
|
||||
diagnostics; a small final assistant result is therefore selected even when large
|
||||
prior tool content exists. Pi evidence remains documented JSONL and is never
|
||||
parsed as one JSON object. This describes current source behavior, not a guarantee
|
||||
that a user's installed server includes the fix. An updated skill/client is not
|
||||
deployment evidence; never rebuild/restart services or log into providers without
|
||||
specific authorization.
|
||||
|
||||
Retained model-stage failures now carry a fixed sanitized category. A strictly
|
||||
invalid final result (a duplicate nested field, or malformed/ambiguous JSON with a
|
||||
byte offset) is labeled `FINAL RESULT VALIDATION`; request, V2 active-session poll
|
||||
and final-readback failures are labeled `HARNESS TRANSPORT`. A pre-admission
|
||||
request failure replays nothing and is retried for any stage; a post-admission
|
||||
failure is retried only on an explicit read-only stage (see the 2026-10-17 section
|
||||
below). A validation failure
|
||||
preserves the admitted session, transcript and evidence and performs no second tool
|
||||
execution. Confirmed ordinary-endpoint failures on 2026-10-11: an all-history
|
||||
readback of exactly 262144 bytes of incomplete JSON (`invalid agent messages`), a
|
||||
reviewer result with a duplicate nested `performance_review` field, and an
|
||||
onboarding visual-review result with `invalid character 'z' after object key:value
|
||||
pair at byte 3968` where unsafe lossless repair was rejected. These are **mocked
|
||||
`internal/stage` fixtures**: executable source-level evidence, not live-provider
|
||||
evidence and not a deployment. OpenCode V2 prose `output_contract` does not enforce
|
||||
structured output; V2 results are validated locally and no unsupported V2 format
|
||||
field is sent. A review-only fresh execution is a future recovery idea, **not
|
||||
implemented**. Onboarding UI visual approval remains unresolved and must not be
|
||||
reported complete.
|
||||
|
||||
### Bounded malformed-result recovery: 2026-10-16
|
||||
|
||||
Source-level, mocked `internal/stage` regression coverage (not live-provider
|
||||
evidence and not a deployment): malformed final-result recovery now tries a
|
||||
deterministic local pass first (append missing closers, drop one trailing comma,
|
||||
or insert the single comma/colon the parser expected), then at most one deny-all
|
||||
formatter request whose deadline is derived from the payload size, the stage
|
||||
timeout and the attempt's remaining budget rather than a fixed 30-second
|
||||
constant. Every path
|
||||
records `repaired`, `partial` or `unrecoverable` in `format_repairs`; a `partial`
|
||||
or `unrecoverable` payload is never returned as a completed result, and an
|
||||
ambiguous payload (an unescaped quote/colon inside a string value) is never
|
||||
guessed at. A shell-enabled stage's malformed but deterministically recoverable
|
||||
result now completes without replaying admitted tool work; `safeToReplay` is
|
||||
unchanged. The run-55 `ui_reviewer` payload was retrieved read-only with
|
||||
`pipelines inspect 55 --json` and reproduced its exact byte offset 7837; it is the
|
||||
ambiguous error class, so it is reported unrecoverable rather than repaired. This
|
||||
does not update or restart any running server.
|
||||
|
||||
### Harness transport reach retry and repair deadline: 2026-10-17
|
||||
|
||||
Source-level, mocked `internal/stage`, `internal/opencode` and `internal/server`
|
||||
regression coverage (not live-provider evidence and not a deployment): the
|
||||
harness transport path now distinguishes a pre-admission reach failure
|
||||
(connecting or creating the session) from a post-admission readback/poll/agent
|
||||
failure under the one `HARNESS TRANSPORT` category. A pre-admission reach failure
|
||||
replays nothing, so it is retried within `retries` for any stage with a fresh
|
||||
session and no replayed prompt; a post-admission failure is retried only on an
|
||||
explicit read-only stage, and an `edit`/`shell`/legacy stage still never replays
|
||||
admitted work. A context that expires during a request keeps both the sanitized
|
||||
transport detail (operation/status/exit) and
|
||||
`errors.Is(err, context.DeadlineExceeded)`; a bare attempt-boundary deadline is
|
||||
not treated as a reach failure and stays terminal for a mutating stage. An
|
||||
unrecognized provider execution-error name is reported as a bounded, sanitized
|
||||
identifier plus any HTTP status. The isolated formatter deadline is now
|
||||
proportional to the stage timeout (floored, capped, and clamped to
|
||||
`remaining - margin` so it can never extend the stage/attempt deadline). These are
|
||||
mocked fixtures; a deployed server is not updated by this change.
|
||||
|
||||
### Failure-handling source changes: 2026-10-15
|
||||
|
||||
Source-level, mocked `internal/stage` and `internal/server` regression coverage
|
||||
(not live-provider evidence): an explicit read-only tools stage now actually
|
||||
retries a post-admission stage deadline or harness transport failure, creating a
|
||||
fresh bounded session per attempt, while a stage with `edit`/`shell` or no tools
|
||||
block still never replays admitted work; a malformed final result is re-emitted
|
||||
up to `retries` only on such a read-only stage with its `format_repairs` evidence
|
||||
retained. A timed-out agent stage settles knowable V2 usage and retains one
|
||||
bounded, redacted latest-assistant readback plus a `retained_session` pointer in
|
||||
its stored transcript. A declared outcome routed to `#failure` with no step
|
||||
diagnostic now yields a run diagnostic like `stage <key> returned outcome
|
||||
"blocked"`; an existing step diagnostic is still used verbatim.
|
||||
|
||||
### Dated source-level coverage: 2026-10-11
|
||||
|
||||
Source-level/mocked regression coverage (not live-provider evidence):
|
||||
`resolve-releases` now uses a single bounded
|
||||
`GET .../git/matching-refs/tags/v2` response that the GitHub REST docs describe as
|
||||
complete with no pagination; offline fixtures cover a 100+ ref array,
|
||||
`MAX_REFS`/`MAX_BODY` caps, ignored prerelease/build tags, malformed refs and a
|
||||
missing newest-tag digest failing closed with no older fallback, with no
|
||||
`tags/list` cross-check. Picker tests cover blink/index preservation,
|
||||
ANSI-readable selection without a colliding background, and
|
||||
40x16/80x24/120x36 height budgets. The compiled picker PTY check (opt-in via
|
||||
`PIPELINES_HARNESS_PTY_DIR`) waits through blink intervals and asserts exact
|
||||
saved/arrow-scrolled models. `setup-docker.py` accepts the actual
|
||||
`opencode v2.0.26` probe token and compares source identity only on a full
|
||||
install; model/login-only reconfiguration compares the verified release image
|
||||
identity. Python fixtures bind temporary HOME directories so the suite passes
|
||||
under the container default `/home/pipelines` without weakening mount guards.
|
||||
The Docker authoring transport is also covered at source level: the shared
|
||||
prefix picks `/usr/local/bin/pipelines` for a Docker-linked profile. Loopback is
|
||||
recognized only via `net.ParseIP(...).IsLoopback()` and an exact case-insensitive
|
||||
`localhost`, so a remote DNS name such as `127.remote.example` stays remote with
|
||||
no probe. A single bounded `docker compose port` probe parses complete
|
||||
`host:port` bindings, and scoped source/target aliases are embedded only when a
|
||||
binding covers the requested loopback address and port; malformed or non-matching
|
||||
bindings and a loopback `wss` endpoint fail closed, and `client.Dial` rewrites
|
||||
only an exact source match. `provider-check` dispatches through a Docker-linked
|
||||
profile's existing service with `-T`, the original cwd/model/timeout, exact
|
||||
stdout/exit propagation and `--json` passed only in JSON mode (fixture tests, no
|
||||
real provider), while explicit native profiles keep the host generator. The
|
||||
picker uses a true `x/term` TTY check so `/dev/null` and pipes fail fast, with a
|
||||
short context deadline so a regression fails rather than hangs. Real Docker
|
||||
rebuild/reconfigure against live registries, a real
|
||||
40x16/80x24/120x36 terminal capture, and real model inference remain **not tested**
|
||||
in this change.
|
||||
|
||||
#### 2026-10-11 Pi sign-in handoff (source + Root PTY observation)
|
||||
|
||||
Root separately exercised current-source `pipelines harness setup pi --login`
|
||||
under a real Docker PTY with copied harness preferences. Root observed that
|
||||
Pi 1.1.0 defaults to fullscreen and clears the Pipelines login instructions, so
|
||||
the user sees a coding interface without the `/login` guidance, and that
|
||||
`/login` first shows "Select authentication method" (account / API key / Radius)
|
||||
rather than a provider picker. The same handler run with a temporary executable
|
||||
wrapper passing `pi --tui-mode regular` retained the Pipelines header and
|
||||
instructions above Pi's interface. Installed `pi --help` documents the supported
|
||||
`--tui-mode regular` flag and the fullscreen default; `pi auth --help` confirms
|
||||
there is no standalone login (only print/check, never run). These are source/root
|
||||
observations with copied preferences: **no provider was selected, no key was
|
||||
entered and no inference occurred**, and this source is **not deployed** and is
|
||||
not live-provider evidence. A first reviewer capture at 80x30 found the longer
|
||||
handoff plus Pi's own banner scrolled the Pipelines header off the top, so the
|
||||
copy was shortened to a title plus three short lines (existing-credential reuse
|
||||
folded into the `/login` line, with the account/API key/Radius method choice and
|
||||
`ctrl+c`/`ctrl+d` exit keys kept) and the redundant trailing blank removed.
|
||||
Independent PTY re-capture at 80x30/100x36 after this shortening is **not
|
||||
tested** in this change. The matching exact login-argv and copy assertions are
|
||||
covered by fixture tests in `internal/cli`/`internal/harness`.
|
||||
|
||||
#### 2026-10-11 server CLI fixture isolation (source-level + Root-reported run)
|
||||
|
||||
Root-reported runtime evidence supplied for this change (not re-executed by the
|
||||
authoring stages): the deterministic seeded-profile regression
|
||||
`go test ./tmp/preview-verification -run TestServerCLIFixtureDoesNotInheritUserDockerProfile -count=1`
|
||||
seeds a synthetic Docker-linked harness profile in a child
|
||||
`go test ./internal/server -run '^TestCLIHeadlessRunInspectionCancellationAndRPC$'`.
|
||||
It previously failed because the child inherited the caller's real
|
||||
`PIPELINES_HARNESS_CONFIG`, so the CLI `debug` prefix failed closed
|
||||
("refusing to redirect") against the test's ephemeral httptest endpoint. Root's
|
||||
four independent preview tests for the bounded transcript helpers pass; Root's
|
||||
full host race run produced no race warning and only this fixture failure, and
|
||||
`go vet ./...` passed. Current source therefore isolates the server CLI fixture
|
||||
with the same missing-temp-file
|
||||
`t.Setenv("PIPELINES_HARNESS_CONFIG", filepath.Join(t.TempDir(), "harnesses.json"))`
|
||||
pattern already used by `internal/cli`'s setup, so the host executable stays
|
||||
authoritative and no Docker probe runs for an ephemeral test endpoint. The
|
||||
production fail-closed redirect guard in `internal/harness/prefix.go` and its
|
||||
`refusing to redirect` / loopback-wss regressions are unchanged; the failing
|
||||
verification was fixed rather than removed.
|
||||
|
||||
This is a current-source and Root-reported observation. At the time of that
|
||||
observation, independent re-execution of the full `go build`/`go vet`/`go test`
|
||||
matrix (including Docker-only race results) and the final installed-host Docker
|
||||
verification were **pending**; they are now recorded in the 2026-10-11 FINAL
|
||||
INSTALLED section below, so this phase-specific wording is superseded by that dated
|
||||
install. The packaged reference describes source behavior; a source fix or updated
|
||||
skill is not deployment evidence.
|
||||
|
||||
#### 2026-10-11 retained run11 reviewer re-capture (source-level, not a final install)
|
||||
|
||||
Independently reading the retained run11 verification at
|
||||
`tmp/maintenance-run11-verification.json` records the source-level review
|
||||
sequence: an earlier reviewer PTY capture rejected the longer handoff because at
|
||||
80x30 it scrolled the Pipelines header off the top; the implementer then
|
||||
shortened `AuthHandoff` to a title plus three short lines; and the re-run
|
||||
`reviewer` and `ui_reviewer` stages approved from real current-source Pipelines
|
||||
PTY captures at 80x30 and 100x36 that showed the header and handoff visible above
|
||||
installed Pi 1.1.0's regular interface. A labelled controlled fixture that
|
||||
stripped `--tui-mode regular` reproduced the pre-fix fullscreen clear, and the
|
||||
review stages report no provider selected, no key entered and no inference. This
|
||||
is source-level reviewer evidence, read from retained output; it was not a
|
||||
deployed/installed state at that time, and the final full Docker install
|
||||
verification was then **pending** — it is now recorded in the 2026-10-11 FINAL
|
||||
INSTALLED section below, so this phase-specific wording is superseded by that dated
|
||||
install.
|
||||
|
||||
Root-reported actual runs (source-level observations supplied by Root for this
|
||||
change and not re-verified here): run7 showed both configured providers' exact
|
||||
handoff markers; runs6/9/10 exercised live maintenance with final-assistant
|
||||
polling/readback succeeding. These are Root observations, not mocked-regression
|
||||
coverage and not a completed installation; no final deployed/installed state is
|
||||
claimed.
|
||||
|
||||
#### 2026-10-11 run13 interruption, Docker race limitation and state separation
|
||||
|
||||
Four states are kept explicitly separate and must not be conflated:
|
||||
|
||||
1. **Current source** (this checkout, not deployed): bounded 32 KiB
|
||||
(`transcriptBound = 32<<10`) redacted transcript previews produce a valid JSON
|
||||
envelope with redaction applied before truncation and a rune-safe 8 KiB preview;
|
||||
session/debug evidence is scoped `latest_bounded_evidence` with the single
|
||||
bounded latest-assistant readback and at most the 3 most recent session-bearing
|
||||
steps; an unbudgeted V2 audited agent stage records reported usage best-effort
|
||||
through exactly one 2s-bounded final session-aggregate read, leaving
|
||||
missing/invalid usage unknown; and the isolated server CLI fixture uses the
|
||||
missing-temp-file `PIPELINES_HARNESS_CONFIG` pattern while the production
|
||||
fail-closed redirect guard in `internal/harness/prefix.go` is unchanged. Pi JSONL
|
||||
is never parsed as one JSON object; native V1 keeps its existing path.
|
||||
2. **Host verification evidence independently recorded by Root**: after the fixture
|
||||
fix, Root ran the full `go test -race -count=1 ./...` and `go vet ./...`; after
|
||||
three harness-neutral evidence labels and Step wording changes Root re-ran the
|
||||
affected `tui`/`author`/`cli` race and vet checks, the Python suite (80) and the
|
||||
opt-in compiled picker PTY check (4). These are Root-recorded host results, not
|
||||
re-executed by the authoring stages.
|
||||
3. **Deployed old image**: the running Docker image predates these source changes
|
||||
and does not include them.
|
||||
4. **Final install (completed 2026-10-11)**: the final install and installed
|
||||
authoring/provider verification were **pending** when this separation was
|
||||
written and are now recorded in the 2026-10-11 FINAL INSTALLED section below; an
|
||||
updated skill/client or source checkout is still not deployment evidence on its
|
||||
own.
|
||||
|
||||
Docker race limitation (confirmed): the Docker runtime is built with
|
||||
`CGO_ENABLED=0` and has **no C compiler** (the Alpine runtime stage installs no
|
||||
GCC/musl-dev), so `go test -race` cannot run there. In-Docker reviewers must use
|
||||
ordinary `go build`/`go test`/`go vet` and rely on independently executed host
|
||||
`-race` runs. Race checks must not be attempted in Docker and no compiler may be
|
||||
installed or searched for.
|
||||
|
||||
Run13 interruption (dated evidence, not a fix): the previous reviewer ran `find /`
|
||||
looking for GCC; the tool timeout left a process in disk sleep, Docker could not
|
||||
stop the container, and an approved Desktop restart recovered run13 as interrupted
|
||||
(`executing=false`, every step `agent_active=false`) with **no automatic replay**.
|
||||
This is an environment/interruption observation only: no general cancellation fix
|
||||
is inferred and process termination is unchanged. Docker startup recovery already
|
||||
sets running steps to `interrupted` with `agent_active=0`.
|
||||
|
||||
#### 2026-10-11 short-lived CLI inherited-pipe cancellation (current source)
|
||||
|
||||
Current source (not deployed) adds one shared, dependency-free constructor in
|
||||
`internal/opencode`: `CommandContext(ctx, executable, args...)` sets the single
|
||||
finite `os/exec.Cmd.WaitDelay` constant `ShortLivedWaitDelay = 250ms` and is the
|
||||
only place that value is defined. It is reused by the `internal/stage`
|
||||
`OpenCodeClient.call()` API/agent path (including `Interrupt`), `opencode.Detect()`
|
||||
`--help` and the native `Connection.Messages()` export fallback. It bounds
|
||||
**draining of inherited output pipes only**: after the child exits os/exec waits at
|
||||
most the grace for descendants holding the inherited stdout/stderr pipes, then
|
||||
closes them and reports `exec.ErrWaitDelay`; it does not terminate or confirm
|
||||
cleanup of descendants.
|
||||
|
||||
`context.Canceled`/`context.DeadlineExceeded` identities, `exec.ErrWaitDelay` on
|
||||
the sanitized errors, the ≤64 KiB request / ≤1 MiB response / 8 KiB stderr bounds,
|
||||
`boundedBuffer`/`boundedOutput` drain-with-overflow behavior, redaction, strict
|
||||
result validation and cleanup-unconfirmed active ownership (`ErrStageCleanup`, no
|
||||
replay) are unchanged; no new module dependency is added and a parent that exits
|
||||
successfully while a descendant still holds partial stdout is never treated as
|
||||
success.
|
||||
|
||||
Dated synthetic reproduction (`tmp/interrupt-verification`, synthetic
|
||||
real-subprocess diagnostics, **not provider evidence**): with Root waiting for the
|
||||
descendant itself to start before cancelling public
|
||||
`OpenCodeClient{Backend:"v2"}.Interrupt`, two Docker/Linux returns took
|
||||
3996/3995ms despite `context.Canceled` (`descendant-started.jsonl`); the stronger
|
||||
host reproduction took 4003/4000ms (`reproduction.log`). Added source-level
|
||||
regressions use real `os/exec` fixtures with start-marker barriers, bounded
|
||||
watchdogs and naturally terminating children for cancel-after-descendant,
|
||||
deadline identity, descendant-held pipes → `ErrWaitDelay`, ordinary
|
||||
output/exit/redaction, failed-interrupt ownership/no replay, plus `Detect`/export
|
||||
wait-delay and cancellation cases. This is **current source, not deployed**:
|
||||
descendant termination, confirmed remote interruption and any run13 Docker disk
|
||||
sleep fix are **not claimed**, and Docker race remains unavailable (no C
|
||||
compiler).
|
||||
|
||||
#### 2026-10-11 FINAL INSTALLED verification (friend-ready preview)
|
||||
|
||||
This is the dated receipt for the installed runtime, recorded by Root from
|
||||
`tmp/friend-final-verification/summary.json`; it supersedes the earlier
|
||||
"pending final install" statements above, which stay as phase-specific history.
|
||||
A normal interactive `./install --model opencode-go/deepseek-v4.1-flash
|
||||
--harness-model opencode-go/deepseek-v4.1-flash` exited **0**, resolving the current
|
||||
releases live and building fresh dirty source into actual image
|
||||
`sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136`
|
||||
(OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1) with real tool-free provider
|
||||
validation, ntfy authenticated publish, server + host health, home/workspace entry,
|
||||
preserved authoritative endpoint and `ok` pre-install database/home backup
|
||||
integrity. Post-install preservation all passed (original definitions 1–4, history
|
||||
1–16, host/container default Pi and both models/profiles, logical endpoint,
|
||||
service URLs/timezone/ports/models, Pi credential hash, OpenCode connected accounts,
|
||||
ntfy token); neither original-repo commit/push nor credential/data cleanup ran.
|
||||
Installed host-CLI authoring with **both** Docker harnesses (separate retained
|
||||
drafts, each changing only the worker prompt) performed real `ping` → `pong` and
|
||||
`validate` → `validated` through the generated Linux `/usr/local/bin/pipelines` and
|
||||
scoped transport aliases preserving the logical `8081` endpoint and draft metadata,
|
||||
independently inspected from native OpenCode tool outputs and retained Pi JSONL
|
||||
`toolResults`, with no draft submit/run/publication. Docker-proxy submit on
|
||||
verification-only definition 5 published v2, a second stale v1 draft was rejected as
|
||||
conflict, and **Run17** succeeded in 7 seconds with the exact
|
||||
`pipelines-docker-verified` markers in OpenCode then Pi, both inactive and exactly
|
||||
one completed marker read verified from retained native history (not the
|
||||
latest-assistant preview); best-effort unbudgeted usage was OpenCode 4011
|
||||
tokens/$0.00029769599999999996 and Pi 1071 tokens/$0.000142002, and
|
||||
`/api/session/active` was empty after exit. Installed-UI captures
|
||||
(`installed-pi-*`, `pi-signin-*`, `installed-pi-models-*`) cover the actual run17
|
||||
80x30/120x36 neutral-Pi/green-both-stage renders, the Pi auth handoff/method menu
|
||||
viewed/canceled/exit0 with no sign-in, and the model wizard using copied
|
||||
preferences only while production preferences stayed preserved. Root's independent
|
||||
checks passed: full host `go test -race -count=1 ./...` and `go vet ./...`,
|
||||
Python 80 tests (1 optional PTY skipped), Node policy, isolated developer lifecycle
|
||||
fixture (no inference/deploy) and author package tests; earlier independent
|
||||
reviewers ran ordinary Docker build/vet/tests and focused subprocess regressions
|
||||
(no Docker race, no GCC search, no global `find`). **UNVERIFIED** here: fresh
|
||||
OAuth/provider login completion, phone delivery, other real providers, the
|
||||
D-disk-sleep fix and active lease expiry; authentication menus and
|
||||
existing-credential inference are not fresh sign-in proof. The stronger synthetic
|
||||
PUBLIC `Interrupt` child-start barrier (host 252/252 ms, Docker 250/251 ms, context
|
||||
canceled true, bounded true vs ~4 s before) is not proof of descendant termination,
|
||||
remote cleanup or a Docker VM disk fix. Archive extraction, build-identity and
|
||||
compile verification remain pending until Root executes them.
|
||||
|
||||
#### 2026-10-11 generated release receipt ignore (source-hygiene regression)
|
||||
|
||||
Current source adds two root-anchored `.gitignore` rules for the installer's
|
||||
machine-local generated artifacts: `/.pipelines-releases.json` (the receipt path
|
||||
from `setup-docker.py` `release_receipt_path()`) and `/.pipelines-releases-*` (the
|
||||
`tempfile.mkstemp` `prefix=".pipelines-releases-"` in `record_release_receipt()`);
|
||||
no installer/runtime code changed and the pre-existing receipt and other files are
|
||||
retained. A new `scripts/test_build_metadata.py` case is a controlled, isolated Git
|
||||
fixture using the repository's actual `.gitignore`: a clean tracked source plus a
|
||||
representative generated receipt/`.pipelines-releases-*` temporary file leave `git
|
||||
status --porcelain` empty and `build_metadata.docker_args` at
|
||||
`PIPELINES_DIRTY=false` with no `-dirty` version; editing the tracked source then
|
||||
makes the status non-empty and `docker_args` report `PIPELINES_DIRTY=true` with a
|
||||
`-dirty` version. This is **offline source-level regression coverage**, not the
|
||||
recorded actual new-account install/provider login above and not live-provider or
|
||||
deployment evidence; this docs-and-ignore change has no rendered UI.
|
||||
|
||||
### Dated live reference validation: 2026-10-10
|
||||
|
||||
Environment: Docker Linux aarch64, OpenCode v2.0.20,
|
||||
**opencode-go/deepseek-v4.1-flash**. Isolated databases/fixtures and local HTTP
|
||||
notification endpoints; normal published pipelines/repositories were not changed.
|
||||
|
||||
- Incident extraction → report: exact IDs/severities/counts, unchanged handoff,
|
||||
retained session readback and one local success receipt passed (6.63s).
|
||||
- File edit → verification: exact final release-file bytes, unchanged fixtures,
|
||||
independently executed `cmp`, completed read/edit/shell evidence, retained edit
|
||||
diff and no verifier edits passed (14.60s).
|
||||
- Missing required input: blocked route ended failed; missing file was not invented;
|
||||
retained read attempt and one local failure receipt passed (4.21s).
|
||||
- Accounting/one-token budget initially failed its acceptance check because session
|
||||
zeros let the run succeed. After the tracked-prompt fix, a focused live rerun
|
||||
passed: **952 reported tokens, $0.00016440**, budget-exceeded failure, stopped
|
||||
owned session (2.43s). This is not an independently reconciled provider bill.
|
||||
|
||||
The earlier three workflows were rechecked from retained artifacts with current
|
||||
assertions, without more inference. There was **no fresh full-suite live run of
|
||||
all four workflows after the fix**. Do not generalize these results to all five
|
||||
starter templates, other models/providers, or compatible native schema/tool usage.
|
||||
Real ntfy authentication/storage, phone reception, provider outages/timeouts,
|
||||
long-stream interruption and promptness of cleanup were **not tested** by this trial.
|
||||
The trial ran rebuilt test binaries; it did not deploy the fix to the ordinary server.
|
||||
|
||||
### Dated live reference validation: 2026-10-11 (run7)
|
||||
|
||||
One existing real run7 (2026-10-11) is the retained live evidence dated for this
|
||||
change: it executed both configured harnesses, OpenCode 2.0.26 and Pi 1.1.0, with
|
||||
`opencode-go/deepseek-v4.1-flash` stages on the Docker server, and both steps
|
||||
succeeded (6s; the retained Pi evidence panel reports 1054/10000 tokens). The
|
||||
captures are retained at `tmp/friend-ui-verification/source-pi-80-*` and
|
||||
`source-pi-120-*`, and a current-source TUI inspection of that run recorded actual
|
||||
80x30 and 120x36 renderings, including the run graph and the bounded evidence
|
||||
panel.
|
||||
|
||||
This is a single real run, **not** a fresh full reference suite. Other providers,
|
||||
real outages/long-stream interruption, fresh OAuth and phone delivery remain
|
||||
**not tested**; the new-source installer and installed-authoring verification were
|
||||
separately completed by the dated 2026-10-11 FINAL INSTALLED section above. The
|
||||
[2026-10-10 OpenCode v2.0.20 trial](#dated-live-reference-validation-2026-10-10)
|
||||
above remains dated history.
|
||||
|
||||
### Reference suite when explicitly requested
|
||||
|
||||
The source checkout's `TestReferenceWorkflowsLive` uses six model stage attempts
|
||||
plus possible compatibility/format-repair calls. Retries are disabled and stages
|
||||
have two-minute timeouts. It uses real existing credentials and retains private
|
||||
`runs.db`, pinned definitions, fixture directories, run/session JSON and report.
|
||||
The accounting case has a one-token/$0.25 budget; other cases have no spending
|
||||
limit. Test notifications use a local endpoint, not the configured phone topic.
|
||||
|
||||
Live inference requires explicit `PIPELINES_REFERENCE_LIVE=1`, a chosen
|
||||
`PIPELINES_REFERENCE_MODEL=PROVIDER/MODEL`, and an existing private
|
||||
`PIPELINES_REFERENCE_DIR`. Get consent for chargeable inference and confirm which
|
||||
execution computer/account to use. Ordinary tests skip live inference. Do not
|
||||
search the repository or run the suite during normal draft authoring.
|
||||
|
||||
If the user explicitly asks to run the suite and provides a source checkout:
|
||||
|
||||
```sh
|
||||
PIPELINES_REFERENCE_LIVE=1 \
|
||||
PIPELINES_REFERENCE_MODEL=PROVIDER/MODEL \
|
||||
PIPELINES_REFERENCE_DIR=/private/approved/artifacts \
|
||||
go test ./internal/server -run '^TestReferenceWorkflowsLive$' -count=1 -v -timeout 15m
|
||||
```
|
||||
|
||||
Docker credentials require running the correctly compiled test binary in that
|
||||
container; do not copy host credentials or publish tests to the normal database.
|
||||
`TestReferenceDefinitions` checks definitions without inference.
|
||||
`TestReferenceArtifactsReview` rechecks copied run/fixture evidence without
|
||||
inference using `PIPELINES_REFERENCE_REVIEW_DIR`; select only cases present in
|
||||
that artifact directory. Missing cases skip and must not be reported as passed.
|
||||
|
||||
### 2026-10-12 run22 success, run23 live CSV fixture and run24 interruption
|
||||
|
||||
Source/structural and live-fixture evidence stays separate from runtime
|
||||
deployment; nothing here is a deployment claim.
|
||||
|
||||
- **run22 (structural/source checks, success):** the retained run22 verification
|
||||
passed its structural/source checks of the scoped template and prompt changes.
|
||||
- **run23 (live Pi CSV fixture):** a real exact-template Pi CSV run fetched the
|
||||
official Python `csv` documentation through the conditional official-source
|
||||
webfetch and implemented CSV parsing with the standard library's `csv.reader`
|
||||
through an existing shared helper, adding no duplicate preview parser; both the
|
||||
implementer and reviewer checks actually passed. Root independently verified the
|
||||
full retained native tool results and protected-file hashes against its saved
|
||||
baseline, rather than relying on model-written reports.
|
||||
- **run24 (interrupted, not a pass):** the run24 implementation completed, but its
|
||||
reviewer ignored the scope instruction and ran a prohibited filesystem-root scan
|
||||
(`find /`), leaving a process in uninterruptible D-state and stalling Docker
|
||||
mount/snapshot reads. Root canceled the run, made a consistent SQLite backup
|
||||
(`integrity ok`), recovered Docker with the supported stop/start path, and
|
||||
restored the existing ntfy/Pipelines services with the original definitions, run
|
||||
history, credentials and settings verified unchanged. run24 is retained as
|
||||
**interrupted**, not a success.
|
||||
|
||||
- **run25 (live Pi source-only review):** the reviewer identified duplicated
|
||||
implementation, a hardcoded special case, suppressed errors and test gaps.
|
||||
Root inspected retained native calls: only local read/search tools ran, with no
|
||||
shell/edit/network calls. Original fixture hashes were unchanged; runtime
|
||||
behavior was explicitly marked untested.
|
||||
- **run26 (final maintenance review, success):** the scoped plan/implement/review
|
||||
workflow passed package tests, vet, build and formatting checks. Root inspected
|
||||
the actual retained outputs and changes. No rendered UI change was claimed.
|
||||
|
||||
The scoped prompt guidance asks stages to keep inspection/verification to relevant
|
||||
repository paths and supported commands; it is a working instruction, **not** an OS
|
||||
sandbox, and neither fixes Docker VM/disk bugs nor universally enforces tool
|
||||
boundaries. At these recorded checkpoints the source changes had not been deployed.
|
||||
Catalog deployment requires a separate check of the running server and its actual
|
||||
template contents; these observations do not establish publication or deployment.
|
||||
Other providers and phone delivery remain **not tested** by this record.
|
||||
|
||||
### 2026-10-12 final template verification and Docker catalog deployment
|
||||
|
||||
The final three-stage template ran as live Pi run27 (verification definition7,
|
||||
version2) in a fresh disposable CSV fixture. Root inspected full retained native
|
||||
results: the planner fetched official Python CSV documentation, the implementer
|
||||
used the standard library through the existing shared parser, and implementer and
|
||||
reviewer independently ran `python3 checks.py` successfully. Root reran those
|
||||
checks, verified the protected fixture hashes and planner-owned review criteria,
|
||||
and confirmed the preview reused the shared parser.
|
||||
|
||||
The Docker server was then refreshed from clean source commit
|
||||
`143f237268dbde2efd6c5ce5722a3e1fd7e08c9c`, retaining OpenCode 2.0.26, Pi 1.1.0
|
||||
and Node 24.18.1. The running server reported that exact commit and `dirty=false`;
|
||||
all five served templates matched current source with the saved Pi/model defaults.
|
||||
A consistent pre-deployment database backup passed integrity checks. Every stored
|
||||
definition version/source, retained run ID, saved setting, endpoint and provider
|
||||
credential/account checked by Root was preserved. This record is a documentation
|
||||
follow-up to that deployed build; it changes no runtime code or template source.
|
||||
|
||||
These bounded fixtures establish the observed behavior on this instance, not a
|
||||
guarantee of every future model review, provider, task or platform. They do not
|
||||
establish fresh account sign-in or phone notification reception.
|
||||
|
||||
|
||||
## From the skill harness reference
|
||||
|
||||
2026-10-10 verification: `scripts/check-pi-rpc.py` checked installed Pi 0.87.1
|
||||
readiness, metadata/config RPC and **idle** lease cleanup without inference.
|
||||
Go RPC fixtures cover result/budget/missing-accounting/timeout/format-repair paths;
|
||||
Node tests cover permission checks. `PIPELINES_PI_LOCAL=1 go test ./internal/stage
|
||||
-run TestPiInstalledLocalIntegration` also exercised the **installed Pi** against
|
||||
scripted loopback responses: actual allowed/denied writes, budget exhaustion,
|
||||
and active-shell cancellation (including no delayed write after cleanup).
|
||||
The installed-Pi loopback suite also covers isolated deny-all syntax repair with
|
||||
retained ownership and cumulative usage. The compiled fuzzy picker passed PTY
|
||||
selection/cancellation; installed Pi catalog discovery/configuration passed with
|
||||
an unreachable local endpoint and no prompt.
|
||||
This is integration evidence, not real-provider/model evidence. Active lease-expiry
|
||||
recovery, web search delivery and fresh interactive Docker Pi setup remain
|
||||
**not tested**. Pi catalogs exclude ambient provider extensions for parity with
|
||||
audited stages; compatible custom endpoints can use Pi's private `models.json`.
|
||||
An isolated built Docker image also passed non-root, network-disabled policy/RPC
|
||||
readiness and idle lease cleanup, without credentials/mounts or inference.
|
||||
Codex/Claude
|
||||
adapters and AI-assisted server guidelines validation remain future work.
|
||||
|
||||
2026-10-11 source-level coverage (source-level regressions; the separate dated
|
||||
2026-10-11 final install is live evidence): the
|
||||
shared Docker-aware CLI prefix, exact-match client transport rewriting,
|
||||
Docker-linked `provider-check` dispatch and the `x/term` TTY check have
|
||||
source-level regression tests (`internal/harness`, `internal/client`,
|
||||
`internal/cli`). The prefix tests cover the remote-DNS negative
|
||||
(`127.remote.example` stays unmapped with no probe), IPv4/IPv6/`localhost`
|
||||
positives, complete `host:port` binding matching (a different service on the same
|
||||
port, or a family mismatch, does not match), malformed/empty output failing
|
||||
clearly, multiple bindings under a single probe, and loopback `wss` rejection.
|
||||
Terminal Pick regression tests use a short deadline so a TTY regression fails
|
||||
fast rather than hanging. Docker dispatch tests assert `--json` is passed only in
|
||||
JSON mode while plain text stays readable. Python fixtures cover arbitrary-digit
|
||||
OpenCode patches (a parsed `2.0.260` is still rejected against an expected
|
||||
`2.0.26` by exact equality; 260/100 are illustrative valid/future examples, not
|
||||
verified releases) and reject both raw and canonical reserved homes. Real
|
||||
in-container `provider-check`/ping confirmation, a compiled PTY run and a real
|
||||
Docker reconfigure remain **not tested** in this change. These source-level
|
||||
regressions are not a claim about the deployed image; the separate dated 2026-10-11
|
||||
FINAL INSTALLED verification recorded that the runtime was actually installed and
|
||||
used (OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1, image
|
||||
`sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136`, with
|
||||
both real stages executed in run17 and passed in Root's full host race/vet runs).
|
||||
Active lease-expiry recovery, web search delivery and fresh interactive Docker Pi
|
||||
sign-in remain **not tested**.
|
||||
Reference in New Issue
Block a user