Files
pipelines/docs/verification-history.md
T
alan 5090f1cf4e Pipelines skill: outcome-first output, aligned review and loop guidance, operator material moved to docs
Rewrite format.md around turn files, the output.md order (# outcome, edge word, # handoff, detail), checks, configs, human stages and the land stage delivery gate (sh -c in a fresh detached worktree, clean checkouts incl. untracked, 64 KB output cap, opencode only). Align review guidance across practices, verification, orchestration and the feedback-loop example. Fix loop-limit semantics in control.md against runner.go, budget_tokens sizing, read for large inputs, blocked-else-error and one stage naming set. Move dated verification logs to docs/verification-history.md and brokered chat, TUI keys and harness internals to docs/operator-reference.md; label operator-only references in SKILL.md.
2026-11-04 17:27:01 +00:00

613 lines
40 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Verification history
Dated verification logs and backend caveats moved out of the packaged
Pipelines skill (`internal/author/skill/references/verification.md` and
`harnesses.md`) so authoring agents do not load them. They are a historical
record: the JSON/line-format result envelope and the format-repair ladder they
describe are retired. Model stages now write `output.md` and only its
`# outcome` section is parsed (`internal/author/skill/references/format.md`,
`docs/model-results.md`).
## From the skill verification reference
### Budgets and backend caveats
Defaults remain two retries and one hour per stage attempt; a run-level wall-clock
deadline is not added. Budgets are optional, positive and independent USD/token
reported-usage limits shared across attempts and repeat visits.
Accounting failures stop budget-limited stages. In-flight calls can overshoot;
these are **not provider-side hard caps or a guaranteed bill ceiling**. Reported
USD may validly be zero; do not fabricate prices/accounting from output length.
Per-run usage is recorded as typed step evidence from the usage the stage already
measured for its budget (no second harness read and no transcript parse), and
totalled for the run and over all history. A stage revisited within a run shares
one budget, so each step records only that visit's increment of it and the run
total is not the cumulative stage budget counted again; a revisit that reports
nothing new (a failure or timeout before any usage poll, or a Pi visit that never
reaches a usage stat) records no usage for that visit rather than a known zero.
`CLI_PREFIX inspect RUN_ID --json`
exposes each model step's `tokens_total`, `tokens_in`/`tokens_out` (only when
`split_known`), `cost_usd` (only when `cost_known`) and the model it ran with,
plus the totalled `usage`; `CLI_PREFIX stats --json` adds the history-wide totals,
which equal the sum of the per-run totals. A stage that reported no cost exposes
its tokens and marks the cost unavailable rather than showing zero; a harness that
reports only a total (Pi) exposes the total and marks the in/out split
unavailable rather than presenting the total as input tokens. An upgraded or
historical database gains an empty usage table with no backfill, so old runs omit
`usage` instead of showing zero. This is current source, not deployment evidence.
Dated current-source evidence (2026-10-17, this machine). A throwaway read-only
reader (`internal` store API) over a scratch copy of
`~/.local/share/pipelines/data/pipelines.db`, taken with
`sqlite3 "file:$HOME/.local/share/pipelines/data/pipelines.db?mode=ro" ".backup <scratch>"`
so the live DB is never mutated, opened 228 runs and found **no** `step_usage`
rows: every stored run predates this change, so per-run figures are unavailable
rather than zero. Run 314 (this item's own run) and run 239 (whose implementer
step ran `opencode-go/kimi-k2.7-code`, a stage model different from the pinned
`opencode-go/deepseek-v4.1-flash`) both report `usage=(none recorded)`, and
`stats` omits the history-wide totals. The recorded figures themselves are
covered by the passing current-source round trips — `internal/pipeline/usage_test.go`,
`internal/server/usage_test.go` (including `inspect --json` over a websocket) and
the pre-upgrade/read-bounded/stats-sum cases — not by a live run, because the
deployed server predates this change and was not restarted. No live model call,
deploy or rebuild produced this evidence.
A live V2 trial found `/generate` returned text while its retained session had no
messages and zero counters. Current source uses tracked `/prompt` → bounded
active-session polling → message readback for **budgeted V2 text-only calls, with
all tools denied**. Completed token-budgeted responses reporting zero tokens fail
closed. No new definition flags or agent tools are required for this fix.
Unbudgeted tool-free V2 generation can still have empty backend message history;
pipeline inputs, results and command audit remain retained. An unbudgeted V2
audited agent stage makes exactly one extra bounded final session-aggregate read
to record any reported USD/token usage best-effort: missing or invalid usage stays
unknown and never turns a valid run into a failure, cumulative sessions/retries
are keyed per session so they cannot double count, and totals are never inferred
from final-assistant-only or native-partial data. Budgeted settlement and native
V1 enforcement are unchanged. Native V1 uses its existing path.
The V2 `.../experimental/session/{id}/wait` request is **no longer used**: a dated
Docker log shows it timing out at the CLI's own request timeout after about six
minutes (run started 2026-10-10T23:07:16.547Z, `TimeoutError` logged at
2026-10-10T23:13:17.262Z, server `http.status=499`), which aborted a long
implementation stage. Instead current source polls `GET /api/session/active`
about once per second with a context-aware delay and no busy loop, until the
admitted session is absent (inactive). Missing/null/non-object `data`, or a state
other than the documented `running` type, fails closed and is never treated as
inactive. This is **mocked regression coverage** in `internal/stage` (fake CLI),
not live-provider evidence; V1 native settlement is unchanged.
Current source reads the V2 final result with a single bounded descending
final-assistant readback (`order=desc&limit=1&type=assistant`) after the settled
poll, for both audited agent stages and budgeted V2 text-only calls, so large
retained history (including prior tool-read contents) is no longer fetched just
to select the completed result. A truncated/incomplete readback, an empty data
list, or a latest message that is neither completed `stop` nor a completed
structured tool-call fails closed with a sanitized diagnostic that omits
provider content; retained audit previews are **not** executable final results.
Session/debug evidence now reuses the same single latest-assistant readback
instead of a recent-20 all-message fetch, and returns each payload as a labelled
bounded, credential-redacted envelope with explicit preview/truncation/missing
diagnostics; a small final assistant result is therefore selected even when large
prior tool content exists. Pi evidence remains documented JSONL and is never
parsed as one JSON object. This describes current source behavior, not a guarantee
that a user's installed server includes the fix. An updated skill/client is not
deployment evidence; never rebuild/restart services or log into providers without
specific authorization.
Retained model-stage failures now carry a fixed sanitized category. A strictly
invalid final result (a duplicate nested field, or malformed/ambiguous JSON with a
byte offset) is labeled `FINAL RESULT VALIDATION`; request, V2 active-session poll
and final-readback failures are labeled `HARNESS TRANSPORT`. A pre-admission
request failure replays nothing and is retried for any stage; a post-admission
failure is retried only on an explicit read-only stage (see the 2026-10-17 section
below). A validation failure
preserves the admitted session, transcript and evidence and performs no second tool
execution. Confirmed ordinary-endpoint failures on 2026-10-11: an all-history
readback of exactly 262144 bytes of incomplete JSON (`invalid agent messages`), a
reviewer result with a duplicate nested `performance_review` field, and an
onboarding visual-review result with `invalid character 'z' after object key:value
pair at byte 3968` where unsafe lossless repair was rejected. These are **mocked
`internal/stage` fixtures**: executable source-level evidence, not live-provider
evidence and not a deployment. OpenCode V2 prose `output_contract` does not enforce
structured output; V2 results are validated locally and no unsupported V2 format
field is sent. A review-only fresh execution is a future recovery idea, **not
implemented**. Onboarding UI visual approval remains unresolved and must not be
reported complete.
### Bounded malformed-result recovery: 2026-10-16
Source-level, mocked `internal/stage` regression coverage (not live-provider
evidence and not a deployment): malformed final-result recovery now tries a
deterministic local pass first (append missing closers, drop one trailing comma,
or insert the single comma/colon the parser expected), then at most one deny-all
formatter request whose deadline is derived from the payload size, the stage
timeout and the attempt's remaining budget rather than a fixed 30-second
constant. Every path
records `repaired`, `partial` or `unrecoverable` in `format_repairs`; a `partial`
or `unrecoverable` payload is never returned as a completed result, and an
ambiguous payload (an unescaped quote/colon inside a string value) is never
guessed at. A shell-enabled stage's malformed but deterministically recoverable
result now completes without replaying admitted tool work; `safeToReplay` is
unchanged. The run-55 `ui_reviewer` payload was retrieved read-only with
`pipelines inspect 55 --json` and reproduced its exact byte offset 7837; it is the
ambiguous error class, so it is reported unrecoverable rather than repaired. This
does not update or restart any running server.
### Harness transport reach retry and repair deadline: 2026-10-17
Source-level, mocked `internal/stage`, `internal/opencode` and `internal/server`
regression coverage (not live-provider evidence and not a deployment): the
harness transport path now distinguishes a pre-admission reach failure
(connecting or creating the session) from a post-admission readback/poll/agent
failure under the one `HARNESS TRANSPORT` category. A pre-admission reach failure
replays nothing, so it is retried within `retries` for any stage with a fresh
session and no replayed prompt; a post-admission failure is retried only on an
explicit read-only stage, and an `edit`/`shell`/legacy stage still never replays
admitted work. A context that expires during a request keeps both the sanitized
transport detail (operation/status/exit) and
`errors.Is(err, context.DeadlineExceeded)`; a bare attempt-boundary deadline is
not treated as a reach failure and stays terminal for a mutating stage. An
unrecognized provider execution-error name is reported as a bounded, sanitized
identifier plus any HTTP status. The isolated formatter deadline is now
proportional to the stage timeout (floored, capped, and clamped to
`remaining - margin` so it can never extend the stage/attempt deadline). These are
mocked fixtures; a deployed server is not updated by this change.
### Failure-handling source changes: 2026-10-15
Source-level, mocked `internal/stage` and `internal/server` regression coverage
(not live-provider evidence): an explicit read-only tools stage now actually
retries a post-admission stage deadline or harness transport failure, creating a
fresh bounded session per attempt, while a stage with `edit`/`shell` or no tools
block still never replays admitted work; a malformed final result is re-emitted
up to `retries` only on such a read-only stage with its `format_repairs` evidence
retained. A timed-out agent stage settles knowable V2 usage and retains one
bounded, redacted latest-assistant readback plus a `retained_session` pointer in
its stored transcript. A declared outcome routed to `#failure` with no step
diagnostic now yields a run diagnostic like `stage <key> returned outcome
"blocked"`; an existing step diagnostic is still used verbatim.
### Dated source-level coverage: 2026-10-11
Source-level/mocked regression coverage (not live-provider evidence):
`resolve-releases` now uses a single bounded
`GET .../git/matching-refs/tags/v2` response that the GitHub REST docs describe as
complete with no pagination; offline fixtures cover a 100+ ref array,
`MAX_REFS`/`MAX_BODY` caps, ignored prerelease/build tags, malformed refs and a
missing newest-tag digest failing closed with no older fallback, with no
`tags/list` cross-check. Picker tests cover blink/index preservation,
ANSI-readable selection without a colliding background, and
40x16/80x24/120x36 height budgets. The compiled picker PTY check (opt-in via
`PIPELINES_HARNESS_PTY_DIR`) waits through blink intervals and asserts exact
saved/arrow-scrolled models. `setup-docker.py` accepts the actual
`opencode v2.0.26` probe token and compares source identity only on a full
install; model/login-only reconfiguration compares the verified release image
identity. Python fixtures bind temporary HOME directories so the suite passes
under the container default `/home/pipelines` without weakening mount guards.
The Docker authoring transport is also covered at source level: the shared
prefix picks `/usr/local/bin/pipelines` for a Docker-linked profile. Loopback is
recognized only via `net.ParseIP(...).IsLoopback()` and an exact case-insensitive
`localhost`, so a remote DNS name such as `127.remote.example` stays remote with
no probe. A single bounded `docker compose port` probe parses complete
`host:port` bindings, and scoped source/target aliases are embedded only when a
binding covers the requested loopback address and port; malformed or non-matching
bindings and a loopback `wss` endpoint fail closed, and `client.Dial` rewrites
only an exact source match. `provider-check` dispatches through a Docker-linked
profile's existing service with `-T`, the original cwd/model/timeout, exact
stdout/exit propagation and `--json` passed only in JSON mode (fixture tests, no
real provider), while explicit native profiles keep the host generator. The
picker uses a true `x/term` TTY check so `/dev/null` and pipes fail fast, with a
short context deadline so a regression fails rather than hangs. Real Docker
rebuild/reconfigure against live registries, a real
40x16/80x24/120x36 terminal capture, and real model inference remain **not tested**
in this change.
#### 2026-10-11 Pi sign-in handoff (source + Root PTY observation)
Root separately exercised current-source `pipelines harness setup pi --login`
under a real Docker PTY with copied harness preferences. Root observed that
Pi 1.1.0 defaults to fullscreen and clears the Pipelines login instructions, so
the user sees a coding interface without the `/login` guidance, and that
`/login` first shows "Select authentication method" (account / API key / Radius)
rather than a provider picker. The same handler run with a temporary executable
wrapper passing `pi --tui-mode regular` retained the Pipelines header and
instructions above Pi's interface. Installed `pi --help` documents the supported
`--tui-mode regular` flag and the fullscreen default; `pi auth --help` confirms
there is no standalone login (only print/check, never run). These are source/root
observations with copied preferences: **no provider was selected, no key was
entered and no inference occurred**, and this source is **not deployed** and is
not live-provider evidence. A first reviewer capture at 80x30 found the longer
handoff plus Pi's own banner scrolled the Pipelines header off the top, so the
copy was shortened to a title plus three short lines (existing-credential reuse
folded into the `/login` line, with the account/API key/Radius method choice and
`ctrl+c`/`ctrl+d` exit keys kept) and the redundant trailing blank removed.
Independent PTY re-capture at 80x30/100x36 after this shortening is **not
tested** in this change. The matching exact login-argv and copy assertions are
covered by fixture tests in `internal/cli`/`internal/harness`.
#### 2026-10-11 server CLI fixture isolation (source-level + Root-reported run)
Root-reported runtime evidence supplied for this change (not re-executed by the
authoring stages): the deterministic seeded-profile regression
`go test ./tmp/preview-verification -run TestServerCLIFixtureDoesNotInheritUserDockerProfile -count=1`
seeds a synthetic Docker-linked harness profile in a child
`go test ./internal/server -run '^TestCLIHeadlessRunInspectionCancellationAndRPC$'`.
It previously failed because the child inherited the caller's real
`PIPELINES_HARNESS_CONFIG`, so the CLI `debug` prefix failed closed
("refusing to redirect") against the test's ephemeral httptest endpoint. Root's
four independent preview tests for the bounded transcript helpers pass; Root's
full host race run produced no race warning and only this fixture failure, and
`go vet ./...` passed. Current source therefore isolates the server CLI fixture
with the same missing-temp-file
`t.Setenv("PIPELINES_HARNESS_CONFIG", filepath.Join(t.TempDir(), "harnesses.json"))`
pattern already used by `internal/cli`'s setup, so the host executable stays
authoritative and no Docker probe runs for an ephemeral test endpoint. The
production fail-closed redirect guard in `internal/harness/prefix.go` and its
`refusing to redirect` / loopback-wss regressions are unchanged; the failing
verification was fixed rather than removed.
This is a current-source and Root-reported observation. At the time of that
observation, independent re-execution of the full `go build`/`go vet`/`go test`
matrix (including Docker-only race results) and the final installed-host Docker
verification were **pending**; they are now recorded in the 2026-10-11 FINAL
INSTALLED section below, so this phase-specific wording is superseded by that dated
install. The packaged reference describes source behavior; a source fix or updated
skill is not deployment evidence.
#### 2026-10-11 retained run11 reviewer re-capture (source-level, not a final install)
Independently reading the retained run11 verification at
`tmp/maintenance-run11-verification.json` records the source-level review
sequence: an earlier reviewer PTY capture rejected the longer handoff because at
80x30 it scrolled the Pipelines header off the top; the implementer then
shortened `AuthHandoff` to a title plus three short lines; and the re-run
`reviewer` and `ui_reviewer` stages approved from real current-source Pipelines
PTY captures at 80x30 and 100x36 that showed the header and handoff visible above
installed Pi 1.1.0's regular interface. A labelled controlled fixture that
stripped `--tui-mode regular` reproduced the pre-fix fullscreen clear, and the
review stages report no provider selected, no key entered and no inference. This
is source-level reviewer evidence, read from retained output; it was not a
deployed/installed state at that time, and the final full Docker install
verification was then **pending** — it is now recorded in the 2026-10-11 FINAL
INSTALLED section below, so this phase-specific wording is superseded by that dated
install.
Root-reported actual runs (source-level observations supplied by Root for this
change and not re-verified here): run7 showed both configured providers' exact
handoff markers; runs6/9/10 exercised live maintenance with final-assistant
polling/readback succeeding. These are Root observations, not mocked-regression
coverage and not a completed installation; no final deployed/installed state is
claimed.
#### 2026-10-11 run13 interruption, Docker race limitation and state separation
Four states are kept explicitly separate and must not be conflated:
1. **Current source** (this checkout, not deployed): bounded 32 KiB
(`transcriptBound = 32<<10`) redacted transcript previews produce a valid JSON
envelope with redaction applied before truncation and a rune-safe 8 KiB preview;
session/debug evidence is scoped `latest_bounded_evidence` with the single
bounded latest-assistant readback and at most the 3 most recent session-bearing
steps; an unbudgeted V2 audited agent stage records reported usage best-effort
through exactly one 2s-bounded final session-aggregate read, leaving
missing/invalid usage unknown; and the isolated server CLI fixture uses the
missing-temp-file `PIPELINES_HARNESS_CONFIG` pattern while the production
fail-closed redirect guard in `internal/harness/prefix.go` is unchanged. Pi JSONL
is never parsed as one JSON object; native V1 keeps its existing path.
2. **Host verification evidence independently recorded by Root**: after the fixture
fix, Root ran the full `go test -race -count=1 ./...` and `go vet ./...`; after
three harness-neutral evidence labels and Step wording changes Root re-ran the
affected `tui`/`author`/`cli` race and vet checks, the Python suite (80) and the
opt-in compiled picker PTY check (4). These are Root-recorded host results, not
re-executed by the authoring stages.
3. **Deployed old image**: the running Docker image predates these source changes
and does not include them.
4. **Final install (completed 2026-10-11)**: the final install and installed
authoring/provider verification were **pending** when this separation was
written and are now recorded in the 2026-10-11 FINAL INSTALLED section below; an
updated skill/client or source checkout is still not deployment evidence on its
own.
Docker race limitation (confirmed): the Docker runtime is built with
`CGO_ENABLED=0` and has **no C compiler** (the Alpine runtime stage installs no
GCC/musl-dev), so `go test -race` cannot run there. In-Docker reviewers must use
ordinary `go build`/`go test`/`go vet` and rely on independently executed host
`-race` runs. Race checks must not be attempted in Docker and no compiler may be
installed or searched for.
Run13 interruption (dated evidence, not a fix): the previous reviewer ran `find /`
looking for GCC; the tool timeout left a process in disk sleep, Docker could not
stop the container, and an approved Desktop restart recovered run13 as interrupted
(`executing=false`, every step `agent_active=false`) with **no automatic replay**.
This is an environment/interruption observation only: no general cancellation fix
is inferred and process termination is unchanged. Docker startup recovery already
sets running steps to `interrupted` with `agent_active=0`.
#### 2026-10-11 short-lived CLI inherited-pipe cancellation (current source)
Current source (not deployed) adds one shared, dependency-free constructor in
`internal/opencode`: `CommandContext(ctx, executable, args...)` sets the single
finite `os/exec.Cmd.WaitDelay` constant `ShortLivedWaitDelay = 250ms` and is the
only place that value is defined. It is reused by the `internal/stage`
`OpenCodeClient.call()` API/agent path (including `Interrupt`), `opencode.Detect()`
`--help` and the native `Connection.Messages()` export fallback. It bounds
**draining of inherited output pipes only**: after the child exits os/exec waits at
most the grace for descendants holding the inherited stdout/stderr pipes, then
closes them and reports `exec.ErrWaitDelay`; it does not terminate or confirm
cleanup of descendants.
`context.Canceled`/`context.DeadlineExceeded` identities, `exec.ErrWaitDelay` on
the sanitized errors, the ≤64 KiB request / ≤1 MiB response / 8 KiB stderr bounds,
`boundedBuffer`/`boundedOutput` drain-with-overflow behavior, redaction, strict
result validation and cleanup-unconfirmed active ownership (`ErrStageCleanup`, no
replay) are unchanged; no new module dependency is added and a parent that exits
successfully while a descendant still holds partial stdout is never treated as
success.
Dated synthetic reproduction (`tmp/interrupt-verification`, synthetic
real-subprocess diagnostics, **not provider evidence**): with Root waiting for the
descendant itself to start before cancelling public
`OpenCodeClient{Backend:"v2"}.Interrupt`, two Docker/Linux returns took
3996/3995ms despite `context.Canceled` (`descendant-started.jsonl`); the stronger
host reproduction took 4003/4000ms (`reproduction.log`). Added source-level
regressions use real `os/exec` fixtures with start-marker barriers, bounded
watchdogs and naturally terminating children for cancel-after-descendant,
deadline identity, descendant-held pipes → `ErrWaitDelay`, ordinary
output/exit/redaction, failed-interrupt ownership/no replay, plus `Detect`/export
wait-delay and cancellation cases. This is **current source, not deployed**:
descendant termination, confirmed remote interruption and any run13 Docker disk
sleep fix are **not claimed**, and Docker race remains unavailable (no C
compiler).
#### 2026-10-11 FINAL INSTALLED verification (friend-ready preview)
This is the dated receipt for the installed runtime, recorded by Root from
`tmp/friend-final-verification/summary.json`; it supersedes the earlier
"pending final install" statements above, which stay as phase-specific history.
A normal interactive `./install --model opencode-go/deepseek-v4.1-flash
--harness-model opencode-go/deepseek-v4.1-flash` exited **0**, resolving the current
releases live and building fresh dirty source into actual image
`sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136`
(OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1) with real tool-free provider
validation, ntfy authenticated publish, server + host health, home/workspace entry,
preserved authoritative endpoint and `ok` pre-install database/home backup
integrity. Post-install preservation all passed (original definitions 1–4, history
1–16, host/container default Pi and both models/profiles, logical endpoint,
service URLs/timezone/ports/models, Pi credential hash, OpenCode connected accounts,
ntfy token); neither original-repo commit/push nor credential/data cleanup ran.
Installed host-CLI authoring with **both** Docker harnesses (separate retained
drafts, each changing only the worker prompt) performed real `ping` → `pong` and
`validate` → `validated` through the generated Linux `/usr/local/bin/pipelines` and
scoped transport aliases preserving the logical `8081` endpoint and draft metadata,
independently inspected from native OpenCode tool outputs and retained Pi JSONL
`toolResults`, with no draft submit/run/publication. Docker-proxy submit on
verification-only definition 5 published v2, a second stale v1 draft was rejected as
conflict, and **Run17** succeeded in 7 seconds with the exact
`pipelines-docker-verified` markers in OpenCode then Pi, both inactive and exactly
one completed marker read verified from retained native history (not the
latest-assistant preview); best-effort unbudgeted usage was OpenCode 4011
tokens/$0.00029769599999999996 and Pi 1071 tokens/$0.000142002, and
`/api/session/active` was empty after exit. Installed-UI captures
(`installed-pi-*`, `pi-signin-*`, `installed-pi-models-*`) cover the actual run17
80x30/120x36 neutral-Pi/green-both-stage renders, the Pi auth handoff/method menu
viewed/canceled/exit0 with no sign-in, and the model wizard using copied
preferences only while production preferences stayed preserved. Root's independent
checks passed: full host `go test -race -count=1 ./...` and `go vet ./...`,
Python 80 tests (1 optional PTY skipped), Node policy, isolated developer lifecycle
fixture (no inference/deploy) and author package tests; earlier independent
reviewers ran ordinary Docker build/vet/tests and focused subprocess regressions
(no Docker race, no GCC search, no global `find`). **UNVERIFIED** here: fresh
OAuth/provider login completion, phone delivery, other real providers, the
D-disk-sleep fix and active lease expiry; authentication menus and
existing-credential inference are not fresh sign-in proof. The stronger synthetic
PUBLIC `Interrupt` child-start barrier (host 252/252 ms, Docker 250/251 ms, context
canceled true, bounded true vs ~4 s before) is not proof of descendant termination,
remote cleanup or a Docker VM disk fix. Archive extraction, build-identity and
compile verification remain pending until Root executes them.
#### 2026-10-11 generated release receipt ignore (source-hygiene regression)
Current source adds two root-anchored `.gitignore` rules for the installer's
machine-local generated artifacts: `/.pipelines-releases.json` (the receipt path
from `setup-docker.py` `release_receipt_path()`) and `/.pipelines-releases-*` (the
`tempfile.mkstemp` `prefix=".pipelines-releases-"` in `record_release_receipt()`);
no installer/runtime code changed and the pre-existing receipt and other files are
retained. A new `scripts/test_build_metadata.py` case is a controlled, isolated Git
fixture using the repository's actual `.gitignore`: a clean tracked source plus a
representative generated receipt/`.pipelines-releases-*` temporary file leave `git
status --porcelain` empty and `build_metadata.docker_args` at
`PIPELINES_DIRTY=false` with no `-dirty` version; editing the tracked source then
makes the status non-empty and `docker_args` report `PIPELINES_DIRTY=true` with a
`-dirty` version. This is **offline source-level regression coverage**, not the
recorded actual new-account install/provider login above and not live-provider or
deployment evidence; this docs-and-ignore change has no rendered UI.
### Dated live reference validation: 2026-10-10
Environment: Docker Linux aarch64, OpenCode v2.0.20,
**opencode-go/deepseek-v4.1-flash**. Isolated databases/fixtures and local HTTP
notification endpoints; normal published pipelines/repositories were not changed.
- Incident extraction → report: exact IDs/severities/counts, unchanged handoff,
retained session readback and one local success receipt passed (6.63s).
- File edit → verification: exact final release-file bytes, unchanged fixtures,
independently executed `cmp`, completed read/edit/shell evidence, retained edit
diff and no verifier edits passed (14.60s).
- Missing required input: blocked route ended failed; missing file was not invented;
retained read attempt and one local failure receipt passed (4.21s).
- Accounting/one-token budget initially failed its acceptance check because session
zeros let the run succeed. After the tracked-prompt fix, a focused live rerun
passed: **952 reported tokens, $0.00016440**, budget-exceeded failure, stopped
owned session (2.43s). This is not an independently reconciled provider bill.
The earlier three workflows were rechecked from retained artifacts with current
assertions, without more inference. There was **no fresh full-suite live run of
all four workflows after the fix**. Do not generalize these results to all five
starter templates, other models/providers, or compatible native schema/tool usage.
Real ntfy authentication/storage, phone reception, provider outages/timeouts,
long-stream interruption and promptness of cleanup were **not tested** by this trial.
The trial ran rebuilt test binaries; it did not deploy the fix to the ordinary server.
### Dated live reference validation: 2026-10-11 (run7)
One existing real run7 (2026-10-11) is the retained live evidence dated for this
change: it executed both configured harnesses, OpenCode 2.0.26 and Pi 1.1.0, with
`opencode-go/deepseek-v4.1-flash` stages on the Docker server, and both steps
succeeded (6s; the retained Pi evidence panel reports 1054/10000 tokens). The
captures are retained at `tmp/friend-ui-verification/source-pi-80-*` and
`source-pi-120-*`, and a current-source TUI inspection of that run recorded actual
80x30 and 120x36 renderings, including the run graph and the bounded evidence
panel.
This is a single real run, **not** a fresh full reference suite. Other providers,
real outages/long-stream interruption, fresh OAuth and phone delivery remain
**not tested**; the new-source installer and installed-authoring verification were
separately completed by the dated 2026-10-11 FINAL INSTALLED section above. The
[2026-10-10 OpenCode v2.0.20 trial](#dated-live-reference-validation-2026-10-10)
above remains dated history.
### Reference suite when explicitly requested
The source checkout's `TestReferenceWorkflowsLive` uses six model stage attempts
plus possible compatibility/format-repair calls. Retries are disabled and stages
have two-minute timeouts. It uses real existing credentials and retains private
`runs.db`, pinned definitions, fixture directories, run/session JSON and report.
The accounting case has a one-token/$0.25 budget; other cases have no spending
limit. Test notifications use a local endpoint, not the configured phone topic.
Live inference requires explicit `PIPELINES_REFERENCE_LIVE=1`, a chosen
`PIPELINES_REFERENCE_MODEL=PROVIDER/MODEL`, and an existing private
`PIPELINES_REFERENCE_DIR`. Get consent for chargeable inference and confirm which
execution computer/account to use. Ordinary tests skip live inference. Do not
search the repository or run the suite during normal draft authoring.
If the user explicitly asks to run the suite and provides a source checkout:
```sh
PIPELINES_REFERENCE_LIVE=1 \
PIPELINES_REFERENCE_MODEL=PROVIDER/MODEL \
PIPELINES_REFERENCE_DIR=/private/approved/artifacts \
go test ./internal/server -run '^TestReferenceWorkflowsLive$' -count=1 -v -timeout 15m
```
Docker credentials require running the correctly compiled test binary in that
container; do not copy host credentials or publish tests to the normal database.
`TestReferenceDefinitions` checks definitions without inference.
`TestReferenceArtifactsReview` rechecks copied run/fixture evidence without
inference using `PIPELINES_REFERENCE_REVIEW_DIR`; select only cases present in
that artifact directory. Missing cases skip and must not be reported as passed.
### 2026-10-12 run22 success, run23 live CSV fixture and run24 interruption
Source/structural and live-fixture evidence stays separate from runtime
deployment; nothing here is a deployment claim.
- **run22 (structural/source checks, success):** the retained run22 verification
passed its structural/source checks of the scoped template and prompt changes.
- **run23 (live Pi CSV fixture):** a real exact-template Pi CSV run fetched the
official Python `csv` documentation through the conditional official-source
webfetch and implemented CSV parsing with the standard library's `csv.reader`
through an existing shared helper, adding no duplicate preview parser; both the
implementer and reviewer checks actually passed. Root independently verified the
full retained native tool results and protected-file hashes against its saved
baseline, rather than relying on model-written reports.
- **run24 (interrupted, not a pass):** the run24 implementation completed, but its
reviewer ignored the scope instruction and ran a prohibited filesystem-root scan
(`find /`), leaving a process in uninterruptible D-state and stalling Docker
mount/snapshot reads. Root canceled the run, made a consistent SQLite backup
(`integrity ok`), recovered Docker with the supported stop/start path, and
restored the existing ntfy/Pipelines services with the original definitions, run
history, credentials and settings verified unchanged. run24 is retained as
**interrupted**, not a success.
- **run25 (live Pi source-only review):** the reviewer identified duplicated
implementation, a hardcoded special case, suppressed errors and test gaps.
Root inspected retained native calls: only local read/search tools ran, with no
shell/edit/network calls. Original fixture hashes were unchanged; runtime
behavior was explicitly marked untested.
- **run26 (final maintenance review, success):** the scoped plan/implement/review
workflow passed package tests, vet, build and formatting checks. Root inspected
the actual retained outputs and changes. No rendered UI change was claimed.
The scoped prompt guidance asks stages to keep inspection/verification to relevant
repository paths and supported commands; it is a working instruction, **not** an OS
sandbox, and neither fixes Docker VM/disk bugs nor universally enforces tool
boundaries. At these recorded checkpoints the source changes had not been deployed.
Catalog deployment requires a separate check of the running server and its actual
template contents; these observations do not establish publication or deployment.
Other providers and phone delivery remain **not tested** by this record.
### 2026-10-12 final template verification and Docker catalog deployment
The final three-stage template ran as live Pi run27 (verification definition7,
version2) in a fresh disposable CSV fixture. Root inspected full retained native
results: the planner fetched official Python CSV documentation, the implementer
used the standard library through the existing shared parser, and implementer and
reviewer independently ran `python3 checks.py` successfully. Root reran those
checks, verified the protected fixture hashes and planner-owned review criteria,
and confirmed the preview reused the shared parser.
The Docker server was then refreshed from clean source commit
`143f237268dbde2efd6c5ce5722a3e1fd7e08c9c`, retaining OpenCode 2.0.26, Pi 1.1.0
and Node 24.18.1. The running server reported that exact commit and `dirty=false`;
all five served templates matched current source with the saved Pi/model defaults.
A consistent pre-deployment database backup passed integrity checks. Every stored
definition version/source, retained run ID, saved setting, endpoint and provider
credential/account checked by Root was preserved. This record is a documentation
follow-up to that deployed build; it changes no runtime code or template source.
These bounded fixtures establish the observed behavior on this instance, not a
guarantee of every future model review, provider, task or platform. They do not
establish fresh account sign-in or phone notification reception.
## From the skill harness reference
2026-10-10 verification: `scripts/check-pi-rpc.py` checked installed Pi 0.87.1
readiness, metadata/config RPC and **idle** lease cleanup without inference.
Go RPC fixtures cover result/budget/missing-accounting/timeout/format-repair paths;
Node tests cover permission checks. `PIPELINES_PI_LOCAL=1 go test ./internal/stage
-run TestPiInstalledLocalIntegration` also exercised the **installed Pi** against
scripted loopback responses: actual allowed/denied writes, budget exhaustion,
and active-shell cancellation (including no delayed write after cleanup).
The installed-Pi loopback suite also covers isolated deny-all syntax repair with
retained ownership and cumulative usage. The compiled fuzzy picker passed PTY
selection/cancellation; installed Pi catalog discovery/configuration passed with
an unreachable local endpoint and no prompt.
This is integration evidence, not real-provider/model evidence. Active lease-expiry
recovery, web search delivery and fresh interactive Docker Pi setup remain
**not tested**. Pi catalogs exclude ambient provider extensions for parity with
audited stages; compatible custom endpoints can use Pi's private `models.json`.
An isolated built Docker image also passed non-root, network-disabled policy/RPC
readiness and idle lease cleanup, without credentials/mounts or inference.
Codex/Claude
adapters and AI-assisted server guidelines validation remain future work.
2026-10-11 source-level coverage (source-level regressions; the separate dated
2026-10-11 final install is live evidence): the
shared Docker-aware CLI prefix, exact-match client transport rewriting,
Docker-linked `provider-check` dispatch and the `x/term` TTY check have
source-level regression tests (`internal/harness`, `internal/client`,
`internal/cli`). The prefix tests cover the remote-DNS negative
(`127.remote.example` stays unmapped with no probe), IPv4/IPv6/`localhost`
positives, complete `host:port` binding matching (a different service on the same
port, or a family mismatch, does not match), malformed/empty output failing
clearly, multiple bindings under a single probe, and loopback `wss` rejection.
Terminal Pick regression tests use a short deadline so a TTY regression fails
fast rather than hanging. Docker dispatch tests assert `--json` is passed only in
JSON mode while plain text stays readable. Python fixtures cover arbitrary-digit
OpenCode patches (a parsed `2.0.260` is still rejected against an expected
`2.0.26` by exact equality; 260/100 are illustrative valid/future examples, not
verified releases) and reject both raw and canonical reserved homes. Real
in-container `provider-check`/ping confirmation, a compiled PTY run and a real
Docker reconfigure remain **not tested** in this change. These source-level
regressions are not a claim about the deployed image; the separate dated 2026-10-11
FINAL INSTALLED verification recorded that the runtime was actually installed and
used (OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1, image
`sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136`, with
both real stages executed in run17 and passed in Root's full host race/vet runs).
Active lease-expiry recovery, web search delivery and fresh interactive Docker Pi
sign-in remain **not tested**.