Pipelines skill: outcome-first output, aligned review and loop guidance, operator material moved to docs
Rewrite format.md around turn files, the output.md order (# outcome, edge word, # handoff, detail), checks, configs, human stages and the land stage delivery gate (sh -c in a fresh detached worktree, clean checkouts incl. untracked, 64 KB output cap, opencode only). Align review guidance across practices, verification, orchestration and the feedback-loop example. Fix loop-limit semantics in control.md against runner.go, budget_tokens sizing, read for large inputs, blocked-else-error and one stage naming set. Move dated verification logs to docs/verification-history.md and brokered chat, TUI keys and harness internals to docs/operator-reference.md; label operator-only references in SKILL.md.
This commit is contained in:
@@ -0,0 +1,291 @@
|
||||
# Operator reference
|
||||
|
||||
Operator and developer material moved out of the packaged Pipelines skill
|
||||
(`internal/author/skill/references/operations.md`) because authoring agents
|
||||
never need it.
|
||||
|
||||
## TUI keys
|
||||
|
||||
In the TUI, Home advertises `/` omni search and `:` commands. Home `/` opens one
|
||||
popup over pipelines, versions, runs, steps/visits, artifact names and stage
|
||||
input/output/evidence text; each row shows a short type badge and a text match
|
||||
shows a bracketed snippet, and a bare status word or date selects runs. The same
|
||||
query is available as `pipelines search QUERY [--type KIND]`, where `--type`
|
||||
narrows the kind. Run Tab switches only
|
||||
Details ↔ Graph. `n/p` select recorded visits, including repeated stages; `d`
|
||||
focuses/toggles file changes, `A` opens the run's grouped artifacts view (name,
|
||||
content type, size and URL per entry, `PgUp`/`PgDn` scroll so a trailing visit
|
||||
that recorded none stays reachable, `Enter`/`o` to open the selected URL and
|
||||
`y` to copy it), `a` selects the active/latest retained
|
||||
configured-harness agent session's bounded, redacted tool evidence, `D`/`:debug`
|
||||
opens fresh investigation, and `P` opens the pinned pipeline version. `:quit`
|
||||
exits; `q` does not. Results are retained reports, not independently confirmed
|
||||
achievements. Distinguish partial output from full success.
|
||||
|
||||
## Brokered OpenCode chat
|
||||
|
||||
The protocol declares the brokered `chat_*` surface (`chat_sessions`,
|
||||
`chat_create`, `chat_directories`, `chat_prompt`, `chat_messages`,
|
||||
`chat_interrupt`, `chat_permission`, `chat_reply`). The server dispatches all of
|
||||
them, and pushes `chat_event` frames derived from OpenCode's global SSE firehose
|
||||
(`/event` on the native serve, `/api/event` on the V2 API), with retained
|
||||
per-session/per-message state so a reconnecting client re-syncs instead of
|
||||
replaying deltas.
|
||||
|
||||
At startup `cmd/pipelines-server` calls `opencode.OpenChat`, which detects the
|
||||
installed backend and opens exactly one owned `opencode serve` endpoint for it:
|
||||
the native wire for v1, the V2 API for the installed V2 CLI. Chat is enabled only
|
||||
when that connection carries a URL-bearing, SSE-capable event endpoint; otherwise
|
||||
chat stays disabled and every `chat_*` request is answered with a protocol error
|
||||
naming the missing configuration rather than hanging. Stage generation still
|
||||
reaches V2 through the `opencode api` CLI, so the owned serve process is chat's
|
||||
single transport and starts no process for stages. The adapter skips firehose
|
||||
events it does not surface before decoding their data, so an unrelated event
|
||||
whose `delta` has a different shape (`session.instructions.updated` carries an
|
||||
object) never fails the stream.
|
||||
|
||||
The operator can instead broker chat from their own OpenCode service with the
|
||||
opt-in `-opencode-endpoint` flag or `PIPELINES_OPENCODE_ENDPOINT`, with precedence
|
||||
flag > environment > the endpoint recorded on a previous run and the spawned
|
||||
instance as the default. `opencode.OpenEndpoint` validates the reference once
|
||||
(the established http(s)/host/no-credentials/no-query/no-path idiom), detects the
|
||||
backend from the service's own `/api/info` (V2) or `/global/health` (native)
|
||||
contract and requires a streamable `text/event-stream` event path; it returns a
|
||||
connection with **no owned process** and never calls `startServe`. A malformed
|
||||
reference, an unreachable host, a reachable non-OpenCode service or a
|
||||
health-ok-but-no-SSE service each disable chat with an error naming the endpoint
|
||||
and never fall back to the spawned instance. An authenticating endpoint accepts an
|
||||
optional password only through the private file
|
||||
`PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE` (regular, 0600, single-owner), and
|
||||
`PIPELINES_OPENCODE_ENDPOINT_USERNAME` overrides the detected default user. The
|
||||
validated choice is recorded in the additive `server_settings` table so a restart
|
||||
with no flag/environment reuses it; an unreachable recorded endpoint disables chat
|
||||
with a clear message and no fallback. `pipelines service enable` preserves the
|
||||
endpoint and the password-file reference in autostart.
|
||||
|
||||
`chat_sessions` lists the operator's own chat sessions, not the sessions pipeline
|
||||
runs create. Operator sessions are recorded in the dedicated `operator_sessions`
|
||||
table (`session_id` primary key, plus `directory`, `title`, `tag`, `created_at`,
|
||||
`last_used_at`), separate from `run_steps`/`run_session_owners`, so an operator
|
||||
chat is a first-class thing and a pipeline stage session — recorded as run
|
||||
ownership, never in the operator registry — is never listed. The handler reads the
|
||||
registry once (bounded to 128 rows, most-recently-used first), calls OpenCode's
|
||||
session list once, and keeps only registry rows the endpoint still reports, so a
|
||||
session with no recorded tag is never listed and a deleted upstream session does
|
||||
not linger. It fills each session's `directory`, `tag` and `created_at`/
|
||||
`updated_at` from the registry and marks `busy` when another client currently
|
||||
holds the session (an in-server claim, released when the stream ends or the client
|
||||
disconnects); a second client's `chat_prompt` for a held session is rejected
|
||||
before any OpenCode call. It is fail-closed: with no registry configured it
|
||||
returns an empty list, never the endpoint's whole session list. The OpenCode
|
||||
session list is read unscoped by directory (the V2 `GET /api/session` otherwise
|
||||
defaults to the caller's directory), because an operator session may live in any
|
||||
directory.
|
||||
|
||||
`chat_create` creates one operator session and records it in the same registry
|
||||
with the stored tag `operator` (the display label `Operator` is derived from it,
|
||||
so there is no second tag vocabulary). It takes a `title` (default "New chat") and
|
||||
a `directory` (default `~`): an omitted or empty value, `~`, and any `~/...` value
|
||||
all resolve to `os.UserHomeDir`, never the server's working directory; a relative
|
||||
path or a missing directory is rejected before any OpenCode call, and a session
|
||||
created but not recordable is reported as an error naming its id. Create posts the
|
||||
verified V2 shape `POST /api/session` `{"title":...,"location":{"directory":...}}`.
|
||||
`chat_directories` returns the operator's home first, then the directories of
|
||||
previous operator sessions most-recently-used first, deduplicated and bounded (8);
|
||||
it is a dedicated request so the create dialog can load the choices without first
|
||||
listing sessions and `chat_sessions` stays a pure, fail-closed session list. A
|
||||
successful `chat_prompt` touches the session's `last_used_at` in one UPDATE, so it
|
||||
moves to the front of the list; a non-operator session id touches no row.
|
||||
|
||||
`chat_adopt` records a session that already exists on the configured endpoint in
|
||||
the same operator registry, using the endpoint's own directory and title
|
||||
verbatim (never a guess, the server's working directory or the home directory).
|
||||
It reads the endpoint once through the shared `Chat.Session` (which reuses
|
||||
`Sessions`/`decodeChatSessions`), records through the existing
|
||||
`RecordOperatorSession` with the `operator` tag, and echoes the adopted session
|
||||
plus its registry row like `chat_create`, so `chat_sessions` lists it and
|
||||
`chat_prompt` continues it. A nil payload, a blank or foreign-shaped id, a nil
|
||||
broker or a nil registry is refused before any endpoint call; an id the endpoint
|
||||
does not report is refused naming the id and records nothing; an endpoint record
|
||||
with an empty directory is refused rather than substituting a directory. The CLI
|
||||
equivalent is `pipelines chat-adopt <session-id>` (`--json` prints the full
|
||||
response, otherwise a short `Adopted <id> · <directory>` line).
|
||||
|
||||
While a `chat_prompt` subscription is open the server also polls
|
||||
`GET /api/permission/request` on a fixed small interval (first poll immediately) and
|
||||
pushes one `chat_permission` frame per newly pending approval request, deduped by
|
||||
request id so a repeated poll never re-pushes the same request. The app answers with
|
||||
`chat_reply`, which the server forwards as
|
||||
`POST /api/session/{id}/permission/{requestID}/reply` with
|
||||
`{"decision":"once"|"always"|"reject"}`. A missing session/request id or an invalid
|
||||
decision is rejected before any OpenCode call, and the poller stops when the stream
|
||||
ends or the client disconnects.
|
||||
|
||||
The permission list and reply are implemented against the V2 HTTP API only: their
|
||||
field names match the service's `Permission.Request`/`Permission.Reply` schemas
|
||||
(verified against the installed service's `/openapi.json`). The native v1 serve
|
||||
translation rejects them explicitly instead of guessing an endpoint, so on v1 the
|
||||
permission poller stops quietly rather than reporting the documented limitation as
|
||||
a per-prompt error.
|
||||
|
||||
### Verification evidence (2026-10-17)
|
||||
|
||||
Verified against the installed OpenCode 2.0.20 on 2026-10-17:
|
||||
|
||||
- `opencode --version` reports `2.0.20`; `opencode --help` advertises `api` and
|
||||
`serve`. `GET /openapi.json` on an owned `opencode serve` lists `GET /api/event`
|
||||
(SSE) and the session/prompt/interrupt/permission endpoints.
|
||||
- A live turn streamed token-level `session.text.delta` and
|
||||
`session.reasoning.delta` events (each with `assistantMessageID` and `ordinal`),
|
||||
plus `session.step.*` and `session.usage.updated`; the V2 API server
|
||||
authenticates the fixed `opencode` basic-auth user.
|
||||
- The opt-in live end-to-end test (`PIPELINES_TEST_OPENCODE_CHAT_LIVE=1 go test
|
||||
./internal/server -run TestChatLiveEndToEnd -v`) opened an owned V2 endpoint,
|
||||
created an operator session through `chat_create` in a chosen directory, listed
|
||||
exactly that session through `chat_sessions` (with its directory and tag
|
||||
`operator`), recorded a genuine pipeline stage session in
|
||||
`run_steps`/`run_session_owners` and showed it never appeared in the list, and
|
||||
streamed a completed turn as `[step:started,text,step:streamed,step:ended]`
|
||||
(first delta `"pong"`). The operator-session registry, the filter, directory
|
||||
defaulting and the recents bound are also covered by the `pipeline`, `opencode`
|
||||
and `server` unit tests, which run without the live gate.
|
||||
- Not verified live: a genuine pending-approval round trip and any backend other
|
||||
than the installed 2.0.20. The permission wire shapes are unit-tested over an
|
||||
httptest fixture, not against a live pending request.
|
||||
- Operator-supplied endpoint selection and `chat_adopt` (added 2026-10-17):
|
||||
`opencode.OpenEndpoint` and the flag/env/recorded precedence, the URL-shape
|
||||
validation failures (scheme, empty host, userinfo, query, fragment, non-empty
|
||||
path), the unreachable/non-OpenCode/no-SSE/auth-mismatch failures, the
|
||||
no-owned-process property (`conn.process == nil`) and the password-file rule
|
||||
are unit-tested over httptest fixtures in `internal/opencode/endpoint_test.go`.
|
||||
`chat_adopt` (endpoint directory/title recorded with the `operator` tag, echo,
|
||||
missing id refused naming the id and recording nothing, nil payload/blank id/nil
|
||||
broker/nil registry refused before any endpoint call, empty endpoint directory
|
||||
refused) is unit-tested over the server fake broker in
|
||||
`internal/server/chat_test.go`; the recorded-endpoint round-trip across a store
|
||||
Close/Open is in `internal/pipeline/settings_test.go`; the protocol round-trip
|
||||
and the CLI dispatch (`pipelines chat-adopt`) are covered in
|
||||
`internal/protocol/chat_test.go` and `internal/cli/chat_adopt_test.go`. The
|
||||
live scratch-server check against the operator's real endpoint (adopt, list,
|
||||
prompt) was **not** run in this change; only the unit fixtures above were
|
||||
executed.
|
||||
|
||||
Deployment state: not deployed. This is reviewed and committed locally only;
|
||||
deploying it (and restarting the server) is the operator's step. Do not describe
|
||||
brokered chat as already deployed.
|
||||
|
||||
Decide a waiting human stage in the TUI: open the run, press `c`, pick an option,
|
||||
optionally type feedback, Enter.
|
||||
|
||||
## Docker release resolution
|
||||
|
||||
Docker setup resolves the current supported OpenCode V2 release from the single
|
||||
documented complete `GET .../git/matching-refs/tags/v2` response (a bounded
|
||||
complete array with no pagination parameters; never the moving `latest` tag or
|
||||
the registry's insertion order) and the current Pi release from npm, resolves the
|
||||
selected tag's immutable GHCR `sha256` digest, builds `FROM` that digest, and
|
||||
verifies the built image reports exactly those executable versions plus a
|
||||
Node-compatible Pi engine.
|
||||
It fails closed, keeping the last verified Dockerfile fallback pins, when discovery,
|
||||
digest resolution, completeness, build or compatibility cannot be established: never
|
||||
a hardcoded fallback, never a silent stale-pin substitution. A full install skips a
|
||||
rebuild only when the release receipt **and** the actual built image's release and
|
||||
source identity labels already match; a dirty source always invokes the Docker
|
||||
build and relies on layer caching. A model/login-only reconfiguration compares only
|
||||
the verified release image identity, so a changed source alone does not rebuild.
|
||||
`pipelines harness setup opencode|pi` reconfigures inside the existing container and
|
||||
refuses an image that does not match; it does not rebuild or restart services (an
|
||||
explicit ordinary `./install` does when the user asks).
|
||||
|
||||
## Harness internals
|
||||
|
||||
CLI `new/edit/debug` and TUI author/debug build one shared, Docker-aware CLI
|
||||
prefix. A Docker-linked profile embeds the in-container `/usr/local/bin/pipelines`
|
||||
(never the host binary: Linux can see but cannot execute a mounted macOS host
|
||||
executable). Loopback is recognized only via `net.ParseIP(...).IsLoopback()` plus
|
||||
an exact case-insensitive `localhost`, so a remote DNS name such as
|
||||
`127.remote.example` stays remote and runs no probe. For a loopback logical
|
||||
`--url`, a single bounded `docker compose port` probe parses each returned line
|
||||
as a complete `host:port` binding (IPv4/IPv6/wildcard) and the prefix maps only
|
||||
when a binding covers the requested loopback address and port; malformed or
|
||||
non-matching bindings fail closed. A loopback `wss` endpoint is rejected as
|
||||
having no supported local TLS mapping rather than downgraded to plaintext. On a
|
||||
match it embeds scoped `PIPELINES_TRANSPORT_SOURCE`/`PIPELINES_TRANSPORT_TARGET`
|
||||
aliases in the prefix string (not `cmd.Env`, so the pre-existing harness
|
||||
server's tools inherit them). The client rewrites an endpoint only on an exact
|
||||
source match; remote endpoints stay remote. Draft metadata (server, base
|
||||
version, create-only guard) is preserved unchanged.
|
||||
`provider-check` with a Docker-linked profile runs inside that profile's existing
|
||||
Compose service with `-T`, the original cwd/model/timeout and exact stdout/exit
|
||||
propagation; container profiles have no `DockerDirectory` and cannot recurse, and
|
||||
explicit native profiles keep the host path. The picker uses a true `x/term` TTY
|
||||
check, so `/dev/null` and pipes fail fast instead of hanging.
|
||||
|
||||
On an existing Docker setup, `pipelines harness setup opencode|pi` configures inside its container and link the host
|
||||
launcher without restarting/deploying services. They resolve and verify the current
|
||||
OpenCode V2 release from the single documented complete
|
||||
`GET .../git/matching-refs/tags/v2` response (a bounded complete array with no
|
||||
pagination parameters; never the moving `latest` tag or the registry's insertion
|
||||
order) and the current Pi release from npm, resolve the selected tag's immutable
|
||||
GHCR `sha256` digest, and require a rebuild only when the verified release image
|
||||
identity no longer matches. A full install always invokes the Docker build for a
|
||||
dirty source (relying on layer caching); a model/login-only reconfiguration does
|
||||
not rebuild. Setup writes its generated release receipt to the machine-local root
|
||||
file `.pipelines-releases.json` (with atomic `.pipelines-releases-*` temporary
|
||||
files); both are git-ignored, so a fresh clean checkout stays clean and reinstall
|
||||
still reuses the verified release/image identity. The actual built image's labels
|
||||
stay authoritative and a dirty source still rebuilds. If discovery, digest
|
||||
resolution or completeness cannot be established they stop with an error (keeping
|
||||
the last verified Dockerfile fallback pins); they never guess, fall back to a host
|
||||
runtime or install a stale pin.
|
||||
(`./install --opencode`/`--pi` were removed and only point to that command.) The native installation reuses the host OpenCode
|
||||
configuration, writes **host** (non-Docker) profiles for the harnesses it detects
|
||||
into XDG config home's `pipelines/harnesses.json`, and clears a stale Docker link
|
||||
so a Docker-era profile is not left Docker-linked. It never installs, upgrades or
|
||||
downloads a harness. Container and host credentials and accounts remain independent and
|
||||
are reused on reinstall/reconfigure. `./install` installs natively by default
|
||||
(`--native` is equivalent); explicit `./install --docker` selects the Docker
|
||||
server + ntfy installation.
|
||||
Fresh setup accepts `./install --docker --harness pi` to skip OpenCode
|
||||
login. Existing setup can configure either harness independently. Pi setup does
|
||||
not run a generation check; `provider-check --harness pi --model PROVIDER/MODEL`
|
||||
is a separate confirmed, potentially chargeable check.
|
||||
Profiles live in XDG config home's `pipelines/harnesses.json`, store preferences
|
||||
only, and preserve unrelated configuration. A profile launches a native host
|
||||
harness unless it records a Docker checkout directory, in which case it launches
|
||||
through that installation's Compose service. `harness setup <opencode|pi> --host`
|
||||
configures or converts a host profile and clears any Docker link; without
|
||||
`--host`, reconfiguring a Docker-linked profile re-verifies that the recorded
|
||||
directory still has `compose.yaml` and fails naming it rather than saving a link
|
||||
that cannot launch. Setup's summary and `harness list` report the resulting mode
|
||||
(and, for Docker, the directory). Harness CLI setup/default commands
|
||||
always configure the local execution user, not a remote `--url` server.
|
||||
OpenCode interactive launch uses
|
||||
`mini --model`: verified V2 full CLI lacks a model flag. Pi uses its CLI with the
|
||||
packaged skill explicitly loaded. Pi has no standalone `auth login` command
|
||||
(verified `pi auth --help`: only credential check/print): setup prints a
|
||||
title-plus-three-lines Pi sign-in handoff and opens a fresh, no-context,
|
||||
extension-free, no-prompt-templates, no-tools session with Pi's documented
|
||||
`--tui-mode regular` flag, so the header and handoff stay visible above Pi's
|
||||
interface at 80x30 instead of the default fullscreen clear. The user types
|
||||
`/login`, chooses an authentication method ("Sign in with an API key" for
|
||||
provider API credentials; account and Radius remain the user's choice), finishes
|
||||
provider sign-in, then exits with `ctrl+c`/`ctrl+d` to return to Pipelines model
|
||||
configuration. Existing sign-ins are reused by exiting when no new sign-in is
|
||||
needed; completing a sign-in can update the credentials stored in the Docker
|
||||
volume. No coding/task prompt or scripted keystrokes are injected.
|
||||
OpenCode V2 model stages settle long sessions by polling `GET /api/session/active`
|
||||
about once per second under the stage context, instead of the `.../wait` request
|
||||
whose CLI-side request timeout failed long stages after roughly six minutes
|
||||
(HTTP 499, dated 2026-10-10). V1 native settlement is unchanged. Final results
|
||||
and debug session evidence use the same single bounded latest-assistant readback
|
||||
(`order=desc&limit=1&type=assistant`), so a small final result is visible behind
|
||||
large prior tool content; retained transcripts over 32 KiB are stored as valid
|
||||
bounded preview envelopes that preserve stage limits and diagnostics, and exact
|
||||
outputs remain authoritative. Pi evidence is JSONL and is not parsed as one JSON
|
||||
object. Short-lived context-controlled CLI calls (`--help` detection, API calls
|
||||
and the native export fallback) share one finite inherited-pipe drain grace
|
||||
(`os/exec.Cmd.WaitDelay` ~250ms) so a canceled context or a descendant that still
|
||||
holds stdout/stderr cannot block for the child's natural lifetime; the
|
||||
`context.Canceled`/`DeadlineExceeded` and `exec.ErrWaitDelay` identities are
|
||||
preserved and partial stdout after the grace is never accepted as success.
|
||||
@@ -0,0 +1,612 @@
|
||||
# Verification history
|
||||
|
||||
Dated verification logs and backend caveats moved out of the packaged
|
||||
Pipelines skill (`internal/author/skill/references/verification.md` and
|
||||
`harnesses.md`) so authoring agents do not load them. They are a historical
|
||||
record: the JSON/line-format result envelope and the format-repair ladder they
|
||||
describe are retired. Model stages now write `output.md` and only its
|
||||
`# outcome` section is parsed (`internal/author/skill/references/format.md`,
|
||||
`docs/model-results.md`).
|
||||
|
||||
## From the skill verification reference
|
||||
|
||||
### Budgets and backend caveats
|
||||
|
||||
Defaults remain two retries and one hour per stage attempt; a run-level wall-clock
|
||||
deadline is not added. Budgets are optional, positive and independent USD/token
|
||||
reported-usage limits shared across attempts and repeat visits.
|
||||
Accounting failures stop budget-limited stages. In-flight calls can overshoot;
|
||||
these are **not provider-side hard caps or a guaranteed bill ceiling**. Reported
|
||||
USD may validly be zero; do not fabricate prices/accounting from output length.
|
||||
|
||||
Per-run usage is recorded as typed step evidence from the usage the stage already
|
||||
measured for its budget (no second harness read and no transcript parse), and
|
||||
totalled for the run and over all history. A stage revisited within a run shares
|
||||
one budget, so each step records only that visit's increment of it and the run
|
||||
total is not the cumulative stage budget counted again; a revisit that reports
|
||||
nothing new (a failure or timeout before any usage poll, or a Pi visit that never
|
||||
reaches a usage stat) records no usage for that visit rather than a known zero.
|
||||
`CLI_PREFIX inspect RUN_ID --json`
|
||||
exposes each model step's `tokens_total`, `tokens_in`/`tokens_out` (only when
|
||||
`split_known`), `cost_usd` (only when `cost_known`) and the model it ran with,
|
||||
plus the totalled `usage`; `CLI_PREFIX stats --json` adds the history-wide totals,
|
||||
which equal the sum of the per-run totals. A stage that reported no cost exposes
|
||||
its tokens and marks the cost unavailable rather than showing zero; a harness that
|
||||
reports only a total (Pi) exposes the total and marks the in/out split
|
||||
unavailable rather than presenting the total as input tokens. An upgraded or
|
||||
historical database gains an empty usage table with no backfill, so old runs omit
|
||||
`usage` instead of showing zero. This is current source, not deployment evidence.
|
||||
|
||||
Dated current-source evidence (2026-10-17, this machine). A throwaway read-only
|
||||
reader (`internal` store API) over a scratch copy of
|
||||
`~/.local/share/pipelines/data/pipelines.db`, taken with
|
||||
`sqlite3 "file:$HOME/.local/share/pipelines/data/pipelines.db?mode=ro" ".backup <scratch>"`
|
||||
so the live DB is never mutated, opened 228 runs and found **no** `step_usage`
|
||||
rows: every stored run predates this change, so per-run figures are unavailable
|
||||
rather than zero. Run 314 (this item's own run) and run 239 (whose implementer
|
||||
step ran `opencode-go/kimi-k2.7-code`, a stage model different from the pinned
|
||||
`opencode-go/deepseek-v4.1-flash`) both report `usage=(none recorded)`, and
|
||||
`stats` omits the history-wide totals. The recorded figures themselves are
|
||||
covered by the passing current-source round trips — `internal/pipeline/usage_test.go`,
|
||||
`internal/server/usage_test.go` (including `inspect --json` over a websocket) and
|
||||
the pre-upgrade/read-bounded/stats-sum cases — not by a live run, because the
|
||||
deployed server predates this change and was not restarted. No live model call,
|
||||
deploy or rebuild produced this evidence.
|
||||
|
||||
A live V2 trial found `/generate` returned text while its retained session had no
|
||||
messages and zero counters. Current source uses tracked `/prompt` → bounded
|
||||
active-session polling → message readback for **budgeted V2 text-only calls, with
|
||||
all tools denied**. Completed token-budgeted responses reporting zero tokens fail
|
||||
closed. No new definition flags or agent tools are required for this fix.
|
||||
Unbudgeted tool-free V2 generation can still have empty backend message history;
|
||||
pipeline inputs, results and command audit remain retained. An unbudgeted V2
|
||||
audited agent stage makes exactly one extra bounded final session-aggregate read
|
||||
to record any reported USD/token usage best-effort: missing or invalid usage stays
|
||||
unknown and never turns a valid run into a failure, cumulative sessions/retries
|
||||
are keyed per session so they cannot double count, and totals are never inferred
|
||||
from final-assistant-only or native-partial data. Budgeted settlement and native
|
||||
V1 enforcement are unchanged. Native V1 uses its existing path.
|
||||
|
||||
The V2 `.../experimental/session/{id}/wait` request is **no longer used**: a dated
|
||||
Docker log shows it timing out at the CLI's own request timeout after about six
|
||||
minutes (run started 2026-10-10T23:07:16.547Z, `TimeoutError` logged at
|
||||
2026-10-10T23:13:17.262Z, server `http.status=499`), which aborted a long
|
||||
implementation stage. Instead current source polls `GET /api/session/active`
|
||||
about once per second with a context-aware delay and no busy loop, until the
|
||||
admitted session is absent (inactive). Missing/null/non-object `data`, or a state
|
||||
other than the documented `running` type, fails closed and is never treated as
|
||||
inactive. This is **mocked regression coverage** in `internal/stage` (fake CLI),
|
||||
not live-provider evidence; V1 native settlement is unchanged.
|
||||
|
||||
Current source reads the V2 final result with a single bounded descending
|
||||
final-assistant readback (`order=desc&limit=1&type=assistant`) after the settled
|
||||
poll, for both audited agent stages and budgeted V2 text-only calls, so large
|
||||
retained history (including prior tool-read contents) is no longer fetched just
|
||||
to select the completed result. A truncated/incomplete readback, an empty data
|
||||
list, or a latest message that is neither completed `stop` nor a completed
|
||||
structured tool-call fails closed with a sanitized diagnostic that omits
|
||||
provider content; retained audit previews are **not** executable final results.
|
||||
Session/debug evidence now reuses the same single latest-assistant readback
|
||||
instead of a recent-20 all-message fetch, and returns each payload as a labelled
|
||||
bounded, credential-redacted envelope with explicit preview/truncation/missing
|
||||
diagnostics; a small final assistant result is therefore selected even when large
|
||||
prior tool content exists. Pi evidence remains documented JSONL and is never
|
||||
parsed as one JSON object. This describes current source behavior, not a guarantee
|
||||
that a user's installed server includes the fix. An updated skill/client is not
|
||||
deployment evidence; never rebuild/restart services or log into providers without
|
||||
specific authorization.
|
||||
|
||||
Retained model-stage failures now carry a fixed sanitized category. A strictly
|
||||
invalid final result (a duplicate nested field, or malformed/ambiguous JSON with a
|
||||
byte offset) is labeled `FINAL RESULT VALIDATION`; request, V2 active-session poll
|
||||
and final-readback failures are labeled `HARNESS TRANSPORT`. A pre-admission
|
||||
request failure replays nothing and is retried for any stage; a post-admission
|
||||
failure is retried only on an explicit read-only stage (see the 2026-10-17 section
|
||||
below). A validation failure
|
||||
preserves the admitted session, transcript and evidence and performs no second tool
|
||||
execution. Confirmed ordinary-endpoint failures on 2026-10-11: an all-history
|
||||
readback of exactly 262144 bytes of incomplete JSON (`invalid agent messages`), a
|
||||
reviewer result with a duplicate nested `performance_review` field, and an
|
||||
onboarding visual-review result with `invalid character 'z' after object key:value
|
||||
pair at byte 3968` where unsafe lossless repair was rejected. These are **mocked
|
||||
`internal/stage` fixtures**: executable source-level evidence, not live-provider
|
||||
evidence and not a deployment. OpenCode V2 prose `output_contract` does not enforce
|
||||
structured output; V2 results are validated locally and no unsupported V2 format
|
||||
field is sent. A review-only fresh execution is a future recovery idea, **not
|
||||
implemented**. Onboarding UI visual approval remains unresolved and must not be
|
||||
reported complete.
|
||||
|
||||
### Bounded malformed-result recovery: 2026-10-16
|
||||
|
||||
Source-level, mocked `internal/stage` regression coverage (not live-provider
|
||||
evidence and not a deployment): malformed final-result recovery now tries a
|
||||
deterministic local pass first (append missing closers, drop one trailing comma,
|
||||
or insert the single comma/colon the parser expected), then at most one deny-all
|
||||
formatter request whose deadline is derived from the payload size, the stage
|
||||
timeout and the attempt's remaining budget rather than a fixed 30-second
|
||||
constant. Every path
|
||||
records `repaired`, `partial` or `unrecoverable` in `format_repairs`; a `partial`
|
||||
or `unrecoverable` payload is never returned as a completed result, and an
|
||||
ambiguous payload (an unescaped quote/colon inside a string value) is never
|
||||
guessed at. A shell-enabled stage's malformed but deterministically recoverable
|
||||
result now completes without replaying admitted tool work; `safeToReplay` is
|
||||
unchanged. The run-55 `ui_reviewer` payload was retrieved read-only with
|
||||
`pipelines inspect 55 --json` and reproduced its exact byte offset 7837; it is the
|
||||
ambiguous error class, so it is reported unrecoverable rather than repaired. This
|
||||
does not update or restart any running server.
|
||||
|
||||
### Harness transport reach retry and repair deadline: 2026-10-17
|
||||
|
||||
Source-level, mocked `internal/stage`, `internal/opencode` and `internal/server`
|
||||
regression coverage (not live-provider evidence and not a deployment): the
|
||||
harness transport path now distinguishes a pre-admission reach failure
|
||||
(connecting or creating the session) from a post-admission readback/poll/agent
|
||||
failure under the one `HARNESS TRANSPORT` category. A pre-admission reach failure
|
||||
replays nothing, so it is retried within `retries` for any stage with a fresh
|
||||
session and no replayed prompt; a post-admission failure is retried only on an
|
||||
explicit read-only stage, and an `edit`/`shell`/legacy stage still never replays
|
||||
admitted work. A context that expires during a request keeps both the sanitized
|
||||
transport detail (operation/status/exit) and
|
||||
`errors.Is(err, context.DeadlineExceeded)`; a bare attempt-boundary deadline is
|
||||
not treated as a reach failure and stays terminal for a mutating stage. An
|
||||
unrecognized provider execution-error name is reported as a bounded, sanitized
|
||||
identifier plus any HTTP status. The isolated formatter deadline is now
|
||||
proportional to the stage timeout (floored, capped, and clamped to
|
||||
`remaining - margin` so it can never extend the stage/attempt deadline). These are
|
||||
mocked fixtures; a deployed server is not updated by this change.
|
||||
|
||||
### Failure-handling source changes: 2026-10-15
|
||||
|
||||
Source-level, mocked `internal/stage` and `internal/server` regression coverage
|
||||
(not live-provider evidence): an explicit read-only tools stage now actually
|
||||
retries a post-admission stage deadline or harness transport failure, creating a
|
||||
fresh bounded session per attempt, while a stage with `edit`/`shell` or no tools
|
||||
block still never replays admitted work; a malformed final result is re-emitted
|
||||
up to `retries` only on such a read-only stage with its `format_repairs` evidence
|
||||
retained. A timed-out agent stage settles knowable V2 usage and retains one
|
||||
bounded, redacted latest-assistant readback plus a `retained_session` pointer in
|
||||
its stored transcript. A declared outcome routed to `#failure` with no step
|
||||
diagnostic now yields a run diagnostic like `stage <key> returned outcome
|
||||
"blocked"`; an existing step diagnostic is still used verbatim.
|
||||
|
||||
### Dated source-level coverage: 2026-10-11
|
||||
|
||||
Source-level/mocked regression coverage (not live-provider evidence):
|
||||
`resolve-releases` now uses a single bounded
|
||||
`GET .../git/matching-refs/tags/v2` response that the GitHub REST docs describe as
|
||||
complete with no pagination; offline fixtures cover a 100+ ref array,
|
||||
`MAX_REFS`/`MAX_BODY` caps, ignored prerelease/build tags, malformed refs and a
|
||||
missing newest-tag digest failing closed with no older fallback, with no
|
||||
`tags/list` cross-check. Picker tests cover blink/index preservation,
|
||||
ANSI-readable selection without a colliding background, and
|
||||
40x16/80x24/120x36 height budgets. The compiled picker PTY check (opt-in via
|
||||
`PIPELINES_HARNESS_PTY_DIR`) waits through blink intervals and asserts exact
|
||||
saved/arrow-scrolled models. `setup-docker.py` accepts the actual
|
||||
`opencode v2.0.26` probe token and compares source identity only on a full
|
||||
install; model/login-only reconfiguration compares the verified release image
|
||||
identity. Python fixtures bind temporary HOME directories so the suite passes
|
||||
under the container default `/home/pipelines` without weakening mount guards.
|
||||
The Docker authoring transport is also covered at source level: the shared
|
||||
prefix picks `/usr/local/bin/pipelines` for a Docker-linked profile. Loopback is
|
||||
recognized only via `net.ParseIP(...).IsLoopback()` and an exact case-insensitive
|
||||
`localhost`, so a remote DNS name such as `127.remote.example` stays remote with
|
||||
no probe. A single bounded `docker compose port` probe parses complete
|
||||
`host:port` bindings, and scoped source/target aliases are embedded only when a
|
||||
binding covers the requested loopback address and port; malformed or non-matching
|
||||
bindings and a loopback `wss` endpoint fail closed, and `client.Dial` rewrites
|
||||
only an exact source match. `provider-check` dispatches through a Docker-linked
|
||||
profile's existing service with `-T`, the original cwd/model/timeout, exact
|
||||
stdout/exit propagation and `--json` passed only in JSON mode (fixture tests, no
|
||||
real provider), while explicit native profiles keep the host generator. The
|
||||
picker uses a true `x/term` TTY check so `/dev/null` and pipes fail fast, with a
|
||||
short context deadline so a regression fails rather than hangs. Real Docker
|
||||
rebuild/reconfigure against live registries, a real
|
||||
40x16/80x24/120x36 terminal capture, and real model inference remain **not tested**
|
||||
in this change.
|
||||
|
||||
#### 2026-10-11 Pi sign-in handoff (source + Root PTY observation)
|
||||
|
||||
Root separately exercised current-source `pipelines harness setup pi --login`
|
||||
under a real Docker PTY with copied harness preferences. Root observed that
|
||||
Pi 1.1.0 defaults to fullscreen and clears the Pipelines login instructions, so
|
||||
the user sees a coding interface without the `/login` guidance, and that
|
||||
`/login` first shows "Select authentication method" (account / API key / Radius)
|
||||
rather than a provider picker. The same handler run with a temporary executable
|
||||
wrapper passing `pi --tui-mode regular` retained the Pipelines header and
|
||||
instructions above Pi's interface. Installed `pi --help` documents the supported
|
||||
`--tui-mode regular` flag and the fullscreen default; `pi auth --help` confirms
|
||||
there is no standalone login (only print/check, never run). These are source/root
|
||||
observations with copied preferences: **no provider was selected, no key was
|
||||
entered and no inference occurred**, and this source is **not deployed** and is
|
||||
not live-provider evidence. A first reviewer capture at 80x30 found the longer
|
||||
handoff plus Pi's own banner scrolled the Pipelines header off the top, so the
|
||||
copy was shortened to a title plus three short lines (existing-credential reuse
|
||||
folded into the `/login` line, with the account/API key/Radius method choice and
|
||||
`ctrl+c`/`ctrl+d` exit keys kept) and the redundant trailing blank removed.
|
||||
Independent PTY re-capture at 80x30/100x36 after this shortening is **not
|
||||
tested** in this change. The matching exact login-argv and copy assertions are
|
||||
covered by fixture tests in `internal/cli`/`internal/harness`.
|
||||
|
||||
#### 2026-10-11 server CLI fixture isolation (source-level + Root-reported run)
|
||||
|
||||
Root-reported runtime evidence supplied for this change (not re-executed by the
|
||||
authoring stages): the deterministic seeded-profile regression
|
||||
`go test ./tmp/preview-verification -run TestServerCLIFixtureDoesNotInheritUserDockerProfile -count=1`
|
||||
seeds a synthetic Docker-linked harness profile in a child
|
||||
`go test ./internal/server -run '^TestCLIHeadlessRunInspectionCancellationAndRPC$'`.
|
||||
It previously failed because the child inherited the caller's real
|
||||
`PIPELINES_HARNESS_CONFIG`, so the CLI `debug` prefix failed closed
|
||||
("refusing to redirect") against the test's ephemeral httptest endpoint. Root's
|
||||
four independent preview tests for the bounded transcript helpers pass; Root's
|
||||
full host race run produced no race warning and only this fixture failure, and
|
||||
`go vet ./...` passed. Current source therefore isolates the server CLI fixture
|
||||
with the same missing-temp-file
|
||||
`t.Setenv("PIPELINES_HARNESS_CONFIG", filepath.Join(t.TempDir(), "harnesses.json"))`
|
||||
pattern already used by `internal/cli`'s setup, so the host executable stays
|
||||
authoritative and no Docker probe runs for an ephemeral test endpoint. The
|
||||
production fail-closed redirect guard in `internal/harness/prefix.go` and its
|
||||
`refusing to redirect` / loopback-wss regressions are unchanged; the failing
|
||||
verification was fixed rather than removed.
|
||||
|
||||
This is a current-source and Root-reported observation. At the time of that
|
||||
observation, independent re-execution of the full `go build`/`go vet`/`go test`
|
||||
matrix (including Docker-only race results) and the final installed-host Docker
|
||||
verification were **pending**; they are now recorded in the 2026-10-11 FINAL
|
||||
INSTALLED section below, so this phase-specific wording is superseded by that dated
|
||||
install. The packaged reference describes source behavior; a source fix or updated
|
||||
skill is not deployment evidence.
|
||||
|
||||
#### 2026-10-11 retained run11 reviewer re-capture (source-level, not a final install)
|
||||
|
||||
Independently reading the retained run11 verification at
|
||||
`tmp/maintenance-run11-verification.json` records the source-level review
|
||||
sequence: an earlier reviewer PTY capture rejected the longer handoff because at
|
||||
80x30 it scrolled the Pipelines header off the top; the implementer then
|
||||
shortened `AuthHandoff` to a title plus three short lines; and the re-run
|
||||
`reviewer` and `ui_reviewer` stages approved from real current-source Pipelines
|
||||
PTY captures at 80x30 and 100x36 that showed the header and handoff visible above
|
||||
installed Pi 1.1.0's regular interface. A labelled controlled fixture that
|
||||
stripped `--tui-mode regular` reproduced the pre-fix fullscreen clear, and the
|
||||
review stages report no provider selected, no key entered and no inference. This
|
||||
is source-level reviewer evidence, read from retained output; it was not a
|
||||
deployed/installed state at that time, and the final full Docker install
|
||||
verification was then **pending** — it is now recorded in the 2026-10-11 FINAL
|
||||
INSTALLED section below, so this phase-specific wording is superseded by that dated
|
||||
install.
|
||||
|
||||
Root-reported actual runs (source-level observations supplied by Root for this
|
||||
change and not re-verified here): run7 showed both configured providers' exact
|
||||
handoff markers; runs6/9/10 exercised live maintenance with final-assistant
|
||||
polling/readback succeeding. These are Root observations, not mocked-regression
|
||||
coverage and not a completed installation; no final deployed/installed state is
|
||||
claimed.
|
||||
|
||||
#### 2026-10-11 run13 interruption, Docker race limitation and state separation
|
||||
|
||||
Four states are kept explicitly separate and must not be conflated:
|
||||
|
||||
1. **Current source** (this checkout, not deployed): bounded 32 KiB
|
||||
(`transcriptBound = 32<<10`) redacted transcript previews produce a valid JSON
|
||||
envelope with redaction applied before truncation and a rune-safe 8 KiB preview;
|
||||
session/debug evidence is scoped `latest_bounded_evidence` with the single
|
||||
bounded latest-assistant readback and at most the 3 most recent session-bearing
|
||||
steps; an unbudgeted V2 audited agent stage records reported usage best-effort
|
||||
through exactly one 2s-bounded final session-aggregate read, leaving
|
||||
missing/invalid usage unknown; and the isolated server CLI fixture uses the
|
||||
missing-temp-file `PIPELINES_HARNESS_CONFIG` pattern while the production
|
||||
fail-closed redirect guard in `internal/harness/prefix.go` is unchanged. Pi JSONL
|
||||
is never parsed as one JSON object; native V1 keeps its existing path.
|
||||
2. **Host verification evidence independently recorded by Root**: after the fixture
|
||||
fix, Root ran the full `go test -race -count=1 ./...` and `go vet ./...`; after
|
||||
three harness-neutral evidence labels and Step wording changes Root re-ran the
|
||||
affected `tui`/`author`/`cli` race and vet checks, the Python suite (80) and the
|
||||
opt-in compiled picker PTY check (4). These are Root-recorded host results, not
|
||||
re-executed by the authoring stages.
|
||||
3. **Deployed old image**: the running Docker image predates these source changes
|
||||
and does not include them.
|
||||
4. **Final install (completed 2026-10-11)**: the final install and installed
|
||||
authoring/provider verification were **pending** when this separation was
|
||||
written and are now recorded in the 2026-10-11 FINAL INSTALLED section below; an
|
||||
updated skill/client or source checkout is still not deployment evidence on its
|
||||
own.
|
||||
|
||||
Docker race limitation (confirmed): the Docker runtime is built with
|
||||
`CGO_ENABLED=0` and has **no C compiler** (the Alpine runtime stage installs no
|
||||
GCC/musl-dev), so `go test -race` cannot run there. In-Docker reviewers must use
|
||||
ordinary `go build`/`go test`/`go vet` and rely on independently executed host
|
||||
`-race` runs. Race checks must not be attempted in Docker and no compiler may be
|
||||
installed or searched for.
|
||||
|
||||
Run13 interruption (dated evidence, not a fix): the previous reviewer ran `find /`
|
||||
looking for GCC; the tool timeout left a process in disk sleep, Docker could not
|
||||
stop the container, and an approved Desktop restart recovered run13 as interrupted
|
||||
(`executing=false`, every step `agent_active=false`) with **no automatic replay**.
|
||||
This is an environment/interruption observation only: no general cancellation fix
|
||||
is inferred and process termination is unchanged. Docker startup recovery already
|
||||
sets running steps to `interrupted` with `agent_active=0`.
|
||||
|
||||
#### 2026-10-11 short-lived CLI inherited-pipe cancellation (current source)
|
||||
|
||||
Current source (not deployed) adds one shared, dependency-free constructor in
|
||||
`internal/opencode`: `CommandContext(ctx, executable, args...)` sets the single
|
||||
finite `os/exec.Cmd.WaitDelay` constant `ShortLivedWaitDelay = 250ms` and is the
|
||||
only place that value is defined. It is reused by the `internal/stage`
|
||||
`OpenCodeClient.call()` API/agent path (including `Interrupt`), `opencode.Detect()`
|
||||
`--help` and the native `Connection.Messages()` export fallback. It bounds
|
||||
**draining of inherited output pipes only**: after the child exits os/exec waits at
|
||||
most the grace for descendants holding the inherited stdout/stderr pipes, then
|
||||
closes them and reports `exec.ErrWaitDelay`; it does not terminate or confirm
|
||||
cleanup of descendants.
|
||||
|
||||
`context.Canceled`/`context.DeadlineExceeded` identities, `exec.ErrWaitDelay` on
|
||||
the sanitized errors, the ≤64 KiB request / ≤1 MiB response / 8 KiB stderr bounds,
|
||||
`boundedBuffer`/`boundedOutput` drain-with-overflow behavior, redaction, strict
|
||||
result validation and cleanup-unconfirmed active ownership (`ErrStageCleanup`, no
|
||||
replay) are unchanged; no new module dependency is added and a parent that exits
|
||||
successfully while a descendant still holds partial stdout is never treated as
|
||||
success.
|
||||
|
||||
Dated synthetic reproduction (`tmp/interrupt-verification`, synthetic
|
||||
real-subprocess diagnostics, **not provider evidence**): with Root waiting for the
|
||||
descendant itself to start before cancelling public
|
||||
`OpenCodeClient{Backend:"v2"}.Interrupt`, two Docker/Linux returns took
|
||||
3996/3995ms despite `context.Canceled` (`descendant-started.jsonl`); the stronger
|
||||
host reproduction took 4003/4000ms (`reproduction.log`). Added source-level
|
||||
regressions use real `os/exec` fixtures with start-marker barriers, bounded
|
||||
watchdogs and naturally terminating children for cancel-after-descendant,
|
||||
deadline identity, descendant-held pipes → `ErrWaitDelay`, ordinary
|
||||
output/exit/redaction, failed-interrupt ownership/no replay, plus `Detect`/export
|
||||
wait-delay and cancellation cases. This is **current source, not deployed**:
|
||||
descendant termination, confirmed remote interruption and any run13 Docker disk
|
||||
sleep fix are **not claimed**, and Docker race remains unavailable (no C
|
||||
compiler).
|
||||
|
||||
#### 2026-10-11 FINAL INSTALLED verification (friend-ready preview)
|
||||
|
||||
This is the dated receipt for the installed runtime, recorded by Root from
|
||||
`tmp/friend-final-verification/summary.json`; it supersedes the earlier
|
||||
"pending final install" statements above, which stay as phase-specific history.
|
||||
A normal interactive `./install --model opencode-go/deepseek-v4.1-flash
|
||||
--harness-model opencode-go/deepseek-v4.1-flash` exited **0**, resolving the current
|
||||
releases live and building fresh dirty source into actual image
|
||||
`sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136`
|
||||
(OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1) with real tool-free provider
|
||||
validation, ntfy authenticated publish, server + host health, home/workspace entry,
|
||||
preserved authoritative endpoint and `ok` pre-install database/home backup
|
||||
integrity. Post-install preservation all passed (original definitions 1–4, history
|
||||
1–16, host/container default Pi and both models/profiles, logical endpoint,
|
||||
service URLs/timezone/ports/models, Pi credential hash, OpenCode connected accounts,
|
||||
ntfy token); neither original-repo commit/push nor credential/data cleanup ran.
|
||||
Installed host-CLI authoring with **both** Docker harnesses (separate retained
|
||||
drafts, each changing only the worker prompt) performed real `ping` → `pong` and
|
||||
`validate` → `validated` through the generated Linux `/usr/local/bin/pipelines` and
|
||||
scoped transport aliases preserving the logical `8081` endpoint and draft metadata,
|
||||
independently inspected from native OpenCode tool outputs and retained Pi JSONL
|
||||
`toolResults`, with no draft submit/run/publication. Docker-proxy submit on
|
||||
verification-only definition 5 published v2, a second stale v1 draft was rejected as
|
||||
conflict, and **Run17** succeeded in 7 seconds with the exact
|
||||
`pipelines-docker-verified` markers in OpenCode then Pi, both inactive and exactly
|
||||
one completed marker read verified from retained native history (not the
|
||||
latest-assistant preview); best-effort unbudgeted usage was OpenCode 4011
|
||||
tokens/$0.00029769599999999996 and Pi 1071 tokens/$0.000142002, and
|
||||
`/api/session/active` was empty after exit. Installed-UI captures
|
||||
(`installed-pi-*`, `pi-signin-*`, `installed-pi-models-*`) cover the actual run17
|
||||
80x30/120x36 neutral-Pi/green-both-stage renders, the Pi auth handoff/method menu
|
||||
viewed/canceled/exit0 with no sign-in, and the model wizard using copied
|
||||
preferences only while production preferences stayed preserved. Root's independent
|
||||
checks passed: full host `go test -race -count=1 ./...` and `go vet ./...`,
|
||||
Python 80 tests (1 optional PTY skipped), Node policy, isolated developer lifecycle
|
||||
fixture (no inference/deploy) and author package tests; earlier independent
|
||||
reviewers ran ordinary Docker build/vet/tests and focused subprocess regressions
|
||||
(no Docker race, no GCC search, no global `find`). **UNVERIFIED** here: fresh
|
||||
OAuth/provider login completion, phone delivery, other real providers, the
|
||||
D-disk-sleep fix and active lease expiry; authentication menus and
|
||||
existing-credential inference are not fresh sign-in proof. The stronger synthetic
|
||||
PUBLIC `Interrupt` child-start barrier (host 252/252 ms, Docker 250/251 ms, context
|
||||
canceled true, bounded true vs ~4 s before) is not proof of descendant termination,
|
||||
remote cleanup or a Docker VM disk fix. Archive extraction, build-identity and
|
||||
compile verification remain pending until Root executes them.
|
||||
|
||||
#### 2026-10-11 generated release receipt ignore (source-hygiene regression)
|
||||
|
||||
Current source adds two root-anchored `.gitignore` rules for the installer's
|
||||
machine-local generated artifacts: `/.pipelines-releases.json` (the receipt path
|
||||
from `setup-docker.py` `release_receipt_path()`) and `/.pipelines-releases-*` (the
|
||||
`tempfile.mkstemp` `prefix=".pipelines-releases-"` in `record_release_receipt()`);
|
||||
no installer/runtime code changed and the pre-existing receipt and other files are
|
||||
retained. A new `scripts/test_build_metadata.py` case is a controlled, isolated Git
|
||||
fixture using the repository's actual `.gitignore`: a clean tracked source plus a
|
||||
representative generated receipt/`.pipelines-releases-*` temporary file leave `git
|
||||
status --porcelain` empty and `build_metadata.docker_args` at
|
||||
`PIPELINES_DIRTY=false` with no `-dirty` version; editing the tracked source then
|
||||
makes the status non-empty and `docker_args` report `PIPELINES_DIRTY=true` with a
|
||||
`-dirty` version. This is **offline source-level regression coverage**, not the
|
||||
recorded actual new-account install/provider login above and not live-provider or
|
||||
deployment evidence; this docs-and-ignore change has no rendered UI.
|
||||
|
||||
### Dated live reference validation: 2026-10-10
|
||||
|
||||
Environment: Docker Linux aarch64, OpenCode v2.0.20,
|
||||
**opencode-go/deepseek-v4.1-flash**. Isolated databases/fixtures and local HTTP
|
||||
notification endpoints; normal published pipelines/repositories were not changed.
|
||||
|
||||
- Incident extraction → report: exact IDs/severities/counts, unchanged handoff,
|
||||
retained session readback and one local success receipt passed (6.63s).
|
||||
- File edit → verification: exact final release-file bytes, unchanged fixtures,
|
||||
independently executed `cmp`, completed read/edit/shell evidence, retained edit
|
||||
diff and no verifier edits passed (14.60s).
|
||||
- Missing required input: blocked route ended failed; missing file was not invented;
|
||||
retained read attempt and one local failure receipt passed (4.21s).
|
||||
- Accounting/one-token budget initially failed its acceptance check because session
|
||||
zeros let the run succeed. After the tracked-prompt fix, a focused live rerun
|
||||
passed: **952 reported tokens, $0.00016440**, budget-exceeded failure, stopped
|
||||
owned session (2.43s). This is not an independently reconciled provider bill.
|
||||
|
||||
The earlier three workflows were rechecked from retained artifacts with current
|
||||
assertions, without more inference. There was **no fresh full-suite live run of
|
||||
all four workflows after the fix**. Do not generalize these results to all five
|
||||
starter templates, other models/providers, or compatible native schema/tool usage.
|
||||
Real ntfy authentication/storage, phone reception, provider outages/timeouts,
|
||||
long-stream interruption and promptness of cleanup were **not tested** by this trial.
|
||||
The trial ran rebuilt test binaries; it did not deploy the fix to the ordinary server.
|
||||
|
||||
### Dated live reference validation: 2026-10-11 (run7)
|
||||
|
||||
One existing real run7 (2026-10-11) is the retained live evidence dated for this
|
||||
change: it executed both configured harnesses, OpenCode 2.0.26 and Pi 1.1.0, with
|
||||
`opencode-go/deepseek-v4.1-flash` stages on the Docker server, and both steps
|
||||
succeeded (6s; the retained Pi evidence panel reports 1054/10000 tokens). The
|
||||
captures are retained at `tmp/friend-ui-verification/source-pi-80-*` and
|
||||
`source-pi-120-*`, and a current-source TUI inspection of that run recorded actual
|
||||
80x30 and 120x36 renderings, including the run graph and the bounded evidence
|
||||
panel.
|
||||
|
||||
This is a single real run, **not** a fresh full reference suite. Other providers,
|
||||
real outages/long-stream interruption, fresh OAuth and phone delivery remain
|
||||
**not tested**; the new-source installer and installed-authoring verification were
|
||||
separately completed by the dated 2026-10-11 FINAL INSTALLED section above. The
|
||||
[2026-10-10 OpenCode v2.0.20 trial](#dated-live-reference-validation-2026-10-10)
|
||||
above remains dated history.
|
||||
|
||||
### Reference suite when explicitly requested
|
||||
|
||||
The source checkout's `TestReferenceWorkflowsLive` uses six model stage attempts
|
||||
plus possible compatibility/format-repair calls. Retries are disabled and stages
|
||||
have two-minute timeouts. It uses real existing credentials and retains private
|
||||
`runs.db`, pinned definitions, fixture directories, run/session JSON and report.
|
||||
The accounting case has a one-token/$0.25 budget; other cases have no spending
|
||||
limit. Test notifications use a local endpoint, not the configured phone topic.
|
||||
|
||||
Live inference requires explicit `PIPELINES_REFERENCE_LIVE=1`, a chosen
|
||||
`PIPELINES_REFERENCE_MODEL=PROVIDER/MODEL`, and an existing private
|
||||
`PIPELINES_REFERENCE_DIR`. Get consent for chargeable inference and confirm which
|
||||
execution computer/account to use. Ordinary tests skip live inference. Do not
|
||||
search the repository or run the suite during normal draft authoring.
|
||||
|
||||
If the user explicitly asks to run the suite and provides a source checkout:
|
||||
|
||||
```sh
|
||||
PIPELINES_REFERENCE_LIVE=1 \
|
||||
PIPELINES_REFERENCE_MODEL=PROVIDER/MODEL \
|
||||
PIPELINES_REFERENCE_DIR=/private/approved/artifacts \
|
||||
go test ./internal/server -run '^TestReferenceWorkflowsLive$' -count=1 -v -timeout 15m
|
||||
```
|
||||
|
||||
Docker credentials require running the correctly compiled test binary in that
|
||||
container; do not copy host credentials or publish tests to the normal database.
|
||||
`TestReferenceDefinitions` checks definitions without inference.
|
||||
`TestReferenceArtifactsReview` rechecks copied run/fixture evidence without
|
||||
inference using `PIPELINES_REFERENCE_REVIEW_DIR`; select only cases present in
|
||||
that artifact directory. Missing cases skip and must not be reported as passed.
|
||||
|
||||
### 2026-10-12 run22 success, run23 live CSV fixture and run24 interruption
|
||||
|
||||
Source/structural and live-fixture evidence stays separate from runtime
|
||||
deployment; nothing here is a deployment claim.
|
||||
|
||||
- **run22 (structural/source checks, success):** the retained run22 verification
|
||||
passed its structural/source checks of the scoped template and prompt changes.
|
||||
- **run23 (live Pi CSV fixture):** a real exact-template Pi CSV run fetched the
|
||||
official Python `csv` documentation through the conditional official-source
|
||||
webfetch and implemented CSV parsing with the standard library's `csv.reader`
|
||||
through an existing shared helper, adding no duplicate preview parser; both the
|
||||
implementer and reviewer checks actually passed. Root independently verified the
|
||||
full retained native tool results and protected-file hashes against its saved
|
||||
baseline, rather than relying on model-written reports.
|
||||
- **run24 (interrupted, not a pass):** the run24 implementation completed, but its
|
||||
reviewer ignored the scope instruction and ran a prohibited filesystem-root scan
|
||||
(`find /`), leaving a process in uninterruptible D-state and stalling Docker
|
||||
mount/snapshot reads. Root canceled the run, made a consistent SQLite backup
|
||||
(`integrity ok`), recovered Docker with the supported stop/start path, and
|
||||
restored the existing ntfy/Pipelines services with the original definitions, run
|
||||
history, credentials and settings verified unchanged. run24 is retained as
|
||||
**interrupted**, not a success.
|
||||
|
||||
- **run25 (live Pi source-only review):** the reviewer identified duplicated
|
||||
implementation, a hardcoded special case, suppressed errors and test gaps.
|
||||
Root inspected retained native calls: only local read/search tools ran, with no
|
||||
shell/edit/network calls. Original fixture hashes were unchanged; runtime
|
||||
behavior was explicitly marked untested.
|
||||
- **run26 (final maintenance review, success):** the scoped plan/implement/review
|
||||
workflow passed package tests, vet, build and formatting checks. Root inspected
|
||||
the actual retained outputs and changes. No rendered UI change was claimed.
|
||||
|
||||
The scoped prompt guidance asks stages to keep inspection/verification to relevant
|
||||
repository paths and supported commands; it is a working instruction, **not** an OS
|
||||
sandbox, and neither fixes Docker VM/disk bugs nor universally enforces tool
|
||||
boundaries. At these recorded checkpoints the source changes had not been deployed.
|
||||
Catalog deployment requires a separate check of the running server and its actual
|
||||
template contents; these observations do not establish publication or deployment.
|
||||
Other providers and phone delivery remain **not tested** by this record.
|
||||
|
||||
### 2026-10-12 final template verification and Docker catalog deployment
|
||||
|
||||
The final three-stage template ran as live Pi run27 (verification definition7,
|
||||
version2) in a fresh disposable CSV fixture. Root inspected full retained native
|
||||
results: the planner fetched official Python CSV documentation, the implementer
|
||||
used the standard library through the existing shared parser, and implementer and
|
||||
reviewer independently ran `python3 checks.py` successfully. Root reran those
|
||||
checks, verified the protected fixture hashes and planner-owned review criteria,
|
||||
and confirmed the preview reused the shared parser.
|
||||
|
||||
The Docker server was then refreshed from clean source commit
|
||||
`143f237268dbde2efd6c5ce5722a3e1fd7e08c9c`, retaining OpenCode 2.0.26, Pi 1.1.0
|
||||
and Node 24.18.1. The running server reported that exact commit and `dirty=false`;
|
||||
all five served templates matched current source with the saved Pi/model defaults.
|
||||
A consistent pre-deployment database backup passed integrity checks. Every stored
|
||||
definition version/source, retained run ID, saved setting, endpoint and provider
|
||||
credential/account checked by Root was preserved. This record is a documentation
|
||||
follow-up to that deployed build; it changes no runtime code or template source.
|
||||
|
||||
These bounded fixtures establish the observed behavior on this instance, not a
|
||||
guarantee of every future model review, provider, task or platform. They do not
|
||||
establish fresh account sign-in or phone notification reception.
|
||||
|
||||
|
||||
## From the skill harness reference
|
||||
|
||||
2026-10-10 verification: `scripts/check-pi-rpc.py` checked installed Pi 0.87.1
|
||||
readiness, metadata/config RPC and **idle** lease cleanup without inference.
|
||||
Go RPC fixtures cover result/budget/missing-accounting/timeout/format-repair paths;
|
||||
Node tests cover permission checks. `PIPELINES_PI_LOCAL=1 go test ./internal/stage
|
||||
-run TestPiInstalledLocalIntegration` also exercised the **installed Pi** against
|
||||
scripted loopback responses: actual allowed/denied writes, budget exhaustion,
|
||||
and active-shell cancellation (including no delayed write after cleanup).
|
||||
The installed-Pi loopback suite also covers isolated deny-all syntax repair with
|
||||
retained ownership and cumulative usage. The compiled fuzzy picker passed PTY
|
||||
selection/cancellation; installed Pi catalog discovery/configuration passed with
|
||||
an unreachable local endpoint and no prompt.
|
||||
This is integration evidence, not real-provider/model evidence. Active lease-expiry
|
||||
recovery, web search delivery and fresh interactive Docker Pi setup remain
|
||||
**not tested**. Pi catalogs exclude ambient provider extensions for parity with
|
||||
audited stages; compatible custom endpoints can use Pi's private `models.json`.
|
||||
An isolated built Docker image also passed non-root, network-disabled policy/RPC
|
||||
readiness and idle lease cleanup, without credentials/mounts or inference.
|
||||
Codex/Claude
|
||||
adapters and AI-assisted server guidelines validation remain future work.
|
||||
|
||||
2026-10-11 source-level coverage (source-level regressions; the separate dated
|
||||
2026-10-11 final install is live evidence): the
|
||||
shared Docker-aware CLI prefix, exact-match client transport rewriting,
|
||||
Docker-linked `provider-check` dispatch and the `x/term` TTY check have
|
||||
source-level regression tests (`internal/harness`, `internal/client`,
|
||||
`internal/cli`). The prefix tests cover the remote-DNS negative
|
||||
(`127.remote.example` stays unmapped with no probe), IPv4/IPv6/`localhost`
|
||||
positives, complete `host:port` binding matching (a different service on the same
|
||||
port, or a family mismatch, does not match), malformed/empty output failing
|
||||
clearly, multiple bindings under a single probe, and loopback `wss` rejection.
|
||||
Terminal Pick regression tests use a short deadline so a TTY regression fails
|
||||
fast rather than hanging. Docker dispatch tests assert `--json` is passed only in
|
||||
JSON mode while plain text stays readable. Python fixtures cover arbitrary-digit
|
||||
OpenCode patches (a parsed `2.0.260` is still rejected against an expected
|
||||
`2.0.26` by exact equality; 260/100 are illustrative valid/future examples, not
|
||||
verified releases) and reject both raw and canonical reserved homes. Real
|
||||
in-container `provider-check`/ping confirmation, a compiled PTY run and a real
|
||||
Docker reconfigure remain **not tested** in this change. These source-level
|
||||
regressions are not a claim about the deployed image; the separate dated 2026-10-11
|
||||
FINAL INSTALLED verification recorded that the runtime was actually installed and
|
||||
used (OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1, image
|
||||
`sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136`, with
|
||||
both real stages executed in run17 and passed in Root's full host race/vet runs).
|
||||
Active lease-expiry recovery, web search delivery and fresh interactive Docker Pi
|
||||
sign-in remain **not tested**.
|
||||
@@ -37,9 +37,11 @@ error -> #failure
|
||||
# stage review
|
||||
|
||||
```prompt
|
||||
Independently review the actual change against the original request in
|
||||
memory.md, not just the implementer's summary: `git diff --stat` first, then the
|
||||
changed hunks and only the code around them. Run no test suites. Return
|
||||
Review the actual change against the original request in memory.md, not just
|
||||
the implementer's summary: start from the server-captured changed files and
|
||||
`git diff --stat`, then read the per-file diffs and only the code around them.
|
||||
Judge the checks the implementer reported and name missing coverage; run no
|
||||
test suites. Return
|
||||
rejected only for a must-fix problem, giving its location, evidence and fix.
|
||||
Otherwise return approved, with any non-blocking notes in the body. Do not edit,
|
||||
commit or push.
|
||||
|
||||
@@ -29,7 +29,7 @@ error -> #failure
|
||||
# stage check
|
||||
|
||||
```prompt
|
||||
input.md is the previous stage's draft; memory.md holds the original request.
|
||||
input.md holds the original request and the previous stage's draft.
|
||||
Check the draft against its sources and the request, correct mistakes and gaps,
|
||||
and return the final text under `# Final` with a short list of corrections.
|
||||
Change nothing. Return blocked if the draft cannot be checked.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: Pipelines
|
||||
description: Use existing Pipelines workflows to run, inspect, wait, recover and deliver project work. Use only when the work belongs in an existing pipeline; authoring is a separate, explicit task.
|
||||
description: Use existing Pipelines workflows to run, inspect, wait, recover and deliver work, and author definitions when the user explicitly asks.
|
||||
---
|
||||
|
||||
# Pipelines
|
||||
@@ -34,63 +34,44 @@ explain the gap; ask before authoring. Do not add a task-specific pipeline as a
|
||||
|
||||
## Authoring
|
||||
|
||||
A definition is Markdown: `# stage <Name>` sections with `prompt`, `configs`,
|
||||
`tools`, `edges`, and `limits`/`checks` where required: every stage that can
|
||||
modify the workspace declares `checks` (commands on the `delivery=true` stage,
|
||||
otherwise `none: <reason>`), and every loop has `limits`. Start from
|
||||
`examples/` and read `references/practices.md` before writing prompts. Use stage
|
||||
names that describe the work:
|
||||
|
||||
`Plan` · `Code` · `Review` · `UIReview` · `Land` (`Prepare` and `Survey` are
|
||||
useful variants).
|
||||
Start from `pipelines templates` and `pipelines new NAME --template KEY`
|
||||
(`one-stage`, `two-stages`, `feedback-loop`, `human-decision`); these fill in
|
||||
the server's provider/model. Read `references/format.md` and
|
||||
`references/practices.md` before writing prompts. Stage names: `plan` ·
|
||||
`implement` · `review` · `ui-review` · `decide` (human) · `land`.
|
||||
|
||||
Keep prompts short and objective: what the stage should produce and any real
|
||||
constraints. The runner enforces turns, tools and limits; prompts do not need to
|
||||
repeat them. Only `# outcome` is parsed from `output.md`; everything else is
|
||||
free markdown. Use `links` and `memory` only when they clarify a handoff.
|
||||
constraints. The runner enforces turns, tools and limits and explains the output
|
||||
format; prompts do not repeat them. Only `# outcome` is parsed from `output.md`;
|
||||
everything else is free markdown. There are no memory, links, input or output
|
||||
blocks; stages communicate through `output.md` (`# handoff`) and `public/`.
|
||||
|
||||
`pipelines validate DRAFT` checks a draft, including the authoring rules in
|
||||
`references/practices.md` (errors block submit; `warning:` lines do not but should
|
||||
be fixed); `pipelines submit DRAFT` publishes a new immutable version.
|
||||
`pipelines validate DRAFT` checks a draft, including the authoring rules
|
||||
(errors block submit; `warning:` lines do not but should be fixed);
|
||||
`pipelines submit DRAFT` publishes a new immutable version.
|
||||
`pipelines help <command>` has flags.
|
||||
|
||||
Schedules are separate saved run configurations: pipeline/version, prompt,
|
||||
workspace and other run options, plus timing. Never add `# settings` or schedule
|
||||
blocks to a workflow to arrange recurring work. One generic pipeline can have
|
||||
many independent topics and timers. Read `references/schedules.md` when the user
|
||||
explicitly asks to manage a schedule; scheduling alone is not workflow authoring.
|
||||
|
||||
## Turns and artifacts
|
||||
|
||||
Each visit has `input.md` (the run prompt; later visits add the previous body), `memory.md`
|
||||
(short essential memory inlined in the prompt: task, previous stage's `# handoff`,
|
||||
outcome ledger, changed files; 4 KB cap), `history.md` (completed turns in full,
|
||||
read on demand), `output.md`, and `private/`. Every run also has one shared
|
||||
`public/` directory, addressed by its absolute path; files there are visible to
|
||||
later stages and are the stage's real artifacts. `input.md`, `memory.md`,
|
||||
`history.md`, `output.md` and `session.html` are system evidence for auditing, not stage
|
||||
output.
|
||||
|
||||
## References
|
||||
|
||||
Read only what the task needs:
|
||||
Read only what the task needs. Rows marked operator only are for setting up or
|
||||
debugging the server; skip them when authoring.
|
||||
|
||||
| Need | Reference |
|
||||
| --- | --- |
|
||||
| Definition syntax and turn files | `references/format.md` |
|
||||
| Definition syntax, turn files, output.md, checks, landing | `references/format.md` |
|
||||
| Good practices (tokens, checks, prompts, loops) | `references/practices.md` |
|
||||
| Limits, tools and permissions | `references/control.md` |
|
||||
| Loop limits | `references/control.md` |
|
||||
| Stages, parallel work, recovery | `references/orchestration.md` |
|
||||
| CLI, storage and evidence | `references/operations.md` |
|
||||
| Verification and accounting | `references/verification.md` |
|
||||
| Verification and evidence | `references/verification.md` |
|
||||
| Failures | `references/debugging.md` |
|
||||
| Delivery and installation | `references/delivery.md` |
|
||||
| Harnesses and models | `references/harnesses.md` |
|
||||
| Notifications | `references/notifications.md` |
|
||||
| Independent saved run schedules | `references/schedules.md` |
|
||||
| Saved run schedules (never in a definition) | `references/schedules.md` |
|
||||
| Transfer and versions | `references/transfer.md` |
|
||||
| Hosting, auth and deployment | `references/access.md` |
|
||||
| Queue and work items | `references/work.md`, `references/queue.md` |
|
||||
| CLI, storage and run evidence (operator only; not needed for authoring) | `references/operations.md` |
|
||||
| Harnesses and models (operator only) | `references/harnesses.md` |
|
||||
| Hosting, auth and deployment (operator only) | `references/access.md` |
|
||||
|
||||
Ask about real ambiguity, destructive actions or changes outside the request;
|
||||
routine steps of an authorized task do not need another confirmation.
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
# Runner-owned limits and stage memory
|
||||
# Runner-owned limits and loops
|
||||
|
||||
Use these blocks for bounded feedback loops. Never ask a model to count visits,
|
||||
increment revision counters, or enforce retry caps. Model verdicts concern current
|
||||
work only. Counters are runner audit data, not model input. A fresh session alone
|
||||
does not guarantee independent review: explicitly select its input fields.
|
||||
Use `limits` for bounded feedback loops. Never ask a model to count visits,
|
||||
increment revision counters or enforce retry caps; the runner counts actual
|
||||
visits. Model verdicts concern the current work only.
|
||||
|
||||
Put the cap on the reviewer, and leave the loop target without `limits` (it then
|
||||
has the runner's default of five visits) or give it a cap at least as large as the
|
||||
reviewer's:
|
||||
|
||||
````markdown
|
||||
# stage review
|
||||
|
||||
```limits
|
||||
{"max_visits":3,"on":["rejected"]}
|
||||
```
|
||||
@@ -18,48 +23,32 @@ error -> #failure
|
||||
```
|
||||
````
|
||||
|
||||
These blocks accompany a normal review stage's prompt/tools/configs.
|
||||
Pipelines counts actual visits, ignoring model-generated counters. On visit three,
|
||||
`approved` can succeed; `rejected` retains its verdict but routes to `#failure`.
|
||||
`rejected` is for a must-fix finding with a concrete location, evidence and remedy;
|
||||
`advisory` carries non-blocking notes and continues the run without another
|
||||
implementer visit, so an advisory-only review never buys a full re-implementation.
|
||||
An implementer may likewise declare `blocked_retryable -> implementer` for a
|
||||
transient environment, tooling or device problem a fresh attempt could survive,
|
||||
keeping `blocked -> #failure` for an acceptance criterion that genuinely cannot be
|
||||
met. `max_visits` caps every visit of that stage, including re-entries from the
|
||||
reviewer's `rejected` edge, so an implementer that is also the rejection target must
|
||||
use a cap at least as large as the reviewer's budget; the published pipelines use
|
||||
`{"max_visits":5,"on":["blocked_retryable"]}`. A lower implementer cap stops the loop
|
||||
before the reviewer's own cap fires, making the reviewer's `rejected` control
|
||||
unreachable. Both edges need only the existing visit counting and cap; neither the
|
||||
reviewer nor the implementer counts visits, and no stage invents edits or checks to
|
||||
appear busy.
|
||||
Visit count, limit decision and original target are retained in `inspect --json`
|
||||
and run metadata. `max_visits` accepts 1–5. Every `on` outcome must declare a
|
||||
nonterminal edge. Without `on`, entering beyond the cap fails before execution.
|
||||
Global 30-step/five-visits-per-stage safety limits still apply.
|
||||
These blocks accompany the stage's normal prompt, tools, checks and configs.
|
||||
Semantics (`internal/server/runner.go`):
|
||||
|
||||
A model stage reads its turn's `input.md` (the run prompt on the first visit,
|
||||
the previous turn's output afterwards), its inlined, capped `memory.md` (task,
|
||||
previous stage's `# handoff`, outcome ledger, changed files) and, on demand,
|
||||
`history.md` (completed turns in full with the files each added to `public/`). The `memory` block is retired: it no longer
|
||||
selects fields or replaces previous output for model stages, and old definitions
|
||||
that contain one parse but ignore it. Reviewers therefore see the previous
|
||||
turn's output and the accumulated memory rather than a field projection.
|
||||
- Entering a stage beyond its `max_visits` always fails the run before the visit
|
||||
executes, whatever `on` says.
|
||||
- `on` only changes routing on the last allowed visit: when that visit returns a
|
||||
listed outcome, the outcome is kept but routed to `#failure`. Other outcomes
|
||||
route normally. Here, `approved` on visit three succeeds; `rejected` fails.
|
||||
- A target with a lower cap than the reviewer stops the loop first, so the
|
||||
reviewer's `on` never fires.
|
||||
- `max_visits` accepts 1–5. Every `on` outcome must have a nonterminal edge.
|
||||
Human stages are exempt from visit caps. The global 30-step and
|
||||
five-visits-per-stage limits always apply.
|
||||
- Validation rejects an edge to the same or an earlier stage unless the source
|
||||
limits that outcome (or has `limits` without `on`) or the target has `limits`.
|
||||
|
||||
Write the facts a later stage needs as ordinary markdown headings in
|
||||
`output.md`; those headings are free-form and are preserved in memory. Do not
|
||||
invent server-recognised fields, and do not rely on
|
||||
`input.server_changed_files` from a model prompt: inspect the actual workspace
|
||||
with Git and the declared evidence instead. Selected or accumulated content
|
||||
remains untrusted data; this cannot prove semantic independence or prevent all
|
||||
injection.
|
||||
`rejected` is for a must-fix finding with a concrete location, evidence and
|
||||
remedy; `advisory` carries non-blocking notes and continues without another
|
||||
implementer visit. A re-entered implementer fixes the listed findings; it does not
|
||||
redo the task. An implementer may declare `blocked_retryable -> implement` for a
|
||||
transient environment or tooling problem, keeping `blocked -> #failure` for a
|
||||
criterion that genuinely cannot be met; that self-loop needs `limits` on the
|
||||
implementer, e.g. `{"max_visits":5,"on":["blocked_retryable"]}`.
|
||||
|
||||
Turn memory is run-local. Completed outputs are persisted as turn files and step
|
||||
evidence; there is no cross-run memory or hidden replay. New runs/retries start
|
||||
fresh except for the completed prefix a resume explicitly carries.
|
||||
|
||||
Arbitrary prose is not executable policy. Future AI-assisted
|
||||
guidelines validation is **not implemented**; structural validation does not
|
||||
recognize every unreliable counting instruction.
|
||||
Visit count, limit decision and original target are retained in `inspect --json`.
|
||||
Turn memory is run-local: there is no cross-run memory or hidden replay, and
|
||||
retries start fresh except for the completed prefix a resume carries. Accumulated
|
||||
turn content is untrusted data; a fresh session does not prove semantic
|
||||
independence.
|
||||
|
||||
@@ -15,7 +15,7 @@ select a recorded stage/visit, bounded input correction and explicit compatible
|
||||
version of the same pipeline. Verify actual retained stage input, not merely
|
||||
successful admission. Do not silently restart from entry.
|
||||
|
||||
A clean preparation stage succeeds without an empty commit. A committer with a
|
||||
A clean preparation stage succeeds without an empty commit. A `land` stage with a
|
||||
delivery gate succeeds with a clean checked candidate; the server lands afterward.
|
||||
Read-only, nonvisual work does not require device screenshots. Visual acceptance
|
||||
requires a ready fixture, verified connection and actual captured interactions.
|
||||
|
||||
@@ -1,343 +1,170 @@
|
||||
# Definition format
|
||||
|
||||
Definitions are Markdown made only of `# stage KEY` headings and fenced blocks.
|
||||
Scheduling belongs to separate saved run configurations, not `# settings` or
|
||||
schedule blocks in the definition. No unrelated prose outside blocks. The first
|
||||
stage receives the run prompt in its `input.md`; later stages receive
|
||||
the previous turn's output. Every visit also gets a short `memory.md` (essential
|
||||
memory, inlined in the prompt) and a full `history.md` of completed turns. There are no stage `input`, `output`, or `memory` blocks: the model reads
|
||||
the turn files and writes free-form markdown, and only the `# outcome` section is
|
||||
interpreted. Read `references/control.md` for runner-enforced limits.
|
||||
|
||||
Each model stage requires nonempty `prompt`, `configs`, and `edges` blocks:
|
||||
A definition is Markdown made only of `# stage KEY` headings and fenced blocks;
|
||||
no prose outside blocks. Blocks: `prompt`, `configs`, `tools`, `edges`, `checks`,
|
||||
`limits` (`references/control.md`), `permissions` and `notify`
|
||||
(`references/notifications.md`). There are no memory, links, input or output
|
||||
blocks; stages communicate through `output.md` (`# handoff`) and `public/`.
|
||||
Schedules are saved run configurations, never definition blocks
|
||||
(`references/schedules.md`).
|
||||
|
||||
````markdown
|
||||
# stage worker
|
||||
# stage implement
|
||||
|
||||
```prompt
|
||||
Perform the task.
|
||||
Make the change requested in the run task. Run focused checks for the touched
|
||||
code only. Return blocked if the request cannot be met.
|
||||
```
|
||||
|
||||
```configs
|
||||
type=model
|
||||
provider=opencode-go
|
||||
model=deepseek-v4.1-flash
|
||||
retries=2
|
||||
timeout=1h
|
||||
```
|
||||
|
||||
```tools
|
||||
```
|
||||
|
||||
```edges
|
||||
success -> #success
|
||||
blocked -> #failure
|
||||
error -> #failure
|
||||
```
|
||||
````
|
||||
|
||||
Stage/outcome keys begin with a letter and contain letters, digits, `_`, or `-`.
|
||||
Each outcome has exactly one target: another stage, `#success`, or `#failure`.
|
||||
An omitted error edge defaults to failure. Execution is sequential; forks choose
|
||||
one outcome, never execute parallel branches. Cycles are allowed; every stage
|
||||
must be reachable from entry and have a possible path to a terminal. Use bounded
|
||||
runner-owned `limits` for feedback loops, never revision counters in model
|
||||
prompts/outputs: publishing rejects an edge to the same or an earlier stage
|
||||
(definition order) unless the source stage limits that outcome or the target
|
||||
stage has `limits` (a human stage's choices are exempt). Read
|
||||
`references/control.md` for the exact format.
|
||||
|
||||
## Checks
|
||||
|
||||
Every model stage that can modify the workspace (it declares `edit` or `shell`,
|
||||
has no `tools` block, or sets `delivery=true`) must declare a `checks` block, so
|
||||
the author decides where verification happens. Read-only stages need none.
|
||||
|
||||
- A `delivery=true` stage lists 1–20 commands, one per line. The server runs them
|
||||
on the committed candidate (and again on a merged tree) before landing; a
|
||||
failing command blocks landing. These are the run's deterministic checks.
|
||||
- Every other stage writes exactly one line, `none: <reason>`, naming where
|
||||
verification happens instead, for example
|
||||
`none: focused tests only; the Land delivery gate runs the suite` or
|
||||
`none: read-only review; shell is used only for git diff`. Commands on a
|
||||
non-delivery stage are rejected because nothing would run them.
|
||||
|
||||
A check must fail when it cannot run. Validation rejects guards that pass
|
||||
without running anything: `|| true`, `|| :`, `|| exit 0`, `|| echo`,
|
||||
`[ ! -x tool ] || ...`, `test ! -f ... || ...`, `! command -v tool || ...`,
|
||||
an `if ...; then ...; fi` with no failing else, and `set +e`. Cover every
|
||||
language the run can change (a Go-only check list does not verify Flutter work).
|
||||
`pipelines validate` reports these with the remaining authoring rules
|
||||
(`references/practices.md`).
|
||||
|
||||
## Result format
|
||||
|
||||
Each visit gets `input.md`, `memory.md`, `history.md`, `output.md` and
|
||||
`private/` in `<run>/stages/<seq>-<stage>/`, plus the run-level shared
|
||||
`public/`. What a stage receives:
|
||||
|
||||
- `memory.md` (essential, hard cap 4 KB, always inlined in the prompt): the run
|
||||
task, the previous stage's handoff with its outcome and public/ additions, a
|
||||
one-line-per-stage outcome ledger (latest visit; a revisit is marked) and the
|
||||
server-captured changed files. A stage must be able to work from this alone.
|
||||
- `input.md` (on demand): the full run task and the previous turn's full output.
|
||||
- `history.md` (on demand, never inlined): every completed turn in full, the
|
||||
format `memory.md` had before essential memory existed.
|
||||
|
||||
Stages with `read` are told the two on-demand paths; stages without it get
|
||||
`input.md` inlined too (capped at 32 KB) but never `history.md`.
|
||||
|
||||
The previous stage's handoff is its `# handoff` section (any level, up to the
|
||||
next heading of the same or higher level); without one the server uses the head
|
||||
of its output, marked as such. Either is truncated to fit with a pointer to
|
||||
`input.md`, so nothing depends on the model following the convention. The
|
||||
prompt asks every stage to start its output with a short handoff: what it did,
|
||||
files changed, open issues or findings. The only required section is the
|
||||
outcome heading, followed by one of the stage's declared edge words:
|
||||
|
||||
````markdown
|
||||
# handoff
|
||||
|
||||
Fixed both review findings in cache.go; tests pass. No open issues.
|
||||
|
||||
# outcome
|
||||
|
||||
success
|
||||
|
||||
# Result
|
||||
|
||||
Free-form markdown for the next stage and the operator.
|
||||
````
|
||||
|
||||
`internal/turn.ParseOutput` matches the heading case-insensitively at any ATX
|
||||
level and ignores headings or words inside fenced code blocks. A missing,
|
||||
duplicate, empty or undeclared outcome fails the stage with
|
||||
`FINAL RESULT VALIDATION`. Every other heading is uninterpreted and preserved
|
||||
verbatim in the body. There is no JSON envelope, output schema, field selection
|
||||
or format-repair pass: do not ask for JSON fields or invent server sections.
|
||||
Keep turn-local files in `private/`; put reports, logs and screenshots for later
|
||||
stages in `public/`. Never fabricate a result.
|
||||
|
||||
Model configs may set `harness=opencode` or `harness=pi`; omission preserves
|
||||
historical OpenCode execution. Read `references/harnesses.md` for setup/limits.
|
||||
Use the exact provider/model IDs validated during server setup. New drafts/templates
|
||||
inherit that selection; do not assume the host's login matches the container's.
|
||||
Other OpenCode provider/model IDs are accepted syntactically; actual availability,
|
||||
authentication and inference require `pipelines provider-check` on the execution
|
||||
computer/container. Optional `retries` defaults to 2 (three total attempts),
|
||||
and accepts integers 0–100; explicit 0 disables execution retries. Optional
|
||||
`timeout` defaults to `1h` per attempt and accepts positive durations such as
|
||||
`30m` or `90s`.
|
||||
|
||||
A stage that declares a `skipped` edge (for example a UI reviewer) is eligible
|
||||
for the server skip fallback: when every earlier visit's file-change capture was
|
||||
complete and recorded no changed files or screenshots, the server completes the
|
||||
stage as `skipped` without a model call (`reason_source: fallback`). Any recorded
|
||||
change or incomplete capture runs the model. Optional `auto_skip=false` (values
|
||||
`true`/`false`, default `true`) always runs the model. Read
|
||||
`references/operations.md` ("Server skip fallback") for the exact rule.
|
||||
|
||||
Optional `reasoning_effort=high` selects a model's reasoning/thinking level for
|
||||
this stage. Values are `default`, `none` (alias `off`), `minimal`, `low`, `medium`,
|
||||
`high`, `xhigh`, or `max`. Omission or `default` preserves the harness/model
|
||||
default; never add a level automatically to existing workflows. Availability is
|
||||
model-specific: OpenCode V2 selects the corresponding named model variant and
|
||||
rejects unavailable variants; Pi uses its native thinking level (`none` becomes
|
||||
`off`) and verifies the effective level before prompting, rejecting unsupported
|
||||
or silently clamped levels. Explicit effort on OpenCode V1 is unsupported.
|
||||
Do not instruct the agent to “think harder” as a substitute for this native
|
||||
setting. Effort is not a token budget or a guaranteed quality improvement; keep
|
||||
usage budgets/provider spending limits separate. Both text-only and tool-using
|
||||
model stages honor the setting, without changing their tools or permissions.
|
||||
|
||||
Automatic retries cover retryable transport failures and
|
||||
timeouts; an explicit read-only stage may additionally retry a post-admission
|
||||
deadline or transport failure, while a stage with `edit`/`shell` (or no tools
|
||||
block) never replays admitted work. Optional `budget_usd=1.50` and
|
||||
`budget_tokens=50000` independently cap reported usage.
|
||||
They accumulate across attempts and repeated visits
|
||||
to this stage in the same run. Omit either budget to leave that resource unlimited.
|
||||
Both budgets must be positive; tokens include input, output, reasoning and cache
|
||||
reads/writes. Accounting failures stop budget-limited stages. In-flight requests
|
||||
may exceed limits before usage is reported; these are not provider-side hard caps.
|
||||
In current source, all OpenCode V2 text-only calls use tracked prompting with
|
||||
all tools denied, rather than the usage-less `/generate` path. Completed
|
||||
token-budgeted responses reporting zero tokens fail closed. This is an execution
|
||||
implementation detail, not a new Markdown field: never add backend/format/API
|
||||
settings to definitions. Read `references/verification.md` for the dated live
|
||||
accounting result and its provider/deployment limitations when relevant.
|
||||
The run has no additional wall-clock deadline; step/visit caps and cancellation
|
||||
still apply. A native ntfy stage accepts `timeout` but never automatically retries
|
||||
delivery and has no model budget.
|
||||
|
||||
Explicit empty `tools` denies all; allowed tools are `read`, `glob`,
|
||||
`grep`, `webfetch`, `websearch`, `edit`, `shell`, one per line. No wildcards,
|
||||
delegation, or browser tools. Tools are per-stage and automatically enabled by
|
||||
starting a run. `edit` supports any file type and creating files/directories in the
|
||||
run directory; no file list is required. `shell` supports commands appropriate to
|
||||
the repository's language and tooling, not a language-specific allowlist. A missing
|
||||
tools block uses local read/glob/grep/edit/shell tools; prefer explicit stage roles.
|
||||
Planning stages must not label a task blocked simply because they defer edits to
|
||||
an implementer stage. The intended role split: an implementer may
|
||||
read/glob/grep/edit/shell; a planner reads, plus conditional web search/fetch only
|
||||
when external knowledge is needed; a reviewer reads and may run read-only checks
|
||||
with shell but never edits; a source-only review stage has no edit/shell; a
|
||||
research stage uses web search/fetch; a committer has read/glob/grep and shell but
|
||||
no edit. The bundled examples (`one-stage`, `two-stages`, `feedback-loop`,
|
||||
`human-decision`) follow it without web tools. Never force web access for
|
||||
trivial, fully understood local edits, and return blocked only when required
|
||||
reliable evidence cannot be obtained rather than whenever one search tool fails.
|
||||
Keep examples and prompts language-neutral unless the task itself asks for a
|
||||
specific language.
|
||||
|
||||
Optional `permissions` is a JSON object with `edit` (run-directory-relative
|
||||
patterns) and `shell` (exact commands) arrays. It narrows declared tools and any
|
||||
run-level file scope; it never enables a missing tool. Empty arrays deny actions.
|
||||
Nulls, duplicate/unknown keys, traversal and shell wildcards/chaining/redirects or
|
||||
expansion are rejected. If edits are restricted and shell is enabled, provide an
|
||||
explicit shell list too. Example: `{"edit":["README.md","plugin/*.qml"],"shell":["bash -n install.sh"]}`.
|
||||
Use this block for intentional file/command restrictions; prose is not a runtime
|
||||
permission policy. Keep ordinary workflows language-neutral and unrestricted
|
||||
within their run directory unless the user asks for a narrower scope.
|
||||
|
||||
## Human (decision) stages
|
||||
|
||||
A `type=human` stage pauses its run until a person chooses one of its outgoing
|
||||
options. It has no model, agent, tools or configs: it needs a nonempty `prompt`
|
||||
block (the operator instructions) and at least one non-`error` `edges` outcome.
|
||||
Its outcomes are exactly its non-error edges, so a draft can express choices such
|
||||
as `deploy`, `revise` or `abort`; the mandatory error edge is a failure route,
|
||||
not a choice. It validates, renders, exports and imports like every other type.
|
||||
|
||||
````markdown
|
||||
# stage gate
|
||||
|
||||
```prompt
|
||||
Review the screenshots and artifacts above, then choose where the run goes.
|
||||
```
|
||||
|
||||
```edges
|
||||
deploy -> publish
|
||||
revise -> worker
|
||||
abort -> #failure
|
||||
error -> #failure
|
||||
```
|
||||
````
|
||||
|
||||
When a run reaches a human stage its step becomes `waiting`, the run reports it is
|
||||
waiting on a person, and nothing later runs. Decide from the TUI (open the run,
|
||||
press `c`, pick an option, optionally type feedback, Enter) or the CLI:
|
||||
`pipelines decide <run-id> [stage] --option KEY [--feedback TEXT]`. The decision
|
||||
and feedback are recorded as the step's evidence (the feedback stays in the step
|
||||
output, never an error diagnostic) and the run resumes along that
|
||||
edge. The wait is persisted, so it survives a server restart. An unknown option is
|
||||
refused with the valid list; a decision to a stage that is not waiting is refused;
|
||||
a repeated decision for an already-decided visit does not double-apply. Cancel and
|
||||
retry keep working while waiting and after a decision, and a human stage is never
|
||||
killed by a stage timeout or `max_visits` (the shared run budget still applies).
|
||||
|
||||
## Committer stages
|
||||
|
||||
A workflow that changes the repository ends in a committer: a normal model stage
|
||||
whose prompt commits the run's change locally and never pushes, tags, deploys or
|
||||
rewrites history. It is not a native type. Declare `shell` (and `read`/`glob`/`grep`
|
||||
when it inspects content) and no `edit`, so a failed check is reported rather than
|
||||
patched, and select the runner's captured change set alongside the producer's
|
||||
declared list:
|
||||
|
||||
````markdown
|
||||
# stage committer
|
||||
|
||||
```prompt
|
||||
Commit this run's change locally; never push.
|
||||
provider=default
|
||||
model=default
|
||||
```
|
||||
|
||||
```tools
|
||||
read
|
||||
glob
|
||||
grep
|
||||
edit
|
||||
shell
|
||||
```
|
||||
|
||||
```checks
|
||||
none: commits locally; the manager lands and runs the suite
|
||||
none: focused tests only; the land delivery gate runs the suite
|
||||
```
|
||||
|
||||
```edges
|
||||
success -> #success
|
||||
success -> review
|
||||
blocked -> #failure
|
||||
error -> #failure
|
||||
```
|
||||
````
|
||||
|
||||
A landing committer instead sets `delivery=true` in `configs` and lists the
|
||||
repository's required check commands in `checks`; the server runs them, so its
|
||||
prompt does not run the suite again.
|
||||
Stage and outcome keys begin with a letter and contain letters, digits, `_` or
|
||||
`-`. Name roles `plan`, `implement`, `review`, `ui-review`, `decide` (human) and
|
||||
`land`; other jobs take a plain verb (`draft`, `check`, `act`). Each outcome has exactly one target: another
|
||||
stage, `#success` or `#failure`. An omitted `error` edge defaults to `#failure`
|
||||
for every stage type, human included. Execution is sequential; an outcome picks
|
||||
one edge, never parallel branches. Every stage must be reachable from the entry
|
||||
and have a path to a terminal; loops need runner-owned `limits`.
|
||||
|
||||
State this pre-commit checklist in the prompt:
|
||||
## Turn files
|
||||
|
||||
1. Inspect `git status` and the full diff. Stage this run's change, including
|
||||
inherited work, restricted to paths that still exist, and keep every
|
||||
exclusion. Do not rely on a model-declared file list.
|
||||
2. Inspect the staged diff before committing; never commit unrelated work, secrets
|
||||
or generated artifacts.
|
||||
3. Without a delivery gate, run the repository's required checks once (summary
|
||||
output) and record the exact commands and observed results as evidence. Do not
|
||||
commit when a required check fails; return `blocked` with the failing output
|
||||
instead. With `delivery=true` the server runs the `checks` block; skip this step.
|
||||
4. Create no empty commit: when nothing is staged, return `success` with a clear
|
||||
note.
|
||||
5. Commit with a concise message naming the run's change and report the commit hash;
|
||||
never push, tag, rebase or amend.
|
||||
6. Ensure the item's per-item documentation note (`docs/notes/<slug>.md`, written when
|
||||
the change makes a shared document stale) is staged with the change. Never edit a
|
||||
shared document (a README or topic doc, or a packaged skill reference) here: a single
|
||||
documentation writer folds notes later (`references/orchestration.md`). An item that
|
||||
staled no shared document has no note and needs no step 6.
|
||||
Each visit gets `input.md`, `memory.md`, `history.md`, `output.md` and `private/`
|
||||
in `<run>/stages/<seq>-<stage>/`, plus the run-level shared `public/` addressed by
|
||||
its absolute path.
|
||||
|
||||
This list is the canonical checklist. Repeat it in each committer prompt, because a
|
||||
definition is published on its own; keep those copies in sync with this list. The
|
||||
bundled examples leave changes uncommitted and have no committer.
|
||||
- `memory.md` (capped at 4 KB, always inlined in the prompt): the run task, the
|
||||
previous stage's `# handoff` with its outcome and `public/` additions, a
|
||||
one-line-per-stage outcome ledger and the server-captured changed files.
|
||||
- `input.md`: the run task, plus on later turns the previous turn's full output
|
||||
(`internal/turn/render.go`). A stage with `read` reads it on demand; a stage
|
||||
without `read` gets it inlined (up to 32 KB) on every call, so give `read` to
|
||||
stages that follow a large output.
|
||||
- `history.md`: every completed turn in full; on demand for stages with `read`,
|
||||
never inlined.
|
||||
|
||||
The committer prepares the complete clean candidate and resolves integration conflicts.
|
||||
When `delivery=true`, the server executes the declared checks and lands after the
|
||||
stage returns success; do not run the post-landing ancestry check beforehand.
|
||||
If canonical `main` moved meanwhile, the server merges it into the item (a normal
|
||||
merge commit), re-runs the checks on the merged tree and lands that; a conflicting
|
||||
catch-up is aborted with the workspace unchanged and the conflicting files named.
|
||||
Without a delivery gate, the original workflow or manager completes the merge
|
||||
and verifies ancestry. No separate merger pipeline is required (`references/queue.md`).
|
||||
Keep turn-local files in `private/`; put reports, logs and screenshots for later
|
||||
stages in `public/`. `input.md`, `memory.md`, `history.md`, `output.md` and
|
||||
`session.html` are server-written evidence, not stage output.
|
||||
|
||||
A stage returns the markdown above, never the retired
|
||||
`{ "outcome": ..., "output": ... }` envelope. Do not treat tool-call/DSML markup,
|
||||
plain prose, or a missing outcome as success, and never rerun tool work merely to
|
||||
fix a response's formatting.
|
||||
## output.md
|
||||
|
||||
## Scheduling
|
||||
The stage returns the complete `output.md` in this order: `# outcome`, the edge
|
||||
word alone on the next non-blank line, then `# handoff` (a few lines: what it did,
|
||||
files changed, open findings), then any detail:
|
||||
|
||||
Publishing a workflow does not schedule it. Create a separate saved schedule
|
||||
with an explicit pipeline/version and complete run information (prompt, server
|
||||
workspace, model overrides, notifications, output format and write restrictions
|
||||
when used). Multiple saved schedules can reuse one workflow with different
|
||||
research items or timers; updates and deletion do not rewrite the definition.
|
||||
Use exactly one timing rule: `times` (24-hour clock values) or `every` (a duration
|
||||
of at least one minute), with optional weekdays and timezone.
|
||||
````markdown
|
||||
# outcome
|
||||
|
||||
New definitions with embedded schedules are rejected. Historical immutable
|
||||
versions remain readable; their latest legacy slots migrate into separate saved
|
||||
schedules without changing source/history or replaying old occurrences. Never
|
||||
reintroduce legacy scheduling blocks or make a new workflow just to set another
|
||||
topic/timer. Warn explicitly before enabling unattended runs. See
|
||||
`references/schedules.md` for the API and retry/concurrency guards.
|
||||
success
|
||||
|
||||
## Notifications
|
||||
# handoff
|
||||
|
||||
Stage hooks use the global phone connection, never store credentials. Optional
|
||||
fenced `notify` block: `on=success,failure` (add `waiting` to fire when a human
|
||||
stage parks the run), `artifacts=markdown`.
|
||||
Read this skill's `references/notifications.md` for the minimal terminal-path
|
||||
recipe and optional native publisher format. Prefer hooks for ordinary success/error
|
||||
alerts; a request to notify is not a request for a new delivery stage. Do not scan
|
||||
the user's repository for Pipelines docs. Bundled examples are generic and manual-only.
|
||||
Notification attachment/report support and model availability need real runtime
|
||||
tests; structural validation alone cannot confirm delivery.
|
||||
Fixed both review findings in cache.go; focused tests pass. No open issues.
|
||||
|
||||
Free-form detail for the operator and later stages.
|
||||
````
|
||||
|
||||
The outcome heading matches case-insensitively at any ATX level, outside fenced
|
||||
code. Never add a second heading named Outcome. If the outcome is missing, the
|
||||
runner sends one repair follow-up asking only for it; a still-missing, duplicate,
|
||||
empty or undeclared outcome fails with `FINAL RESULT VALIDATION`. Every other
|
||||
heading is free-form and preserved. The handoff is the `# handoff` section (or,
|
||||
without one, the head of the output, marked as such), truncated to fit memory.
|
||||
The `{ "outcome": ..., "output": ... }` JSON envelope and its format-repair ladder
|
||||
are retired: never ask for JSON fields or invent server sections. Missing tools or
|
||||
evidence route to `blocked` when declared, else `error`; never fabricate a result.
|
||||
|
||||
## Configs
|
||||
|
||||
- `provider`/`model`: exact IDs validated during server setup. Templates
|
||||
(`pipelines new NAME --template KEY`) fill them in; copy those IDs into a
|
||||
hand-written draft. Availability needs `pipelines provider-check`.
|
||||
- `harness=opencode|pi`; omission means OpenCode (`references/harnesses.md`).
|
||||
- `retries` 0–100, default 2 (three attempts). Retries cover transport failures
|
||||
and timeouts; a stage with `edit`/`shell` (or no tools block) never replays
|
||||
admitted work.
|
||||
- `timeout` per attempt, default `1h` (e.g. `30m`, `90s`).
|
||||
- `reasoning_effort`: `default`, `none` (`off`), `minimal`, `low`, `medium`,
|
||||
`high`, `xhigh`, `max`. Omit unless asked; unsupported levels are rejected.
|
||||
Do not prompt "think harder" instead.
|
||||
- `budget_usd`, `budget_tokens`: positive caps on reported usage, shared across
|
||||
attempts and visits of this stage; totals include cache reads and writes. Set
|
||||
only when the user asks or a measured visit exists (`references/practices.md`).
|
||||
- `delivery=true`: the landing stage's gate (below). OpenCode harness only.
|
||||
- `auto_skip=false`: disable the server skip fallback for a stage that declares a
|
||||
`skipped` edge (`references/operations.md`, Server skip fallback).
|
||||
|
||||
## Tools and permissions
|
||||
|
||||
`tools` lists `read`, `glob`, `grep`, `webfetch`, `websearch`, `edit`, `shell`,
|
||||
one per line; an empty block denies all and a missing block grants local
|
||||
read/glob/grep/edit/shell (prefer explicit). Roles: `plan` reads (web only when
|
||||
external knowledge is needed); `implement` reads, edits and runs shell; `review`
|
||||
reads and may use shell for `git diff`, never edits; `land` has read/glob/grep and
|
||||
shell, no edit. Planning stages do not return `blocked` merely because edits come
|
||||
later. Keep prompts language-neutral unless the task names a language.
|
||||
|
||||
Optional `permissions` is a JSON object with `edit` (run-directory-relative
|
||||
patterns) and `shell` (exact commands) arrays. It narrows declared tools; it never
|
||||
enables a missing one. Empty arrays deny; wildcards, chaining, redirects and
|
||||
traversal are rejected. With restricted edits and shell enabled, list shell too:
|
||||
`{"edit":["README.md","plugin/*.qml"],"shell":["bash -n install.sh"]}`.
|
||||
|
||||
## Checks
|
||||
|
||||
Every model stage that can modify the workspace (`edit` or `shell`, no `tools`
|
||||
block, or `delivery=true`) declares `checks`. A `delivery=true` stage lists 1–20
|
||||
commands, one per line. Every other stage writes exactly one line,
|
||||
`none: <reason>`, e.g. `none: read-only review; shell is used only for git diff`.
|
||||
A check must fail when it cannot run: validation rejects `|| true`, `|| :`,
|
||||
`|| exit 0`, `|| echo`, negated file tests, `! command -v tool ||`, an `if` with no
|
||||
failing else, and `set +e`. Cover every language the run can change.
|
||||
|
||||
## Human stages
|
||||
|
||||
`type=human` pauses the run until a person picks one of its non-`error` edges
|
||||
(e.g. `approve -> act`, `revise -> plan`, `abort -> #failure`; see the
|
||||
`human-decision` template). It needs only `prompt` (operator instructions),
|
||||
`configs` with `type=human` and `edges`; no model or tools. Decide with
|
||||
`pipelines decide RUN_ID [STAGE] --option KEY [--feedback TEXT]` or the apps. The
|
||||
wait survives restarts and is never killed by a stage timeout or `max_visits`.
|
||||
|
||||
## Land stage and delivery gate
|
||||
|
||||
A workflow that commits its change ends in `land`: a model stage with
|
||||
`read`/`glob`/`grep`/`shell`, no `edit`, that commits locally. Its prompt states:
|
||||
|
||||
1. Commit only this run's changes (server list + git status).
|
||||
2. Check the staged diff for unrelated work and secrets.
|
||||
3. No empty commit, push, amend or rebase. Report the hash.
|
||||
|
||||
With `delivery=true` and `checks` commands, after `land` succeeds the server:
|
||||
requires the item and main checkouts to be clean, untracked files included; merges
|
||||
a moved `main` into the item (a conflict aborts with the workspace unchanged);
|
||||
runs each check with `sh -c` from the root of a fresh detached worktree of the
|
||||
commit, so include dependency bootstrap (`npm ci && npm test`); caps each check's
|
||||
output at 64 KB; and fast-forwards `main`. A failing check blocks landing. Do not
|
||||
check ancestry before the gate runs. Without a gate, the manager merges and
|
||||
verifies ancestry; no separate merger pipeline is needed (`references/queue.md`).
|
||||
|
||||
@@ -1,191 +1,45 @@
|
||||
# Configured harnesses
|
||||
# Configured harnesses (operator only)
|
||||
|
||||
Current source supports OpenCode and Pi interactive authoring, debugging and
|
||||
model stages. nvim remains a file editor/read-only debug packet viewer, not a
|
||||
stage backend. Only configured harnesses appear in TUI pickers, default first.
|
||||
CLI `new/edit/debug --harness opencode|pi|nvim` overrides the default.
|
||||
Authoring needs only the `harness=` config (`references/format.md`). The rest is
|
||||
for an operator setting up or debugging the execution computer.
|
||||
|
||||
OpenCode and Pi run interactive authoring, debugging and model stages; Claude Code
|
||||
and Codex are chat backends. nvim is a file editor/read-only debug packet viewer,
|
||||
not a stage backend. Only configured harnesses appear in pickers, default first.
|
||||
CLI `new/edit/debug --harness opencode|pi|nvim` overrides the default, and
|
||||
`pipelines harness` launches the configured default in the invocation directory.
|
||||
|
||||
CLI `new/edit/debug` and TUI author/debug build one shared, Docker-aware CLI
|
||||
prefix. A Docker-linked profile embeds the in-container `/usr/local/bin/pipelines`
|
||||
(never the host binary: Linux can see but cannot execute a mounted macOS host
|
||||
executable). Loopback is recognized only via `net.ParseIP(...).IsLoopback()` plus
|
||||
an exact case-insensitive `localhost`, so a remote DNS name such as
|
||||
`127.remote.example` stays remote and runs no probe. For a loopback logical
|
||||
`--url`, a single bounded `docker compose port` probe parses each returned line
|
||||
as a complete `host:port` binding (IPv4/IPv6/wildcard) and the prefix maps only
|
||||
when a binding covers the requested loopback address and port; malformed or
|
||||
non-matching bindings fail closed. A loopback `wss` endpoint is rejected as
|
||||
having no supported local TLS mapping rather than downgraded to plaintext. On a
|
||||
match it embeds scoped `PIPELINES_TRANSPORT_SOURCE`/`PIPELINES_TRANSPORT_TARGET`
|
||||
aliases in the prefix string (not `cmd.Env`, so the pre-existing harness
|
||||
server's tools inherit them). The client rewrites an endpoint only on an exact
|
||||
source match; remote endpoints stay remote. Draft metadata (server, base
|
||||
version, create-only guard) is preserved unchanged.
|
||||
`provider-check` with a Docker-linked profile runs inside that profile's existing
|
||||
Compose service with `-T`, the original cwd/model/timeout and exact stdout/exit
|
||||
propagation; container profiles have no `DockerDirectory` and cannot recurse, and
|
||||
explicit native profiles keep the host path. The picker uses a true `x/term` TTY
|
||||
check, so `/dev/null` and pipes fail fast instead of hanging.
|
||||
|
||||
`pipelines harness setup <claude|codex|opencode|pi>` is the single setup path on
|
||||
native and Docker installations: it checks the CLI is reachable from the server
|
||||
(printing the install command when missing), signs in only when needed (`--login`
|
||||
forces it) and verifies with the server's own check. Claude Code and Codex have
|
||||
no models to pick. For OpenCode/Pi it runs a styled onboarding
|
||||
wizard (bold title, verified Docker harness versions) and selects separate
|
||||
harness and new-pipeline models using type-to-filter fuzzy pickers, not numbered
|
||||
prompts. Each picker itself shows its numbered step with the title and purpose
|
||||
(no duplicate numbered prose is printed outside it); the interactive harness
|
||||
model is `Step 1 of 2` and the new-pipeline model is `Step 2 of 2`, matching
|
||||
fresh Docker setup. Typing filters, arrow keys or Tab navigate, Enter confirms,
|
||||
and Esc cancels. The saved value is preselected and marked `(current)`; the window
|
||||
follows the active row on narrow terminals and the heading, selected row and
|
||||
no-match state stay within the real terminal height.
|
||||
Automation: `--model PROVIDER/MODEL --pipeline-model PROVIDER/MODEL`.
|
||||
`harness default [opencode|pi]` switches defaults; `harness list` shows installed /
|
||||
signed-in / configured / default status for all four (`--json` adds profiles).
|
||||
Setup reads catalogs/login state without inference. Changing defaults never
|
||||
rewrites published definitions or retained drafts. Canceling or failing either
|
||||
picker leaves saved preferences byte-identical. Non-terminal automation with a
|
||||
missing model fails clearly instead of starting a picker.
|
||||
forces it) and verifies with the server's own check. For OpenCode/Pi it picks the
|
||||
interactive harness model and the new-pipeline model; automation passes
|
||||
`--model PROVIDER/MODEL --pipeline-model PROVIDER/MODEL`, and a non-terminal run
|
||||
with a missing model fails instead of opening a picker. `harness default
|
||||
[opencode|pi]` switches defaults; `harness list` shows installed / signed-in /
|
||||
configured / default status (`--json` adds profiles). Setup reads catalogs and
|
||||
login state without inference. Changing defaults never rewrites published
|
||||
definitions or drafts; canceling a picker leaves saved preferences unchanged.
|
||||
Harness setup/default commands configure the local execution user, not a remote
|
||||
`--url` server. Pi setup runs no generation check; `provider-check --harness pi
|
||||
--model PROVIDER/MODEL` is a separate, potentially chargeable check.
|
||||
|
||||
On an existing Docker setup, `pipelines harness setup opencode|pi` configures inside its container and link the host
|
||||
launcher without restarting/deploying services. They resolve and verify the current
|
||||
OpenCode V2 release from the single documented complete
|
||||
`GET .../git/matching-refs/tags/v2` response (a bounded complete array with no
|
||||
pagination parameters; never the moving `latest` tag or the registry's insertion
|
||||
order) and the current Pi release from npm, resolve the selected tag's immutable
|
||||
GHCR `sha256` digest, and require a rebuild only when the verified release image
|
||||
identity no longer matches. A full install always invokes the Docker build for a
|
||||
dirty source (relying on layer caching); a model/login-only reconfiguration does
|
||||
not rebuild. Setup writes its generated release receipt to the machine-local root
|
||||
file `.pipelines-releases.json` (with atomic `.pipelines-releases-*` temporary
|
||||
files); both are git-ignored, so a fresh clean checkout stays clean and reinstall
|
||||
still reuses the verified release/image identity. The actual built image's labels
|
||||
stay authoritative and a dirty source still rebuilds. If discovery, digest
|
||||
resolution or completeness cannot be established they stop with an error (keeping
|
||||
the last verified Dockerfile fallback pins); they never guess, fall back to a host
|
||||
runtime or install a stale pin.
|
||||
(`./install --opencode`/`--pi` were removed and only point to that command.) The native installation reuses the host OpenCode
|
||||
configuration, writes **host** (non-Docker) profiles for the harnesses it detects
|
||||
into XDG config home's `pipelines/harnesses.json`, and clears a stale Docker link
|
||||
so a Docker-era profile is not left Docker-linked. It never installs, upgrades or
|
||||
downloads a harness. Container and host credentials and accounts remain independent and
|
||||
are reused on reinstall/reconfigure. `./install` installs natively by default
|
||||
(`--native` is equivalent); explicit `./install --docker` selects the Docker
|
||||
server + ntfy installation.
|
||||
Fresh setup accepts `./install --docker --harness pi` to skip OpenCode
|
||||
login. Existing setup can configure either harness independently. Pi setup does
|
||||
not run a generation check; `provider-check --harness pi --model PROVIDER/MODEL`
|
||||
is a separate confirmed, potentially chargeable check.
|
||||
Profiles live in XDG config home's `pipelines/harnesses.json`, store preferences
|
||||
only, and preserve unrelated configuration. A profile launches a native host
|
||||
harness unless it records a Docker checkout directory, in which case it launches
|
||||
through that installation's Compose service. `harness setup <opencode|pi> --host`
|
||||
configures or converts a host profile and clears any Docker link; without
|
||||
`--host`, reconfiguring a Docker-linked profile re-verifies that the recorded
|
||||
directory still has `compose.yaml` and fails naming it rather than saving a link
|
||||
that cannot launch. Setup's summary and `harness list` report the resulting mode
|
||||
(and, for Docker, the directory). Harness CLI setup/default commands
|
||||
always configure the local execution user, not a remote `--url` server.
|
||||
OpenCode interactive launch uses
|
||||
`mini --model`: verified V2 full CLI lacks a model flag. Pi uses its CLI with the
|
||||
packaged skill explicitly loaded. Pi has no standalone `auth login` command
|
||||
(verified `pi auth --help`: only credential check/print): setup prints a
|
||||
title-plus-three-lines Pi sign-in handoff and opens a fresh, no-context,
|
||||
extension-free, no-prompt-templates, no-tools session with Pi's documented
|
||||
`--tui-mode regular` flag, so the header and handoff stay visible above Pi's
|
||||
interface at 80x30 instead of the default fullscreen clear. The user types
|
||||
`/login`, chooses an authentication method ("Sign in with an API key" for
|
||||
provider API credentials; account and Radius remain the user's choice), finishes
|
||||
provider sign-in, then exits with `ctrl+c`/`ctrl+d` to return to Pipelines model
|
||||
configuration. Existing sign-ins are reused by exiting when no new sign-in is
|
||||
needed; completing a sign-in can update the credentials stored in the Docker
|
||||
volume. No coding/task prompt or scripted keystrokes are injected.
|
||||
OpenCode V2 model stages settle long sessions by polling `GET /api/session/active`
|
||||
about once per second under the stage context, instead of the `.../wait` request
|
||||
whose CLI-side request timeout failed long stages after roughly six minutes
|
||||
(HTTP 499, dated 2026-10-10). V1 native settlement is unchanged. Final results
|
||||
and debug session evidence use the same single bounded latest-assistant readback
|
||||
(`order=desc&limit=1&type=assistant`), so a small final result is visible behind
|
||||
large prior tool content; retained transcripts over 32 KiB are stored as valid
|
||||
bounded preview envelopes that preserve stage limits and diagnostics, and exact
|
||||
outputs remain authoritative. Pi evidence is JSONL and is not parsed as one JSON
|
||||
object. Short-lived context-controlled CLI calls (`--help` detection, API calls
|
||||
and the native export fallback) share one finite inherited-pipe drain grace
|
||||
(`os/exec.Cmd.WaitDelay` ~250ms) so a canceled context or a descendant that still
|
||||
holds stdout/stderr cannot block for the child's natural lifetime; the
|
||||
`context.Canceled`/`DeadlineExceeded` and `exec.ErrWaitDelay` identities are
|
||||
preserved and partial stdout after the grace is never accepted as success.
|
||||
Docker image resolution, profile storage, the Docker-aware CLI prefix, Pi sign-in
|
||||
and OpenCode V2 settlement internals are in `docs/operator-reference.md` in the
|
||||
source checkout; dated verification is in `docs/verification-history.md`.
|
||||
|
||||
Stages set `harness=pi` or `harness=opencode` in configs; omission preserves
|
||||
historical OpenCode behavior. Pi must be configured on the execution server.
|
||||
A run may override the pinned model at launch: `pipelines run --model
|
||||
PROVIDER/MODEL` covers every model stage and repeatable `--stage-model
|
||||
STAGE=PROVIDER/MODEL` overrides one stage, with precedence **per-stage >
|
||||
whole-run > the model pinned in the pipeline version**. Overrides are validated
|
||||
against the pinned version before the run starts (an unknown stage key fails
|
||||
naming the valid keys; a non-model stage cannot be overridden), apply to that run
|
||||
only, and are recorded on the run so `retry` reproduces them. The effective
|
||||
provider/model is stamped on the step and exposed by the run's graph, and the
|
||||
stage's repair/formatter path uses the same resolved model because both read the
|
||||
stage's effective configs.
|
||||
The RPC adapter starts new retained sessions per visit with explicit tools and
|
||||
a Pipelines permission extension. Ambient extensions, skills, context files,
|
||||
prompt templates, automatic retries/compaction are disabled. Tool gates are not
|
||||
an OS sandbox: approved shell can have broad side effects. Usage accumulates
|
||||
across attempts/visits; missing required accounting fails closed. Strict envelope
|
||||
validation and bounded syntax-only repair never replay tools. A model-driven
|
||||
repair, when needed, uses a separate deny-all session with retained ownership
|
||||
and the same cumulative budget. Its output is checked locally for value changes.
|
||||
STAGE=PROVIDER/MODEL` overrides one stage (per-stage > whole-run > pinned).
|
||||
Overrides are validated against the pinned version before the run starts (an
|
||||
unknown stage key fails naming the valid keys; a non-model stage cannot be
|
||||
overridden), apply to that run only, and are recorded so `retry` reproduces them.
|
||||
The effective provider/model is stamped on the step.
|
||||
|
||||
Session/RPC audit lives in the execution user's cache under `pipelines/pi-sessions`.
|
||||
Ownership persists before prompts. Cleanup waits for settlement; expired leases
|
||||
request abort through the extension. Unconfirmed cleanup blocks continuation/retry.
|
||||
|
||||
2026-10-10 verification: `scripts/check-pi-rpc.py` checked installed Pi 0.87.1
|
||||
readiness, metadata/config RPC and **idle** lease cleanup without inference.
|
||||
Go RPC fixtures cover result/budget/missing-accounting/timeout/format-repair paths;
|
||||
Node tests cover permission checks. `PIPELINES_PI_LOCAL=1 go test ./internal/stage
|
||||
-run TestPiInstalledLocalIntegration` also exercised the **installed Pi** against
|
||||
scripted loopback responses: actual allowed/denied writes, budget exhaustion,
|
||||
and active-shell cancellation (including no delayed write after cleanup).
|
||||
The installed-Pi loopback suite also covers isolated deny-all syntax repair with
|
||||
retained ownership and cumulative usage. The compiled fuzzy picker passed PTY
|
||||
selection/cancellation; installed Pi catalog discovery/configuration passed with
|
||||
an unreachable local endpoint and no prompt.
|
||||
This is integration evidence, not real-provider/model evidence. Active lease-expiry
|
||||
recovery, web search delivery and fresh interactive Docker Pi setup remain
|
||||
**not tested**. Pi catalogs exclude ambient provider extensions for parity with
|
||||
audited stages; compatible custom endpoints can use Pi's private `models.json`.
|
||||
An isolated built Docker image also passed non-root, network-disabled policy/RPC
|
||||
readiness and idle lease cleanup, without credentials/mounts or inference.
|
||||
Codex/Claude
|
||||
adapters and AI-assisted server guidelines validation remain future work.
|
||||
|
||||
2026-10-11 source-level coverage (source-level regressions; the separate dated
|
||||
2026-10-11 final install is live evidence): the
|
||||
shared Docker-aware CLI prefix, exact-match client transport rewriting,
|
||||
Docker-linked `provider-check` dispatch and the `x/term` TTY check have
|
||||
source-level regression tests (`internal/harness`, `internal/client`,
|
||||
`internal/cli`). The prefix tests cover the remote-DNS negative
|
||||
(`127.remote.example` stays unmapped with no probe), IPv4/IPv6/`localhost`
|
||||
positives, complete `host:port` binding matching (a different service on the same
|
||||
port, or a family mismatch, does not match), malformed/empty output failing
|
||||
clearly, multiple bindings under a single probe, and loopback `wss` rejection.
|
||||
Terminal Pick regression tests use a short deadline so a TTY regression fails
|
||||
fast rather than hanging. Docker dispatch tests assert `--json` is passed only in
|
||||
JSON mode while plain text stays readable. Python fixtures cover arbitrary-digit
|
||||
OpenCode patches (a parsed `2.0.260` is still rejected against an expected
|
||||
`2.0.26` by exact equality; 260/100 are illustrative valid/future examples, not
|
||||
verified releases) and reject both raw and canonical reserved homes. Real
|
||||
in-container `provider-check`/ping confirmation, a compiled PTY run and a real
|
||||
Docker reconfigure remain **not tested** in this change. These source-level
|
||||
regressions are not a claim about the deployed image; the separate dated 2026-10-11
|
||||
FINAL INSTALLED verification recorded that the runtime was actually installed and
|
||||
used (OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1, image
|
||||
`sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136`, with
|
||||
both real stages executed in run17 and passed in Root's full host race/vet runs).
|
||||
Active lease-expiry recovery, web search delivery and fresh interactive Docker Pi
|
||||
sign-in remain **not tested**.
|
||||
Pi stages start a new retained session per visit with explicit tools and a
|
||||
Pipelines permission extension; ambient extensions, skills, context files, prompt
|
||||
templates, automatic retries and compaction are disabled. Tool gates are not an
|
||||
OS sandbox: approved shell can have broad side effects. Usage accumulates across
|
||||
attempts and visits; missing required accounting fails closed. Session/RPC audit
|
||||
lives in the execution user's cache under `pipelines/pi-sessions`; unconfirmed
|
||||
cleanup blocks continuation and retry.
|
||||
|
||||
@@ -38,23 +38,6 @@ parallel/distributed task engine or a guarantee that model reports are correct.
|
||||
- Services are native or Docker-based. With an existing Docker installation do not
|
||||
start a competing native server. Server APIs are unauthenticated: trusted networks
|
||||
only; never recommend public exposure. Tool scope is not a universal OS sandbox.
|
||||
- Docker setup resolves the current supported OpenCode V2 release from the single
|
||||
documented complete `GET .../git/matching-refs/tags/v2` response (a bounded
|
||||
complete array with no pagination parameters; never the moving `latest` tag or
|
||||
the registry's insertion order) and the current Pi release from npm, resolves the
|
||||
selected tag's immutable GHCR `sha256` digest, builds `FROM` that digest, and
|
||||
verifies the built image reports exactly those executable versions plus a
|
||||
Node-compatible Pi engine.
|
||||
It fails closed, keeping the last verified Dockerfile fallback pins, when discovery,
|
||||
digest resolution, completeness, build or compatibility cannot be established: never
|
||||
a hardcoded fallback, never a silent stale-pin substitution. A full install skips a
|
||||
rebuild only when the release receipt **and** the actual built image's release and
|
||||
source identity labels already match; a dirty source always invokes the Docker
|
||||
build and relies on layer caching. A model/login-only reconfiguration compares only
|
||||
the verified release image identity, so a changed source alone does not rebuild.
|
||||
`pipelines harness setup opencode|pi` reconfigures inside the existing container and
|
||||
refuses an image that does not match; it does not rebuild or restart services (an
|
||||
explicit ordinary `./install` does when the user asks).
|
||||
|
||||
## Where data lives
|
||||
|
||||
@@ -72,7 +55,6 @@ delete databases, sessions, volumes or credentials to repair a workflow.
|
||||
|
||||
Use the authoritative `CLI_PREFIX` from the prompt:
|
||||
|
||||
- `CLI_PREFIX runs --json`: run history (optional `--pipeline NAME` scope).
|
||||
- `CLI_PREFIX search QUERY [--type KIND] [--limit N] --json`: one read-only query
|
||||
across pipelines, versions, runs, steps, field text and artifacts. QUERY is free
|
||||
text, a bare status word (`queued`/`running`/`succeeded`/`failed`/`canceled`/
|
||||
@@ -112,17 +94,7 @@ Use the authoritative `CLI_PREFIX` from the prompt:
|
||||
limits, failure/format diagnostics and the retained session pointer), not the
|
||||
complete tool history.
|
||||
- `CLI_PREFIX decide RUN_ID [STAGE] --option KEY [--feedback TEXT] [--json]`: answer
|
||||
a run that is `waiting` on a `type=human` stage. The status and the stage's
|
||||
instructions/options are visible in `runs` and `inspect --json` (and the TUI
|
||||
picker); the stage is inferred from the run's current stage unless named. The
|
||||
choice and feedback are recorded as the step's evidence (its output and derived
|
||||
verdict; the feedback is never an error diagnostic) and the run resumes along
|
||||
`edges[KEY]`. An unknown option is
|
||||
refused with the valid list; a stage that is not waiting is refused; a repeated
|
||||
decision for an already-decided visit does not double-apply. The wait is persisted,
|
||||
so `decide` still works after a server restart. `cancel` and `retry` keep working
|
||||
while waiting and after a decision; a human stage is never killed by a stage
|
||||
timeout or `max_visits` (the shared 30-step run budget still applies).
|
||||
a run `waiting` on a `type=human` stage (`references/format.md`).
|
||||
- `CLI_PREFIX run ... --json --wait --wait-timeout DURATION`: `--wait` polls until
|
||||
terminal and `--wait-timeout` bounds only the client wait after admission,
|
||||
independent of per-request `--timeout`. A reached wait limit leaves the admitted run
|
||||
@@ -160,22 +132,6 @@ runs natively through `./install` by default (`scripts/setup-native.py`, with
|
||||
`--native` equivalent); the Docker server + ntfy installation is explicit with
|
||||
`./install --docker`.
|
||||
|
||||
In the TUI, Home advertises `/` omni search and `:` commands. Home `/` opens one
|
||||
popup over pipelines, versions, runs, steps/visits, artifact names and stage
|
||||
input/output/evidence text; each row shows a short type badge and a text match
|
||||
shows a bracketed snippet, and a bare status word or date selects runs. The same
|
||||
query is available as `pipelines search QUERY [--type KIND]`, where `--type`
|
||||
narrows the kind. Run Tab switches only
|
||||
Details ↔ Graph. `n/p` select recorded visits, including repeated stages; `d`
|
||||
focuses/toggles file changes, `A` opens the run's grouped artifacts view (name,
|
||||
content type, size and URL per entry, `PgUp`/`PgDn` scroll so a trailing visit
|
||||
that recorded none stays reachable, `Enter`/`o` to open the selected URL and
|
||||
`y` to copy it), `a` selects the active/latest retained
|
||||
configured-harness agent session's bounded, redacted tool evidence, `D`/`:debug`
|
||||
opens fresh investigation, and `P` opens the pinned pipeline version. `:quit`
|
||||
exits; `q` does not. Results are retained reports, not independently confirmed
|
||||
achievements. Distinguish partial output from full success.
|
||||
|
||||
## Retained stage file changes
|
||||
|
||||
Each newly executed visit compares a bounded, read-only workspace snapshot just
|
||||
@@ -228,18 +184,11 @@ file diffs: current capture is server-owned. Do not promise full files/rollback.
|
||||
|
||||
## Retained stage artifacts
|
||||
|
||||
Every model visit gets a file-based turn under the data directory
|
||||
(`<data>/artifacts/<run>/stages/<seq>-<stage>/`) with `input.md`, `memory.md`
|
||||
(capped essential memory, inlined), `history.md` (full earlier turns, read on
|
||||
demand), `output.md` and `private/`; `public/` is run-level and shared by every stage.
|
||||
The root is configurable and defaults next to the database. The writable
|
||||
`private/` and shared `public/` directories are stated in the agent prompt for
|
||||
every harness, and `PIPELINES_ARTIFACTS_DIR` points at the turn directory for a
|
||||
locally spawned Pi process. OpenCode V2 tools run inside the shared serve
|
||||
process, so there the prompt, not the environment, is the reliable carrier.
|
||||
Turn-local files belong in `private/`; reports, logs and screenshots for later
|
||||
stages belong in `public/`. The server owns `input.md`, `memory.md`,
|
||||
`history.md` and `output.md` and never asks the model to write them directly.
|
||||
Turn files (`input.md`, `memory.md`, `history.md`, `output.md`, `private/`) and
|
||||
the run-level `public/` are described in `references/format.md`. They live under
|
||||
`<data>/artifacts/<run>/stages/<seq>-<stage>/`; the root is configurable and
|
||||
defaults next to the database. The server owns `input.md`, `memory.md`,
|
||||
`history.md` and `output.md`.
|
||||
|
||||
After the visit the server makes one bounded inventory of the turn and public
|
||||
directories and records each file's name, size, content type (extension, sniff
|
||||
@@ -247,9 +196,8 @@ fallback) and a relative url; these appear under each step's `artifacts` in
|
||||
`inspect RUN_ID --json` and the TUI's `A` artifacts view, which groups them by
|
||||
stage and visit, marks images and shows the recorded count,
|
||||
capture-pending/not-recorded wording and partial-capture notes. Public entries
|
||||
carry a `public/` prefix. `input.md`, `memory.md`, `history.md` and `output.md`
|
||||
are the primary artifacts of every model visit. To see what a late stage was
|
||||
given, read its `memory.md`; `history.md` is the full record it could consult.
|
||||
carry a `public/` prefix. To see what a late stage was given, read its
|
||||
`memory.md`; `history.md` is the full record it could consult.
|
||||
`GET /artifacts/<run>/<step>/<name>` serves the bytes (correct Content-Type and
|
||||
Content-Length, HEAD, and Range/206/416 via `http.ServeContent`), refusing
|
||||
traversal, absolute paths, symlink escapes and unknown runs/steps. A pre-turn
|
||||
@@ -302,158 +250,6 @@ and one WAL checkpoint, reporting measured bytes. Pipeline definitions and their
|
||||
immutable versions are never touched. The commands are local, require an existing
|
||||
`--db`, and need no running server.
|
||||
|
||||
## Brokered OpenCode chat
|
||||
|
||||
The protocol declares the brokered `chat_*` surface (`chat_sessions`,
|
||||
`chat_create`, `chat_directories`, `chat_prompt`, `chat_messages`,
|
||||
`chat_interrupt`, `chat_permission`, `chat_reply`). The server dispatches all of
|
||||
them, and pushes `chat_event` frames derived from OpenCode's global SSE firehose
|
||||
(`/event` on the native serve, `/api/event` on the V2 API), with retained
|
||||
per-session/per-message state so a reconnecting client re-syncs instead of
|
||||
replaying deltas.
|
||||
|
||||
At startup `cmd/pipelines-server` calls `opencode.OpenChat`, which detects the
|
||||
installed backend and opens exactly one owned `opencode serve` endpoint for it:
|
||||
the native wire for v1, the V2 API for the installed V2 CLI. Chat is enabled only
|
||||
when that connection carries a URL-bearing, SSE-capable event endpoint; otherwise
|
||||
chat stays disabled and every `chat_*` request is answered with a protocol error
|
||||
naming the missing configuration rather than hanging. Stage generation still
|
||||
reaches V2 through the `opencode api` CLI, so the owned serve process is chat's
|
||||
single transport and starts no process for stages. The adapter skips firehose
|
||||
events it does not surface before decoding their data, so an unrelated event
|
||||
whose `delta` has a different shape (`session.instructions.updated` carries an
|
||||
object) never fails the stream.
|
||||
|
||||
The operator can instead broker chat from their own OpenCode service with the
|
||||
opt-in `-opencode-endpoint` flag or `PIPELINES_OPENCODE_ENDPOINT`, with precedence
|
||||
flag > environment > the endpoint recorded on a previous run and the spawned
|
||||
instance as the default. `opencode.OpenEndpoint` validates the reference once
|
||||
(the established http(s)/host/no-credentials/no-query/no-path idiom), detects the
|
||||
backend from the service's own `/api/info` (V2) or `/global/health` (native)
|
||||
contract and requires a streamable `text/event-stream` event path; it returns a
|
||||
connection with **no owned process** and never calls `startServe`. A malformed
|
||||
reference, an unreachable host, a reachable non-OpenCode service or a
|
||||
health-ok-but-no-SSE service each disable chat with an error naming the endpoint
|
||||
and never fall back to the spawned instance. An authenticating endpoint accepts an
|
||||
optional password only through the private file
|
||||
`PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE` (regular, 0600, single-owner), and
|
||||
`PIPELINES_OPENCODE_ENDPOINT_USERNAME` overrides the detected default user. The
|
||||
validated choice is recorded in the additive `server_settings` table so a restart
|
||||
with no flag/environment reuses it; an unreachable recorded endpoint disables chat
|
||||
with a clear message and no fallback. `pipelines service enable` preserves the
|
||||
endpoint and the password-file reference in autostart.
|
||||
|
||||
`chat_sessions` lists the operator's own chat sessions, not the sessions pipeline
|
||||
runs create. Operator sessions are recorded in the dedicated `operator_sessions`
|
||||
table (`session_id` primary key, plus `directory`, `title`, `tag`, `created_at`,
|
||||
`last_used_at`), separate from `run_steps`/`run_session_owners`, so an operator
|
||||
chat is a first-class thing and a pipeline stage session — recorded as run
|
||||
ownership, never in the operator registry — is never listed. The handler reads the
|
||||
registry once (bounded to 128 rows, most-recently-used first), calls OpenCode's
|
||||
session list once, and keeps only registry rows the endpoint still reports, so a
|
||||
session with no recorded tag is never listed and a deleted upstream session does
|
||||
not linger. It fills each session's `directory`, `tag` and `created_at`/
|
||||
`updated_at` from the registry and marks `busy` when another client currently
|
||||
holds the session (an in-server claim, released when the stream ends or the client
|
||||
disconnects); a second client's `chat_prompt` for a held session is rejected
|
||||
before any OpenCode call. It is fail-closed: with no registry configured it
|
||||
returns an empty list, never the endpoint's whole session list. The OpenCode
|
||||
session list is read unscoped by directory (the V2 `GET /api/session` otherwise
|
||||
defaults to the caller's directory), because an operator session may live in any
|
||||
directory.
|
||||
|
||||
`chat_create` creates one operator session and records it in the same registry
|
||||
with the stored tag `operator` (the display label `Operator` is derived from it,
|
||||
so there is no second tag vocabulary). It takes a `title` (default "New chat") and
|
||||
a `directory` (default `~`): an omitted or empty value, `~`, and any `~/...` value
|
||||
all resolve to `os.UserHomeDir`, never the server's working directory; a relative
|
||||
path or a missing directory is rejected before any OpenCode call, and a session
|
||||
created but not recordable is reported as an error naming its id. Create posts the
|
||||
verified V2 shape `POST /api/session` `{"title":...,"location":{"directory":...}}`.
|
||||
`chat_directories` returns the operator's home first, then the directories of
|
||||
previous operator sessions most-recently-used first, deduplicated and bounded (8);
|
||||
it is a dedicated request so the create dialog can load the choices without first
|
||||
listing sessions and `chat_sessions` stays a pure, fail-closed session list. A
|
||||
successful `chat_prompt` touches the session's `last_used_at` in one UPDATE, so it
|
||||
moves to the front of the list; a non-operator session id touches no row.
|
||||
|
||||
`chat_adopt` records a session that already exists on the configured endpoint in
|
||||
the same operator registry, using the endpoint's own directory and title
|
||||
verbatim (never a guess, the server's working directory or the home directory).
|
||||
It reads the endpoint once through the shared `Chat.Session` (which reuses
|
||||
`Sessions`/`decodeChatSessions`), records through the existing
|
||||
`RecordOperatorSession` with the `operator` tag, and echoes the adopted session
|
||||
plus its registry row like `chat_create`, so `chat_sessions` lists it and
|
||||
`chat_prompt` continues it. A nil payload, a blank or foreign-shaped id, a nil
|
||||
broker or a nil registry is refused before any endpoint call; an id the endpoint
|
||||
does not report is refused naming the id and records nothing; an endpoint record
|
||||
with an empty directory is refused rather than substituting a directory. The CLI
|
||||
equivalent is `pipelines chat-adopt <session-id>` (`--json` prints the full
|
||||
response, otherwise a short `Adopted <id> · <directory>` line).
|
||||
|
||||
While a `chat_prompt` subscription is open the server also polls
|
||||
`GET /api/permission/request` on a fixed small interval (first poll immediately) and
|
||||
pushes one `chat_permission` frame per newly pending approval request, deduped by
|
||||
request id so a repeated poll never re-pushes the same request. The app answers with
|
||||
`chat_reply`, which the server forwards as
|
||||
`POST /api/session/{id}/permission/{requestID}/reply` with
|
||||
`{"decision":"once"|"always"|"reject"}`. A missing session/request id or an invalid
|
||||
decision is rejected before any OpenCode call, and the poller stops when the stream
|
||||
ends or the client disconnects.
|
||||
|
||||
The permission list and reply are implemented against the V2 HTTP API only: their
|
||||
field names match the service's `Permission.Request`/`Permission.Reply` schemas
|
||||
(verified against the installed service's `/openapi.json`). The native v1 serve
|
||||
translation rejects them explicitly instead of guessing an endpoint, so on v1 the
|
||||
permission poller stops quietly rather than reporting the documented limitation as
|
||||
a per-prompt error.
|
||||
|
||||
### Verification evidence (2026-10-17)
|
||||
|
||||
Verified against the installed OpenCode 2.0.20 on 2026-10-17:
|
||||
|
||||
- `opencode --version` reports `2.0.20`; `opencode --help` advertises `api` and
|
||||
`serve`. `GET /openapi.json` on an owned `opencode serve` lists `GET /api/event`
|
||||
(SSE) and the session/prompt/interrupt/permission endpoints.
|
||||
- A live turn streamed token-level `session.text.delta` and
|
||||
`session.reasoning.delta` events (each with `assistantMessageID` and `ordinal`),
|
||||
plus `session.step.*` and `session.usage.updated`; the V2 API server
|
||||
authenticates the fixed `opencode` basic-auth user.
|
||||
- The opt-in live end-to-end test (`PIPELINES_TEST_OPENCODE_CHAT_LIVE=1 go test
|
||||
./internal/server -run TestChatLiveEndToEnd -v`) opened an owned V2 endpoint,
|
||||
created an operator session through `chat_create` in a chosen directory, listed
|
||||
exactly that session through `chat_sessions` (with its directory and tag
|
||||
`operator`), recorded a genuine pipeline stage session in
|
||||
`run_steps`/`run_session_owners` and showed it never appeared in the list, and
|
||||
streamed a completed turn as `[step:started,text,step:streamed,step:ended]`
|
||||
(first delta `"pong"`). The operator-session registry, the filter, directory
|
||||
defaulting and the recents bound are also covered by the `pipeline`, `opencode`
|
||||
and `server` unit tests, which run without the live gate.
|
||||
- Not verified live: a genuine pending-approval round trip and any backend other
|
||||
than the installed 2.0.20. The permission wire shapes are unit-tested over an
|
||||
httptest fixture, not against a live pending request.
|
||||
- Operator-supplied endpoint selection and `chat_adopt` (added 2026-10-17):
|
||||
`opencode.OpenEndpoint` and the flag/env/recorded precedence, the URL-shape
|
||||
validation failures (scheme, empty host, userinfo, query, fragment, non-empty
|
||||
path), the unreachable/non-OpenCode/no-SSE/auth-mismatch failures, the
|
||||
no-owned-process property (`conn.process == nil`) and the password-file rule
|
||||
are unit-tested over httptest fixtures in `internal/opencode/endpoint_test.go`.
|
||||
`chat_adopt` (endpoint directory/title recorded with the `operator` tag, echo,
|
||||
missing id refused naming the id and recording nothing, nil payload/blank id/nil
|
||||
broker/nil registry refused before any endpoint call, empty endpoint directory
|
||||
refused) is unit-tested over the server fake broker in
|
||||
`internal/server/chat_test.go`; the recorded-endpoint round-trip across a store
|
||||
Close/Open is in `internal/pipeline/settings_test.go`; the protocol round-trip
|
||||
and the CLI dispatch (`pipelines chat-adopt`) are covered in
|
||||
`internal/protocol/chat_test.go` and `internal/cli/chat_adopt_test.go`. The
|
||||
live scratch-server check against the operator's real endpoint (adopt, list,
|
||||
prompt) was **not** run in this change; only the unit fixtures above were
|
||||
executed.
|
||||
|
||||
Deployment state: not deployed. This is reviewed and committed locally only;
|
||||
deploying it (and restarting the server) is the operator's step. Do not describe
|
||||
brokered chat as already deployed.
|
||||
|
||||
## Debug and retry
|
||||
|
||||
Debug opens a new conversation with retained failure context, exact definition and
|
||||
@@ -471,18 +267,11 @@ side effects remain and can happen again. Never describe retry or cancellation a
|
||||
rollback, or resume/replay admitted tool work just to recover formatting.
|
||||
|
||||
Retained model-stage failures carry a fixed sanitized category. `FINAL RESULT
|
||||
VALIDATION` names the exact duplicate-field/syntax detail and byte offset for a
|
||||
strictly invalid final result; `HARNESS TRANSPORT` covers request, active-session
|
||||
poll and final-readback failures. A validation failure never re-prompts the
|
||||
admitted tool-using stage: completed edits and tool side effects are retained and
|
||||
not undone, so a full retry repeats work. An explicit read-only stage is the one
|
||||
exception: a malformed result there mutates nothing, so it may be re-emitted up
|
||||
to `retries` with its `format_repairs` evidence retained, and a post-admission
|
||||
deadline or transport failure may be retried. A failure to reach the harness
|
||||
before any prompt or tool call is admitted (connecting, or creating the session)
|
||||
replays nothing, so it is retried within `retries` for any stage, mutating or not.
|
||||
A stage whose tools include
|
||||
`edit`/`shell`, or that has no tools block (the legacy read/glob/grep/edit/shell
|
||||
bundle), is never replayed. Neither category exposes provider/model bodies or
|
||||
credentials. The isolated formatter's deadline is proportional to the stage
|
||||
timeout and clamped so it never extends the stage's own deadline.
|
||||
VALIDATION` means `output.md` had a missing, duplicate, empty or undeclared
|
||||
outcome (`references/format.md`); `HARNESS TRANSPORT` covers request, session-poll
|
||||
and readback failures. Completed edits and tool side effects are retained, not
|
||||
undone, so a full retry repeats work. A failure before any prompt is admitted is
|
||||
retried within `retries` for any stage; a post-admission deadline or transport
|
||||
failure is retried only on an explicit read-only stage; a stage with `edit`/`shell`
|
||||
(or no tools block) is never replayed. Neither category exposes provider/model
|
||||
bodies or credentials.
|
||||
|
||||
@@ -13,11 +13,13 @@ at integration. Serialize only actual dependency or resource conflicts.
|
||||
|
||||
## Stage jobs
|
||||
|
||||
- Plan: a short approach and acceptance checks when they help.
|
||||
- Code: the focused change, tests and focused checks.
|
||||
- Review: the whole task diff and independent verification.
|
||||
- UIReview: rendered behavior with controlled data and captures.
|
||||
- Land: commit the reviewed work, integrate main and resolve conflicts.
|
||||
- `plan`: a short approach and acceptance checks when they help.
|
||||
- `implement`: the focused change, tests and focused checks.
|
||||
- `review`: the changed hunks (server-captured changed list, `git diff --stat`,
|
||||
then per-file diffs) and the reported checks; it runs no suites.
|
||||
- `ui-review`: rendered behavior with controlled data and captures.
|
||||
- `decide`: a human choice between edges.
|
||||
- `land`: commit the reviewed work; the delivery gate checks and lands it.
|
||||
|
||||
Names are a convention, not a requirement; a pipeline can use any clear stage
|
||||
names. Keep each stage's job narrow enough to judge its output.
|
||||
|
||||
@@ -8,22 +8,23 @@ below (errors block publishing; warnings print and do not).
|
||||
## Where work belongs
|
||||
|
||||
- **Deterministic checks belong in the server `checks` block, not the model.**
|
||||
The landing stage sets `delivery=true` and lists the full check commands; the
|
||||
server runs them on the committed candidate. A model running the suite puts
|
||||
every log line into its context, and every later call pays for it again.
|
||||
- **Code runs focused tests only**: the test file or package it touched, with
|
||||
quiet or summary output. Never the full suite.
|
||||
- **Review reads the diff, not the repository.** Start from the server-captured
|
||||
changed-file list the runner adds to the prompt and `git diff --stat`, then read
|
||||
only the changed hunks and the code around them. Review runs no suites; it judges
|
||||
the reported checks and names missing coverage.
|
||||
- **Land/committer relies on server checks.** With a delivery gate its prompt
|
||||
commits and stops; it does not re-run what the gate runs. Do not claim "the
|
||||
server lands it" unless a stage has `delivery=true` (error otherwise).
|
||||
The `land` stage sets `delivery=true` and lists the full check commands; the
|
||||
server runs them on the committed candidate (`references/format.md`, Delivery
|
||||
gate). A model running the suite puts every log line into its context, and
|
||||
every later call pays for it again.
|
||||
- **`implement` runs focused tests only**: the test file or package it touched,
|
||||
with quiet or summary output. Never the full suite.
|
||||
- **`review` reads the changed hunks, not the repository.** Start from the
|
||||
server-captured changed-file list the runner adds to the prompt and
|
||||
`git diff --stat`, then read per-file diffs and only the code around them.
|
||||
Review runs no suites; it judges the reported checks and names missing coverage.
|
||||
- **`land` relies on server checks.** With a delivery gate its prompt commits and
|
||||
stops; it does not re-run what the gate runs. Do not claim "the server lands
|
||||
it" unless a stage has `delivery=true` (error otherwise).
|
||||
- Every stage that can modify the workspace declares `checks`, even if only
|
||||
`none: <reason>` (`references/format.md`, Checks). Checks must fail when a tool
|
||||
is missing: no `|| true`, `[ ! -x tool ] || ...` or similar guards (error).
|
||||
Cover every language the run can change.
|
||||
`none: <reason>`. Checks must fail when a tool is missing: no `|| true`,
|
||||
`[ ! -x tool ] || ...` or similar guards (error). Cover every language the run
|
||||
can change.
|
||||
|
||||
## Prompts
|
||||
|
||||
@@ -31,34 +32,41 @@ below (errors block publishing; warnings print and do not).
|
||||
warns. Shared procedure belongs in project docs or this skill.
|
||||
- **No runner boilerplate.** The runner already explains `output.md`, the
|
||||
`# outcome` and `# handoff` sections, allowed outcomes, turn files, tools and
|
||||
honesty rules (`internal/stage/model_prompt.go`). Repeating them warns.
|
||||
honesty rules. Repeating them warns.
|
||||
- **Every outcome a prompt names is a declared edge.** "Return skipped" with no
|
||||
`skipped` edge, or an undeclared `blocked_retryable`, fails the stage at run
|
||||
time; validation rejects it (unquoted outcome words such as `skipped`,
|
||||
`blocked`, `rejected`, `approved`, `advisory`; other back-quoted words after
|
||||
return/report warn).
|
||||
- **Missing tools or evidence**: declare `blocked` where a stage can lack required
|
||||
input; the runner tells the model to choose `blocked` when declared, else
|
||||
`error`.
|
||||
|
||||
## Memory and handoffs
|
||||
|
||||
- The runner inlines the **essential memory** (`memory.md`: task, previous
|
||||
stage's handoff, outcome ledger, changed files; capped) into every prompt, and
|
||||
asks each stage to start its output with a short `# handoff`: what it did,
|
||||
outcome, files changed, open findings, in a few lines. Do not repeat that
|
||||
instruction; say only what this stage's handoff must carry if it is special.
|
||||
- The **full history** (`history.md`) and full `input.md` are on demand. Never
|
||||
tell a stage to read the whole history every visit (warning); say when it
|
||||
- The runner inlines the essential `memory.md` into every prompt and asks each
|
||||
stage for a short `# handoff` after its `# outcome` (`references/format.md`).
|
||||
Do not repeat that instruction; say only what this stage's handoff must carry
|
||||
if it is special.
|
||||
- The full `history.md` and `input.md` are on demand for stages with `read`.
|
||||
Never tell a stage to read the whole history every visit (warning); say when it
|
||||
needs it, e.g. "read history.md only if the handoff lacks a detail you need".
|
||||
- A stage without `read` gets `input.md` inlined (up to 32 KB) on every call. Give
|
||||
`read` to any stage that follows a large output so it reads detail only when
|
||||
needed.
|
||||
|
||||
## Loops and budgets
|
||||
|
||||
- **Loops get limits.** An edge back to the same or an earlier stage needs
|
||||
`limits` on the source outcome or the target (error). Use `max_visits` 2–3.
|
||||
- **Loops get limits.** Put `{"max_visits":3,"on":["rejected"]}` on the reviewer;
|
||||
leave the loop target unlimited or capped at least as high
|
||||
(`references/control.md`).
|
||||
- **Re-entry gets only findings**: a rejected review hands back must-fix items
|
||||
with location and fix, not a full re-review, and the implementer fixes those
|
||||
items rather than redoing the task. Use `advisory` for non-blocking notes so
|
||||
they do not buy another loop.
|
||||
- Set **per-stage `budget_tokens`** (and `budget_usd` if wanted) as a runaway
|
||||
guard on stages that loop or explore, sized a few times a normal visit.
|
||||
with location and fix, and the implementer fixes those items rather than
|
||||
redoing the task. Use `advisory` for non-blocking notes so they do not buy
|
||||
another loop.
|
||||
- **`budget_tokens` only when the user asks or a measured visit exists**
|
||||
(`pipelines inspect RUN_ID --json`, `usage.tokens_total`). Size it 3–5× that
|
||||
visit, remembering totals include cache reads, so a normal visit never trips it.
|
||||
|
||||
## Commands and scope
|
||||
|
||||
|
||||
@@ -1,609 +1,59 @@
|
||||
# Verification, accounting and real-provider evidence
|
||||
|
||||
Historical note: model stages now use file-based turns and parse only the
|
||||
`# outcome` section. The JSON/line-format envelope and format-repair ladder
|
||||
described in the dated sections below are retired; see `docs/model-results.md`
|
||||
for the current contract. The dated evidence remains as a record.
|
||||
# Verification and evidence
|
||||
|
||||
Structural validation is not inference, tool execution, correct task completion,
|
||||
phone delivery or correct billing. Distinguish observed artifacts/command results
|
||||
from model-written reports. State the command/check actually run and say **not
|
||||
tested** for anything not executed. Never present mocked results as live-provider
|
||||
evidence or a different backend/model as already validated.
|
||||
phone delivery or correct billing. Distinguish observed artifacts and command
|
||||
results from model-written reports. State the command or check actually run and
|
||||
say **not tested** for anything not executed. Never present mocked results as
|
||||
live-provider evidence or a different backend/model as already validated. Dated
|
||||
verification logs live in `docs/verification-history.md` in the source checkout.
|
||||
|
||||
## Authoring useful verification
|
||||
## Where verification happens in a pipeline
|
||||
|
||||
Preserve the facts later stages need in free-form output headings; only the
|
||||
`# outcome` section is interpreted. Implementer/reviewer stages
|
||||
can use appropriate permitted commands; do not accept a preceding stage's success
|
||||
summary as independent proof. Actual file bytes, command exit status, retained tool
|
||||
parts and server-captured diffs are stronger evidence than prose. A read failure
|
||||
can prove a deliberately missing input, but is not a successful read. Tool names
|
||||
mentioned in text do not prove tools executed. A reviewer stage must inspect the
|
||||
actual files and run its own checks; a source-only review runs no commands and
|
||||
must state that runtime behavior was not tested. Structural, mocked or
|
||||
authoring-time checks are not deployed or live-provider evidence.
|
||||
- The `delivery=true` stage's `checks` block is the run's deterministic suite;
|
||||
the server runs it (`references/format.md`, Checks).
|
||||
- The implementer runs focused checks for what it touched and reports the exact
|
||||
commands and results.
|
||||
- The reviewer reads the changed hunks (the server-captured changed-file list,
|
||||
then `git diff --stat`, then per-file diffs), judges the reported checks and
|
||||
names missing coverage. It runs no suites. Its verdict concerns the current
|
||||
change only.
|
||||
- Actual file bytes, command exit status, retained tool parts and server-captured
|
||||
diffs are stronger evidence than prose. Tool names mentioned in text do not prove
|
||||
tools executed. Do not accept a preceding stage's summary as proof of runtime
|
||||
behavior.
|
||||
|
||||
Do not add technology-specific checks unless the task/repository requires them.
|
||||
Keep runtime verification separate from authoring. Execute relevant code and fixture
|
||||
checks within the authorized task; ask before unrelated chargeable calls or notifications.
|
||||
Do not weaken an acceptance check just to get a passing run.
|
||||
`provider-check` must run on the intended execution computer/container with its
|
||||
credentials; a client-side check is not proof of a remote server's provider health.
|
||||
Do not add technology-specific checks unless the task or repository requires
|
||||
them, and do not weaken an acceptance check just to get a passing run. Ask before
|
||||
unrelated chargeable calls or notifications. `provider-check` must run on the
|
||||
intended execution computer/container with its credentials; a client-side check
|
||||
is not proof of a remote server's provider health.
|
||||
|
||||
## Budgets and backend caveats
|
||||
## Usage evidence
|
||||
|
||||
Defaults remain two retries and one hour per stage attempt; a run-level wall-clock
|
||||
deadline is not added. Budgets are optional, positive and independent USD/token
|
||||
reported-usage limits shared across attempts and repeat visits.
|
||||
Accounting failures stop budget-limited stages. In-flight calls can overshoot;
|
||||
these are **not provider-side hard caps or a guaranteed bill ceiling**. Reported
|
||||
USD may validly be zero; do not fabricate prices/accounting from output length.
|
||||
|
||||
Per-run usage is recorded as typed step evidence from the usage the stage already
|
||||
measured for its budget (no second harness read and no transcript parse), and
|
||||
totalled for the run and over all history. A stage revisited within a run shares
|
||||
one budget, so each step records only that visit's increment of it and the run
|
||||
total is not the cumulative stage budget counted again; a revisit that reports
|
||||
nothing new (a failure or timeout before any usage poll, or a Pi visit that never
|
||||
reaches a usage stat) records no usage for that visit rather than a known zero.
|
||||
`CLI_PREFIX inspect RUN_ID --json`
|
||||
exposes each model step's `tokens_total`, `tokens_in`/`tokens_out` (only when
|
||||
`split_known`), `cost_usd` (only when `cost_known`) and the model it ran with,
|
||||
plus the totalled `usage`; `CLI_PREFIX stats --json` adds the history-wide totals,
|
||||
which equal the sum of the per-run totals. A stage that reported no cost exposes
|
||||
its tokens and marks the cost unavailable rather than showing zero; a harness that
|
||||
reports only a total (Pi) exposes the total and marks the in/out split
|
||||
unavailable rather than presenting the total as input tokens. An upgraded or
|
||||
historical database gains an empty usage table with no backfill, so old runs omit
|
||||
`usage` instead of showing zero. This is current source, not deployment evidence.
|
||||
|
||||
Dated current-source evidence (2026-10-17, this machine). A throwaway read-only
|
||||
reader (`internal` store API) over a scratch copy of
|
||||
`~/.local/share/pipelines/data/pipelines.db`, taken with
|
||||
`sqlite3 "file:$HOME/.local/share/pipelines/data/pipelines.db?mode=ro" ".backup <scratch>"`
|
||||
so the live DB is never mutated, opened 228 runs and found **no** `step_usage`
|
||||
rows: every stored run predates this change, so per-run figures are unavailable
|
||||
rather than zero. Run 314 (this item's own run) and run 239 (whose implementer
|
||||
step ran `opencode-go/kimi-k2.7-code`, a stage model different from the pinned
|
||||
`opencode-go/deepseek-v4.1-flash`) both report `usage=(none recorded)`, and
|
||||
`stats` omits the history-wide totals. The recorded figures themselves are
|
||||
covered by the passing current-source round trips — `internal/pipeline/usage_test.go`,
|
||||
`internal/server/usage_test.go` (including `inspect --json` over a websocket) and
|
||||
the pre-upgrade/read-bounded/stats-sum cases — not by a live run, because the
|
||||
deployed server predates this change and was not restarted. No live model call,
|
||||
deploy or rebuild produced this evidence.
|
||||
|
||||
A live V2 trial found `/generate` returned text while its retained session had no
|
||||
messages and zero counters. Current source uses tracked `/prompt` → bounded
|
||||
active-session polling → message readback for **budgeted V2 text-only calls, with
|
||||
all tools denied**. Completed token-budgeted responses reporting zero tokens fail
|
||||
closed. No new definition flags or agent tools are required for this fix.
|
||||
Unbudgeted tool-free V2 generation can still have empty backend message history;
|
||||
pipeline inputs, results and command audit remain retained. An unbudgeted V2
|
||||
audited agent stage makes exactly one extra bounded final session-aggregate read
|
||||
to record any reported USD/token usage best-effort: missing or invalid usage stays
|
||||
unknown and never turns a valid run into a failure, cumulative sessions/retries
|
||||
are keyed per session so they cannot double count, and totals are never inferred
|
||||
from final-assistant-only or native-partial data. Budgeted settlement and native
|
||||
V1 enforcement are unchanged. Native V1 uses its existing path.
|
||||
|
||||
The V2 `.../experimental/session/{id}/wait` request is **no longer used**: a dated
|
||||
Docker log shows it timing out at the CLI's own request timeout after about six
|
||||
minutes (run started 2026-10-10T23:07:16.547Z, `TimeoutError` logged at
|
||||
2026-10-10T23:13:17.262Z, server `http.status=499`), which aborted a long
|
||||
implementation stage. Instead current source polls `GET /api/session/active`
|
||||
about once per second with a context-aware delay and no busy loop, until the
|
||||
admitted session is absent (inactive). Missing/null/non-object `data`, or a state
|
||||
other than the documented `running` type, fails closed and is never treated as
|
||||
inactive. This is **mocked regression coverage** in `internal/stage` (fake CLI),
|
||||
not live-provider evidence; V1 native settlement is unchanged.
|
||||
|
||||
Current source reads the V2 final result with a single bounded descending
|
||||
final-assistant readback (`order=desc&limit=1&type=assistant`) after the settled
|
||||
poll, for both audited agent stages and budgeted V2 text-only calls, so large
|
||||
retained history (including prior tool-read contents) is no longer fetched just
|
||||
to select the completed result. A truncated/incomplete readback, an empty data
|
||||
list, or a latest message that is neither completed `stop` nor a completed
|
||||
structured tool-call fails closed with a sanitized diagnostic that omits
|
||||
provider content; retained audit previews are **not** executable final results.
|
||||
Session/debug evidence now reuses the same single latest-assistant readback
|
||||
instead of a recent-20 all-message fetch, and returns each payload as a labelled
|
||||
bounded, credential-redacted envelope with explicit preview/truncation/missing
|
||||
diagnostics; a small final assistant result is therefore selected even when large
|
||||
prior tool content exists. Pi evidence remains documented JSONL and is never
|
||||
parsed as one JSON object. This describes current source behavior, not a guarantee
|
||||
that a user's installed server includes the fix. An updated skill/client is not
|
||||
deployment evidence; never rebuild/restart services or log into providers without
|
||||
specific authorization.
|
||||
|
||||
Retained model-stage failures now carry a fixed sanitized category. A strictly
|
||||
invalid final result (a duplicate nested field, or malformed/ambiguous JSON with a
|
||||
byte offset) is labeled `FINAL RESULT VALIDATION`; request, V2 active-session poll
|
||||
and final-readback failures are labeled `HARNESS TRANSPORT`. A pre-admission
|
||||
request failure replays nothing and is retried for any stage; a post-admission
|
||||
failure is retried only on an explicit read-only stage (see the 2026-10-17 section
|
||||
below). A validation failure
|
||||
preserves the admitted session, transcript and evidence and performs no second tool
|
||||
execution. Confirmed ordinary-endpoint failures on 2026-10-11: an all-history
|
||||
readback of exactly 262144 bytes of incomplete JSON (`invalid agent messages`), a
|
||||
reviewer result with a duplicate nested `performance_review` field, and an
|
||||
onboarding visual-review result with `invalid character 'z' after object key:value
|
||||
pair at byte 3968` where unsafe lossless repair was rejected. These are **mocked
|
||||
`internal/stage` fixtures**: executable source-level evidence, not live-provider
|
||||
evidence and not a deployment. OpenCode V2 prose `output_contract` does not enforce
|
||||
structured output; V2 results are validated locally and no unsupported V2 format
|
||||
field is sent. A review-only fresh execution is a future recovery idea, **not
|
||||
implemented**. Onboarding UI visual approval remains unresolved and must not be
|
||||
reported complete.
|
||||
|
||||
## Bounded malformed-result recovery: 2026-10-16
|
||||
|
||||
Source-level, mocked `internal/stage` regression coverage (not live-provider
|
||||
evidence and not a deployment): malformed final-result recovery now tries a
|
||||
deterministic local pass first (append missing closers, drop one trailing comma,
|
||||
or insert the single comma/colon the parser expected), then at most one deny-all
|
||||
formatter request whose deadline is derived from the payload size, the stage
|
||||
timeout and the attempt's remaining budget rather than a fixed 30-second
|
||||
constant. Every path
|
||||
records `repaired`, `partial` or `unrecoverable` in `format_repairs`; a `partial`
|
||||
or `unrecoverable` payload is never returned as a completed result, and an
|
||||
ambiguous payload (an unescaped quote/colon inside a string value) is never
|
||||
guessed at. A shell-enabled stage's malformed but deterministically recoverable
|
||||
result now completes without replaying admitted tool work; `safeToReplay` is
|
||||
unchanged. The run-55 `ui_reviewer` payload was retrieved read-only with
|
||||
`pipelines inspect 55 --json` and reproduced its exact byte offset 7837; it is the
|
||||
ambiguous error class, so it is reported unrecoverable rather than repaired. This
|
||||
does not update or restart any running server.
|
||||
|
||||
## Harness transport reach retry and repair deadline: 2026-10-17
|
||||
|
||||
Source-level, mocked `internal/stage`, `internal/opencode` and `internal/server`
|
||||
regression coverage (not live-provider evidence and not a deployment): the
|
||||
harness transport path now distinguishes a pre-admission reach failure
|
||||
(connecting or creating the session) from a post-admission readback/poll/agent
|
||||
failure under the one `HARNESS TRANSPORT` category. A pre-admission reach failure
|
||||
replays nothing, so it is retried within `retries` for any stage with a fresh
|
||||
session and no replayed prompt; a post-admission failure is retried only on an
|
||||
explicit read-only stage, and an `edit`/`shell`/legacy stage still never replays
|
||||
admitted work. A context that expires during a request keeps both the sanitized
|
||||
transport detail (operation/status/exit) and
|
||||
`errors.Is(err, context.DeadlineExceeded)`; a bare attempt-boundary deadline is
|
||||
not treated as a reach failure and stays terminal for a mutating stage. An
|
||||
unrecognized provider execution-error name is reported as a bounded, sanitized
|
||||
identifier plus any HTTP status. The isolated formatter deadline is now
|
||||
proportional to the stage timeout (floored, capped, and clamped to
|
||||
`remaining - margin` so it can never extend the stage/attempt deadline). These are
|
||||
mocked fixtures; a deployed server is not updated by this change.
|
||||
|
||||
## Failure-handling source changes: 2026-10-15
|
||||
|
||||
Source-level, mocked `internal/stage` and `internal/server` regression coverage
|
||||
(not live-provider evidence): an explicit read-only tools stage now actually
|
||||
retries a post-admission stage deadline or harness transport failure, creating a
|
||||
fresh bounded session per attempt, while a stage with `edit`/`shell` or no tools
|
||||
block still never replays admitted work; a malformed final result is re-emitted
|
||||
up to `retries` only on such a read-only stage with its `format_repairs` evidence
|
||||
retained. A timed-out agent stage settles knowable V2 usage and retains one
|
||||
bounded, redacted latest-assistant readback plus a `retained_session` pointer in
|
||||
its stored transcript. A declared outcome routed to `#failure` with no step
|
||||
diagnostic now yields a run diagnostic like `stage <key> returned outcome
|
||||
"blocked"`; an existing step diagnostic is still used verbatim.
|
||||
|
||||
## Dated source-level coverage: 2026-10-11
|
||||
|
||||
Source-level/mocked regression coverage (not live-provider evidence):
|
||||
`resolve-releases` now uses a single bounded
|
||||
`GET .../git/matching-refs/tags/v2` response that the GitHub REST docs describe as
|
||||
complete with no pagination; offline fixtures cover a 100+ ref array,
|
||||
`MAX_REFS`/`MAX_BODY` caps, ignored prerelease/build tags, malformed refs and a
|
||||
missing newest-tag digest failing closed with no older fallback, with no
|
||||
`tags/list` cross-check. Picker tests cover blink/index preservation,
|
||||
ANSI-readable selection without a colliding background, and
|
||||
40x16/80x24/120x36 height budgets. The compiled picker PTY check (opt-in via
|
||||
`PIPELINES_HARNESS_PTY_DIR`) waits through blink intervals and asserts exact
|
||||
saved/arrow-scrolled models. `setup-docker.py` accepts the actual
|
||||
`opencode v2.0.26` probe token and compares source identity only on a full
|
||||
install; model/login-only reconfiguration compares the verified release image
|
||||
identity. Python fixtures bind temporary HOME directories so the suite passes
|
||||
under the container default `/home/pipelines` without weakening mount guards.
|
||||
The Docker authoring transport is also covered at source level: the shared
|
||||
prefix picks `/usr/local/bin/pipelines` for a Docker-linked profile. Loopback is
|
||||
recognized only via `net.ParseIP(...).IsLoopback()` and an exact case-insensitive
|
||||
`localhost`, so a remote DNS name such as `127.remote.example` stays remote with
|
||||
no probe. A single bounded `docker compose port` probe parses complete
|
||||
`host:port` bindings, and scoped source/target aliases are embedded only when a
|
||||
binding covers the requested loopback address and port; malformed or non-matching
|
||||
bindings and a loopback `wss` endpoint fail closed, and `client.Dial` rewrites
|
||||
only an exact source match. `provider-check` dispatches through a Docker-linked
|
||||
profile's existing service with `-T`, the original cwd/model/timeout, exact
|
||||
stdout/exit propagation and `--json` passed only in JSON mode (fixture tests, no
|
||||
real provider), while explicit native profiles keep the host generator. The
|
||||
picker uses a true `x/term` TTY check so `/dev/null` and pipes fail fast, with a
|
||||
short context deadline so a regression fails rather than hangs. Real Docker
|
||||
rebuild/reconfigure against live registries, a real
|
||||
40x16/80x24/120x36 terminal capture, and real model inference remain **not tested**
|
||||
in this change.
|
||||
|
||||
### 2026-10-11 Pi sign-in handoff (source + Root PTY observation)
|
||||
|
||||
Root separately exercised current-source `pipelines harness setup pi --login`
|
||||
under a real Docker PTY with copied harness preferences. Root observed that
|
||||
Pi 1.1.0 defaults to fullscreen and clears the Pipelines login instructions, so
|
||||
the user sees a coding interface without the `/login` guidance, and that
|
||||
`/login` first shows "Select authentication method" (account / API key / Radius)
|
||||
rather than a provider picker. The same handler run with a temporary executable
|
||||
wrapper passing `pi --tui-mode regular` retained the Pipelines header and
|
||||
instructions above Pi's interface. Installed `pi --help` documents the supported
|
||||
`--tui-mode regular` flag and the fullscreen default; `pi auth --help` confirms
|
||||
there is no standalone login (only print/check, never run). These are source/root
|
||||
observations with copied preferences: **no provider was selected, no key was
|
||||
entered and no inference occurred**, and this source is **not deployed** and is
|
||||
not live-provider evidence. A first reviewer capture at 80x30 found the longer
|
||||
handoff plus Pi's own banner scrolled the Pipelines header off the top, so the
|
||||
copy was shortened to a title plus three short lines (existing-credential reuse
|
||||
folded into the `/login` line, with the account/API key/Radius method choice and
|
||||
`ctrl+c`/`ctrl+d` exit keys kept) and the redundant trailing blank removed.
|
||||
Independent PTY re-capture at 80x30/100x36 after this shortening is **not
|
||||
tested** in this change. The matching exact login-argv and copy assertions are
|
||||
covered by fixture tests in `internal/cli`/`internal/harness`.
|
||||
|
||||
### 2026-10-11 server CLI fixture isolation (source-level + Root-reported run)
|
||||
|
||||
Root-reported runtime evidence supplied for this change (not re-executed by the
|
||||
authoring stages): the deterministic seeded-profile regression
|
||||
`go test ./tmp/preview-verification -run TestServerCLIFixtureDoesNotInheritUserDockerProfile -count=1`
|
||||
seeds a synthetic Docker-linked harness profile in a child
|
||||
`go test ./internal/server -run '^TestCLIHeadlessRunInspectionCancellationAndRPC$'`.
|
||||
It previously failed because the child inherited the caller's real
|
||||
`PIPELINES_HARNESS_CONFIG`, so the CLI `debug` prefix failed closed
|
||||
("refusing to redirect") against the test's ephemeral httptest endpoint. Root's
|
||||
four independent preview tests for the bounded transcript helpers pass; Root's
|
||||
full host race run produced no race warning and only this fixture failure, and
|
||||
`go vet ./...` passed. Current source therefore isolates the server CLI fixture
|
||||
with the same missing-temp-file
|
||||
`t.Setenv("PIPELINES_HARNESS_CONFIG", filepath.Join(t.TempDir(), "harnesses.json"))`
|
||||
pattern already used by `internal/cli`'s setup, so the host executable stays
|
||||
authoritative and no Docker probe runs for an ephemeral test endpoint. The
|
||||
production fail-closed redirect guard in `internal/harness/prefix.go` and its
|
||||
`refusing to redirect` / loopback-wss regressions are unchanged; the failing
|
||||
verification was fixed rather than removed.
|
||||
|
||||
This is a current-source and Root-reported observation. At the time of that
|
||||
observation, independent re-execution of the full `go build`/`go vet`/`go test`
|
||||
matrix (including Docker-only race results) and the final installed-host Docker
|
||||
verification were **pending**; they are now recorded in the 2026-10-11 FINAL
|
||||
INSTALLED section below, so this phase-specific wording is superseded by that dated
|
||||
install. The packaged reference describes source behavior; a source fix or updated
|
||||
skill is not deployment evidence.
|
||||
|
||||
### 2026-10-11 retained run11 reviewer re-capture (source-level, not a final install)
|
||||
|
||||
Independently reading the retained run11 verification at
|
||||
`tmp/maintenance-run11-verification.json` records the source-level review
|
||||
sequence: an earlier reviewer PTY capture rejected the longer handoff because at
|
||||
80x30 it scrolled the Pipelines header off the top; the implementer then
|
||||
shortened `AuthHandoff` to a title plus three short lines; and the re-run
|
||||
`reviewer` and `ui_reviewer` stages approved from real current-source Pipelines
|
||||
PTY captures at 80x30 and 100x36 that showed the header and handoff visible above
|
||||
installed Pi 1.1.0's regular interface. A labelled controlled fixture that
|
||||
stripped `--tui-mode regular` reproduced the pre-fix fullscreen clear, and the
|
||||
review stages report no provider selected, no key entered and no inference. This
|
||||
is source-level reviewer evidence, read from retained output; it was not a
|
||||
deployed/installed state at that time, and the final full Docker install
|
||||
verification was then **pending** — it is now recorded in the 2026-10-11 FINAL
|
||||
INSTALLED section below, so this phase-specific wording is superseded by that dated
|
||||
install.
|
||||
|
||||
Root-reported actual runs (source-level observations supplied by Root for this
|
||||
change and not re-verified here): run7 showed both configured providers' exact
|
||||
handoff markers; runs6/9/10 exercised live maintenance with final-assistant
|
||||
polling/readback succeeding. These are Root observations, not mocked-regression
|
||||
coverage and not a completed installation; no final deployed/installed state is
|
||||
claimed.
|
||||
|
||||
### 2026-10-11 run13 interruption, Docker race limitation and state separation
|
||||
|
||||
Four states are kept explicitly separate and must not be conflated:
|
||||
|
||||
1. **Current source** (this checkout, not deployed): bounded 32 KiB
|
||||
(`transcriptBound = 32<<10`) redacted transcript previews produce a valid JSON
|
||||
envelope with redaction applied before truncation and a rune-safe 8 KiB preview;
|
||||
session/debug evidence is scoped `latest_bounded_evidence` with the single
|
||||
bounded latest-assistant readback and at most the 3 most recent session-bearing
|
||||
steps; an unbudgeted V2 audited agent stage records reported usage best-effort
|
||||
through exactly one 2s-bounded final session-aggregate read, leaving
|
||||
missing/invalid usage unknown; and the isolated server CLI fixture uses the
|
||||
missing-temp-file `PIPELINES_HARNESS_CONFIG` pattern while the production
|
||||
fail-closed redirect guard in `internal/harness/prefix.go` is unchanged. Pi JSONL
|
||||
is never parsed as one JSON object; native V1 keeps its existing path.
|
||||
2. **Host verification evidence independently recorded by Root**: after the fixture
|
||||
fix, Root ran the full `go test -race -count=1 ./...` and `go vet ./...`; after
|
||||
three harness-neutral evidence labels and Step wording changes Root re-ran the
|
||||
affected `tui`/`author`/`cli` race and vet checks, the Python suite (80) and the
|
||||
opt-in compiled picker PTY check (4). These are Root-recorded host results, not
|
||||
re-executed by the authoring stages.
|
||||
3. **Deployed old image**: the running Docker image predates these source changes
|
||||
and does not include them.
|
||||
4. **Final install (completed 2026-10-11)**: the final install and installed
|
||||
authoring/provider verification were **pending** when this separation was
|
||||
written and are now recorded in the 2026-10-11 FINAL INSTALLED section below; an
|
||||
updated skill/client or source checkout is still not deployment evidence on its
|
||||
own.
|
||||
|
||||
Docker race limitation (confirmed): the Docker runtime is built with
|
||||
`CGO_ENABLED=0` and has **no C compiler** (the Alpine runtime stage installs no
|
||||
GCC/musl-dev), so `go test -race` cannot run there. In-Docker reviewers must use
|
||||
ordinary `go build`/`go test`/`go vet` and rely on independently executed host
|
||||
`-race` runs. Race checks must not be attempted in Docker and no compiler may be
|
||||
installed or searched for.
|
||||
|
||||
Run13 interruption (dated evidence, not a fix): the previous reviewer ran `find /`
|
||||
looking for GCC; the tool timeout left a process in disk sleep, Docker could not
|
||||
stop the container, and an approved Desktop restart recovered run13 as interrupted
|
||||
(`executing=false`, every step `agent_active=false`) with **no automatic replay**.
|
||||
This is an environment/interruption observation only: no general cancellation fix
|
||||
is inferred and process termination is unchanged. Docker startup recovery already
|
||||
sets running steps to `interrupted` with `agent_active=0`.
|
||||
|
||||
### 2026-10-11 short-lived CLI inherited-pipe cancellation (current source)
|
||||
|
||||
Current source (not deployed) adds one shared, dependency-free constructor in
|
||||
`internal/opencode`: `CommandContext(ctx, executable, args...)` sets the single
|
||||
finite `os/exec.Cmd.WaitDelay` constant `ShortLivedWaitDelay = 250ms` and is the
|
||||
only place that value is defined. It is reused by the `internal/stage`
|
||||
`OpenCodeClient.call()` API/agent path (including `Interrupt`), `opencode.Detect()`
|
||||
`--help` and the native `Connection.Messages()` export fallback. It bounds
|
||||
**draining of inherited output pipes only**: after the child exits os/exec waits at
|
||||
most the grace for descendants holding the inherited stdout/stderr pipes, then
|
||||
closes them and reports `exec.ErrWaitDelay`; it does not terminate or confirm
|
||||
cleanup of descendants.
|
||||
|
||||
`context.Canceled`/`context.DeadlineExceeded` identities, `exec.ErrWaitDelay` on
|
||||
the sanitized errors, the ≤64 KiB request / ≤1 MiB response / 8 KiB stderr bounds,
|
||||
`boundedBuffer`/`boundedOutput` drain-with-overflow behavior, redaction, strict
|
||||
result validation and cleanup-unconfirmed active ownership (`ErrStageCleanup`, no
|
||||
replay) are unchanged; no new module dependency is added and a parent that exits
|
||||
successfully while a descendant still holds partial stdout is never treated as
|
||||
success.
|
||||
|
||||
Dated synthetic reproduction (`tmp/interrupt-verification`, synthetic
|
||||
real-subprocess diagnostics, **not provider evidence**): with Root waiting for the
|
||||
descendant itself to start before cancelling public
|
||||
`OpenCodeClient{Backend:"v2"}.Interrupt`, two Docker/Linux returns took
|
||||
3996/3995ms despite `context.Canceled` (`descendant-started.jsonl`); the stronger
|
||||
host reproduction took 4003/4000ms (`reproduction.log`). Added source-level
|
||||
regressions use real `os/exec` fixtures with start-marker barriers, bounded
|
||||
watchdogs and naturally terminating children for cancel-after-descendant,
|
||||
deadline identity, descendant-held pipes → `ErrWaitDelay`, ordinary
|
||||
output/exit/redaction, failed-interrupt ownership/no replay, plus `Detect`/export
|
||||
wait-delay and cancellation cases. This is **current source, not deployed**:
|
||||
descendant termination, confirmed remote interruption and any run13 Docker disk
|
||||
sleep fix are **not claimed**, and Docker race remains unavailable (no C
|
||||
compiler).
|
||||
|
||||
### 2026-10-11 FINAL INSTALLED verification (friend-ready preview)
|
||||
|
||||
This is the dated receipt for the installed runtime, recorded by Root from
|
||||
`tmp/friend-final-verification/summary.json`; it supersedes the earlier
|
||||
"pending final install" statements above, which stay as phase-specific history.
|
||||
A normal interactive `./install --model opencode-go/deepseek-v4.1-flash
|
||||
--harness-model opencode-go/deepseek-v4.1-flash` exited **0**, resolving the current
|
||||
releases live and building fresh dirty source into actual image
|
||||
`sha256:1810ab50f3ea3991e7f3548c79250e65a114bb1821eac52efd872b91cf7c3136`
|
||||
(OpenCode 2.0.26 / Pi 1.1.0 / Node 24.18.1) with real tool-free provider
|
||||
validation, ntfy authenticated publish, server + host health, home/workspace entry,
|
||||
preserved authoritative endpoint and `ok` pre-install database/home backup
|
||||
integrity. Post-install preservation all passed (original definitions 1–4, history
|
||||
1–16, host/container default Pi and both models/profiles, logical endpoint,
|
||||
service URLs/timezone/ports/models, Pi credential hash, OpenCode connected accounts,
|
||||
ntfy token); neither original-repo commit/push nor credential/data cleanup ran.
|
||||
Installed host-CLI authoring with **both** Docker harnesses (separate retained
|
||||
drafts, each changing only the worker prompt) performed real `ping` → `pong` and
|
||||
`validate` → `validated` through the generated Linux `/usr/local/bin/pipelines` and
|
||||
scoped transport aliases preserving the logical `8081` endpoint and draft metadata,
|
||||
independently inspected from native OpenCode tool outputs and retained Pi JSONL
|
||||
`toolResults`, with no draft submit/run/publication. Docker-proxy submit on
|
||||
verification-only definition 5 published v2, a second stale v1 draft was rejected as
|
||||
conflict, and **Run17** succeeded in 7 seconds with the exact
|
||||
`pipelines-docker-verified` markers in OpenCode then Pi, both inactive and exactly
|
||||
one completed marker read verified from retained native history (not the
|
||||
latest-assistant preview); best-effort unbudgeted usage was OpenCode 4011
|
||||
tokens/$0.00029769599999999996 and Pi 1071 tokens/$0.000142002, and
|
||||
`/api/session/active` was empty after exit. Installed-UI captures
|
||||
(`installed-pi-*`, `pi-signin-*`, `installed-pi-models-*`) cover the actual run17
|
||||
80x30/120x36 neutral-Pi/green-both-stage renders, the Pi auth handoff/method menu
|
||||
viewed/canceled/exit0 with no sign-in, and the model wizard using copied
|
||||
preferences only while production preferences stayed preserved. Root's independent
|
||||
checks passed: full host `go test -race -count=1 ./...` and `go vet ./...`,
|
||||
Python 80 tests (1 optional PTY skipped), Node policy, isolated developer lifecycle
|
||||
fixture (no inference/deploy) and author package tests; earlier independent
|
||||
reviewers ran ordinary Docker build/vet/tests and focused subprocess regressions
|
||||
(no Docker race, no GCC search, no global `find`). **UNVERIFIED** here: fresh
|
||||
OAuth/provider login completion, phone delivery, other real providers, the
|
||||
D-disk-sleep fix and active lease expiry; authentication menus and
|
||||
existing-credential inference are not fresh sign-in proof. The stronger synthetic
|
||||
PUBLIC `Interrupt` child-start barrier (host 252/252 ms, Docker 250/251 ms, context
|
||||
canceled true, bounded true vs ~4 s before) is not proof of descendant termination,
|
||||
remote cleanup or a Docker VM disk fix. Archive extraction, build-identity and
|
||||
compile verification remain pending until Root executes them.
|
||||
|
||||
### 2026-10-11 generated release receipt ignore (source-hygiene regression)
|
||||
|
||||
Current source adds two root-anchored `.gitignore` rules for the installer's
|
||||
machine-local generated artifacts: `/.pipelines-releases.json` (the receipt path
|
||||
from `setup-docker.py` `release_receipt_path()`) and `/.pipelines-releases-*` (the
|
||||
`tempfile.mkstemp` `prefix=".pipelines-releases-"` in `record_release_receipt()`);
|
||||
no installer/runtime code changed and the pre-existing receipt and other files are
|
||||
retained. A new `scripts/test_build_metadata.py` case is a controlled, isolated Git
|
||||
fixture using the repository's actual `.gitignore`: a clean tracked source plus a
|
||||
representative generated receipt/`.pipelines-releases-*` temporary file leave `git
|
||||
status --porcelain` empty and `build_metadata.docker_args` at
|
||||
`PIPELINES_DIRTY=false` with no `-dirty` version; editing the tracked source then
|
||||
makes the status non-empty and `docker_args` report `PIPELINES_DIRTY=true` with a
|
||||
`-dirty` version. This is **offline source-level regression coverage**, not the
|
||||
recorded actual new-account install/provider login above and not live-provider or
|
||||
deployment evidence; this docs-and-ignore change has no rendered UI.
|
||||
|
||||
## Dated live reference validation: 2026-10-10
|
||||
|
||||
Environment: Docker Linux aarch64, OpenCode v2.0.20,
|
||||
**opencode-go/deepseek-v4.1-flash**. Isolated databases/fixtures and local HTTP
|
||||
notification endpoints; normal published pipelines/repositories were not changed.
|
||||
|
||||
- Incident extraction → report: exact IDs/severities/counts, unchanged handoff,
|
||||
retained session readback and one local success receipt passed (6.63s).
|
||||
- File edit → verification: exact final release-file bytes, unchanged fixtures,
|
||||
independently executed `cmp`, completed read/edit/shell evidence, retained edit
|
||||
diff and no verifier edits passed (14.60s).
|
||||
- Missing required input: blocked route ended failed; missing file was not invented;
|
||||
retained read attempt and one local failure receipt passed (4.21s).
|
||||
- Accounting/one-token budget initially failed its acceptance check because session
|
||||
zeros let the run succeed. After the tracked-prompt fix, a focused live rerun
|
||||
passed: **952 reported tokens, $0.00016440**, budget-exceeded failure, stopped
|
||||
owned session (2.43s). This is not an independently reconciled provider bill.
|
||||
|
||||
The earlier three workflows were rechecked from retained artifacts with current
|
||||
assertions, without more inference. There was **no fresh full-suite live run of
|
||||
all four workflows after the fix**. Do not generalize these results to all five
|
||||
starter templates, other models/providers, or compatible native schema/tool usage.
|
||||
Real ntfy authentication/storage, phone reception, provider outages/timeouts,
|
||||
long-stream interruption and promptness of cleanup were **not tested** by this trial.
|
||||
The trial ran rebuilt test binaries; it did not deploy the fix to the ordinary server.
|
||||
|
||||
## Dated live reference validation: 2026-10-11 (run7)
|
||||
|
||||
One existing real run7 (2026-10-11) is the retained live evidence dated for this
|
||||
change: it executed both configured harnesses, OpenCode 2.0.26 and Pi 1.1.0, with
|
||||
`opencode-go/deepseek-v4.1-flash` stages on the Docker server, and both steps
|
||||
succeeded (6s; the retained Pi evidence panel reports 1054/10000 tokens). The
|
||||
captures are retained at `tmp/friend-ui-verification/source-pi-80-*` and
|
||||
`source-pi-120-*`, and a current-source TUI inspection of that run recorded actual
|
||||
80x30 and 120x36 renderings, including the run graph and the bounded evidence
|
||||
panel.
|
||||
|
||||
This is a single real run, **not** a fresh full reference suite. Other providers,
|
||||
real outages/long-stream interruption, fresh OAuth and phone delivery remain
|
||||
**not tested**; the new-source installer and installed-authoring verification were
|
||||
separately completed by the dated 2026-10-11 FINAL INSTALLED section above. The
|
||||
[2026-10-10 OpenCode v2.0.20 trial](#dated-live-reference-validation-2026-10-10)
|
||||
above remains dated history.
|
||||
|
||||
## Reference suite when explicitly requested
|
||||
|
||||
The source checkout's `TestReferenceWorkflowsLive` uses six model stage attempts
|
||||
plus possible compatibility/format-repair calls. Retries are disabled and stages
|
||||
have two-minute timeouts. It uses real existing credentials and retains private
|
||||
`runs.db`, pinned definitions, fixture directories, run/session JSON and report.
|
||||
The accounting case has a one-token/$0.25 budget; other cases have no spending
|
||||
limit. Test notifications use a local endpoint, not the configured phone topic.
|
||||
|
||||
Live inference requires explicit `PIPELINES_REFERENCE_LIVE=1`, a chosen
|
||||
`PIPELINES_REFERENCE_MODEL=PROVIDER/MODEL`, and an existing private
|
||||
`PIPELINES_REFERENCE_DIR`. Get consent for chargeable inference and confirm which
|
||||
execution computer/account to use. Ordinary tests skip live inference. Do not
|
||||
search the repository or run the suite during normal draft authoring.
|
||||
|
||||
If the user explicitly asks to run the suite and provides a source checkout:
|
||||
|
||||
```sh
|
||||
PIPELINES_REFERENCE_LIVE=1 \
|
||||
PIPELINES_REFERENCE_MODEL=PROVIDER/MODEL \
|
||||
PIPELINES_REFERENCE_DIR=/private/approved/artifacts \
|
||||
go test ./internal/server -run '^TestReferenceWorkflowsLive$' -count=1 -v -timeout 15m
|
||||
```
|
||||
|
||||
Docker credentials require running the correctly compiled test binary in that
|
||||
container; do not copy host credentials or publish tests to the normal database.
|
||||
`TestReferenceDefinitions` checks definitions without inference.
|
||||
`TestReferenceArtifactsReview` rechecks copied run/fixture evidence without
|
||||
inference using `PIPELINES_REFERENCE_REVIEW_DIR`; select only cases present in
|
||||
that artifact directory. Missing cases skip and must not be reported as passed.
|
||||
|
||||
## 2026-10-12 run22 success, run23 live CSV fixture and run24 interruption
|
||||
|
||||
Source/structural and live-fixture evidence stays separate from runtime
|
||||
deployment; nothing here is a deployment claim.
|
||||
|
||||
- **run22 (structural/source checks, success):** the retained run22 verification
|
||||
passed its structural/source checks of the scoped template and prompt changes.
|
||||
- **run23 (live Pi CSV fixture):** a real exact-template Pi CSV run fetched the
|
||||
official Python `csv` documentation through the conditional official-source
|
||||
webfetch and implemented CSV parsing with the standard library's `csv.reader`
|
||||
through an existing shared helper, adding no duplicate preview parser; both the
|
||||
implementer and reviewer checks actually passed. Root independently verified the
|
||||
full retained native tool results and protected-file hashes against its saved
|
||||
baseline, rather than relying on model-written reports.
|
||||
- **run24 (interrupted, not a pass):** the run24 implementation completed, but its
|
||||
reviewer ignored the scope instruction and ran a prohibited filesystem-root scan
|
||||
(`find /`), leaving a process in uninterruptible D-state and stalling Docker
|
||||
mount/snapshot reads. Root canceled the run, made a consistent SQLite backup
|
||||
(`integrity ok`), recovered Docker with the supported stop/start path, and
|
||||
restored the existing ntfy/Pipelines services with the original definitions, run
|
||||
history, credentials and settings verified unchanged. run24 is retained as
|
||||
**interrupted**, not a success.
|
||||
|
||||
- **run25 (live Pi source-only review):** the reviewer identified duplicated
|
||||
implementation, a hardcoded special case, suppressed errors and test gaps.
|
||||
Root inspected retained native calls: only local read/search tools ran, with no
|
||||
shell/edit/network calls. Original fixture hashes were unchanged; runtime
|
||||
behavior was explicitly marked untested.
|
||||
- **run26 (final maintenance review, success):** the scoped plan/implement/review
|
||||
workflow passed package tests, vet, build and formatting checks. Root inspected
|
||||
the actual retained outputs and changes. No rendered UI change was claimed.
|
||||
|
||||
The scoped prompt guidance asks stages to keep inspection/verification to relevant
|
||||
repository paths and supported commands; it is a working instruction, **not** an OS
|
||||
sandbox, and neither fixes Docker VM/disk bugs nor universally enforces tool
|
||||
boundaries. At these recorded checkpoints the source changes had not been deployed.
|
||||
Catalog deployment requires a separate check of the running server and its actual
|
||||
template contents; these observations do not establish publication or deployment.
|
||||
Other providers and phone delivery remain **not tested** by this record.
|
||||
|
||||
## 2026-10-12 final template verification and Docker catalog deployment
|
||||
|
||||
The final three-stage template ran as live Pi run27 (verification definition7,
|
||||
version2) in a fresh disposable CSV fixture. Root inspected full retained native
|
||||
results: the planner fetched official Python CSV documentation, the implementer
|
||||
used the standard library through the existing shared parser, and implementer and
|
||||
reviewer independently ran `python3 checks.py` successfully. Root reran those
|
||||
checks, verified the protected fixture hashes and planner-owned review criteria,
|
||||
and confirmed the preview reused the shared parser.
|
||||
|
||||
The Docker server was then refreshed from clean source commit
|
||||
`143f237268dbde2efd6c5ce5722a3e1fd7e08c9c`, retaining OpenCode 2.0.26, Pi 1.1.0
|
||||
and Node 24.18.1. The running server reported that exact commit and `dirty=false`;
|
||||
all five served templates matched current source with the saved Pi/model defaults.
|
||||
A consistent pre-deployment database backup passed integrity checks. Every stored
|
||||
definition version/source, retained run ID, saved setting, endpoint and provider
|
||||
credential/account checked by Root was preserved. This record is a documentation
|
||||
follow-up to that deployed build; it changes no runtime code or template source.
|
||||
|
||||
These bounded fixtures establish the observed behavior on this instance, not a
|
||||
guarantee of every future model review, provider, task or platform. They do not
|
||||
establish fresh account sign-in or phone notification reception.
|
||||
`pipelines inspect RUN_ID --json` exposes each model step's `tokens_total`,
|
||||
`tokens_in`/`tokens_out` (only when `split_known`), `cost_usd` (only when
|
||||
`cost_known`) and the model it ran with, plus the run's totalled `usage`;
|
||||
`pipelines stats --json` adds history-wide totals. Token totals include cache
|
||||
reads. An unreported cost or split is shown as unavailable, never zero; runs
|
||||
recorded before usage tracking omit `usage`. Budgets are reported-usage limits,
|
||||
**not provider-side hard caps or a guaranteed bill ceiling**; in-flight calls can
|
||||
overshoot.
|
||||
|
||||
## Orchestration verification habits
|
||||
|
||||
A run's status and a stage's summary are **claims**, not evidence. When the agent
|
||||
manages runs rather than making the change, its job includes independently checking
|
||||
what each run claims.
|
||||
manages runs rather than making the change, its job includes checking what each
|
||||
run claims.
|
||||
|
||||
- **Read the retained evidence yourself** — `inspect RUN_ID --json` for the exact
|
||||
stage outputs, outcomes, diagnostics, file changes and artifacts — instead of
|
||||
trusting the model-written summary of the run.
|
||||
- **Run the check.** Where an item claims a build or tests, run the exact command and
|
||||
read its output; "tests pass" without a command and its observed result is not a
|
||||
check. Never report a check you did not execute.
|
||||
- **Read the images.** Open the screenshots the reviewer cites; do not accept a
|
||||
description of an image you can open. Confirm a capture is a **real image** (magic
|
||||
bytes or the reported content type), not a text file named `.png`
|
||||
(`references/debugging.md`).
|
||||
- **Compare the built artefact with what was delivered.** Check the md5 (or
|
||||
equivalent) of what was built against what is installed or published, so a stale or
|
||||
substituted file cannot pass as the artefact (`references/delivery.md`).
|
||||
- **State "not tested" explicitly.** A run that `succeeded` means its stages reported
|
||||
success; it is not proof that the operator's acceptance criteria were met. Name what
|
||||
was not covered.
|
||||
- **Report failures as failures.** Never re-run until something passes and then call
|
||||
that verified; record the failure and its category.
|
||||
- **Read the retained evidence yourself**: `inspect RUN_ID --json` for the exact
|
||||
stage outputs, outcomes, diagnostics, file changes and artifacts.
|
||||
- **Run the check.** Where an item claims a build or tests, run the exact command
|
||||
and read its output. Never report a check you did not execute.
|
||||
- **Read the images.** Open the screenshots a reviewer cites and confirm each is a
|
||||
real image (magic bytes or content type), not a text file named `.png`.
|
||||
- **Compare the built artefact with what was delivered** (md5 or equivalent), so a
|
||||
stale or substituted file cannot pass (`references/delivery.md`).
|
||||
- **State "not tested" explicitly.** A `succeeded` run means its stages reported
|
||||
success, not that the operator's acceptance criteria were met.
|
||||
- **Report failures as failures.** Never re-run until something passes and then
|
||||
call that verified; record the failure and its category.
|
||||
|
||||
Reference in New Issue
Block a user