The operator's chat list showed ~99 stray "New chat" entries. The listing
was already fail-closed (registry-only); the fault was what got recorded: an
automated UI check drives the real page, so its throwaway chat was recorded as
an operator chat exactly like a manually created one and stayed in the list for
ever (each run added one).
Server: chat_create takes an optional ephemeral flag. The session is created,
bootstrapped, usable, closable and reopenable exactly like any other, but its
registry row is hidden, so chat_sessions never lists it. operator_sessions gains
an ephemeral column (idempotent migration, existing rows default to listed) and
the list query filters it; the single-row lookup does not, so close/reopen keep
working. RecordVerificationSession is the hidden recorder.
Web: the page marks a create ephemeral only when automation asks for it
(__pipelinesEphemeralChat, set by the UI smoke for its throwaway chat), so the
smoke's own runs stop polluting while a manual create is unchanged.
Tests: a hidden row is never listed yet is still looked up and closable; a
normal re-record of the same id lists it again; a pre-ephemeral database
migrates with every existing chat still listed. Live data pruned separately
(107 -> 8 registry rows, backed up).
Add local operator inspection and reconciliation for quarantined resource leases, including atomic audit-backed FIFO handoff and recovery protocol support.\n\nPreserve capability tokens in the authoritative store and reject non-loopback or ordinary client transport for recovery mutations.\n\nChecked packaged templates and packaged authoring skill references; none need updating. The task-owned docs note records shared README and skill reference follow-up for docs-fold.
Use an explicit chat_create operation_id as the durable bootstrap key while retaining envelope IDs for reply correlation. Legacy requests receive fresh server-scoped identities, so independent cli-1/android-1 requests do not collide; clients that retry retain one operation_id.
Update Android creation state and repository calls to preserve an operation identity across retries, and cover durable reconnect retries, duplicate barriers, changed input, legacy correlation reuse, and protocol serialization.
Checked packaged templates: none describe chat_create. Checked packaged authoring-skill references: the existing description is stale, but shared references and README must not be edited by an item; docs/notes/bootstrap-create-identity.md records the required docs-fold update and rollout compatibility.
Add durable FIFO capacity-one resource leases for executable stages, including cleanup-confirmed release, conservative restart quarantine, queue state events, and CLI/TUI/Android waiting presentation. Add scheduler, cleanup, protocol, client, and Android coverage plus a proposed Android v3 resource draft.
Checked packaged templates and authoring skill references: neither needs updating in this item. The per-item docs note records the shared documentation that docs-fold must update; no shared README or packaged reference was edited.
A stage's session was exported twice, as session.md and session.html, and both
files were presented. It is now one html file; the markdown export is retired.
- internal/stage/session_export.go: ExportSession renders and writes exactly one
file, session.html, via writeSessionFile; renderSessionMarkdown and its helpers
are gone; SessionExportResult carries HTML/HTMLBytes only. The source read, the
8 MiB source / 2 MiB rendered bounds, the truncation banner, the
never-overwrite rule and the honest empty-session handling are unchanged.
- internal/protocol/runs.go: SessionExport drops markdown/markdown_bytes (a
record written before the change still decodes; unknown fields are ignored),
and adds LegacySessionMarkdownArtifact with RunStep.PresentedArtifacts(), the
one Go rule that hides the retired session.md from every artifact listing.
- internal/server/runner.go: evidence.Markdown/MarkdownBytes are no longer copied.
- internal/tui: the artifacts panel, the recorded count and the per-visit summary
derive from PresentedArtifacts(), so the terminal never lists session.md.
- web/src/app.js: the spike's renderArtifacts applies the same filter.
- android: the Session section is the html export alone in a WebView — SessionView,
the Markdown/HTML toggle and the session markdown path are removed;
presentedArtifacts(step) filters the retired session.md out of the per-visit
gallery, the run gallery and its count, the verdict/selection lists and the
timeline indicator; data/protocol SessionExport drops the markdown fields.
- Tests pin the single html file, prove no session.md is written and that a
pre-existing one is neither overwritten nor presented, and prove a legacy
export record still loads and its session.md is hidden.
The legacy session.md of an older run stays on disk (dated evidence); it is only
no longer produced and no longer presented.
Checks (macOS, Go 1.24, Android Gradle 8.11.1, 18 Oct 2026): gofmt -l . prints
nothing; go build ./... && go vet ./... clean; go test ./internal/... ./cmd/...
-> 19 packages ok, 0 failures; node --check web/src/app.js ok and node --test
web/test -> 20/20; gradle --offline --no-daemon :app:assembleDebug
:app:testDebugUnitTest -> BUILD SUCCESSFUL.
Packaged references checked: internal/author/skill/references/delivery.md and
operations.md both described the session.md+session.html pair, so both are
updated here to describe the single session.html export and the retired, hidden
session.md. README.md and internal/author/skill/SKILL.md carry no session.md
reference and install.go's managed inventory is unchanged (no reference added or
removed). docs/notes/session-html-only.md records the remaining stale shared
documentation (docs/stage-artifacts.md, docs/app-integration-verification.md) for
the docs-fold writer; shared docs are not edited by this item.
Two gaps in the chat's in-flight feedback.
The turn indicator no longer reads a bare `Working…` for every state. A new
pure derivation (TurnPhase/turnPhase/turnPhaseLabel in ui/chat/ChatFormat.kt)
labels the headline from evidence already on the wire: `Thinking…` while text
or reasoning deltas are still arriving (within TURN_THINKING_WINDOW_MILLIS),
`Running a tool…` while calls are in flight, `Waiting…` otherwise. No
percentage, ETA or invented progress.
The agent's shell commands are now surfaced end to end as their own frame kind,
so a background process (a dev server, a watching build) that outlives its tool
call is visible and clears when it exits.
Server:
- protocol/chat.go adds ChatEventShell, ChatShell (id, status, command, *int
exit — a pointer so exit 0 is not "absent") and ChatEventFrame.shell.
- opencode/chat.go maps shell.created/shell.exited through a new exported
ChatStreamState, attributing created via info.metadata.sessionID and exit via
the ids the stream already surfaced; another session's or an unattributed
shell is ignored, an id-less shell fails the stream. No cwd/file/credential is
carried.
- server/chat_hub.go retains still-running shells (chatShellLimit plus the
shared byte cap) and drops them on exit; chat_stream.go replays them on
chat_subscribe, the shell half of the resume (a shell is not a turn, so the
message snapshot cannot carry it).
App:
- data/protocol/Chat.kt adds ChatShell, ChatEventFrame.shell, EVENT_SHELL,
SHELL_RUNNING/SHELL_EXITED.
- ui/chat/ChatStore.kt stamps ChatThread.lastDeltaAt on each delta (cleared when
the turn settles) and keeps ChatThread.shells across a chat switch; a shell
never marks the thread running. streamResumed drops shells so the subscribe
replay re-syncs the still-running ones.
- ui/chat/ChatTranscript.kt: RunningIndicator prints the phase label; a new
always-visible BackgroundProcessBar names each running shell.
- ui/chat/ChatConversationPane.kt shows the bar whenever a shell is running,
whether or not a turn is.
Tests extend the server's event mapping (opencode/chat_v2_test.go,
server/chat_test.go) and the app's pure folds (new
ui/chat/ChatInFlightStateTest.kt, 15 tests), pinning the shell mapping, the
other-session rejection and the "background process outlives the turn" state.
Checked and intentionally unchanged:
- No packaged OpenCode skill reference documents either surface.
internal/author/skill/references/operations.md describes the brokered chat_*
protocol in prose without naming frame kinds or the app indicator, so it stays
correct as-is; no reference was added, so install.go's managed inventory needs
no change.
- This repository has no packaged templates directory.
- Shared docs are not edited here per the map: docs/notes/chat-shell-background
-process.md records what is now stale in docs/cli.md and android/README.md for
the docs-fold writer.
Verification (this session): gofmt -l clean, go build ./... and go vet ./... OK.
Android unit suite and the on-device capture were run in the preceding review
stage; see the note for the live-verification limitation.
The phone's chat transcript rendered one agent reply as ~36 bubbles, each with
its own thinking block, and a finished tool call lost its name and its input.
- Turn fold (android): ChatFormat.kt gains the pure ChatTurn / TranscriptRow /
transcriptRows fold - each run of consecutive assistant messages between two of
the operator's messages is one agent turn - and ChatTranscript renders one
bubble per turn: the turn's single collapsible thinking block (all its
reasoning, in order, growing live while it streams), then its tool-call chips,
then one answer block of just the turn's text parts. AssistantParts/MessagePart
are replaced by AgentTurnItem/UserMessageItem; ChatConversationPane counts the
folded rows.
- Delta part ids (server + client): internal/protocol/chat.go adds
ChatDeltaPartID(messageID, kind, ordinal), mirrored as ChatProtocol.deltaPartId
in data/protocol/Chat.kt, and NormalizeChatEvent keys a text or reasoning delta
on it. OpenCode numbers a message's text and reasoning parts independently from
0, so the old "%s#%d" gave the first text delta and first reasoning part the
same id: the answer was appended into the thinking block, and the server's
retained state (chat_hub.go) re-collided on resume. ChatStore's upsert now
routes a delta through the kind-aware deltaPartId fold, so it stays robust even
against a peer that still shares one id.
- Tool fields (server + client): NormalizeChatEvent takes a call's arguments from
session.tool.input.ended's text as well as session.tool.called's input object
(toolInput), and chat_hub.go's new mergeChatToolPart folds each tool frame onto
the retained part so the progress/success frames that omit name and input never
blank them. ChatFormat.kt's mergeToolPart mirrors that on the client, so the
chip and panel header keep the real tool name and the panel shows INPUT beside
OUTPUT.
Tests: chat_v2_test.go and chat_test.go add the text/reasoning id-collision and
the tool name+input retention cases; ChatReasoningTurnTest, ChatTranscriptTest
and ChatRunningWorkTest cover the turn fold, the collision and the tool lifecycle.
Checks: Go build, vet, gofmt and the package tests pass (go build ./... &&
go vet ./...; gofmt -l . empty; go test ./internal/server ./internal/opencode
./internal/protocol ./internal/stage ./internal/pipeline; go test ./...).
Android passes offline with --no-daemon (assembleDebug and the full
testDebugUnitTest suite, 660 tests). The on-phone observation was not performed:
the device lock was held by another run and adb was not available here.
Documentation inventory checked: no packaged OpenCode skill reference in
internal/author/skill/references/ and no packaged template documents the app's
chat transcript, this delta part-id derivation or the tool-field retention, and
no internal/author/ managed file inventory entry names any touched file, so none
of them needs updating. The shared android/README.md behaviour statements are
made stale by this change and are recorded instead in
docs/notes/chat-transcript-turn-fold.md for the docs-fold writer, per the repo
rule that an item never edits a shared document itself.
Brokered chat accepted an operator-supplied endpoint (-opencode-endpoint /
PIPELINES_OPENCODE_ENDPOINT) but failed closed against the operator's running
V2 service: that API authenticates every request with HTTP Basic auth (fixed
user "opencode" plus a password), the broker had no credential, so the health
probe answered 401 and startup logged "OpenCode endpoint ... is unreachable or
is not an OpenCode service", disabling chat entirely.
Established from the installed binary (opencode v2.0.20), not from docs:
`opencode --help` advertises `api`, `pair`, `serve`, `service` and `auth`;
`GET /api/info` is unauthenticated-401 with www-authenticate Basic realm
"Secure Area"; Basic auth user `opencode` with the password from
~/.local/state/opencode/service.json answers 200; `POST /api/pair` returns a
code and `GET /auth/connect/{code}` returns a token that is then accepted as
the Basic password.
- internal/opencode/pair.go (new): PairEndpoint does the pair exchange over the
shared nativeRequest transport (same auth, bounds and JSON checks);
ReadServiceCredential reads the operator's local service password under the
private-file rule; OpenEndpointAttached resolves the credential in precedence
order - explicit password file, a paired token stored by an earlier run, then
a fresh pairing bootstrapped by the local service credential - stores the
token owner-only and re-pairs once when a stored token is refused (401).
- internal/opencode/endpoint.go: OpenEndpoint wraps probeEndpoint, which returns
the connection even on failure; Connection.unauthorized (set on HTTP 401 by
the request paths, cleared per probe) distinguishes an authentication refusal
from any other refusal.
- cmd/pipelines-server/main.go: the endpoint attach uses OpenEndpointAttached
with the paired-credential file beside the pipeline database. An endpoint that
cannot be attached logs a line naming it and falls back to the server's own
OpenCode instance instead of leaving chat refused.
- internal/protocol/status.go, internal/server/websocket.go: get_status gains an
opencode object (health = endpoint | fallback | spawned | unavailable, plus
the endpoint url), the honest status evidence for the fallback. No credential
is ever placed there.
- Tests: internal/opencode/pair_test.go, internal/protocol/status_test.go and a
TestStatusReportsOpenCodeAttachment case in internal/server/status_test.go.
Repository checks: go build ./..., go vet ./... and gofmt -l . clean;
go test ./internal/... ./cmd/... green.
Packaged templates and skill references: the repository has no template tree
that mentions the OpenCode endpoint, and no packaged reference file is added, so
internal/author/install.go's managed inventory is unchanged. The packaged
authoring reference internal/author/skill/references/operations.md (the endpoint
paragraph around line 289, the "clear refusal and no fallback" wording and the
password-file/username-only credential story, and the missing opencode status
field) is now stale; per the maintenance convention the item does not edit the
shared document and instead records the stale points, with the exact wording
they should take, in docs/notes/chat-persistence.md for the docs-fold writer.
The item adds chat_close/chat_reopen to the brokered chat protocol and the
operator-session registry's closed state. main replaced the per-session claim
model with shared chat rooms (chat_subscribe, a prompt queue, one broker stream
fanned out to every watcher). This merges the two.
Resolved conflicts:
- internal/protocol/chat.go, internal/protocol/message.go: kept both additions -
ChatClose/ChatReopen and main's ChatSubscribe/ChatQueued.
- internal/server/chat_sessions.go: kept the closed fields on a listed session;
dropped the `Busy` assignment, because main no longer marks a session Busy (any
number of clients may watch and write one).
- internal/server/chat_claims.go: main deleted the claim model, so the file goes;
close's "is a turn running" check is now `chatHub.turnInFlight` in
internal/server/chat_room.go, reading the room's `running` flag. That is the
same question the old claim answered, against main's model.
- internal/server/chat_close.go: calls turnInFlight instead of the removed
`active`.
- android AppViewModel.kt, ChatFormat.kt, ChatListPane.kt, Message.kt: kept both
sides - main's search state, queued-prompt handling, `Busy`-free status label
and row/menu shape, plus the item's close/reopen actions, closed chip and
section, close confirmation and "Hide on this device" rename.
Also in this merge: main's new convention (AGENTS.md) that an item does not edit
a shared document; the item's stale-document record for this change lives in
docs/notes/chat-close.md, and docs/cli.md, android/README.md and
internal/author/skill/references/operations.md are left to the docs-fold writer.
Checks after resolving: gofmt clean, go build ./... and go vet ./... clean, the
Go module passes, and the app's :app:assembleDebug and :app:testDebugUnitTest
pass (--offline --no-daemon).
Add a server-side close for operator chats, so finishing with a chat takes it out
of the active list for every client without deleting anything. The brokered
protocol gains chat_close and chat_reopen; chat_close stops a running turn by
reusing the existing broker interrupt (its result carries an interrupted flag, so
a stream is never left running into nothing) and then sets closed_at on the
session's operator_sessions row. The session, its transcript and its history
survive, and because the closed state lives in the registry chat_sessions reports
it to every client, not just the one that asked. An id that is not a recorded
operator session, including a pipeline-owned run session, is refused before
anything is written. chat_reopen clears the state again, and simply prompting a
closed session also clears it (the store clears closed_at on touch).
The registry gains closed_at for a fresh database and an idempotent migration for
one created before it existed. The app renders closed sessions under a labelled
Closed section with a Closed chip, offers a Close action with a confirmation that
says it stops a running turn, marks the chat closed on every device and deletes
nothing, and a Reopen action on each closed row. The device-local hide is renamed
Hide on this device, with a snackbar saying the chat stays on the server, so it is
never confused with the real close. No new dependency, the installer and Docker
files are untouched, and no run is started from the app.
Checked before committing: the packaged templates carry no chat surface and need
no update, and internal/author/install.go's managed inventory needs none either
(this adds no reference file). The packaged authoring skill reference
internal/author/skill/references/operations.md and the shared docs/cli.md and
android/README.md are stale because of this change, but an item no longer edits a
shared document: what is stale and what should change there is recorded in
docs/notes/chat-close.md for the docs-fold writer, which is committed here with
the change.
Checks: gofmt clean, go build ./... and go vet ./... clean, and
go test ./internal/protocol/... ./internal/pipeline/... ./internal/server/...
./internal/author/... pass. The app side is verified with :app:assembleDebug and
:app:testDebugUnitTest (--offline --no-daemon, BUILD SUCCESSFUL); the on-device
close, second-client and reopen check is deferred to visual review.
Not deployed; deploying is the operator's step.
A session was owned by one connection: the claim map in chat_claims.go let the
first client hold it, every other client saw the session Busy and read-only, and
a second client's prompt was rejected. Replace the claim with a per-session
"room" so the operator's phone, tablet and terminal all watch and write the same
conversation, and a pending approval is a shared, exactly-once object.
Server (internal/server, internal/protocol):
- New chat_room.go: one room per session holding the session's single broker
stream, its one bounded prompt queue (8) and the fan-out to every attached
client. A chat_subscribe frame attaches a connection to a session's shared
stream and replays the still-unresolved approval requests the room holds.
- chat_stream.go: handleChatPrompt accepts a prompt from any client. It attaches
the sender, either starts a turn or queues the prompt behind the in-flight turn
and acknowledges it with chat_queued:true ("queued" is the chosen behaviour,
stated on the wire and in docs/cli.md), and echoes the user turn to every other
client attributed to the sender's label. The prompt handler no longer owns the
stream; one turn runner per room drains the queue, so two turns never
interleave. Per-connection write serialisation is unchanged (each chatSub has
its own writer goroutine and bounded buffer).
- chat_permissions.go / chat_commands.go: an approval is reserved by the first
answer and confirmed once delivered; the first answer wins, is recorded with
the answering client's label, and the resolution is pushed to every watching
client as chat_permission with resolved/decision/resolved_by so their cards
clear. A later answer is refused with an error naming the client that answered,
and never reaches the agent twice; a reservation whose reply failed is
released.
- chat_hub.go / chat_sessions.go / chat.go: the hub now owns rooms and the
user-turn id sequence; the resulting user turns are retained (role and client
carried) so a reconnect re-syncs who said what. chat_sessions no longer marks a
session Busy and its clientID parameter is gone; the protocol keeps the Busy
field for wire compatibility only. websocket.go detaches a connection's chat
subscriptions when the read loop ends.
- chat_claims.go is removed, not left vestigial: the only exclusive thing left is
a turn, and the room's queue is that.
App (android):
- PipelinesRepository carries a stable per-device client label (Build.MODEL, with
a fallback) on prompts and replies, and adds chatSubscribe; AppViewModel
subscribes on opening a chat (best-effort, an older server's error frame does
not stop history loading) and records the queued ack.
- ChatStore folds a user frame from another client into the transcript once,
attributed by the sender's label, and never receives its own prompt back, so
the sender's own message is never duplicated. ChatThread.readOnly is gone:
canSend no longer depends on another client, the composer says "Queued - it
will send when the current turn finishes." instead of disabling input, and the
status label is Running/Idle only. Scroll/follow behaviour is untouched.
- Tests: ChatProtocolTest round-trips the new wire fields, ChatFormatTest asserts
the stale Busy flag cannot produce a read-only label, MultiChatStateTest asserts
a session another client watches stays writable.
Reflected where it belongs: no packaged template changes (no bundled starter
embeds the chat protocol). Checked the packaged authoring skill references: none
of them describes the chat wire beyond operations.md, and
internal/author/skill/references/operations.md is now stale - its "Brokered
OpenCode chat" section still says chat_sessions marks "busy" when another client
holds the session and that a second client's chat_prompt is rejected, and it does
not mention chat_subscribe, the per-session prompt queue/chat_queued, the
per-client label or the shared exactly-once approval (it also still describes the
per-subscription permission poller, which the room's single poller replaced). The
supported semantics are the ones in docs/cli.md, updated in this commit; the
operations.md section needs a rewrite to match. No README or other shared
document was edited here.
Checks run on the committed tree: gofmt -l internal cmd examples (nothing), go
build ./..., go vet ./internal/... ./cmd/..., go test ./internal/server
./internal/protocol ./internal/stage ./internal/pipeline.
main brought pushed run events (emitRun/RunEventFrame across protocol, server
and the Android client) and this branch brought per-run usage recording. Both
touch internal/server/runner.go; git auto-merged cleanly and the result keeps
both intents — each model step is stamped with its effective provider/model and
budget-derived usage (stageUsage, overlayEffectiveModels) while run events are
emitted (emitRun) and applied to the same step loop. No conflict markers, no
side's work discarded.
Post-merge checks: gofmt -l clean, go build ./..., go vet ./... and
go test ./... all pass.
Carry the usage a model stage already measures for its budget
(stage.Budget.Snapshot()) onto the step as typed evidence and total it in
one pass, rather than a second harness read, a transcript parse or an
output-length estimate. A shared budget across stage revisits records only
the visit's increment, and a revisit that observes no new report records no
usage instead of a fabricated known zero.
- protocol: StageUsage/RunUsage/TotalUsage and UsageTotals.
- pipeline: step_usage table + migration, COALESCEd into the one step
projection, FinishStep persistence, one json_extract history aggregate,
retention removes the rows with their run.
- server: stageUsage maps the budget snapshot, stamping per-stage model and
usage for every model stage (including failures).
- cli: inspect/stats human and JSON output.
- budget/pi: retain the reported in/out split, mark a total-only (Pi) report
split-unknown, and keep an unreported cost unknown (never zero).
Evidence and docs updated in the packaged skill references and docs/cli.md;
the live database has no step_usage rows yet (all history predates this), so
per-run figures are reported as unavailable, not zero.
Combines this lane's "make every view live from the pushed run events" change
(android + server run-event scope and frame identity) with main's operator
chat brokering work (opencode endpoint adoption, protocol/settings/CLI chat
support, operations reference). The two touch different files; git's ort
strategy merged them cleanly with no conflict markers and nothing to resolve.
Post-merge checks: go build/vet and go test ./... all pass; gradle
--no-daemon testDebugUnitTest assembleDebug passes (558 tests, 0 failures).
The pipelines list (and every other run-state view) kept the row it had when
the screen was last fetched. Two class-level root causes, fixed here:
- The app subscribed to run events scoped to the one workspace the operator
configured, so a run executing in another workspace (a lane worktree is its
own workspace) never delivered a frame: the rows only changed on a manual
refresh. The app now asks for the blank "any workspace" scope the server
already treats as any (ALL_WORKSPACES in AppViewModel.subscribeRuns, blank
default on PipelinesRepository.subscribeRuns). Workspace isolation on the
server is unchanged: an explicit workspace scope still matches only that
workspace (TestRunEventWorkspaceIsolation); the new
TestRunEventAllWorkspacesSubscription pins the blank "any" scope.
- foldRunEvent refused to insert a run the list had never held, returning
needsResync and leaving the list untouched, so a run started after the
screen loaded appeared only after a fetch. The pushed frame now carries the
pipeline identity a row needs (RunEventFrame.PipelineID/Name/Title/Version),
so the fold inserts the new run at the head from the frame's own identity
without a fetch and without reordering existing rows; a server that predates
those fields still falls back to the one resync. Added fields and why each
is needed: pipeline_id (row's pipeline join), name (row title), title (the
stored operator-facing title a row shows), version (the row's pinned
version) - everything a RunSummary row renders, so no per-event fetch and no
guess from stale cache. The server stamps them on every emitted frame from a
per-run identity remembered at create/decide/cancel/resume (runner.go
runIdentity/rememberRunIdentity/stampRunIdentity), first-write-wins through
the coalescer (mergeRunEventFrame) and on the subscribe snapshot, so every
transition - created/queued, running, each stage start/finish, terminal -
carries them. Reconnect still resyncs once then follows pushes; the
pull-to-refresh path is untouched.
Checks run: go build ./..., go vet ./..., go test ./... (all packages ok),
go test -race -run 'RunEvent|RunEvents|Subscribe' ./internal/protocol/
./internal/server/, and gradle --no-daemon testDebugUnitTest assembleDebug
(558 tests, 0 failures, 0 skipped; BUILD SUCCESSFUL).
Packaged templates and the packaged authoring skill references were checked:
nothing there describes the run_event field list or the client's subscription
scope, so neither needs updating (references/notifications.md only says the
attention surface is derived from the pushed run events, which is unchanged).
Docs are a shared file owned by a single writer and were not edited:
docs/cli.md's description of the pushed `run_event` frame still enumerates
only the older fields (status, outcome, current stage, completed-step count,
timestamps, optional step) and is now incomplete - it should mention the
pipeline identity fields; docs/ntfy.md remains accurate.
Add an opt-in way to broker chat from an operator-supplied OpenCode service
instead of the instance the server spawns for itself, and a way to adopt a
session that already exists on that endpoint so it appears in chat_sessions and
can be continued from the app.
- Endpoint selection: the `-opencode-endpoint` server flag (defaulting to
`PIPELINES_OPENCODE_ENDPOINT`) opts in; precedence is explicit flag >
explicit environment > the endpoint recorded on a previous run, with the
spawned instance as the default. opencode.OpenEndpoint validates the
reference once (http(s), host, no userinfo/query/fragment/path), detects the
backend from its own /api/info (v2) or /global/health (v1) contract, and
requires a streamable text/event-stream event path. It owns no process
(Close is a no-op) and never calls startServe; every failure names the
endpoint and disables chat rather than silently falling back. An
authenticated endpoint takes an optional password only through the private
file PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE, with
PIPELINES_OPENCODE_ENDPOINT_USERNAME overriding the detected default.
- Persistence: the validated choice is recorded in an additive single-row
server_settings table so a restart with no flag/environment reuses it; an
unreachable recorded endpoint disables chat with a clear message and no
fallback.
- chat_adopt: a new protocol request (ChatAdoptRequest) with handleChatAdopt in
internal/server/chat.go. It refuses a nil broker/registry/payload and a
blank or foreign-shaped id before any endpoint call, reads the endpoint via
the shared Chat.Session (which reuses Sessions/decodeChatSessions), records
through the existing RecordOperatorSession with the endpoint's directory and
title verbatim, and echoes the adopted session plus its registry row like
chat_create. A missing id is refused naming the id and records nothing; an
endpoint record with an empty directory is refused rather than substituting
the server's working directory or home.
- CLI: `pipelines chat-adopt <session-id> [--json]` (usage, notes, help
inventory, dispatch) with docs/cli.md updated under Chat sessions.
- Autostart preserves the endpoint variables (and the password-file reference,
never an inline secret) via serviceEnvironmentKeys.
Checks before committing:
- Package location: the change is user-facing chat behaviour, so the packaged
authoring skill reference internal/author/skill/references/operations.md is
updated in the same change (Brokered OpenCode chat section) with a dated
verification-evidence entry and the not-deployed state. No new reference file
is added, so internal/author/install.go's managed inventory is unchanged.
- Packaged templates: this repository packages no template set for this
behaviour and none references the chat endpoint, so no template needs
updating.
- Verification: `go build ./...`, `go vet` on the touched packages, `gofmt -l`
clean and `go test ./...` all pass. The unit tests cover the endpoint
validation/precedence, the adopt happy path and refusals, the recorded
endpoint round-trip, the protocol round-trip and the CLI dispatch. The live
scratch-server check against the operator's real endpoint was not run in this
change.
Nothing unrelated is staged; docs left stale (README and other shared documents)
are not edited here and are for a single folding writer.
Every OpenCode-harness model stage now writes its real session into its own
step artifacts directory as session.md and session.html, so the existing
bounded artifact listing, the /artifacts/<run>/<step>/<name> serving path and
the artifact scan find and serve them with no special casing, while the stage
keeps its bounded preview in the run record unchanged.
- internal/opencode/export.go adds the one shared native export invocation and
the exact session-ID/messages validation used by both Connection.Messages and
the new ExportSession, so the readback path does not diverge. The backend's
published subcommand is used (v1 `export <id>`, v2 `session export <id>`),
the source read is capped at 8 MiB, and a larger session is read as a prefix
and reported truncated instead of failing. boundedOutput.ReadFrom keeps the
cap effective so overflow and the truncation path are real.
- internal/stage/session_export.go renders markdown and html with a metadata
header, every role and every part (unknown part types as raw JSON, every
session text escaped in html), atomic temp+rename writes that never overwrite
an existing file, a visible truncation banner at both ends when the source or
the 2 MiB rendered cap cuts it, and an honest 0-message header for an empty
session rather than fabricated text.
- protocol.SessionExport and RunStep.SessionExport are stored in a new
step_session_exports table (CREATE TABLE IF NOT EXISTS, so an old database
gains an empty table with no migration) loaded through the existing main
SELECT alongside the other per-step evidence and deleted by run retention.
- internal/server/runner.go exports after the post-stage filediff capture and
before scanArtifacts, with context.WithTimeout(context.WithoutCancel(ctx),
10s); it never waits for session idleness and a failure or a Pi stage (no
native export path) records a sanitized diagnostic and continues, never
changing the stage status, outcome, target, timing or result validation.
Reflected where it belongs: the packaged skill references
(internal/author/skill/references/operations.md and delivery.md) carry the new
Session export rules and the run-artifacts bullet, and docs/stage-artifacts.md
documents the names, the <run>/<step> paths, the served URL, the caps, the
truncation labels, failure isolation and the Pi limitation. The packaged
templates (examples/*.md) only tell a stage to save its own artifacts and say
nothing about the server-written session export, so none needed updating; no
new reference file was added, so install.go's managed inventory is unchanged.
Shared documentation: README.md's per-step artifacts sentence is now stale - it
should name the exported session alongside images, screenshots, reports and
logs. It is deliberately not committed here; the shared documents are folded by
the single documentation writer.
Checks: gofmt -l on the touched packages produced no output; go vet ./...
passed; go test -race -count=1 ./... passed.
chat_sessions used to list exactly the OpenCode sessions the pipelines own,
filtered through pipeline.SessionOwner (run_steps/run_session_owners), so an
operator-created chat was an accident of the run-ownership table and the
operator's working sessions were mixed with the internals of running pipelines.
Add a first-class operator session:
- internal/pipeline/operator_sessions.go: a dedicated operator_sessions table
(session_id PK, directory, title, tag, created_at, last_used_at) with
RecordOperatorSession (upsert preserving created_at), a one-statement
TouchOperatorSession (no-op for a non-operator id), OperatorSessions bounded to
128 rows most-recently-used first, and OperatorDirectories (bounded MRU,
deduped). run_steps/run_session_owners and SessionOwner/OwnedSessions are
untouched.
- internal/server/chat.go: chat_sessions returns operator sessions only - it
reads the registry once, intersects it with one broker session-list call, fills
directory/tag/created/updated from the registry, sets busy for a session held
by another connection, keeps MRU order, and stays fail-closed (empty list, never
the endpoint's whole list). A pipeline stage session and an untagged row can
never be listed. chat_create resolves the directory (empty/'~'/'~/...' ->
operator home; relative or missing rejected before any OpenCode call), calls
broker.Create(title, resolved) and records the row with the operator tag.
chat_directories returns home first then bounded (8) deduped recents. A
successful prompt touches the operator session without affecting pipeline
prompts.
- internal/opencode, internal/protocol, internal/server/websocket.go and
message.go: unscoped session list, Create posts {"title":...,
"location":{"directory":...}} to POST /api/session, additive protocol fields
(omitempty) and the ChatDirectories request/response, plus the OperatorRegistry
option defaulting to the first store. One OpenCode transport per server is
unchanged.
Docs and references: docs/cli.md gains a "Chat sessions" section documenting the
operator-only fail-closed list, directory defaulting/rejection, chat_directories
(home first, deduped, bounded 8) and why a dedicated request was chosen.
internal/author/skill/references/operations.md's stale "Brokered OpenCode chat"
text (chat_create unbrokered, run_steps filtering, "listed the owned scratch
session" evidence) is rewritten to the operator-session registry, the fail-closed
filter, directory defaulting and the corrected live-test wording. No new
reference file, so internal/author/install.go's inventory is unchanged.
Checked for staleness elsewhere: README.md, the docs/ topic pages and the
packaged server/TUI templates carry no chat-session ownership claims, so none
needed updating.
Verified: gofmt -l clean; go vet and go build ./... clean; go test -count=1
./internal/pipeline/... ./internal/protocol/... ./internal/opencode/...
./internal/server/... and the full go test -count=1 ./... pass.
Combines lane-go's typed omni search, launch-time model overrides, TUI
artifacts view, run-state websocket push, and the "reflect the change where
it belongs" committer rule with the Templates gallery, chat/omni-search and
other reviewed work already on main. The 12 files both sides touched keep
both intents composed rather than dropping either side: the omni search keeps
the typed search contract (CLI `pipelines search` and the TUI popup with type
badges and marked snippets) alongside the run-list quick filter; cli.md and
ui-ux-review.md document both the search rework and the artifacts view;
orchestration.md, harnesses.md and operations.md keep the model-override and
reflect-where-it-belongs guidance; and the protocol/CLI/server/TUI code keeps
the new search, run-event, override and artifacts paths alongside the existing
run, title and help surfaces.
Checks before committing:
- go build ./... -- pass
- go vet ./... -- pass
- go test -count=1 ./internal/... ./cmd/... -- all pass
Doc check before committing: nothing needs updating. The lane's own docs and
references (docs/cli.md, docs/ui-ux-review.md, and the orchestration,
harnesses and operations skill references) already carry the new features.
format.md and the packaged templates already describe the line-oriented result
format and the human decision stage from the earlier lane-go merge. The root
README already describes the omni search popup. Model overrides, the search
command, run events and the artifacts view are runtime/CLI/TUI surfaces that
cli.md documents; no template, README or topic-doc edit is required by this
merge.
Add a server-pushed run_event frame so a client no longer has to poll
once a second to notice run changes.
- protocol: RunEventFrame/RunStepDelta compact delta (run id, workspace,
status, outcome, current stage, completed-step count, timestamps and an
optional single step delta), plus subscribe/unsubscribe/run_event types
and the optional RunEvent/RunIDs message fields. All fields optional so
older peers ignore them.
- server: runEventHub with per-connection subscriptions scoped to a
workspace (and optional explicit run ids), one bounded writer goroutine
per subscription, slow-subscriber drop, per-client subscription cap and
a ~220ms per-run coalescer that flushes pending deltas before a terminal
frame.
- runner: emit deltas on created/queued, queued->running and step start,
human waiting, step finish (outcome, target, diagnostic, artifacts and
completed count), terminal finish, cancel of a waiting run and
decide/resume; emitRun is a no-op without a hub.
- websocket: subscribe seeds current state before the ack, unsubscribe and
connection teardown drop subscriptions, and pushed frames reuse the
connection write closure.
- tests: protocol golden-JSON wire shape plus websocket coverage for the
started-run frame sequence, subscribe current-state snapshot, silent
unsubscribed/never-subscribed clients, workspace isolation and a
bounded coalescing frame count.
- docs: note subscribe/unsubscribe/run_event in docs/cli.md; TUI and
Android still poll in this release.
Protocol: replace the loose Query/Kind/Matches/Limit search fields with a Search request object and a SearchResults response, and have the server answer search from Store.Search over the shared run projections (runFrom/runStamps).
CLI: add the read-only 'pipelines search QUERY [--type KIND] [--limit N] [--json]' command with usage/help, and document the typed targets, ranking and bounds in docs/cli.md.
TUI: the '/' popup keeps type-to-search, up/down, Enter and Esc, renders one ranked list with kind badges and a bracketed (non-colour) match snippet, and Enter opens the matched pipeline, run, step, artifact or field; footer text updated to match.
TUI: add a dedicated run artifacts view ('A') with open/copy of recorded files, and move artifacts out of the run-detail File changes section (artifactCaptureNotes).
Tests: add Store.OpenReadOnly for read-only inspection of a live database plus pipeline/server/TUI search tests and artifacts tests, including env-gated live checks.
Add `pipelines run --model PROVIDER/MODEL` and repeatable
`--stage-model STAGE=PROVIDER/MODEL`, resolved with precedence
per-stage > whole-run > the model pinned in the pipeline version.
- A shared pure resolver (`pipeline.ParseModelOverride` /
`pipeline.EffectiveModels`) owns the precedence and validation: every value
must be a `provider/model` pair (`stage.ValidModel`), every stage key must
exist in the pinned graph, an unknown key fails naming the valid keys, and a
per-stage key that names a non-model stage is refused.
- The server validates against the pinned version before admission (no run row
is ever inserted for an invalid request), persists `model`/`stage_models` on
the run and `provider`/`model` on each step, applies the effective pair to a
clone of the stage definition so the stage and its repair/formatter path share
it, overlays the returned run graph with the effective pair (presentation
only), and reuses the recorded overrides on retry.
- The run record echoes the overrides it was launched with and each step records
the provider/model it actually ran with; the TUI Evidence pane renders that
effective pair. The pinned pipeline version and `show --version N` are never
modified.
- CLI help, `docs/cli.md` and the packaged authoring skill references document
the flags and the precedence. A TUI override picker remains a documented
follow-up; the CLI is the deliverable.
Tests: pure precedence/validation table tests, store round-trip and
request-key-mismatch plus migration, server admission/application/overlay/retry
with a controlled executor, CLI flag/help/malformed-value tests, and a TUI
evidence test.
Back the TUI "/" popup with the server's search command and add a matching
pipelines search CLI command.
- protocol: SearchRequest/SearchResponse and per-kind SearchResult payloads,
one shared Label for the six-kind badge dispatch and RuneIndexFold for
case-insensitive matching.
- pipeline: the read-only, ranked, bounded Store.Search implementation.
- cli: search QUERY [--type KIND] [--limit N] [--json], printing the shared
label plus a text-field snippet when present.
- tui: type to search, up/down to choose, Enter to open and Esc to cancel over
one mixed ranked list, with short type badges and a bracketed non-colour
match marker; Enter navigates to the matched pipeline, run, step, field
section or artifact entry.
- tests: protocol and TUI coverage, including an env-gated live test against
the running server.
Integrates nine commits of reviewed work from the lane-go branch after the
automated resolver failed on this merge five times (each attempt dying on the
malformed-result defect whose fix is contained in this same merge).
Integration decisions:
- Search: kept main's shipped design (Store.Search + SearchResponse.Results) and
dropped lane-go's parallel, older attempt (an unused Message.Matches field, an
absent runs.search, and two partial TUI plumbing blocks). lane-go itself did not
compile: protocol.SearchMatch was undefined and model.go referenced searchNav,
requestSearch and applyRunNav, which exist nowhere. Those partial TUI blocks are
removed; the queued TUI omni-search item will implement the surface properly on
main's design.
- Both sides' additions are kept where they are independent: run titles and
provenance, the human decision stage and its CLI/TUI surface, the derived stage
verdict, the stale-client guard, the line-oriented result format, and main's
search command.
- The search key's help label is now "/ omni search"; the two tests asserting the
old label are updated rather than the label reverted.
- docs/cli.md and the packaged operations reference carry both sides' text.
Verified: go build ./... , go vet ./... and the full go test ./... suite pass.
Exchange a build/protocol identity so an older CLI can no longer talk to a
newer native server silently:
- protocol carries ProtocolVersion and every server frame (including the
unsolicited welcome) is stamped with the server build and protocol, so a
client that never pings still learns the server identity.
- buildinfo.Compare classifies the relationship as InSync/Warn/Refuse and
names both builds; the client Watch reports one warning per connection and
returns a hard error when the server speaks a newer, uninterpretable
protocol so the caller never acts on the frame.
- client.ReceiveNotice decodes the raw frame so an unmodeled top-level field
produces a one-time "unrecognized field(s)" notice instead of being dropped
silently (the case that hid the artifacts field).
- CLI routes warnings to stderr, keeps machine JSON stdout clean and fails
non-zero on a refused mismatch; ping/connection print the client/server
relationship. The TUI surfaces the same notices through the shared log.
- scripts/setup-native.py now warns when an older `pipelines` earlier on PATH
shadows prefix/bin/pipelines, naming both paths and versions, without
installing or modifying anything else.
- Docs, the packaged debugging skill reference and README no longer imply the
installed CLI always matches the server.
Tests cover the comparison table, unknown-field detection and caching, the
server identity stamping on pong and welcome, the CLI/TUI reporting paths and
the installer PATH shadowing warning.
Refinements on top of the human decision stage (a6aff47):
- `pipelines decide` now reports the feedback the server actually stored for
the chosen option, so a repeated (idempotent) decision shows the recorded
evidence instead of the ignored request text. A repeated decision is covered
by a new assertion that also checks the step output does not change.
- The decision option/feedback output keys are defined once in protocol
(DecisionOptionKey/DecisionFeedbackKey) with a DecisionEvidence reader, and
the store and CLI use them instead of bare string literals.
- piEnvironment drops any inherited PIPELINES_ARTIFACTS_DIR before spawning Pi,
so the artifactsDir argument is authoritative and an ambient directory (for
example Pipelines' own when a stage runs inside another stage) never leaks
into a child that was given none. Covered by a test with an ambient value set.
A new registered stage type `human` (internal/stage/human.go) has no model,
agent, tools or configs: its required `prompt` block carries the operator
instructions and its non-error edges are the selectable options. It parses,
validates, renders, exports and imports like every other type (no parser
special-case, no config keys).
At runtime the runner parks the run instead of executing: the step and the run
persist `waiting` with self-contained {prompt, options} evidence, and the run
returns without starting later stages, capturing file diffs, scanning artifacts,
holding a goroutine/timer or applying a stage timeout. The wait is durable: a
restart leaves the waiting rows untouched (RecoverRuns excludes them) and a
rebuilt runtime over the same database still accepts the decision.
`pipelines decide <run-id> [stage] --option KEY [--feedback TEXT] [--json]` and
the TUI `c` picker resolve the decision from the pinned graph in one store
transaction, record the choice and feedback as the step's evidence and resume
along edges[KEY]. An unknown option is refused with the valid list, a stage that
is not waiting is refused, and a repeated decision for an already-decided visit
does not double-apply. Cancel and retry keep working while waiting and after a
decision; a human stage is never killed by a stage timeout or max_visits, and a
`notify` policy can fire an `on=waiting` event.
Docs: the authoring skill references (format.md, operations.md), docs/cli.md and
README describe the stage, the command and the TUI picker. Tests cover stage
validation, the parsing round-trip, the store transitions, a websocket wait/decide
integration, cancel/retry, waiting notifications and a persistence-layer rebuild
(second Open + newRunner over one database, no process restart).
One query now finds anything in the system over the wire and from the CLI.
- protocol: add Search/SearchResults message types and a SearchRequest
(query, optional kind filter, optional limit) / SearchResponse (typed,
ranked matches plus truncation and scan-bound counters) pair, with typed
payloads for pipeline, version, run, step/visit, field text and artifact.
- store: add Store.Search, a read-only query over pipelines,
pipeline_versions, pipeline_runs, run_steps (+step_control) and
step_artifacts. Ranks exact name (3) > name substring (2) > body text (1),
breaks ties by recency then a stable kind/target order, and reports the
matched field. Bounds work with a 50-default/200-max result cap, a 2000-row
per-kind scan limit and a 2000-step / 1 MiB field-text scan budget.
- terms: a bare status word or lone date expression (today, yesterday, this
week, YYYY-MM-DD) selects runs; anything else stays free text.
- server: dispatch protocol.Search to stores[0].Search; read-only and
unconfirmed.
- cli: add `pipelines search QUERY [--type KIND] [--limit N] [--json]`, help
text and a BROWSE line.
- store: add OpenReadOnly so the live-database check never runs DDL.
- docs: document the grammar, kinds, ranking and bounds in docs/cli.md,
scope the TUI omnisearch to pipelines in docs/ui-ux-review.md, and add a
search bullet to the packaged operations reference.
- tests: protocol round trip, store/ranking/term-parsing/limit/scan-bound
unit tests, a server end-to-end test, and an env-guarded live check.
No new dependencies; go.mod/go.sum unchanged.
Each completed step now exposes a verdict: the chosen option (outcome -> target),
a bounded one-line reason with model/fallback/absent provenance, and the recorded
artifacts the reason refers to. The store persists only the model-generated line
in a new step_verdicts table and derives the fallback at read time (no backfill);
the server generates the line once at completion, best-effort; the TUI renders it
in the stage progress summary. The packaged debugging reference documents it.
Add a bounded, never-empty title to every run. `pipelines run --title "..."`
sets it explicitly and skips model generation; otherwise the server generates
it from the prompt through the shared summariser and falls back to the prompt's
first line, recording title_source (model/fallback/explicit) and title_model.
- protocol/store: RunSummary.Title plus Label(), a set_run_title message, title
columns with an idempotent migration, CreateRunTitled, and SetRunTitle renames.
- summarise: Title/GenerateTitle/DeriveTitle/ExplicitTitle bounded to TitleCap.
- runner: generate the title outside the admission lock and only for a request
that starts a run; a replay reuses the stored title and an explicit title
skips the model entirely.
- CLI: `--title` on run and a `title <run-id> "<title>"` rename command; human
`runs` and `inspect` show the title beside the pipeline name and version.
- TUI: the run list row, run detail header, Home run cards and last-run line,
and the pipeline Results line show the title via Label().
Tests cover storage/migration/fallback, explicit-title and rename, the
end-to-end CLI flow, and title rendering; CLI and operations docs updated.
internal/summarise called protocol.NormalizeText, which was never defined, so the tree did not compile. Add the shared normalisation helper: drop zero-width formatting runes, turn control characters into spaces and collapse whitespace runs (including the non-breaking space variants) into single spaces. Reuse it in OutputPreview instead of duplicating the mapping logic.
Cover the awkward inputs (newlines, tabs, non-breaking spaces, zero-width runes, surrounding quotes, trailing punctuation) in the protocol and summarise tests. Unrelated working-tree changes are left uncommitted.
chat_sessions now brokers GET /api/session but filters the global list to
the sessions this server recorded in run_steps, deduped by session id, and
fails closed to an empty list when no ownership source is configured. A
session another client holds (in-server claim, released on stream end or
disconnect) or an active run is marked busy; a second client's chat_prompt
for a held session is rejected before any OpenCode call. chat_create stays
unbrokered.