Commit Graph

113 Commits

Author SHA1 Message Date
alan 2cdb9a9e93 Stabilize guarded model delivery
Add validated OpenCode-only delivery capabilities, canonical linked-worktree access checks, isolated candidate verification, and guarded main settlement.

Wire the bounded reconnect-foreground controlled HTTP fixture through ordinary server construction so target verification obtains server-controlled provenance and rejects unknown or incompatible fixture evidence.

Checked packaged templates and packaged authoring skill references; no updates are needed because this draft-only infrastructure activates no definition. Recorded the stale reference and activation guidance in docs/notes/delivery-stabilization.md for the docs fold.
2026-10-18 14:50:18 +01:00
alan 66b5f60b28 Merge preserved connection reconnect baseline 2026-10-18 13:44:20 +01:00
alan e9d902213e Preserve background shell visibility across readback races
Reconcile authoritative shell snapshots with bounded lifecycle journals so concurrent exits and creates cannot be overwritten, coalesce observation setup, and retain verified ownership across reconnects. Add deterministic reconciliation and cancellation regressions plus stable limited-visibility handling.\n\nChecked packaged templates and authoring skill references: none need updating. The task-owned docs note records the stale shared chat documentation for docs-fold; shared documentation was not edited.
2026-10-18 12:54:36 +01:00
alan f91145001a Merge main into connection reconnect delivery 2026-10-18 12:37:24 +01:00
alan 3c47b08d98 Restore connections on app foregrounding
Add a process-lifecycle foreground reconnector and pure recovery policy, wiring repository recovery through the validated saved endpoint. Include lifecycle-process and focused unit coverage while preserving explicit Disconnect and stale socket callback guards.

Checked packaged templates and packaged authoring skill references: neither documents Android connection lifecycle behavior, so neither needs updating. Shared android README changes are recorded in docs/notes/foreground-reconnect.md for the docs-fold writer.
2026-10-18 12:37:20 +01:00
alan 9130ae8c7a Add stage-exclusive resource waiting
Add durable FIFO capacity-one resource leases for executable stages, including cleanup-confirmed release, conservative restart quarantine, queue state events, and CLI/TUI/Android waiting presentation. Add scheduler, cleanup, protocol, client, and Android coverage plus a proposed Android v3 resource draft.

Checked packaged templates and authoring skill references: neither needs updating in this item. The per-item docs note records the shared documentation that docs-fold must update; no shared README or packaged reference was edited.
2026-10-18 12:30:30 +01:00
alan 31aec05f99 Scope stage artifact access by run visit
Derive validated same-run visit artifact capabilities in the runner and pass them to OpenCode and Pi clients. Permit supported file tools to read completed prior visits and write only the current visit, preserving workspace, secret, database, and shell restrictions.\n\nChecked packaged templates and authoring skill references: neither needs updating in this item; docs/notes/scoped-artifact-access.md records the shared documentation staleness for the docs-fold writer.
2026-10-18 12:02:37 +01:00
alan 31d8f4a6d5 Distinguish active stage deadlines from harness transport failures
Identify configured attempt expiry during V2 active-session idle waits and retain bounded phase, timeout and session diagnostics without claiming a lost response. Preserve genuine operation failures, error identities, redaction, cleanup and admitted-work replay protections. Add stage regressions and scratch-server CLI/websocket retention coverage.

Checked packaged templates in examples and the pipeline template generator: none needs updating because timeout configuration and contracts are unchanged. Checked packaged authoring skill references: debugging.md needs a STAGE DEADLINE playbook and clarified transport guidance. Recorded that stale shared documentation in docs/notes/stage-timeout-diagnostics.md for docs-fold; shared documentation was deliberately not edited.

Verification: go build ./...; go vet ./...; go test ./...; go test -count=1 ./internal/server ./internal/stage ./internal/pipeline ./internal/protocol; go test -count=1 -run TestStageDeadlineDiagnosticThroughCLIAndWebsocket -v ./internal/server; gofmt -l .; git diff --check. All passed. Live-provider behavior not tested; no deploy or deployed-service restart.
2026-10-18 11:17:01 +01:00
alan 70529926cc Merge main into session-html-only: combine the html-only session export with chat-tool-group and chat-activity 2026-10-18 10:06:54 +01:00
alan 95362134aa session export: write only session.html, retire the markdown export
A stage's session was exported twice, as session.md and session.html, and both
files were presented. It is now one html file; the markdown export is retired.

- internal/stage/session_export.go: ExportSession renders and writes exactly one
  file, session.html, via writeSessionFile; renderSessionMarkdown and its helpers
  are gone; SessionExportResult carries HTML/HTMLBytes only. The source read, the
  8 MiB source / 2 MiB rendered bounds, the truncation banner, the
  never-overwrite rule and the honest empty-session handling are unchanged.
- internal/protocol/runs.go: SessionExport drops markdown/markdown_bytes (a
  record written before the change still decodes; unknown fields are ignored),
  and adds LegacySessionMarkdownArtifact with RunStep.PresentedArtifacts(), the
  one Go rule that hides the retired session.md from every artifact listing.
- internal/server/runner.go: evidence.Markdown/MarkdownBytes are no longer copied.
- internal/tui: the artifacts panel, the recorded count and the per-visit summary
  derive from PresentedArtifacts(), so the terminal never lists session.md.
- web/src/app.js: the spike's renderArtifacts applies the same filter.
- android: the Session section is the html export alone in a WebView — SessionView,
  the Markdown/HTML toggle and the session markdown path are removed;
  presentedArtifacts(step) filters the retired session.md out of the per-visit
  gallery, the run gallery and its count, the verdict/selection lists and the
  timeline indicator; data/protocol SessionExport drops the markdown fields.
- Tests pin the single html file, prove no session.md is written and that a
  pre-existing one is neither overwritten nor presented, and prove a legacy
  export record still loads and its session.md is hidden.

The legacy session.md of an older run stays on disk (dated evidence); it is only
no longer produced and no longer presented.

Checks (macOS, Go 1.24, Android Gradle 8.11.1, 18 Oct 2026): gofmt -l . prints
nothing; go build ./... && go vet ./... clean; go test ./internal/... ./cmd/...
-> 19 packages ok, 0 failures; node --check web/src/app.js ok and node --test
web/test -> 20/20; gradle --offline --no-daemon :app:assembleDebug
:app:testDebugUnitTest -> BUILD SUCCESSFUL.

Packaged references checked: internal/author/skill/references/delivery.md and
operations.md both described the session.md+session.html pair, so both are
updated here to describe the single session.html export and the retired, hidden
session.md. README.md and internal/author/skill/SKILL.md carry no session.md
reference and install.go's managed inventory is unchanged (no reference added or
removed). docs/notes/session-html-only.md records the remaining stale shared
documentation (docs/stage-artifacts.md, docs/app-integration-verification.md) for
the docs-fold writer; shared docs are not edited by this item.
2026-10-18 10:06:49 +01:00
alan 194540b768 Merge main into chat-tool-group: combine the tool-call summary chip with the chat-activity feature
Brings the tool-call summary chip (7fbbd95) together with main's chat in-flight
state and background-process surface (7b0dc94). Both edit ChatFormat.kt and
ChatTranscript.kt; the merge combined them cleanly (no conflict markers) — the
chip's live gating (live = streaming, toolStartedAt) now sits alongside main's
turn-phase labels and background-process bar, and both sides' pure helpers
coexist.

Re-ran the repository checks on the resolved tree: gradle --offline --no-daemon
:app:testDebugUnitTest :app:assembleDebug => BUILD SUCCESSFUL; JUnit XML
aggregate classes=70 tests=680 failures=0 errors=0 skipped=0.
2026-10-18 09:59:13 +01:00
alan 7fbbd95374 chat: one tool-call summary chip per turn, opening the call list
The chat transcript no longer renders a chip per tool call: `ToolCalls`
(`android/.../ui/chat/ChatToolCalls.kt`) now draws a single summary chip
reading `N tools called` (`1 tool called` for one) as the turn's only
tool-call surface. While the turn streams the chip shows the call count
growing and the existing spinner while any call is in flight, plus the
existing `· long` mark for a running call past `TOOL_LONG_RUNNING_MILLIS`;
both are gated on the turn still streaming (`live = streaming` in
`AgentTurnItem`), so a settled chip never spins or fakes a long mark.

Tapping the chip opens the existing `ToolPanel` as a popup listing every
call of the turn, in the order the agent made them: one independently
collapsible row per call naming it, showing its one-line detail, its
`+N -M` edit stat and its duration, expanding in place to the
pretty-printed INPUT, the synthesised edit diff, then the ERROR block for
a failed call or the OUTPUT (`(no output)` when empty). The list is
bounded by the new `TOOL_PANEL_MAX_HEIGHT_DP` (`Dimens.kt`, purpose-named)
and scrolls inside the panel, so many calls never grow the popup past the
window nor re-flow the transcript. The chip announces the count ("N tool
calls, expand for details") through the app's accessibility treatment.

Pure logic lives in `ChatFormat.kt` (`toolCallsLabel`,
`toolCallsAnnouncement`, `toolCallCount`, `toolsRunning`, `toolCallRow`)
with unit cases added to the existing `ChatFormatTest`; no raw dimension
was introduced, so `DimensTest` is unchanged.

Checked for staleness: the packaged authoring skill references
(`internal/author/skill/references/`) and the packaged templates carry no
Android-chat content, so neither needs updating — this change is
`android/`-only. The shared docs this change makes stale
(`android/README.md`'s "Tool-call chips and the expandable panel" section
and `docs/chat-verification.md`'s tool-chip row) are recorded in
`docs/notes/chat-tool-group.md` for the single `docs-fold` writer rather
than edited here.
2026-10-18 09:58:28 +01:00
alan 7d9490bba1 chat: name the in-flight state, and surface background processes
Two gaps in the chat's in-flight feedback.

The turn indicator no longer reads a bare `Working…` for every state. A new
pure derivation (TurnPhase/turnPhase/turnPhaseLabel in ui/chat/ChatFormat.kt)
labels the headline from evidence already on the wire: `Thinking…` while text
or reasoning deltas are still arriving (within TURN_THINKING_WINDOW_MILLIS),
`Running a tool…` while calls are in flight, `Waiting…` otherwise. No
percentage, ETA or invented progress.

The agent's shell commands are now surfaced end to end as their own frame kind,
so a background process (a dev server, a watching build) that outlives its tool
call is visible and clears when it exits.

Server:
- protocol/chat.go adds ChatEventShell, ChatShell (id, status, command, *int
  exit — a pointer so exit 0 is not "absent") and ChatEventFrame.shell.
- opencode/chat.go maps shell.created/shell.exited through a new exported
  ChatStreamState, attributing created via info.metadata.sessionID and exit via
  the ids the stream already surfaced; another session's or an unattributed
  shell is ignored, an id-less shell fails the stream. No cwd/file/credential is
  carried.
- server/chat_hub.go retains still-running shells (chatShellLimit plus the
  shared byte cap) and drops them on exit; chat_stream.go replays them on
  chat_subscribe, the shell half of the resume (a shell is not a turn, so the
  message snapshot cannot carry it).

App:
- data/protocol/Chat.kt adds ChatShell, ChatEventFrame.shell, EVENT_SHELL,
  SHELL_RUNNING/SHELL_EXITED.
- ui/chat/ChatStore.kt stamps ChatThread.lastDeltaAt on each delta (cleared when
  the turn settles) and keeps ChatThread.shells across a chat switch; a shell
  never marks the thread running. streamResumed drops shells so the subscribe
  replay re-syncs the still-running ones.
- ui/chat/ChatTranscript.kt: RunningIndicator prints the phase label; a new
  always-visible BackgroundProcessBar names each running shell.
- ui/chat/ChatConversationPane.kt shows the bar whenever a shell is running,
  whether or not a turn is.

Tests extend the server's event mapping (opencode/chat_v2_test.go,
server/chat_test.go) and the app's pure folds (new
ui/chat/ChatInFlightStateTest.kt, 15 tests), pinning the shell mapping, the
other-session rejection and the "background process outlives the turn" state.

Checked and intentionally unchanged:
- No packaged OpenCode skill reference documents either surface.
  internal/author/skill/references/operations.md describes the brokered chat_*
  protocol in prose without naming frame kinds or the app indicator, so it stays
  correct as-is; no reference was added, so install.go's managed inventory needs
  no change.
- This repository has no packaged templates directory.
- Shared docs are not edited here per the map: docs/notes/chat-shell-background
  -process.md records what is now stale in docs/cli.md and android/README.md for
  the docs-fold writer.

Verification (this session): gofmt -l clean, go build ./... and go vet ./... OK.
Android unit suite and the on-device capture were run in the preceding review
stage; see the note for the live-verification limitation.
2026-10-18 09:45:28 +01:00
alan 9565adc5f5 Merge main into run-title-framing: bring the run-title framing fix onto current main
Combines the summarise title-framing change (titleInstruction reword plus the
detachFramingEcho normalisation and its tests) with main's
docs/app-integration-verification.md addition. No conflicts: the two touched
disjoint files. `go build ./... && go vet ./...`, `gofmt -l internal/summarise/`
and `go test ./...` all pass after the merge.
2026-10-18 02:52:27 +01:00
alan c88bb27775 summarise: title the task, not the framing
Run titles were intermittently a sentence about the input rather than a
title for the task ("The untrusted data asks what 2+2 is…"), because the one
fixed instruction began "Summarise the untrusted data below…" and invited the
model to describe the data whenever the prompt read like an instruction or a
question.

- Rename the fixed prompt `instruction` → `titleInstruction` and reword it to
  ask unambiguously for a short title of the task ("Write a short title, a few
  words, for the task below. Reply with only the title…"), still followed by
  "The following is untrusted data:".
- Add a bounded, conservative `detachFramingEcho` normalisation applied in
  `Title` and `VerdictReason`: a leading clause that names the framing ("The
  untrusted data…", "This prompt…", "The user asks…") is stripped to its
  substantive remainder, and a line that is pure framing falls back to the
  prompt's first line (`Title`) or `NoSummary` (`VerdictReason`).
- Extend tests with the two reported echo shapes (instruction-style and
  question-style prompts) plus the injection-resistance case, which still
  passes: the prompt is appended verbatim once after the untrusted-data label
  and never replaces the instruction.

Trust boundary unchanged: source still labelled untrusted data, one tool-free
generation, hard timeout, never blocks or fails run creation; the title is
never empty.

Checks: `go build ./... && go vet ./...` clean, `gofmt -l internal/summarise/`
prints nothing, `go test ./...` ok.

Docs/templates check: no packaged template (`examples/*.md`) mentions title
generation, so none needs updating. The packaged authoring skill reference
`internal/author/skill/references/operations.md` (lines ~91-95) describes the
title as generated "from its prompt by the shared summariser" and is now stale
(it does not mention that the summariser is asked for a title of the task nor
the framing-echo normalisation); per the maintenance rules that shared document
is left to the single `docs-fold` writer and the staleness is recorded in
`docs/notes/run-title-framing-echo.md` instead. `README.md` and `docs/cli.md`
stay true, and no `internal/author/install.go` managed inventory entry changes.
2026-10-18 02:52:15 +01:00
alan f43115ef0a Merge branch 'main' into verify-gaps 2026-10-18 02:46:48 +01:00
alan db313336b2 docs: record the four end-to-end verification gap checks (run 340)
Add a dated follow-up section to docs/app-integration-verification.md that
closes the four claims the first pass left unproven, each with the command or
capture that proved it against the deployed server (0d1ddc7) on the Pixel 9 Pro
XL 47241FDAS00DLK:

- Chat reply arrives: a throwaway session showed the assistant reply `Done.`
  rendering live, and the close-a-chat flow left the session `closed=true`
  server-side.
- Live pushed run events: disproven on this build. Runs 341/342/343 never
  appeared or advanced without a manual `Refresh runs`; only a local elapsed
  timer ticked. Consistent with the run-event push-hub defect fixed on main by
  cff6f4d, absent from the deployed 0d1ddc7. Recorded as a defect with exact
  reproduction, not an app defect.
- Disconnected state: the closed-port state and the restore to
  ws://10.0.0.75:8080/connect were both captured and Settings verified after.
- Wide windows: the covered-pane region is a constant 553 px at depths 1-3 and
  at 2000x2244 and 2076x2152; `wm size reset` left no override.

Checked for staleness from this change: it touches no source or resource file,
so no packaged template or packaged authoring skill reference
(internal/author/skill/references/) depends on it - none references this
document and none needs updating.

Report-only: the untracked scratch directory .gaps-tmp/ is deliberately left
uncommitted.
2026-10-18 02:46:46 +01:00
alan 93dc0103c3 Merge main into docs-event-driven: bring the agent-driven workflow docs onto current main 2026-10-18 01:50:59 +01:00
alan 01d5dad9b9 docs: describe the agent-driven workflow, drop the scripted driver model
Rewrite the workflow documentation to describe how the system actually works:
the agent (the session) is the brain and advances the queue by hand through the
`pipelines` CLI. There is no orchestration script, no driver, no polling loop and
no lane machinery, and none may be written.

- references/work.md: rewritten end to end. The queue-driver script
  (drivers/workd.sh), the claim-file ledger with atomic renames, the PAUSED
  marker and the poll loop are gone. State is now the server (authoritative for
  run state), the queue file (authoritative for intent) and the conversation.
  The agent reads the queue, picks the item, creates its worktree/branch and
  launches it with `pipelines run <pipeline> <brief> --json --yes`; progress is
  event-driven via a background `pipelines wait <run-id> [...]` (never polled);
  on completion the agent reconciles with `pipelines inspect <run-id> --json`,
  verifies it landed and starts the next item.
- references/queue.md: the landing gate runs the command directly
  (`git merge-base --is-ancestor <branch> main`, exit 0 = landed) instead of a
  `land-check.sh` helper; the never-delete-branch / never-remove-worktree rules
  are kept, and the waiting section states the agent advances the queue by hand.
- references/orchestration.md: intro and "Running and watching" state the
  agent-driven, event-driven model; "Pipeline choice" names the current set
  (Android, TUI, Server, Generic, Plan and land, UI sweep, Probe) by area and
  records the retired names.
- references/debugging.md: "Duplicated work after a driver restart" became
  "…after a session restart", resuming from the queue file not a cursor file.
- SKILL.md: the routing row no longer says "queue driver".
- AGENTS.md: the maintenance workflow names the current pipeline set instead of
  the retired `Pipelines maintenance - plan implement review`.
- internal/tui/AGENTS.md: the restart line no longer names lanes.
- docs/notes/agent-driven-work-model.md: the docs-fold note recording that
  docs/cli.md needs its "Scripted runs"/`--wait` framing aligned to the
  event-driven model, and that the dated evidence docs stay as they are.

Checked before committing: no packaged template directory exists in this repo, so
none needed updating; internal/author/install.go's managed file inventory needs no
change (no packaged reference was added or removed — the 13 listed files all
match internal/author/skill/references/); no stale `queue driver`/`workd.sh`/
`claim-file` references remain anywhere under internal/author. Genuine device
tooling (android/scripts/capture_png.sh-style) and server-side polling prose were
left alone.

Checks: `go test ./internal/author -count=1` ok; `go build ./...` clean;
`go vet ./...` clean; `gofmt -l .` clean.
2026-10-18 01:50:55 +01:00
alan d26db8f830 Merge main into chat-turn-render: pick up the end-to-end app verification report alongside the chat turn fold 2026-10-18 01:47:55 +01:00
alan 798e958cb7 chat: fold an agent turn into one reply, split reasoning from answer, keep tool name/input
The phone's chat transcript rendered one agent reply as ~36 bubbles, each with
its own thinking block, and a finished tool call lost its name and its input.

- Turn fold (android): ChatFormat.kt gains the pure ChatTurn / TranscriptRow /
  transcriptRows fold - each run of consecutive assistant messages between two of
  the operator's messages is one agent turn - and ChatTranscript renders one
  bubble per turn: the turn's single collapsible thinking block (all its
  reasoning, in order, growing live while it streams), then its tool-call chips,
  then one answer block of just the turn's text parts. AssistantParts/MessagePart
  are replaced by AgentTurnItem/UserMessageItem; ChatConversationPane counts the
  folded rows.
- Delta part ids (server + client): internal/protocol/chat.go adds
  ChatDeltaPartID(messageID, kind, ordinal), mirrored as ChatProtocol.deltaPartId
  in data/protocol/Chat.kt, and NormalizeChatEvent keys a text or reasoning delta
  on it. OpenCode numbers a message's text and reasoning parts independently from
  0, so the old "%s#%d" gave the first text delta and first reasoning part the
  same id: the answer was appended into the thinking block, and the server's
  retained state (chat_hub.go) re-collided on resume. ChatStore's upsert now
  routes a delta through the kind-aware deltaPartId fold, so it stays robust even
  against a peer that still shares one id.
- Tool fields (server + client): NormalizeChatEvent takes a call's arguments from
  session.tool.input.ended's text as well as session.tool.called's input object
  (toolInput), and chat_hub.go's new mergeChatToolPart folds each tool frame onto
  the retained part so the progress/success frames that omit name and input never
  blank them. ChatFormat.kt's mergeToolPart mirrors that on the client, so the
  chip and panel header keep the real tool name and the panel shows INPUT beside
  OUTPUT.

Tests: chat_v2_test.go and chat_test.go add the text/reasoning id-collision and
the tool name+input retention cases; ChatReasoningTurnTest, ChatTranscriptTest
and ChatRunningWorkTest cover the turn fold, the collision and the tool lifecycle.

Checks: Go build, vet, gofmt and the package tests pass (go build ./... &&
go vet ./...; gofmt -l . empty; go test ./internal/server ./internal/opencode
./internal/protocol ./internal/stage ./internal/pipeline; go test ./...).
Android passes offline with --no-daemon (assembleDebug and the full
testDebugUnitTest suite, 660 tests). The on-phone observation was not performed:
the device lock was held by another run and adb was not available here.

Documentation inventory checked: no packaged OpenCode skill reference in
internal/author/skill/references/ and no packaged template documents the app's
chat transcript, this delta part-id derivation or the tool-field retention, and
no internal/author/ managed file inventory entry names any touched file, so none
of them needs updating. The shared android/README.md behaviour statements are
made stale by this change and are recorded instead in
docs/notes/chat-transcript-turn-fold.md for the docs-fold writer, per the repo
rule that an item never edits a shared document itself.
2026-10-18 01:47:52 +01:00
alan 8c3efa4143 docs: add end-to-end app verification report against the real server
Adds docs/app-integration-verification.md, an operator-facing report of an
end-to-end verification of the Android app against the real running Pipelines
server on the phone. Report only: no app, server, installer or Docker file was
changed, and nothing was fixed.

The report records, per flow, the result, capture path and the real server data
observed: the pipelines list and its filters/sort, pipeline detail tabs, runs
list aggregate (246 total) and run detail, the stage sheet/timeline and
full-screen artifact viewer gaps, artifacts gallery, omni search, Settings
(endpoint ws://10.0.0.75:8080/connect, server build 0d1ddc73a4), the templates
gallery, chat sessions, live run updates and the disconnected state that were
not verified and why.

Revision tested: phone app at main's build (0d1ddc7, installed md5
722f6a139a8aea6d15b09c89360dbea0), server build
v0.0.0-20261018001257-0d1ddc73a444. The phone was left on main's build.

Checked for staleness: no packaged template or packaged authoring skill
reference (internal/author/skill/references/*) describes or indexes this
document, and the change adds no runtime behaviour, so none of them needed
updating. Per the maintenance convention the shared docs were not edited; the
verification findings live in this document alone.
2026-10-18 01:36:22 +01:00
alan 1e7607c502 Merge main into wait-cmd: combine the wait command with chat-persistence pairing
Brings main (chat-persistence: OpenCode pairing, protocol status, server
websocket/status changes) up to date on this branch so it can land without
conflict. Auto-merged with no conflicts; no comment or intent from either side
was dropped. Re-ran build, vet, gofmt and the full test suite after the merge -
all pass.
2026-10-18 01:11:54 +01:00
alan b7c2ba7d56 cli: add wait <run-id> to block until runs finish, then report them
The harness resumes a session when a background command finishes, so "tell me
when this run is done" needs one command that blocks and exits rather than a
polling script.

- `pipelines wait <run-id> [run-id ...]` resolves every id up front (an unknown
  run is a fast client error), then polls each run to a terminal status -
  `succeeded`, `failed`, `canceled` or `interrupted`, blocking through a run
  parked on a human decision - and prints one line per run as it finishes with
  id, status and outcome, stage and duration.
- A finished-but-unsuccessful run is still a finished run: it is reported and
  the command exits 0. The exit is non-zero only for a client-side failure: an
  unknown id, a connection that cannot be established, or `--timeout` expiring
  while a run is still active. A timeout reports the still-active runs,
  changes nothing, and is never treated as a run's outcome.
- Reuses the existing `run --wait` polling machinery: the per-second `get_run`
  reattachment is factored into `awaitRuns`, used by `finishRun` too. No
  subscription or new protocol message. `--url`/`--timeout` follow the existing
  client conventions; `--timeout` is the overall client limit (default 0).
- Read-only: sends only `get_run`, resolves no workspace, is not refused by the
  workspace guard and writes nothing. `--json` prints the final run envelope
  (one envelope for one id, an array for several).
- `commands.go` and `help.go` add it to the usage inventory and EXECUTE help.

Tests: `wait_test.go` (line format, batch reporting, wait-until-terminal,
unknown id, timeout leaves the run active, `--json` shape) and
`wait_command_test.go` (real CLI against the real websocket server and store).

Packaged assets checked: no packaged template changes. No authoring skill
reference file is added, so `internal/author/install.go`'s managed inventory
needs no change. `internal/author/skill/references/operations.md` (the
"inspect results first" run-operations list) and `docs/cli.md` (the scripted-runs
`--wait` paragraph and the CLI/TUI capability-map row) are now stale and are
recorded - not edited - in `docs/notes/wait-command.md` for the `docs-fold`
writer. `SKILL.md` stays a task-directed index; no shared document is edited
here.

No protocol, server, store, TUI or Android change; no new dependency; no run
started. Committed locally only, not pushed or deployed.
2026-10-18 01:11:26 +01:00
alan 648c9c33d8 Merge main into chat-persist: combine OpenCode pairing chat persistence with main's bottom selector strip
Brings main (78725e9, "bottom-strip": the Android detail and visit selectors
moved to a bottom strip, with its docs/notes/bottom-selector-strip.md) into this
branch on top of the chat-persistence commit. The two sides touch disjoint files
- main changes android/ and its own note, this branch changes
cmd/pipelines-server, internal/opencode, internal/protocol, internal/server and
docs/notes/chat-persistence.md - so the merge applied with no conflict and no
resolution was needed. Repository checks after the merge: go build ./...,
go vet ./... and gofmt -l . clean; go test ./internal/... ./cmd/... green.
2026-10-18 00:55:48 +01:00
alan 0ba1204895 chat: attach to the operator's own OpenCode by pairing, fall back honestly
Brokered chat accepted an operator-supplied endpoint (-opencode-endpoint /
PIPELINES_OPENCODE_ENDPOINT) but failed closed against the operator's running
V2 service: that API authenticates every request with HTTP Basic auth (fixed
user "opencode" plus a password), the broker had no credential, so the health
probe answered 401 and startup logged "OpenCode endpoint ... is unreachable or
is not an OpenCode service", disabling chat entirely.

Established from the installed binary (opencode v2.0.20), not from docs:
`opencode --help` advertises `api`, `pair`, `serve`, `service` and `auth`;
`GET /api/info` is unauthenticated-401 with www-authenticate Basic realm
"Secure Area"; Basic auth user `opencode` with the password from
~/.local/state/opencode/service.json answers 200; `POST /api/pair` returns a
code and `GET /auth/connect/{code}` returns a token that is then accepted as
the Basic password.

- internal/opencode/pair.go (new): PairEndpoint does the pair exchange over the
  shared nativeRequest transport (same auth, bounds and JSON checks);
  ReadServiceCredential reads the operator's local service password under the
  private-file rule; OpenEndpointAttached resolves the credential in precedence
  order - explicit password file, a paired token stored by an earlier run, then
  a fresh pairing bootstrapped by the local service credential - stores the
  token owner-only and re-pairs once when a stored token is refused (401).
- internal/opencode/endpoint.go: OpenEndpoint wraps probeEndpoint, which returns
  the connection even on failure; Connection.unauthorized (set on HTTP 401 by
  the request paths, cleared per probe) distinguishes an authentication refusal
  from any other refusal.
- cmd/pipelines-server/main.go: the endpoint attach uses OpenEndpointAttached
  with the paired-credential file beside the pipeline database. An endpoint that
  cannot be attached logs a line naming it and falls back to the server's own
  OpenCode instance instead of leaving chat refused.
- internal/protocol/status.go, internal/server/websocket.go: get_status gains an
  opencode object (health = endpoint | fallback | spawned | unavailable, plus
  the endpoint url), the honest status evidence for the fallback. No credential
  is ever placed there.
- Tests: internal/opencode/pair_test.go, internal/protocol/status_test.go and a
  TestStatusReportsOpenCodeAttachment case in internal/server/status_test.go.

Repository checks: go build ./..., go vet ./... and gofmt -l . clean;
go test ./internal/... ./cmd/... green.

Packaged templates and skill references: the repository has no template tree
that mentions the OpenCode endpoint, and no packaged reference file is added, so
internal/author/install.go's managed inventory is unchanged. The packaged
authoring reference internal/author/skill/references/operations.md (the endpoint
paragraph around line 289, the "clear refusal and no fallback" wording and the
password-file/username-only credential story, and the missing opencode status
field) is now stale; per the maintenance convention the item does not edit the
shared document and instead records the stale points, with the exact wording
they should take, in docs/notes/chat-persistence.md for the docs-fold writer.
2026-10-18 00:55:26 +01:00
alan 49f40bfe45 android: move the detail and visit selectors to a bottom strip on narrow windows
The two selectors that were a horizontally scrolling Material 3 tab row at
the top of a surface become one deliberate treatment. On a narrow window the
shared detail selector (`DetailTabs`) and the visit selector are now the shared
bottom strip (`ui/components/SelectorStrip.kt`), pinned in the thumb zone above
the system gesture inset with the content filling the area above; a wide window
keeps the left rail for the detail selector and the pinned top row for visits.

- SelectorStrip: the one bottom strip plus its pure model (`SelectorEntry`,
  `SelectorPlacement`, `detailSelectorPlacement`, `visitSelectorPlacement`,
  `selectorStripShown`, `detailSelectorEntries`, `visitSelectorKey/Label`,
  `visitMarker`, `visitSelectorEntries`). Each entry keeps at least the platform
  minimum touch target, shows its whole label, and a visit shows its recorded
  status/outcome as a small `RunStatusIcon` marker rather than an invented one.
  Selection is carried by tab semantics and container colour, not colour alone.
  A single entry (or none) draws no strip.
- `DetailTabs` narrow branch replaced by `SelectorStrip` after the panel in a
  Column; the wide rail is unchanged.
- `StageDetailPane`: `StageVisitStrip` is the narrow visit selector; the bodies
  put a `weight(1f)` scroll above it and keep the top row only on a wide window.
  The stage definition section still shows no strip.
- `SessionExport`: `SessionExportSection` gains `visitTabs` so the narrow pane
  does not render the visit selector twice.
- `PipelinesApp`: the strip publishes its measured height through the new
  `LocalBottomStripInset`/`BottomStripInset`, and the floating pill's overlay
  lifts by `pillBottomOffset(stripHeight) = SPACING_L + stripHeight` (pure, zero
  when no strip is up), so the pill never overlaps the strip.
- Tests: new `SelectorStripTest` (placements, single-entry no strip, detail
  entries keep tab keys/titles/icons, visit labels and markers, stable visit
  keys) and a new `PipelinesNavTest` case pinning the pill lift.

Pill/strip clearance is the brief's "pill moves clear of the strip" option,
implemented once in `pillBottomOffset`. All geometry and motion come from the
existing named tokens and the layout; no new dependency.

Documentation checked: no packaged OpenCode skill reference (`internal/author/
skill/references/`) or packaged template (`examples/*.md`) documents the app's
detail/visit selector presentation, so neither is stale and `internal/author/
install.go`'s managed inventory needs no change. The shared `android/README.md`
is made stale by this change (the narrow tab-row wording at lines ~182-184,
~233-235 and ~487-488) and is recorded for the single `docs-fold` writer in the
new `docs/notes/bottom-selector-strip.md` rather than edited here.
2026-10-18 00:50:15 +01:00
alan d6cb1931d0 Merge main into lane-pill: combine the pill overlay with main's chat-close, omni-search and status work
Brings the branch up to date with main's tip so it can be merged back without
conflict. Merged cleanly, with no conflicted path: git auto-merged
ui/PipelinesApp.kt, ui/chat/ChatListPane.kt and ui/screens/RunHistoryScreen.kt,
keeping main's changes (the omni-search panel, the one status colour, the chat
close/tool-work surfaces) alongside this item's pill overlay, its
LocalPillBottomInset clearance and the SettingsDestination templates stack.

android/README.md is main's version, not this item's: this item records its
documentation change in docs/notes/pill-overlay-and-position.md for the single
docs-fold writer instead of editing that shared document.

Checks re-run on the merged tree (macOS, JAVA_HOME=Android Studio jbr,
ANDROID_HOME=~/Library/Android/sdk, Gradle 8.11.1, --offline --no-daemon):
`gradle :app:assembleDebug :app:testDebugUnitTest` -> BUILD SUCCESSFUL, 67 result
XMLs aggregating to 627 tests, 0 failures, 0 errors, 0 skipped.
2026-10-18 00:21:20 +01:00
alan 5a44ff1568 android: make the destinations pill a true overlay, centred narrow and right when wide
The floating destination pill no longer takes layout space: it is drawn in the
bottom overlay layer above the content, horizontally centred on a narrow window
and anchored to the bottom-right corner on a wide one. Templates leaves the pill
for a Templates row on the Settings page.

- PipelinesApp: the content Column's pill reservation is gone; contentPillPadding
  (pillVisible) is the named zero and a unit test pins both states. The pill and
  the attention chip stay in their own bottom-anchored overlay Column, a sibling
  of the content in the Box, so nothing positions the content. The one placement
  rule is destinationPillAlignment(wide) (CenterHorizontally narrow, End wide);
  the corner margin is the named SPACING_L, so no new number. The attention chip
  keeps its own bottom-right place at every width via Modifier.align(End).
- Trailing clearance moves into each scrollable's own content: pillScrollInset
  (DESTINATION_PILL_RESERVE + ATTENTION_CHIP_RESERVE while drawn, zero while
  hidden) is published through the new LocalPillBottomInset
  (components/PillInset.kt) and applied with pillContentPadding /
  Modifier.pillBottomPadding in every list, grid and scrolling detail, stage and
  settings body. The container keeps its full height, a hidden pill leaves no
  empty band, and the last row still scrolls clear of the pill.
- Templates is no longer a pill destination: TemplatesDestination.kt is deleted
  and destinations is pipelines/runs/chat/settings. The new SettingsDestination
  sliding stack (pane 0 SettingsScreen, 1 TemplateGalleryScreen, 2
  TemplateDetailScreen) with the pure, Bundle-able SettingsSelection model gives
  the gallery the shared covered-pane strip and one-level Back; SettingsScreen
  gains the Templates row that opens it. This re-lands the intent of a2043f9,
  which never reached main.
- Dimens: the two reserve tokens' docs now describe the scroll-content clearance
  and the chip's share of it; no new token and no hardcoded geometry.

Checked where the change belongs: no packaged template (examples/*.md) and no
packaged authoring skill reference (internal/author/skill/references/*) describes
the Android pill, its destinations or the Templates gallery, so none needs
updating and internal/author/install.go's inventory is unchanged. android/README.md
does document the surfaces this item changed, so it is now stale: this item does
not edit that shared document - docs/notes/pill-overlay-and-position.md records
the two places that need it (the Templates bullet and the dated section to append)
for the single docs-fold writer.

Tests: at a297fb6 with these working changes the build and unit suite were green
(593 tests, 0 failures); no device command was run and the geometry stays
token-derived. Visual rendering of the centred/right pill needs the device and
was not verified here.
2026-10-18 00:20:37 +01:00
alan f85c172dc2 Merge main into chat-close: combine the shared chat room with the server-side close
The item adds chat_close/chat_reopen to the brokered chat protocol and the
operator-session registry's closed state. main replaced the per-session claim
model with shared chat rooms (chat_subscribe, a prompt queue, one broker stream
fanned out to every watcher). This merges the two.

Resolved conflicts:
- internal/protocol/chat.go, internal/protocol/message.go: kept both additions -
  ChatClose/ChatReopen and main's ChatSubscribe/ChatQueued.
- internal/server/chat_sessions.go: kept the closed fields on a listed session;
  dropped the `Busy` assignment, because main no longer marks a session Busy (any
  number of clients may watch and write one).
- internal/server/chat_claims.go: main deleted the claim model, so the file goes;
  close's "is a turn running" check is now `chatHub.turnInFlight` in
  internal/server/chat_room.go, reading the room's `running` flag. That is the
  same question the old claim answered, against main's model.
- internal/server/chat_close.go: calls turnInFlight instead of the removed
  `active`.
- android AppViewModel.kt, ChatFormat.kt, ChatListPane.kt, Message.kt: kept both
  sides - main's search state, queued-prompt handling, `Busy`-free status label
  and row/menu shape, plus the item's close/reopen actions, closed chip and
  section, close confirmation and "Hide on this device" rename.

Also in this merge: main's new convention (AGENTS.md) that an item does not edit
a shared document; the item's stale-document record for this change lives in
docs/notes/chat-close.md, and docs/cli.md, android/README.md and
internal/author/skill/references/operations.md are left to the docs-fold writer.

Checks after resolving: gofmt clean, go build ./... and go vet ./... clean, the
Go module passes, and the app's :app:assembleDebug and :app:testDebugUnitTest
pass (--offline --no-daemon).
2026-10-18 00:19:23 +01:00
alan 971e555ceb chat: close and reopen an operator session on the server, keeping its history
Add a server-side close for operator chats, so finishing with a chat takes it out
of the active list for every client without deleting anything. The brokered
protocol gains chat_close and chat_reopen; chat_close stops a running turn by
reusing the existing broker interrupt (its result carries an interrupted flag, so
a stream is never left running into nothing) and then sets closed_at on the
session's operator_sessions row. The session, its transcript and its history
survive, and because the closed state lives in the registry chat_sessions reports
it to every client, not just the one that asked. An id that is not a recorded
operator session, including a pipeline-owned run session, is refused before
anything is written. chat_reopen clears the state again, and simply prompting a
closed session also clears it (the store clears closed_at on touch).

The registry gains closed_at for a fresh database and an idempotent migration for
one created before it existed. The app renders closed sessions under a labelled
Closed section with a Closed chip, offers a Close action with a confirmation that
says it stops a running turn, marks the chat closed on every device and deletes
nothing, and a Reopen action on each closed row. The device-local hide is renamed
Hide on this device, with a snackbar saying the chat stays on the server, so it is
never confused with the real close. No new dependency, the installer and Docker
files are untouched, and no run is started from the app.

Checked before committing: the packaged templates carry no chat surface and need
no update, and internal/author/install.go's managed inventory needs none either
(this adds no reference file). The packaged authoring skill reference
internal/author/skill/references/operations.md and the shared docs/cli.md and
android/README.md are stale because of this change, but an item no longer edits a
shared document: what is stale and what should change there is recorded in
docs/notes/chat-close.md for the docs-fold writer, which is committed here with
the change.

Checks: gofmt clean, go build ./... and go vet ./... clean, and
go test ./internal/protocol/... ./internal/pipeline/... ./internal/server/...
./internal/author/... pass. The app side is verified with :app:assembleDebug and
:app:testDebugUnitTest (--offline --no-daemon, BUILD SUCCESSFUL); the on-device
close, second-client and reopen check is deferred to visual review.

Not deployed; deploying is the operator's step.
2026-10-18 00:16:24 +01:00
alan 672d260b33 Merge main into multiclient: combine the multi-client chat fan-out with main's chat tool-work, notifications, status palette and docs-fold changes
No conflicts: main's chat-surface work (the in-flight tool work in ChatFormat,
ChatStore and ChatTranscript, run notifications, the status palette) sits in
different hunks from this branch's fan-out edits, and main did not touch
docs/cli.md. Checks re-run on the merged tree: gofmt -l . (clean), go build ./...,
go vet ./..., go test -count=1 ./... (all packages pass; the focused multi-client
tests fan-out, per-session serialisation, first-answer-wins, pending replay and
resync pass), and the Android unit tests
(gradle --offline --no-daemon :app:testDebugUnitTest, BUILD SUCCESSFUL, 598 tests,
0 failures).
2026-10-18 00:10:15 +01:00
alan 6cb61578de Fan a chat session out to every client and share approvals exactly-once
A session was owned by one connection: the claim map in chat_claims.go let the
first client hold it, every other client saw the session Busy and read-only, and
a second client's prompt was rejected. Replace the claim with a per-session
"room" so the operator's phone, tablet and terminal all watch and write the same
conversation, and a pending approval is a shared, exactly-once object.

Server (internal/server, internal/protocol):
- New chat_room.go: one room per session holding the session's single broker
  stream, its one bounded prompt queue (8) and the fan-out to every attached
  client. A chat_subscribe frame attaches a connection to a session's shared
  stream and replays the still-unresolved approval requests the room holds.
- chat_stream.go: handleChatPrompt accepts a prompt from any client. It attaches
  the sender, either starts a turn or queues the prompt behind the in-flight turn
  and acknowledges it with chat_queued:true ("queued" is the chosen behaviour,
  stated on the wire and in docs/cli.md), and echoes the user turn to every other
  client attributed to the sender's label. The prompt handler no longer owns the
  stream; one turn runner per room drains the queue, so two turns never
  interleave. Per-connection write serialisation is unchanged (each chatSub has
  its own writer goroutine and bounded buffer).
- chat_permissions.go / chat_commands.go: an approval is reserved by the first
  answer and confirmed once delivered; the first answer wins, is recorded with
  the answering client's label, and the resolution is pushed to every watching
  client as chat_permission with resolved/decision/resolved_by so their cards
  clear. A later answer is refused with an error naming the client that answered,
  and never reaches the agent twice; a reservation whose reply failed is
  released.
- chat_hub.go / chat_sessions.go / chat.go: the hub now owns rooms and the
  user-turn id sequence; the resulting user turns are retained (role and client
  carried) so a reconnect re-syncs who said what. chat_sessions no longer marks a
  session Busy and its clientID parameter is gone; the protocol keeps the Busy
  field for wire compatibility only. websocket.go detaches a connection's chat
  subscriptions when the read loop ends.
- chat_claims.go is removed, not left vestigial: the only exclusive thing left is
  a turn, and the room's queue is that.

App (android):
- PipelinesRepository carries a stable per-device client label (Build.MODEL, with
  a fallback) on prompts and replies, and adds chatSubscribe; AppViewModel
  subscribes on opening a chat (best-effort, an older server's error frame does
  not stop history loading) and records the queued ack.
- ChatStore folds a user frame from another client into the transcript once,
  attributed by the sender's label, and never receives its own prompt back, so
  the sender's own message is never duplicated. ChatThread.readOnly is gone:
  canSend no longer depends on another client, the composer says "Queued - it
  will send when the current turn finishes." instead of disabling input, and the
  status label is Running/Idle only. Scroll/follow behaviour is untouched.
- Tests: ChatProtocolTest round-trips the new wire fields, ChatFormatTest asserts
  the stale Busy flag cannot produce a read-only label, MultiChatStateTest asserts
  a session another client watches stays writable.

Reflected where it belongs: no packaged template changes (no bundled starter
embeds the chat protocol). Checked the packaged authoring skill references: none
of them describes the chat wire beyond operations.md, and
internal/author/skill/references/operations.md is now stale - its "Brokered
OpenCode chat" section still says chat_sessions marks "busy" when another client
holds the session and that a second client's chat_prompt is rejected, and it does
not mention chat_subscribe, the per-session prompt queue/chat_queued, the
per-client label or the shared exactly-once approval (it also still describes the
per-subscription permission poller, which the room's single poller replaced). The
supported semantics are the ones in docs/cli.md, updated in this commit; the
operations.md section needs a rewrite to match. No README or other shared
document was edited here.

Checks run on the committed tree: gofmt -l internal cmd examples (nothing), go
build ./..., go vet ./internal/... ./cmd/..., go test ./internal/server
./internal/protocol ./internal/stage ./internal/pipeline.
2026-10-18 00:09:14 +01:00
alan f339503f68 Merge branch 'main' into session-activity 2026-10-18 00:05:10 +01:00
alan 597c7b5e76 android: show a chat's in-flight tool work instead of a bare Running
A session said "Running" with an empty tool chip while it worked, so a silent
wait was indistinguishable from a stuck one. Derive the in-flight work from the
stream the app already folds and show it on both surfaces.

- ChatFormat: runningToolCalls() reads the tool parts still running, in
  transcript order, so the conversation and the session row share one set;
  toolDetail() takes the command/description the part actually carries (the
  first of description/command/filePath/file_path/path/pattern/query/url),
  collapsed to one line and elided at 72 chars, and returns null rather than a
  fabricated summary; runningToolElapsedMillis() uses the protocol's own
  time.ran/created when present and the local per-part stamp otherwise;
  runningToolStamps() keeps a stable start per part across frames;
  TOOL_LONG_RUNNING_MILLIS (60s) and toolLongRunning() mark an unusually long
  call; sessionWorkLabel() names what a working session is doing while a
  text-streaming turn stays plain "Running" and a busy session never reads
  "Idle".
- ChatStore: ChatThread gains toolStartedAt, stamped when a part turns running
  and dropped when it settles, plus runningTools/toolElapsedMillis; history()
  and the frame fold maintain it; setRunning/turn end clear it.
- ChatTranscript: RunningIndicator lists one line per running call (name,
  detail where present, live elapsed) keyed by part id so concurrent calls
  render together without flicker; no percentage, ETA or progress bar.
- ChatListPane: the session row renders sessionWorkLabel(busy, thread).
- ChatRunningWorkTest covers name/detail/elapsed, clear on completion, error
  state, concurrent calls and the no-detail stream.

Only fields the harness reports are used: tool name, state.status and input;
the live brokered frame carries no timing envelope, so elapsed falls back to the
local stamp. Existing chips, INPUT/OUTPUT, reasoning blocks, stop control,
activity marks and scroll/follow behaviour are untouched.

Checked and unchanged: no packaged template or packaged authoring-skill
reference describes this app surface (the chat references in
internal/author/skill/references/ document the server's brokered chat_* protocol,
which this change does not touch), so none is stale. android/AGENTS.md already
names ui/chat/ and its files. The shared documents this change does make stale
are recorded, not edited, in docs/notes/chat-in-flight-tool-work.md: the
Running-indicator and session-status statements in android/README.md.
2026-10-18 00:05:07 +01:00
alan a3f571d672 docs: make docs-fold the single writer of shared documentation
Every item wrote the shared, append-only docs (android/README.md, docs/cli.md and
the packaged skill references), so every merge conflicted on prose. Keep the duty
to notice stale docs; move the writing to one deliberate place.

- Per-item note, not a shared edit. An item that makes a shared document stale
  writes docs/notes/<slug>.md - unique per item, so two items never touch the same
  file - and commits that note with its change. It never edits a README, topic doc
  or packaged skill reference itself.
- One writer. The manual-only docs-fold pipeline, bundled as the fifth starter
  examples/docs-fold.md, folds every note under docs/notes/ into the shared
  document it names and deletes the consumed notes in its own commit; an empty
  notes directory is a success no-op. Operator trigger:
  pipelines new "Docs fold" --template docs-fold --no-open; pipelines submit
  <draft.md>; pipelines run "Docs fold".
- Implemented where the convention lives: orchestration.md (brief item 5, the
  hard-constraint bullet, a "Single documentation writer" section and the
  "Pipeline choice" committer bullet), the canonical committer checklist in
  format.md (new step 6), the committer prompts in examples/task.md and
  examples/three-stage.md, the new examples/docs-fold.md with its catalog and
  README entries, and the repository guideline AGENTS.md. Test-expectation counts
  (examples/catalog_test.go, internal/server/templates_test.go,
  scripts/smoke-first-install.py) follow the fifth starter.

Reflected where it belongs: the packaged templates (examples/task.md,
examples/three-stage.md, the new examples/docs-fold.md), the packaged skill
references (orchestration.md, format.md) and the repo guideline AGENTS.md all carry
the new convention. No new reference file was added, so install.go's managed
inventory is unchanged. The shared descriptive docs were deliberately not
rewritten: this change makes README.md:191, docs/cli.md:170 and
internal/author/skill/references/operations.md:13 name four starters where there are
now five. Those stay as they are - correcting them now is exactly the per-item
shared edit this change removes - and the stale-doc note
docs/notes/docs-fold-single-writer.md records each one for the docs-fold writer to
fold. Nothing else needed updating. No new dependency; no Android, Gradle, server,
installer, Docker or published-definition change; no definition published over the
running pipelines.

Evidence: go test -count=1 ./... passed; go build ./... and go vet on the touched
packages clean; ./internal/author/... covers embedding, reference links, the managed
inventory and install/upgrade/protection; ./examples/... validates the fifth starter
and its manual-only, committer-terminated graph; ./internal/server/... covers the
served catalog. Not run: the docs-fold writer against the shared docs (nothing to
fold yet); no install or deploy.

(cherry picked from commit f29656cf084d7eee2e033f2a37a9d231d7acaaf0)
2026-10-17 23:59:59 +01:00
alan 4426f8cacd docs: re-verify the chat record at main's build (ec27a2d)
Update docs/chat-verification.md from the earlier `3ffc126` record to the
current revision `ec27a2d`, which is both this lane's HEAD and main:

- revision, APK name/size/sha256 and LAN URL refreshed to the rebuilt
  artifact `pipelines-debug-wip-ec27a2d.apk` (18,585,084 bytes,
  sha256 5ee4b38c…da84);
- unit-test totals refreshed to 61 suites / 558 tests / 0 failures, and the
  server-side `go build ./...` plus the chat/session/adopt Go tests recorded;
- a "Changes since the previous record" section describing the chat-relevant
  delta `3ffc126..ec27a2d` (the `chat_adopt` request, the lazy
  `MarkdownBlocks` renderer, the run-watch scope change) and correcting the
  chat protocol file path to `data/protocol/Chat.kt`.

The change is documentation only: no app or server behaviour changes.
Checked where the change belongs: packaged templates (no template encodes
this record) and the packaged authoring skill references
(internal/author/skill/references/) — none is made stale by a
verification-record update, so none is touched. `go build ./...` passes.
The debug APK lives in the run's artifacts directory outside the repo and is
therefore not committed.

Refs: brief "Verify the chat end to end, on the phone, against the real server".
2026-10-17 20:55:19 +01:00
alan 3f5c11b7bf Record and expose per-run model usage (tokens, split, cost)
Carry the usage a model stage already measures for its budget
(stage.Budget.Snapshot()) onto the step as typed evidence and total it in
one pass, rather than a second harness read, a transcript parse or an
output-length estimate. A shared budget across stage revisits records only
the visit's increment, and a revisit that observes no new report records no
usage instead of a fabricated known zero.

- protocol: StageUsage/RunUsage/TotalUsage and UsageTotals.
- pipeline: step_usage table + migration, COALESCEd into the one step
  projection, FinishStep persistence, one json_extract history aggregate,
  retention removes the rows with their run.
- server: stageUsage maps the budget snapshot, stamping per-stage model and
  usage for every model stage (including failures).
- cli: inspect/stats human and JSON output.
- budget/pi: retain the reported in/out split, mark a total-only (Pi) report
  split-unknown, and keep an unreported cost unknown (never zero).

Evidence and docs updated in the packaged skill references and docs/cli.md;
the live database has no step_usage rows yet (all history predates this), so
per-run figures are reported as unavailable, not zero.
2026-10-17 20:40:56 +01:00
alan 1f75e4797e Broker chat from an operator's own OpenCode endpoint and adopt existing sessions
Add an opt-in way to broker chat from an operator-supplied OpenCode service
instead of the instance the server spawns for itself, and a way to adopt a
session that already exists on that endpoint so it appears in chat_sessions and
can be continued from the app.

- Endpoint selection: the `-opencode-endpoint` server flag (defaulting to
  `PIPELINES_OPENCODE_ENDPOINT`) opts in; precedence is explicit flag >
  explicit environment > the endpoint recorded on a previous run, with the
  spawned instance as the default. opencode.OpenEndpoint validates the
  reference once (http(s), host, no userinfo/query/fragment/path), detects the
  backend from its own /api/info (v2) or /global/health (v1) contract, and
  requires a streamable text/event-stream event path. It owns no process
  (Close is a no-op) and never calls startServe; every failure names the
  endpoint and disables chat rather than silently falling back. An
  authenticated endpoint takes an optional password only through the private
  file PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE, with
  PIPELINES_OPENCODE_ENDPOINT_USERNAME overriding the detected default.
- Persistence: the validated choice is recorded in an additive single-row
  server_settings table so a restart with no flag/environment reuses it; an
  unreachable recorded endpoint disables chat with a clear message and no
  fallback.
- chat_adopt: a new protocol request (ChatAdoptRequest) with handleChatAdopt in
  internal/server/chat.go. It refuses a nil broker/registry/payload and a
  blank or foreign-shaped id before any endpoint call, reads the endpoint via
  the shared Chat.Session (which reuses Sessions/decodeChatSessions), records
  through the existing RecordOperatorSession with the endpoint's directory and
  title verbatim, and echoes the adopted session plus its registry row like
  chat_create. A missing id is refused naming the id and records nothing; an
  endpoint record with an empty directory is refused rather than substituting
  the server's working directory or home.
- CLI: `pipelines chat-adopt <session-id> [--json]` (usage, notes, help
  inventory, dispatch) with docs/cli.md updated under Chat sessions.
- Autostart preserves the endpoint variables (and the password-file reference,
  never an inline secret) via serviceEnvironmentKeys.

Checks before committing:
- Package location: the change is user-facing chat behaviour, so the packaged
  authoring skill reference internal/author/skill/references/operations.md is
  updated in the same change (Brokered OpenCode chat section) with a dated
  verification-evidence entry and the not-deployed state. No new reference file
  is added, so internal/author/install.go's managed inventory is unchanged.
- Packaged templates: this repository packages no template set for this
  behaviour and none references the chat endpoint, so no template needs
  updating.
- Verification: `go build ./...`, `go vet` on the touched packages, `gofmt -l`
  clean and `go test ./...` all pass. The unit tests cover the endpoint
  validation/precedence, the adopt happy path and refusals, the recorded
  endpoint round-trip, the protocol round-trip and the CLI dispatch. The live
  scratch-server check against the operator's real endpoint was not run in this
  change.

Nothing unrelated is staged; docs left stale (README and other shared documents)
are not edited here and are for a single folding writer.
2026-10-17 20:14:11 +01:00
alan 58c6b9b7c6 server: export a finished stage's real session as session.md and session.html
Every OpenCode-harness model stage now writes its real session into its own
step artifacts directory as session.md and session.html, so the existing
bounded artifact listing, the /artifacts/<run>/<step>/<name> serving path and
the artifact scan find and serve them with no special casing, while the stage
keeps its bounded preview in the run record unchanged.

- internal/opencode/export.go adds the one shared native export invocation and
  the exact session-ID/messages validation used by both Connection.Messages and
  the new ExportSession, so the readback path does not diverge. The backend's
  published subcommand is used (v1 `export <id>`, v2 `session export <id>`),
  the source read is capped at 8 MiB, and a larger session is read as a prefix
  and reported truncated instead of failing. boundedOutput.ReadFrom keeps the
  cap effective so overflow and the truncation path are real.
- internal/stage/session_export.go renders markdown and html with a metadata
  header, every role and every part (unknown part types as raw JSON, every
  session text escaped in html), atomic temp+rename writes that never overwrite
  an existing file, a visible truncation banner at both ends when the source or
  the 2 MiB rendered cap cuts it, and an honest 0-message header for an empty
  session rather than fabricated text.
- protocol.SessionExport and RunStep.SessionExport are stored in a new
  step_session_exports table (CREATE TABLE IF NOT EXISTS, so an old database
  gains an empty table with no migration) loaded through the existing main
  SELECT alongside the other per-step evidence and deleted by run retention.
- internal/server/runner.go exports after the post-stage filediff capture and
  before scanArtifacts, with context.WithTimeout(context.WithoutCancel(ctx),
  10s); it never waits for session idleness and a failure or a Pi stage (no
  native export path) records a sanitized diagnostic and continues, never
  changing the stage status, outcome, target, timing or result validation.

Reflected where it belongs: the packaged skill references
(internal/author/skill/references/operations.md and delivery.md) carry the new
Session export rules and the run-artifacts bullet, and docs/stage-artifacts.md
documents the names, the <run>/<step> paths, the served URL, the caps, the
truncation labels, failure isolation and the Pi limitation. The packaged
templates (examples/*.md) only tell a stage to save its own artifacts and say
nothing about the server-written session export, so none needed updating; no
new reference file was added, so install.go's managed inventory is unchanged.

Shared documentation: README.md's per-step artifacts sentence is now stale - it
should name the exported session alongside images, screenshots, reports and
logs. It is deliberately not committed here; the shared documents are folded by
the single documentation writer.

Checks: gofmt -l on the touched packages produced no output; go vet ./...
passed; go test -race -count=1 ./... passed.
2026-10-17 19:40:43 +01:00
alan 976169e413 Record the chat end-to-end verification record
Add docs/chat-verification.md, the required record for the chat verification brief: per-flow table (flow, result, evidence path, notes), the known tree-level gaps, and the section on what could not be verified and why. Verification only; no app behaviour changed.

Checked: packaged templates (internal/author/skill/references/) do not mention this record - no template or skill reference needed updating. Shared README and topic docs are intentionally not touched (single-writer fold).
2026-10-17 18:02:00 +01:00
alan 476ecc781c Merge main into lane-srv3: combine the review-loop control change with main's notification and chat work
Keeps this branch's advisory review edge, bounded blocked_retryable implementer
edge and their tests alongside main's run-attention/ntfy notification work, the
split of the Android chat conversation screen, and the retention/history changes.
No conflicts: the two sides touch disjoint files.
2026-10-17 17:07:17 +01:00
alan e70c94e8dd control: make advisory review findings non-blocking and bound retryable blocks
Cut the review loop's repeat cost and make a blocked stage actionable, at the
definition level only, with the runner's existing visit counting and caps.

The published maintenance, quick and focused pipelines gain a reviewer
`advisory -> next stage` edge for non-blocking notes, so an advisory-only review
continues the run instead of buying another implementer visit; `rejected ->
implementer` and its unchanged {"max_visits":3,"on":["rejected"]} limit keep a
must-fix finding looping and still terminate. The implementer gains
blocked_retryable -> implementer with {"max_visits":5,"on":["blocked_retryable"]}
for a transient environment/tooling/device problem a fresh attempt could
survive, while blocked -> #failure stays terminal. The cap is at least the
reviewer's rejection budget so the implementer's own pre-execution cap cannot
preempt the reviewer's rejected control.

No production Go code changed. New tests in internal/server/review_contract_test.go
cover advisory-only never revisiting the implementer, must-fix still looping to
the reviewer limit, the implementer cap not preempting the reviewer, retryable
blocks bounded then failing closed, a retry that recovers, and a terminal block
failing immediately; concurrency/control/human tests share the new submitRunner
and startRun seams.

Documentation checked: internal/author/skill/references/control.md and
docs/harnesses-and-control.md carry the new edges and the visit-cap rule;
no packaged template defines these pipelines (they are published immutable
versions), and docs/model-results.md needs no change because the {outcome,output}
result contract is unchanged. README and other shared docs deliberately untouched.

Verified: go build ./... and go test -count=1 ./... pass.

Residual limitation: the must-fix-vs-advisory split is carried by the prompt
contract, not a machine-readable marker, because the recorded findings have none;
a model that still returns rejected for advisory-only notes would still buy a
visit. All 18 recorded implementer blocked runs classify terminal, so the
blocked_retryable edge recovers 0 of them.
2026-10-17 17:06:51 +01:00
alan 4908452a74 retention: report removed runs and follow a symlinked artifacts root
Fix two honesty gaps in run-history retention that the implementer stage
left uncommitted on top of 14175eb:

- Populate RemovedRuns, RemovedRunCount, RemovedArtifactFiles/Bytes and
  Applied right after the row transaction commits, before staging cleanup.
  A cleanup failure must not report a successful row/artifact removal as
  zero removed, and staging leftovers are excluded from SpaceReclaimedBytes.
  A failed VACUUM after removal now reports the artifact bytes actually
  removed while claiming no database pages.
- Stage artifacts through os.Stat instead of os.Lstat so a symlinked
  artifacts root is followed: the report sized that directory, so apply
  removes through it too instead of counting bytes it never deleted.

Tests cover post-commit staging-cleanup failure reporting, staging through a
symlinked artifacts root (including no staging directory left in the link
target), and an honest reclaim failure after removal.

Checked documentation surfaces:
- docs/cli.md: measured example refreshed with the current before/apply/after
  output and the symlink/staging/VACUUM reporting described accurately.
- README.md: retention prose (explicit, never automatic, protected runs,
  artifacts go with the run) is unchanged and still correct.
- internal/author/skill/references/operations.md: operator-only retention
  summary describes the same behaviour; no user-facing surface changed, so
  no reference update was needed.
- Packaged templates: retention is a CLI/store capability with no template
  change, so nothing to update. internal/author/install.go inventory
  unchanged (no reference added or renamed).

Verification: gofmt clean on the touched Go files, git diff --check clean,
go vet ./internal/pipeline clean, and go test ./internal/pipeline -run
'TestRetention' -count=1 passing.
2026-10-17 16:19:39 +01:00
alan 890d267c71 run attention: publish to ntfy and surface what needs the operator
Nothing told the operator that a run was waiting for a person or had failed; they
found out by going looking. A run that needs attention now reaches the ntfy they
already run, and the app answers "what needs me now" in one live, ordered surface.

Server (smallest change, reusing the existing publisher):
- internal/server/notify.go adds notifyRun, called from finishRunState (failed)
  and when a run parks on a human (runner.go runLoop). It publishes to the same
  already-configured `system` destination through the unchanged stage.PublishNtfy,
  so there is no second publisher, topic or service and the token handling is the
  stage's own. runAttentionEvent maps only waiting and failed; queued, running,
  succeeded, canceled and interrupted publish nothing.
- No storms: runner.attentionSent records sequence:event per run before publishing,
  so a run that stays waiting is never republished on a re-request or restart, and
  a stage notify policy that already covers the same event marks it first so the
  run-level default is suppressed. The set is dropped when the run is terminal. A
  delivery failure is logged, never fails the run, and is not retried.
- The payload names the run and pipeline and carries a click action
  pipelines://run/<id> (validated: absolute URL, no userinfo, no fragment, no
  credentials). PublishNtfy gained the optional click parameter; existing callers
  pass "".

App:
- AndroidManifest/MainActivity declare the pipelines://run/<id> deep link
  (singleTop); runIdFromDeepLink parses it and PipelinesApp opens that run once,
  clearing the link. It opens a run; it starts nothing.
- ui/Attention.kt is the pure model (status -> Decision/Failed/Finished or
  nothing, ordered by kind then newest-first) and ui/components/AttentionSurface.kt
  is the floating indicator plus bottom sheet. It is derived from the same run
  events the server already pushes, so an entry appears with no refresh and clears
  when resolved; a finished run clears once looked at, and any entry can be
  dismissed, persisted in SettingsRepository. The existing destinations, pill and
  bottom strip are untouched and no competing navigation surface is added.

Checked before committing:
- Packaged authoring skill references: internal/author/skill/references/
  notifications.md documents the automatic run-attention notifications and the
  app surface. The only packaged "template" carrying notify is that reference; no
  pipeline definition template exists to update and install.go's managed file list
  is unchanged (notifications.md already existed).
- Repository guidelines: docs/ntfy.md gains an "Automatic run-attention
  notifications" section and no longer claims "No block means no notifications";
  the top-level README phone-notifications note now says a waiting/failed run
  notifies automatically; android/README.md gains the dated feature section and
  supersedes the previous section's note that the attention surface was absent.
- docker.md, install docs and the CLI/TUI docs describe mechanisms this change
  does not alter; left as they are.

Tests: go test ./internal/server/... ./internal/stage/... ./internal/author/...
all ok; notify_test.go covers the event mapping and the real HTTP delivery
(destination, topic, click, no duplicate). Android :app:testDebugUnitTest ->
BUILD SUCCESSFUL, 58 suites / 531 tests / 0 failures (new AttentionTest, 13 cases,
covering ordering, appearing from a push, clearing on resolution, dismissal, the
empty case, persistence and the deep link). Device screenshots and the phone's own
notification are visual review's per the brief's device policy.

Scope: android/ plus the smallest server change; no new dependency, no run started
from the app, no installer or Docker change.
2026-10-17 16:11:57 +01:00
alan 14175eba70 retention: explicit run-history and artifact retention
Add `pipelines retention report` (strictly read-only) and
`pipelines retention apply` (explicit, approval-gated removal) with an age
and/or count policy, protected-run rules, atomic single-transaction removal
and honest database space reclaim. Nothing prunes automatically.
2026-10-17 15:45:46 +01:00
alan 7b72b8f20f stage: retry pre-admission harness reach failures and widen repair deadline
Treat a failure to reach the harness before any prompt or tool call is
admitted (connecting or creating the session) as a distinct retryable
reach failure, so a transport blip at the first stage no longer ends a
mutating run; keep post-admission readback/poll/agent failures retried
only on an explicit read-only stage and never replay admitted tool work.
Preserve the true cause when the context is already done (join ctx.Err
with the typed transport error) and report a sanitized unknown provider
error name/status instead of a blanket summary.

Make the isolated format-repair deadline proportional to the stage
timeout (floored, capped and clamped to remaining minus a margin) so a
slow-but-repairable result is recovered without ever extending the stage
deadline; the bounded ladder, value guard, deny-all formatter session,
cumulative usage accounting and no-tool-replay rule are unchanged.

Update docs/model-results.md, docs/pipeline-performance-review.md and the
packaged skill references; add stage, opencode and server fixtures.
2026-10-17 14:51:09 +01:00
alan 5fafe75267 Merge finish-z02: keep the lane's reviewed work 2026-10-17 11:15:52 +01:00
alan 6a6c1d65fa android: drive the runs list and run detail from server pushes
Replace the Android app's one-second run poll with the server's
workspace-scoped run_event pushes. The runs list, each pipeline row's
last-run line, the run header and the run detail (status, timeline,
stages, artifacts) now advance from the socket instead of polling or a
manual refresh.

- Wire: Message.kt mirrors internal/protocol/runevents.go (subscribe,
  unsubscribe, run_event; RunEventFrame/RunStepDelta). The frames are
  unsolicited and already arrive on the client's events stream;
  PipelinesRepository.subscribeRuns/unsubscribeRuns add the typed
  command, sent via a new PipelinesClient.sendOnly that does not hold the
  send lock waiting for the running server's unreliable subscribe ack.
- One pure fold: ui/RunEvents.kt folds a frame into the list (in place,
  order preserved) and the open detail with one shared status ladder and
  one shared run-field merge. It is monotonic (a stale or duplicate
  delta is dropped whole), reports needsResync for a run the list does
  not hold instead of inventing a row, and only requests a full get_run
  for a settled step or terminal run.
- Wiring: AppViewModel subscribes once per (re)connect and workspace;
  every (re)subscribe re-reads the list and open detail once, because the
  deployed server's subscribe snapshot is partial, then folds the pushes
  on top. Re-reads are push-driven, coalesced and keyed so no push is
  dropped and no timer remains. LiveBadge (RunStream.kt RunStreamState)
  shows honestly live/reconnecting/not live, confirmed by the ack, a
  push or a successful read and dropped when a read fails.
- ui/RunEventsTest.kt pins the delta sequence as pure cases: created run
  not in the list, step started, step finished, run finished, unknown
  run, duplicate/out-of-order deltas, and a reconnect resync.
- android/README.md records the change, the wire mapping and the
  observed limitation of the running server (partial snapshot, no ack
  within 30s), which the client tolerates without inventing a stream.

Checked where the change belongs: android/README.md gained this entry;
docs/cli.md described the Android app as still polling, which this makes
stale, so that sentence now says the app consumes the pushes while the
TUI still polls. The packaged TUI pipeline templates (internal/tui) do
not touch run-state refresh and need no change, and the packaged
authoring skill references (internal/author/skill/references) document
the authoring workflow, not the Android client's run updates, so none of
them needs updating.

Reviewed and approved by the run's review stage (build, unit tests,
Kotlin compile and lint in android/README.md); no merge/rebase/cherry-pick
was in progress and no conflict was present.
2026-10-17 11:15:21 +01:00
alan 18398dfa5b chat: give the operator their own sessions, separate from run sessions
chat_sessions used to list exactly the OpenCode sessions the pipelines own,
filtered through pipeline.SessionOwner (run_steps/run_session_owners), so an
operator-created chat was an accident of the run-ownership table and the
operator's working sessions were mixed with the internals of running pipelines.

Add a first-class operator session:

- internal/pipeline/operator_sessions.go: a dedicated operator_sessions table
  (session_id PK, directory, title, tag, created_at, last_used_at) with
  RecordOperatorSession (upsert preserving created_at), a one-statement
  TouchOperatorSession (no-op for a non-operator id), OperatorSessions bounded to
  128 rows most-recently-used first, and OperatorDirectories (bounded MRU,
  deduped). run_steps/run_session_owners and SessionOwner/OwnedSessions are
  untouched.
- internal/server/chat.go: chat_sessions returns operator sessions only - it
  reads the registry once, intersects it with one broker session-list call, fills
  directory/tag/created/updated from the registry, sets busy for a session held
  by another connection, keeps MRU order, and stays fail-closed (empty list, never
  the endpoint's whole list). A pipeline stage session and an untagged row can
  never be listed. chat_create resolves the directory (empty/'~'/'~/...' ->
  operator home; relative or missing rejected before any OpenCode call), calls
  broker.Create(title, resolved) and records the row with the operator tag.
  chat_directories returns home first then bounded (8) deduped recents. A
  successful prompt touches the operator session without affecting pipeline
  prompts.
- internal/opencode, internal/protocol, internal/server/websocket.go and
  message.go: unscoped session list, Create posts {"title":...,
  "location":{"directory":...}} to POST /api/session, additive protocol fields
  (omitempty) and the ChatDirectories request/response, plus the OperatorRegistry
  option defaulting to the first store. One OpenCode transport per server is
  unchanged.

Docs and references: docs/cli.md gains a "Chat sessions" section documenting the
operator-only fail-closed list, directory defaulting/rejection, chat_directories
(home first, deduped, bounded 8) and why a dedicated request was chosen.
internal/author/skill/references/operations.md's stale "Brokered OpenCode chat"
text (chat_create unbrokered, run_steps filtering, "listed the owned scratch
session" evidence) is rewritten to the operator-session registry, the fail-closed
filter, directory defaulting and the corrected live-test wording. No new
reference file, so internal/author/install.go's inventory is unchanged.

Checked for staleness elsewhere: README.md, the docs/ topic pages and the
packaged server/TUI templates carry no chat-session ownership claims, so none
needed updating.

Verified: gofmt -l clean; go vet and go build ./... clean; go test -count=1
./internal/pipeline/... ./internal/protocol/... ./internal/opencode/...
./internal/server/... and the full go test -count=1 ./... pass.
2026-10-17 11:14:25 +01:00