7fbbd953746ce48ae960f84065ea9103408cfc8a
66 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
cff6f4dfac |
server: write a subscribe snapshot in full before its ack
handleRunEvents seeded a new subscription by pushing every matching run through the subscription channel (runEventSubscriberBuffer = 32) and only then acked. sendMessage is a non-blocking send that closes the subscription when the channel is full, so a snapshot larger than the buffer dropped the subscription it was seeding: the remaining snapshot frames hit the closed-done early return, the ack was never sent, and no later delta could arrive. The connection stayed open, so the client could not tell it was no longer subscribed. This was the app's only subscription - AppViewModel subscribes with a blank workspace, matching every run - so the app's live state froze. The subscription now starts in a snapshot phase. The subscribing connection itself writes the snapshot frames via writeSnapshot (one direct write per frame, outside the bounded channel, so the drop policy cannot fire on them) and the ack, and ready then moves any deltas held meanwhile into the channel in order and starts the writer goroutine. sendMessage holds deltas in a bounded pending queue while snapshotting instead of closing, so the drop-a-slow-subscriber policy still applies, but only to deltas. Ordering is snapshot, ack, then deltas; a write failure or a listRuns error closes the subscription instead of leaving it registered. Numbers reported by this run: the probe against the deployed server saw blank-scope frames=2 ack=0 (scoped frames=1 ack=0) because the snapshot was ~252 frames into a 32-frame buffer; the scratch-server probe with the fix saw 40 snapshot frames, ack=1 and a delta after a run transition. Guarded now by TestRunEventSubscribeSnapshotLargerThanBuffer (seeds 40 finished runs, asserts every snapshot frame, the ack strictly after them, and a terminal delta for a run started after subscribing), which fails against the previous runevents.go. Checked for staleness: the packaged templates and the packaged authoring skill references internal/author/skill/references/ carry no run-event subscription contract (their "snapshot" mentions are the workspace-change snapshots in operations.md), and README carries none either, so none of them needs updating. No UI surface changed. Checks run before committing: gofmt -l . prints nothing, go build ./..., go vet ./..., go test ./... (all packages ok), go test -race ./internal/server (ok). |
||
|
|
798e958cb7 |
chat: fold an agent turn into one reply, split reasoning from answer, keep tool name/input
The phone's chat transcript rendered one agent reply as ~36 bubbles, each with its own thinking block, and a finished tool call lost its name and its input. - Turn fold (android): ChatFormat.kt gains the pure ChatTurn / TranscriptRow / transcriptRows fold - each run of consecutive assistant messages between two of the operator's messages is one agent turn - and ChatTranscript renders one bubble per turn: the turn's single collapsible thinking block (all its reasoning, in order, growing live while it streams), then its tool-call chips, then one answer block of just the turn's text parts. AssistantParts/MessagePart are replaced by AgentTurnItem/UserMessageItem; ChatConversationPane counts the folded rows. - Delta part ids (server + client): internal/protocol/chat.go adds ChatDeltaPartID(messageID, kind, ordinal), mirrored as ChatProtocol.deltaPartId in data/protocol/Chat.kt, and NormalizeChatEvent keys a text or reasoning delta on it. OpenCode numbers a message's text and reasoning parts independently from 0, so the old "%s#%d" gave the first text delta and first reasoning part the same id: the answer was appended into the thinking block, and the server's retained state (chat_hub.go) re-collided on resume. ChatStore's upsert now routes a delta through the kind-aware deltaPartId fold, so it stays robust even against a peer that still shares one id. - Tool fields (server + client): NormalizeChatEvent takes a call's arguments from session.tool.input.ended's text as well as session.tool.called's input object (toolInput), and chat_hub.go's new mergeChatToolPart folds each tool frame onto the retained part so the progress/success frames that omit name and input never blank them. ChatFormat.kt's mergeToolPart mirrors that on the client, so the chip and panel header keep the real tool name and the panel shows INPUT beside OUTPUT. Tests: chat_v2_test.go and chat_test.go add the text/reasoning id-collision and the tool name+input retention cases; ChatReasoningTurnTest, ChatTranscriptTest and ChatRunningWorkTest cover the turn fold, the collision and the tool lifecycle. Checks: Go build, vet, gofmt and the package tests pass (go build ./... && go vet ./...; gofmt -l . empty; go test ./internal/server ./internal/opencode ./internal/protocol ./internal/stage ./internal/pipeline; go test ./...). Android passes offline with --no-daemon (assembleDebug and the full testDebugUnitTest suite, 660 tests). The on-phone observation was not performed: the device lock was held by another run and adb was not available here. Documentation inventory checked: no packaged OpenCode skill reference in internal/author/skill/references/ and no packaged template documents the app's chat transcript, this delta part-id derivation or the tool-field retention, and no internal/author/ managed file inventory entry names any touched file, so none of them needs updating. The shared android/README.md behaviour statements are made stale by this change and are recorded instead in docs/notes/chat-transcript-turn-fold.md for the docs-fold writer, per the repo rule that an item never edits a shared document itself. |
||
|
|
0ba1204895 |
chat: attach to the operator's own OpenCode by pairing, fall back honestly
Brokered chat accepted an operator-supplied endpoint (-opencode-endpoint /
PIPELINES_OPENCODE_ENDPOINT) but failed closed against the operator's running
V2 service: that API authenticates every request with HTTP Basic auth (fixed
user "opencode" plus a password), the broker had no credential, so the health
probe answered 401 and startup logged "OpenCode endpoint ... is unreachable or
is not an OpenCode service", disabling chat entirely.
Established from the installed binary (opencode v2.0.20), not from docs:
`opencode --help` advertises `api`, `pair`, `serve`, `service` and `auth`;
`GET /api/info` is unauthenticated-401 with www-authenticate Basic realm
"Secure Area"; Basic auth user `opencode` with the password from
~/.local/state/opencode/service.json answers 200; `POST /api/pair` returns a
code and `GET /auth/connect/{code}` returns a token that is then accepted as
the Basic password.
- internal/opencode/pair.go (new): PairEndpoint does the pair exchange over the
shared nativeRequest transport (same auth, bounds and JSON checks);
ReadServiceCredential reads the operator's local service password under the
private-file rule; OpenEndpointAttached resolves the credential in precedence
order - explicit password file, a paired token stored by an earlier run, then
a fresh pairing bootstrapped by the local service credential - stores the
token owner-only and re-pairs once when a stored token is refused (401).
- internal/opencode/endpoint.go: OpenEndpoint wraps probeEndpoint, which returns
the connection even on failure; Connection.unauthorized (set on HTTP 401 by
the request paths, cleared per probe) distinguishes an authentication refusal
from any other refusal.
- cmd/pipelines-server/main.go: the endpoint attach uses OpenEndpointAttached
with the paired-credential file beside the pipeline database. An endpoint that
cannot be attached logs a line naming it and falls back to the server's own
OpenCode instance instead of leaving chat refused.
- internal/protocol/status.go, internal/server/websocket.go: get_status gains an
opencode object (health = endpoint | fallback | spawned | unavailable, plus
the endpoint url), the honest status evidence for the fallback. No credential
is ever placed there.
- Tests: internal/opencode/pair_test.go, internal/protocol/status_test.go and a
TestStatusReportsOpenCodeAttachment case in internal/server/status_test.go.
Repository checks: go build ./..., go vet ./... and gofmt -l . clean;
go test ./internal/... ./cmd/... green.
Packaged templates and skill references: the repository has no template tree
that mentions the OpenCode endpoint, and no packaged reference file is added, so
internal/author/install.go's managed inventory is unchanged. The packaged
authoring reference internal/author/skill/references/operations.md (the endpoint
paragraph around line 289, the "clear refusal and no fallback" wording and the
password-file/username-only credential story, and the missing opencode status
field) is now stale; per the maintenance convention the item does not edit the
shared document and instead records the stale points, with the exact wording
they should take, in docs/notes/chat-persistence.md for the docs-fold writer.
|
||
|
|
f85c172dc2 |
Merge main into chat-close: combine the shared chat room with the server-side close
The item adds chat_close/chat_reopen to the brokered chat protocol and the operator-session registry's closed state. main replaced the per-session claim model with shared chat rooms (chat_subscribe, a prompt queue, one broker stream fanned out to every watcher). This merges the two. Resolved conflicts: - internal/protocol/chat.go, internal/protocol/message.go: kept both additions - ChatClose/ChatReopen and main's ChatSubscribe/ChatQueued. - internal/server/chat_sessions.go: kept the closed fields on a listed session; dropped the `Busy` assignment, because main no longer marks a session Busy (any number of clients may watch and write one). - internal/server/chat_claims.go: main deleted the claim model, so the file goes; close's "is a turn running" check is now `chatHub.turnInFlight` in internal/server/chat_room.go, reading the room's `running` flag. That is the same question the old claim answered, against main's model. - internal/server/chat_close.go: calls turnInFlight instead of the removed `active`. - android AppViewModel.kt, ChatFormat.kt, ChatListPane.kt, Message.kt: kept both sides - main's search state, queued-prompt handling, `Busy`-free status label and row/menu shape, plus the item's close/reopen actions, closed chip and section, close confirmation and "Hide on this device" rename. Also in this merge: main's new convention (AGENTS.md) that an item does not edit a shared document; the item's stale-document record for this change lives in docs/notes/chat-close.md, and docs/cli.md, android/README.md and internal/author/skill/references/operations.md are left to the docs-fold writer. Checks after resolving: gofmt clean, go build ./... and go vet ./... clean, the Go module passes, and the app's :app:assembleDebug and :app:testDebugUnitTest pass (--offline --no-daemon). |
||
|
|
971e555ceb |
chat: close and reopen an operator session on the server, keeping its history
Add a server-side close for operator chats, so finishing with a chat takes it out of the active list for every client without deleting anything. The brokered protocol gains chat_close and chat_reopen; chat_close stops a running turn by reusing the existing broker interrupt (its result carries an interrupted flag, so a stream is never left running into nothing) and then sets closed_at on the session's operator_sessions row. The session, its transcript and its history survive, and because the closed state lives in the registry chat_sessions reports it to every client, not just the one that asked. An id that is not a recorded operator session, including a pipeline-owned run session, is refused before anything is written. chat_reopen clears the state again, and simply prompting a closed session also clears it (the store clears closed_at on touch). The registry gains closed_at for a fresh database and an idempotent migration for one created before it existed. The app renders closed sessions under a labelled Closed section with a Closed chip, offers a Close action with a confirmation that says it stops a running turn, marks the chat closed on every device and deletes nothing, and a Reopen action on each closed row. The device-local hide is renamed Hide on this device, with a snackbar saying the chat stays on the server, so it is never confused with the real close. No new dependency, the installer and Docker files are untouched, and no run is started from the app. Checked before committing: the packaged templates carry no chat surface and need no update, and internal/author/install.go's managed inventory needs none either (this adds no reference file). The packaged authoring skill reference internal/author/skill/references/operations.md and the shared docs/cli.md and android/README.md are stale because of this change, but an item no longer edits a shared document: what is stale and what should change there is recorded in docs/notes/chat-close.md for the docs-fold writer, which is committed here with the change. Checks: gofmt clean, go build ./... and go vet ./... clean, and go test ./internal/protocol/... ./internal/pipeline/... ./internal/server/... ./internal/author/... pass. The app side is verified with :app:assembleDebug and :app:testDebugUnitTest (--offline --no-daemon, BUILD SUCCESSFUL); the on-device close, second-client and reopen check is deferred to visual review. Not deployed; deploying is the operator's step. |
||
|
|
672d260b33 |
Merge main into multiclient: combine the multi-client chat fan-out with main's chat tool-work, notifications, status palette and docs-fold changes
No conflicts: main's chat-surface work (the in-flight tool work in ChatFormat, ChatStore and ChatTranscript, run notifications, the status palette) sits in different hunks from this branch's fan-out edits, and main did not touch docs/cli.md. Checks re-run on the merged tree: gofmt -l . (clean), go build ./..., go vet ./..., go test -count=1 ./... (all packages pass; the focused multi-client tests fan-out, per-session serialisation, first-answer-wins, pending replay and resync pass), and the Android unit tests (gradle --offline --no-daemon :app:testDebugUnitTest, BUILD SUCCESSFUL, 598 tests, 0 failures). |
||
|
|
6cb61578de |
Fan a chat session out to every client and share approvals exactly-once
A session was owned by one connection: the claim map in chat_claims.go let the
first client hold it, every other client saw the session Busy and read-only, and
a second client's prompt was rejected. Replace the claim with a per-session
"room" so the operator's phone, tablet and terminal all watch and write the same
conversation, and a pending approval is a shared, exactly-once object.
Server (internal/server, internal/protocol):
- New chat_room.go: one room per session holding the session's single broker
stream, its one bounded prompt queue (8) and the fan-out to every attached
client. A chat_subscribe frame attaches a connection to a session's shared
stream and replays the still-unresolved approval requests the room holds.
- chat_stream.go: handleChatPrompt accepts a prompt from any client. It attaches
the sender, either starts a turn or queues the prompt behind the in-flight turn
and acknowledges it with chat_queued:true ("queued" is the chosen behaviour,
stated on the wire and in docs/cli.md), and echoes the user turn to every other
client attributed to the sender's label. The prompt handler no longer owns the
stream; one turn runner per room drains the queue, so two turns never
interleave. Per-connection write serialisation is unchanged (each chatSub has
its own writer goroutine and bounded buffer).
- chat_permissions.go / chat_commands.go: an approval is reserved by the first
answer and confirmed once delivered; the first answer wins, is recorded with
the answering client's label, and the resolution is pushed to every watching
client as chat_permission with resolved/decision/resolved_by so their cards
clear. A later answer is refused with an error naming the client that answered,
and never reaches the agent twice; a reservation whose reply failed is
released.
- chat_hub.go / chat_sessions.go / chat.go: the hub now owns rooms and the
user-turn id sequence; the resulting user turns are retained (role and client
carried) so a reconnect re-syncs who said what. chat_sessions no longer marks a
session Busy and its clientID parameter is gone; the protocol keeps the Busy
field for wire compatibility only. websocket.go detaches a connection's chat
subscriptions when the read loop ends.
- chat_claims.go is removed, not left vestigial: the only exclusive thing left is
a turn, and the room's queue is that.
App (android):
- PipelinesRepository carries a stable per-device client label (Build.MODEL, with
a fallback) on prompts and replies, and adds chatSubscribe; AppViewModel
subscribes on opening a chat (best-effort, an older server's error frame does
not stop history loading) and records the queued ack.
- ChatStore folds a user frame from another client into the transcript once,
attributed by the sender's label, and never receives its own prompt back, so
the sender's own message is never duplicated. ChatThread.readOnly is gone:
canSend no longer depends on another client, the composer says "Queued - it
will send when the current turn finishes." instead of disabling input, and the
status label is Running/Idle only. Scroll/follow behaviour is untouched.
- Tests: ChatProtocolTest round-trips the new wire fields, ChatFormatTest asserts
the stale Busy flag cannot produce a read-only label, MultiChatStateTest asserts
a session another client watches stays writable.
Reflected where it belongs: no packaged template changes (no bundled starter
embeds the chat protocol). Checked the packaged authoring skill references: none
of them describes the chat wire beyond operations.md, and
internal/author/skill/references/operations.md is now stale - its "Brokered
OpenCode chat" section still says chat_sessions marks "busy" when another client
holds the session and that a second client's chat_prompt is rejected, and it does
not mention chat_subscribe, the per-session prompt queue/chat_queued, the
per-client label or the shared exactly-once approval (it also still describes the
per-subscription permission poller, which the room's single poller replaced). The
supported semantics are the ones in docs/cli.md, updated in this commit; the
operations.md section needs a rewrite to match. No README or other shared
document was edited here.
Checks run on the committed tree: gofmt -l internal cmd examples (nothing), go
build ./..., go vet ./internal/... ./cmd/..., go test ./internal/server
./internal/protocol ./internal/stage ./internal/pipeline.
|
||
|
|
a3f571d672 |
docs: make docs-fold the single writer of shared documentation
Every item wrote the shared, append-only docs (android/README.md, docs/cli.md and the packaged skill references), so every merge conflicted on prose. Keep the duty to notice stale docs; move the writing to one deliberate place. - Per-item note, not a shared edit. An item that makes a shared document stale writes docs/notes/<slug>.md - unique per item, so two items never touch the same file - and commits that note with its change. It never edits a README, topic doc or packaged skill reference itself. - One writer. The manual-only docs-fold pipeline, bundled as the fifth starter examples/docs-fold.md, folds every note under docs/notes/ into the shared document it names and deletes the consumed notes in its own commit; an empty notes directory is a success no-op. Operator trigger: pipelines new "Docs fold" --template docs-fold --no-open; pipelines submit <draft.md>; pipelines run "Docs fold". - Implemented where the convention lives: orchestration.md (brief item 5, the hard-constraint bullet, a "Single documentation writer" section and the "Pipeline choice" committer bullet), the canonical committer checklist in format.md (new step 6), the committer prompts in examples/task.md and examples/three-stage.md, the new examples/docs-fold.md with its catalog and README entries, and the repository guideline AGENTS.md. Test-expectation counts (examples/catalog_test.go, internal/server/templates_test.go, scripts/smoke-first-install.py) follow the fifth starter. Reflected where it belongs: the packaged templates (examples/task.md, examples/three-stage.md, the new examples/docs-fold.md), the packaged skill references (orchestration.md, format.md) and the repo guideline AGENTS.md all carry the new convention. No new reference file was added, so install.go's managed inventory is unchanged. The shared descriptive docs were deliberately not rewritten: this change makes README.md:191, docs/cli.md:170 and internal/author/skill/references/operations.md:13 name four starters where there are now five. Those stay as they are - correcting them now is exactly the per-item shared edit this change removes - and the stale-doc note docs/notes/docs-fold-single-writer.md records each one for the docs-fold writer to fold. Nothing else needed updating. No new dependency; no Android, Gradle, server, installer, Docker or published-definition change; no definition published over the running pipelines. Evidence: go test -count=1 ./... passed; go build ./... and go vet on the touched packages clean; ./internal/author/... covers embedding, reference links, the managed inventory and install/upgrade/protection; ./examples/... validates the fifth starter and its manual-only, committer-terminated graph; ./internal/server/... covers the served catalog. Not run: the docs-fold writer against the shared docs (nothing to fold yet); no install or deploy. (cherry picked from commit f29656cf084d7eee2e033f2a37a9d231d7acaaf0) |
||
|
|
a4cf7366dd | docs: per-area agent maps for the Android app, the TUI and the server | ||
|
|
c1652e2bc6 |
Split chat/model/run files by concern (pure move)
Pure move: split the three multi-feature server files into per-concern files, with no behaviour change - same package, same exported and unexported identifiers, same call graph, same tests. - internal/server/chat.go -> chat.go (broker seam, client token, dispatcher), chat_hub.go, chat_claims.go, chat_permissions.go, chat_sessions.go, chat_stream.go, chat_commands.go - internal/stage/model.go -> model.go, model_run.go, model_failure.go, model_validation.go - internal/pipeline/runs.go -> runs.go (read side), runs_schema.go, runs_create.go, runs_mutate.go, runs_sessions.go Checked before committing: - Packaged templates and internal/author/skill/references: neither refers to internal/server/chat.go, internal/stage/model.go or internal/pipeline/runs.go, so nothing there needed updating (the change is a move with no behaviour change). - gofmt -l on the three packages: clean. go build ./..., go vet ./... and go test ./... pass. go.mod/go.sum unchanged. - No test, docs, android/, install or Docker files touched. Known stale path pointers deliberately left to the single docs writer (shared docs) and out of scope (android/): docs/chat-verification.md and android/README.md name the old file paths. |
||
|
|
7a28aca1f1 |
Merge main into lane-retry: per-run usage recording alongside live run events
main brought pushed run events (emitRun/RunEventFrame across protocol, server and the Android client) and this branch brought per-run usage recording. Both touch internal/server/runner.go; git auto-merged cleanly and the result keeps both intents — each model step is stamped with its effective provider/model and budget-derived usage (stageUsage, overlayEffectiveModels) while run events are emitted (emitRun) and applied to the same step loop. No conflict markers, no side's work discarded. Post-merge checks: gofmt -l clean, go build ./..., go vet ./... and go test ./... all pass. |
||
|
|
3f5c11b7bf |
Record and expose per-run model usage (tokens, split, cost)
Carry the usage a model stage already measures for its budget (stage.Budget.Snapshot()) onto the step as typed evidence and total it in one pass, rather than a second harness read, a transcript parse or an output-length estimate. A shared budget across stage revisits records only the visit's increment, and a revisit that observes no new report records no usage instead of a fabricated known zero. - protocol: StageUsage/RunUsage/TotalUsage and UsageTotals. - pipeline: step_usage table + migration, COALESCEd into the one step projection, FinishStep persistence, one json_extract history aggregate, retention removes the rows with their run. - server: stageUsage maps the budget snapshot, stamping per-stage model and usage for every model stage (including failures). - cli: inspect/stats human and JSON output. - budget/pi: retain the reported in/out split, mark a total-only (Pi) report split-unknown, and keep an unreported cost unknown (never zero). Evidence and docs updated in the packaged skill references and docs/cli.md; the live database has no step_usage rows yet (all history predates this), so per-run figures are reported as unavailable, not zero. |
||
|
|
8a62b31398 |
Merge main into lane-live: adopt operator chat brokering alongside live run events
Combines this lane's "make every view live from the pushed run events" change (android + server run-event scope and frame identity) with main's operator chat brokering work (opencode endpoint adoption, protocol/settings/CLI chat support, operations reference). The two touch different files; git's ort strategy merged them cleanly with no conflict markers and nothing to resolve. Post-merge checks: go build/vet and go test ./... all pass; gradle --no-daemon testDebugUnitTest assembleDebug passes (558 tests, 0 failures). |
||
|
|
5e2aece1be |
android+server: make every view live from the pushed run events
The pipelines list (and every other run-state view) kept the row it had when the screen was last fetched. Two class-level root causes, fixed here: - The app subscribed to run events scoped to the one workspace the operator configured, so a run executing in another workspace (a lane worktree is its own workspace) never delivered a frame: the rows only changed on a manual refresh. The app now asks for the blank "any workspace" scope the server already treats as any (ALL_WORKSPACES in AppViewModel.subscribeRuns, blank default on PipelinesRepository.subscribeRuns). Workspace isolation on the server is unchanged: an explicit workspace scope still matches only that workspace (TestRunEventWorkspaceIsolation); the new TestRunEventAllWorkspacesSubscription pins the blank "any" scope. - foldRunEvent refused to insert a run the list had never held, returning needsResync and leaving the list untouched, so a run started after the screen loaded appeared only after a fetch. The pushed frame now carries the pipeline identity a row needs (RunEventFrame.PipelineID/Name/Title/Version), so the fold inserts the new run at the head from the frame's own identity without a fetch and without reordering existing rows; a server that predates those fields still falls back to the one resync. Added fields and why each is needed: pipeline_id (row's pipeline join), name (row title), title (the stored operator-facing title a row shows), version (the row's pinned version) - everything a RunSummary row renders, so no per-event fetch and no guess from stale cache. The server stamps them on every emitted frame from a per-run identity remembered at create/decide/cancel/resume (runner.go runIdentity/rememberRunIdentity/stampRunIdentity), first-write-wins through the coalescer (mergeRunEventFrame) and on the subscribe snapshot, so every transition - created/queued, running, each stage start/finish, terminal - carries them. Reconnect still resyncs once then follows pushes; the pull-to-refresh path is untouched. Checks run: go build ./..., go vet ./..., go test ./... (all packages ok), go test -race -run 'RunEvent|RunEvents|Subscribe' ./internal/protocol/ ./internal/server/, and gradle --no-daemon testDebugUnitTest assembleDebug (558 tests, 0 failures, 0 skipped; BUILD SUCCESSFUL). Packaged templates and the packaged authoring skill references were checked: nothing there describes the run_event field list or the client's subscription scope, so neither needs updating (references/notifications.md only says the attention surface is derived from the pushed run events, which is unchanged). Docs are a shared file owned by a single writer and were not edited: docs/cli.md's description of the pushed `run_event` frame still enumerates only the older fields (status, outcome, current stage, completed-step count, timestamps, optional step) and is now incomplete - it should mention the pipeline identity fields; docs/ntfy.md remains accurate. |
||
|
|
1f75e4797e |
Broker chat from an operator's own OpenCode endpoint and adopt existing sessions
Add an opt-in way to broker chat from an operator-supplied OpenCode service instead of the instance the server spawns for itself, and a way to adopt a session that already exists on that endpoint so it appears in chat_sessions and can be continued from the app. - Endpoint selection: the `-opencode-endpoint` server flag (defaulting to `PIPELINES_OPENCODE_ENDPOINT`) opts in; precedence is explicit flag > explicit environment > the endpoint recorded on a previous run, with the spawned instance as the default. opencode.OpenEndpoint validates the reference once (http(s), host, no userinfo/query/fragment/path), detects the backend from its own /api/info (v2) or /global/health (v1) contract, and requires a streamable text/event-stream event path. It owns no process (Close is a no-op) and never calls startServe; every failure names the endpoint and disables chat rather than silently falling back. An authenticated endpoint takes an optional password only through the private file PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE, with PIPELINES_OPENCODE_ENDPOINT_USERNAME overriding the detected default. - Persistence: the validated choice is recorded in an additive single-row server_settings table so a restart with no flag/environment reuses it; an unreachable recorded endpoint disables chat with a clear message and no fallback. - chat_adopt: a new protocol request (ChatAdoptRequest) with handleChatAdopt in internal/server/chat.go. It refuses a nil broker/registry/payload and a blank or foreign-shaped id before any endpoint call, reads the endpoint via the shared Chat.Session (which reuses Sessions/decodeChatSessions), records through the existing RecordOperatorSession with the endpoint's directory and title verbatim, and echoes the adopted session plus its registry row like chat_create. A missing id is refused naming the id and records nothing; an endpoint record with an empty directory is refused rather than substituting the server's working directory or home. - CLI: `pipelines chat-adopt <session-id> [--json]` (usage, notes, help inventory, dispatch) with docs/cli.md updated under Chat sessions. - Autostart preserves the endpoint variables (and the password-file reference, never an inline secret) via serviceEnvironmentKeys. Checks before committing: - Package location: the change is user-facing chat behaviour, so the packaged authoring skill reference internal/author/skill/references/operations.md is updated in the same change (Brokered OpenCode chat section) with a dated verification-evidence entry and the not-deployed state. No new reference file is added, so internal/author/install.go's managed inventory is unchanged. - Packaged templates: this repository packages no template set for this behaviour and none references the chat endpoint, so no template needs updating. - Verification: `go build ./...`, `go vet` on the touched packages, `gofmt -l` clean and `go test ./...` all pass. The unit tests cover the endpoint validation/precedence, the adopt happy path and refusals, the recorded endpoint round-trip, the protocol round-trip and the CLI dispatch. The live scratch-server check against the operator's real endpoint was not run in this change. Nothing unrelated is staged; docs left stale (README and other shared documents) are not edited here and are for a single folding writer. |
||
|
|
58c6b9b7c6 |
server: export a finished stage's real session as session.md and session.html
Every OpenCode-harness model stage now writes its real session into its own step artifacts directory as session.md and session.html, so the existing bounded artifact listing, the /artifacts/<run>/<step>/<name> serving path and the artifact scan find and serve them with no special casing, while the stage keeps its bounded preview in the run record unchanged. - internal/opencode/export.go adds the one shared native export invocation and the exact session-ID/messages validation used by both Connection.Messages and the new ExportSession, so the readback path does not diverge. The backend's published subcommand is used (v1 `export <id>`, v2 `session export <id>`), the source read is capped at 8 MiB, and a larger session is read as a prefix and reported truncated instead of failing. boundedOutput.ReadFrom keeps the cap effective so overflow and the truncation path are real. - internal/stage/session_export.go renders markdown and html with a metadata header, every role and every part (unknown part types as raw JSON, every session text escaped in html), atomic temp+rename writes that never overwrite an existing file, a visible truncation banner at both ends when the source or the 2 MiB rendered cap cuts it, and an honest 0-message header for an empty session rather than fabricated text. - protocol.SessionExport and RunStep.SessionExport are stored in a new step_session_exports table (CREATE TABLE IF NOT EXISTS, so an old database gains an empty table with no migration) loaded through the existing main SELECT alongside the other per-step evidence and deleted by run retention. - internal/server/runner.go exports after the post-stage filediff capture and before scanArtifacts, with context.WithTimeout(context.WithoutCancel(ctx), 10s); it never waits for session idleness and a failure or a Pi stage (no native export path) records a sanitized diagnostic and continues, never changing the stage status, outcome, target, timing or result validation. Reflected where it belongs: the packaged skill references (internal/author/skill/references/operations.md and delivery.md) carry the new Session export rules and the run-artifacts bullet, and docs/stage-artifacts.md documents the names, the <run>/<step> paths, the served URL, the caps, the truncation labels, failure isolation and the Pi limitation. The packaged templates (examples/*.md) only tell a stage to save its own artifacts and say nothing about the server-written session export, so none needed updating; no new reference file was added, so install.go's managed inventory is unchanged. Shared documentation: README.md's per-step artifacts sentence is now stale - it should name the exported session alongside images, screenshots, reports and logs. It is deliberately not committed here; the shared documents are folded by the single documentation writer. Checks: gofmt -l on the touched packages produced no output; go vet ./... passed; go test -race -count=1 ./... passed. |
||
|
|
476ecc781c |
Merge main into lane-srv3: combine the review-loop control change with main's notification and chat work
Keeps this branch's advisory review edge, bounded blocked_retryable implementer edge and their tests alongside main's run-attention/ntfy notification work, the split of the Android chat conversation screen, and the retention/history changes. No conflicts: the two sides touch disjoint files. |
||
|
|
e70c94e8dd |
control: make advisory review findings non-blocking and bound retryable blocks
Cut the review loop's repeat cost and make a blocked stage actionable, at the
definition level only, with the runner's existing visit counting and caps.
The published maintenance, quick and focused pipelines gain a reviewer
`advisory -> next stage` edge for non-blocking notes, so an advisory-only review
continues the run instead of buying another implementer visit; `rejected ->
implementer` and its unchanged {"max_visits":3,"on":["rejected"]} limit keep a
must-fix finding looping and still terminate. The implementer gains
blocked_retryable -> implementer with {"max_visits":5,"on":["blocked_retryable"]}
for a transient environment/tooling/device problem a fresh attempt could
survive, while blocked -> #failure stays terminal. The cap is at least the
reviewer's rejection budget so the implementer's own pre-execution cap cannot
preempt the reviewer's rejected control.
No production Go code changed. New tests in internal/server/review_contract_test.go
cover advisory-only never revisiting the implementer, must-fix still looping to
the reviewer limit, the implementer cap not preempting the reviewer, retryable
blocks bounded then failing closed, a retry that recovers, and a terminal block
failing immediately; concurrency/control/human tests share the new submitRunner
and startRun seams.
Documentation checked: internal/author/skill/references/control.md and
docs/harnesses-and-control.md carry the new edges and the visit-cap rule;
no packaged template defines these pipelines (they are published immutable
versions), and docs/model-results.md needs no change because the {outcome,output}
result contract is unchanged. README and other shared docs deliberately untouched.
Verified: go build ./... and go test -count=1 ./... pass.
Residual limitation: the must-fix-vs-advisory split is carried by the prompt
contract, not a machine-readable marker, because the recorded findings have none;
a model that still returns rejected for advisory-only notes would still buy a
visit. All 18 recorded implementer blocked runs classify terminal, so the
blocked_retryable edge recovers 0 of them.
|
||
|
|
890d267c71 |
run attention: publish to ntfy and surface what needs the operator
Nothing told the operator that a run was waiting for a person or had failed; they found out by going looking. A run that needs attention now reaches the ntfy they already run, and the app answers "what needs me now" in one live, ordered surface. Server (smallest change, reusing the existing publisher): - internal/server/notify.go adds notifyRun, called from finishRunState (failed) and when a run parks on a human (runner.go runLoop). It publishes to the same already-configured `system` destination through the unchanged stage.PublishNtfy, so there is no second publisher, topic or service and the token handling is the stage's own. runAttentionEvent maps only waiting and failed; queued, running, succeeded, canceled and interrupted publish nothing. - No storms: runner.attentionSent records sequence:event per run before publishing, so a run that stays waiting is never republished on a re-request or restart, and a stage notify policy that already covers the same event marks it first so the run-level default is suppressed. The set is dropped when the run is terminal. A delivery failure is logged, never fails the run, and is not retried. - The payload names the run and pipeline and carries a click action pipelines://run/<id> (validated: absolute URL, no userinfo, no fragment, no credentials). PublishNtfy gained the optional click parameter; existing callers pass "". App: - AndroidManifest/MainActivity declare the pipelines://run/<id> deep link (singleTop); runIdFromDeepLink parses it and PipelinesApp opens that run once, clearing the link. It opens a run; it starts nothing. - ui/Attention.kt is the pure model (status -> Decision/Failed/Finished or nothing, ordered by kind then newest-first) and ui/components/AttentionSurface.kt is the floating indicator plus bottom sheet. It is derived from the same run events the server already pushes, so an entry appears with no refresh and clears when resolved; a finished run clears once looked at, and any entry can be dismissed, persisted in SettingsRepository. The existing destinations, pill and bottom strip are untouched and no competing navigation surface is added. Checked before committing: - Packaged authoring skill references: internal/author/skill/references/ notifications.md documents the automatic run-attention notifications and the app surface. The only packaged "template" carrying notify is that reference; no pipeline definition template exists to update and install.go's managed file list is unchanged (notifications.md already existed). - Repository guidelines: docs/ntfy.md gains an "Automatic run-attention notifications" section and no longer claims "No block means no notifications"; the top-level README phone-notifications note now says a waiting/failed run notifies automatically; android/README.md gains the dated feature section and supersedes the previous section's note that the attention surface was absent. - docker.md, install docs and the CLI/TUI docs describe mechanisms this change does not alter; left as they are. Tests: go test ./internal/server/... ./internal/stage/... ./internal/author/... all ok; notify_test.go covers the event mapping and the real HTTP delivery (destination, topic, click, no duplicate). Android :app:testDebugUnitTest -> BUILD SUCCESSFUL, 58 suites / 531 tests / 0 failures (new AttentionTest, 13 cases, covering ordering, appearing from a push, clearing on resolution, dismissal, the empty case, persistence and the deep link). Device screenshots and the phone's own notification are visual review's per the brief's device policy. Scope: android/ plus the smallest server change; no new dependency, no run started from the app, no installer or Docker change. |
||
|
|
14175eba70 |
retention: explicit run-history and artifact retention
Add `pipelines retention report` (strictly read-only) and `pipelines retention apply` (explicit, approval-gated removal) with an age and/or count policy, protected-run rules, atomic single-transaction removal and honest database space reclaim. Nothing prunes automatically. |
||
|
|
7b72b8f20f |
stage: retry pre-admission harness reach failures and widen repair deadline
Treat a failure to reach the harness before any prompt or tool call is admitted (connecting or creating the session) as a distinct retryable reach failure, so a transport blip at the first stage no longer ends a mutating run; keep post-admission readback/poll/agent failures retried only on an explicit read-only stage and never replay admitted tool work. Preserve the true cause when the context is already done (join ctx.Err with the typed transport error) and report a sanitized unknown provider error name/status instead of a blanket summary. Make the isolated format-repair deadline proportional to the stage timeout (floored, capped and clamped to remaining minus a margin) so a slow-but-repairable result is recovered without ever extending the stage deadline; the bounded ladder, value guard, deny-all formatter session, cumulative usage accounting and no-tool-replay rule are unchanged. Update docs/model-results.md, docs/pipeline-performance-review.md and the packaged skill references; add stage, opencode and server fixtures. |
||
|
|
18398dfa5b |
chat: give the operator their own sessions, separate from run sessions
chat_sessions used to list exactly the OpenCode sessions the pipelines own,
filtered through pipeline.SessionOwner (run_steps/run_session_owners), so an
operator-created chat was an accident of the run-ownership table and the
operator's working sessions were mixed with the internals of running pipelines.
Add a first-class operator session:
- internal/pipeline/operator_sessions.go: a dedicated operator_sessions table
(session_id PK, directory, title, tag, created_at, last_used_at) with
RecordOperatorSession (upsert preserving created_at), a one-statement
TouchOperatorSession (no-op for a non-operator id), OperatorSessions bounded to
128 rows most-recently-used first, and OperatorDirectories (bounded MRU,
deduped). run_steps/run_session_owners and SessionOwner/OwnedSessions are
untouched.
- internal/server/chat.go: chat_sessions returns operator sessions only - it
reads the registry once, intersects it with one broker session-list call, fills
directory/tag/created/updated from the registry, sets busy for a session held
by another connection, keeps MRU order, and stays fail-closed (empty list, never
the endpoint's whole list). A pipeline stage session and an untagged row can
never be listed. chat_create resolves the directory (empty/'~'/'~/...' ->
operator home; relative or missing rejected before any OpenCode call), calls
broker.Create(title, resolved) and records the row with the operator tag.
chat_directories returns home first then bounded (8) deduped recents. A
successful prompt touches the operator session without affecting pipeline
prompts.
- internal/opencode, internal/protocol, internal/server/websocket.go and
message.go: unscoped session list, Create posts {"title":...,
"location":{"directory":...}} to POST /api/session, additive protocol fields
(omitempty) and the ChatDirectories request/response, plus the OperatorRegistry
option defaulting to the first store. One OpenCode transport per server is
unchanged.
Docs and references: docs/cli.md gains a "Chat sessions" section documenting the
operator-only fail-closed list, directory defaulting/rejection, chat_directories
(home first, deduped, bounded 8) and why a dedicated request was chosen.
internal/author/skill/references/operations.md's stale "Brokered OpenCode chat"
text (chat_create unbrokered, run_steps filtering, "listed the owned scratch
session" evidence) is rewritten to the operator-session registry, the fail-closed
filter, directory defaulting and the corrected live-test wording. No new
reference file, so internal/author/install.go's inventory is unchanged.
Checked for staleness elsewhere: README.md, the docs/ topic pages and the
packaged server/TUI templates carry no chat-session ownership claims, so none
needed updating.
Verified: gofmt -l clean; go vet and go build ./... clean; go test -count=1
./internal/pipeline/... ./internal/protocol/... ./internal/opencode/...
./internal/server/... and the full go test -count=1 ./... pass.
|
||
|
|
8692561bc7 | chat: say 'not implemented yet' instead of 'not yet brokered' | ||
|
|
4c088c9638 | Merge lane-chat: keep the lane's reviewed work | ||
|
|
36fad7d130 |
Merge lane-go: keep the lane's reviewed work
Combines lane-go's typed omni search, launch-time model overrides, TUI artifacts view, run-state websocket push, and the "reflect the change where it belongs" committer rule with the Templates gallery, chat/omni-search and other reviewed work already on main. The 12 files both sides touched keep both intents composed rather than dropping either side: the omni search keeps the typed search contract (CLI `pipelines search` and the TUI popup with type badges and marked snippets) alongside the run-list quick filter; cli.md and ui-ux-review.md document both the search rework and the artifacts view; orchestration.md, harnesses.md and operations.md keep the model-override and reflect-where-it-belongs guidance; and the protocol/CLI/server/TUI code keeps the new search, run-event, override and artifacts paths alongside the existing run, title and help surfaces. Checks before committing: - go build ./... -- pass - go vet ./... -- pass - go test -count=1 ./internal/... ./cmd/... -- all pass Doc check before committing: nothing needs updating. The lane's own docs and references (docs/cli.md, docs/ui-ux-review.md, and the orchestration, harnesses and operations skill references) already carry the new features. format.md and the packaged templates already describe the line-oriented result format and the human decision stage from the earlier lane-go merge. The root README already describes the omni search popup. Model overrides, the search command, run events and the artifacts view are runtime/CLI/TUI surfaces that cli.md documents; no template, README or topic-doc edit is required by this merge. |
||
|
|
30c1ed442e |
chat: broker OpenCode v2 chat over the owned serve SSE endpoint
The installed OpenCode 2.0.20 exposes the authenticated SSE event firehose at
/api/event and the /api HTTP surface, but Detect selects backend "v2" and Open
returned no owned process/URL, so chat stayed disabled and every chat_* request
was answered with "chat is not configured on this server".
Add opencode.OpenStream/OpenChat: for either backend they start the single owned
opencode serve process, discover the loopback URL and verify the backend health
contract (/api/info for v2, /global/health for v1). main and the live test share
OpenChat so the selection cannot drift. Chat now streams real incremental
session.text.delta/session.reasoning.delta frames via the existing
Connection.EventStream + NormalizeChatEvent, with no polling and no fake
streaming.
- Authenticate the owned endpoint with the per-backend user (opencode for v2,
pipelines for v1); the v2 server ignores OPENCODE_SERVER_USERNAME.
- Send the V2 prompt shape {"text":...}, which the v1 translation also accepts.
- Skip firehose event types the adapter does not surface before decoding their
data, so session.instructions.updated (delta is an object) no longer fails the
stream.
Unit tests cover the per-backend user, the prompt shape/auth, the captured real
V2 event shapes and the selection guards. An opt-in live test drives
chat_sessions/chat_messages/a completed chat_prompt/an interrupted chat_prompt
against the installed OpenCode. Update the packaged operations reference with the
realized capability, dated evidence and deployment state.
Checks before commit: the packaged authoring reference
internal/author/skill/references/operations.md is updated in this change; no new
reference file was added, so internal/author/install.go's managed inventory is
unchanged and the internal/author tests pass. No packaged template describes the
brokered chat surface, and neither README.md nor the topic docs under docs/ (nor
web/) document it, so none of them needed updating.
|
||
|
|
c9b904186a |
Update templates and authoring skill to the current pipeline contract
Bring the packaged starters and the pipelines skill up to the guidelines the project now works to. Templates: - task and three-stage now end in a committer stage that stages the union of the runner's captured input.server_changed_files and the producer's declared changed_files, runs the repository's checks, commits locally and never pushes; their producers no longer commit. - Every model/agent prompt states the line-oriented result contract (outcome on the first non-empty line, "# output" JSON map, other "# <heading>" sections kept verbatim as evidence) and an "# evidence" section for exact commands and results, with proportionate verification and no fabricated checks. - Stages that can write save their reports, logs and screenshots into the runtime artifacts directory. - Remove the summary starter: a tool-free single-prompt text transform with no repository interaction, artifact or commit is a format demonstration, not a reusable workflow. No starter invents a pretend approval stage; a decision point is a type=human stage. Skill: - references/format.md gains a Result format section (line form, legacy JSON precedence, result_parse, raw-output.md, artifacts) and a Committer stages section with the pre-commit checklist; the role-split and envelope notes mention the line form and the committer. - orchestration.md points at the checklist and records that a genuine gate is a human stage; operations.md and the SKILL.md index drop the removed starter and name the result format and committer. No new reference, so the managed inventory is unchanged. Docs/tests: examples/README, README, docs/cli, docs/first-version-testing, the CLI help example and the smoke script track the four starters; catalog, tool-role, server, tui and pipeline tests are updated, with a new test pinning the committer ending and server-captured change set. Commands run: go build ./...; go vet ./...; go test ./...; python3 -m unittest discover -s scripts -p 'test_*.py'; python3 scripts/smoke-first-install.py. All passed. Templates inspected through an isolated CLI server (pipelines templates; pipelines template KEY --source). No server, service or device was restarted, and nothing was pushed. |
||
|
|
8f3036bd3b |
Push run state to subscribed websocket clients
Add a server-pushed run_event frame so a client no longer has to poll once a second to notice run changes. - protocol: RunEventFrame/RunStepDelta compact delta (run id, workspace, status, outcome, current stage, completed-step count, timestamps and an optional single step delta), plus subscribe/unsubscribe/run_event types and the optional RunEvent/RunIDs message fields. All fields optional so older peers ignore them. - server: runEventHub with per-connection subscriptions scoped to a workspace (and optional explicit run ids), one bounded writer goroutine per subscription, slow-subscriber drop, per-client subscription cap and a ~220ms per-run coalescer that flushes pending deltas before a terminal frame. - runner: emit deltas on created/queued, queued->running and step start, human waiting, step finish (outcome, target, diagnostic, artifacts and completed count), terminal finish, cancel of a waiting run and decide/resume; emitRun is a no-op without a hub. - websocket: subscribe seeds current state before the ack, unsubscribe and connection teardown drop subscriptions, and pushed frames reuse the connection write closure. - tests: protocol golden-JSON wire shape plus websocket coverage for the started-run frame sequence, subscribe current-state snapshot, silent unsubscribed/never-subscribed clients, workspace isolation and a bounded coalescing frame count. - docs: note subscribe/unsubscribe/run_event in docs/cli.md; TUI and Android still poll in this release. |
||
|
|
865993b200 |
Rework omni search onto a typed search contract and add a TUI artifacts view
Protocol: replace the loose Query/Kind/Matches/Limit search fields with a Search request object and a SearchResults response, and have the server answer search from Store.Search over the shared run projections (runFrom/runStamps).
CLI: add the read-only 'pipelines search QUERY [--type KIND] [--limit N] [--json]' command with usage/help, and document the typed targets, ranking and bounds in docs/cli.md.
TUI: the '/' popup keeps type-to-search, up/down, Enter and Esc, renders one ranked list with kind badges and a bracketed (non-colour) match snippet, and Enter opens the matched pipeline, run, step, artifact or field; footer text updated to match.
TUI: add a dedicated run artifacts view ('A') with open/copy of recorded files, and move artifacts out of the run-detail File changes section (artifactCaptureNotes).
Tests: add Store.OpenReadOnly for read-only inspection of a live database plus pipeline/server/TUI search tests and artifacts tests, including env-gated live checks.
|
||
|
|
98713e55f5 |
server: hand the committer the captured changed-path set
The committer staged only the implementer's self-declared changed_files, so an incomplete declaration silently left part of the approved change uncommitted. The runner already records the run's real changed paths by comparing the working tree around each step; publish that set to the committer. - runState gains an unbounded, deduplicated, first-seen-ordered captured path accumulator (capturedPaths/capturedSeen), distinct from the bounded changedPaths sample used only by the UI-surface skip reason. - recordChanges folds each captured FileChange path into it in the existing single pass, so no second walk and no new I/O. - publishServerChangedFiles writes the set into the input memory value as input.server_changed_files, called before memoryInput so the binding always resolves (an empty list when nothing was captured) and only stages that select it receive it. rebuildRunState rebuilds it from persisted completed steps, so a resumed run hands the committer the same evidence. - Tests prove the committer receives the captured set distinct from and additive to the implementer's declared changed_files, that an omitted path is still present, and that the field is an empty list when nothing was captured. - control.md and orchestration.md document the reserved field and instruct the committer to select input.server_changed_files alongside the declared changed_files and stage their union. Commands run: go build ./...; go vet ./...; go test -count=1 ./internal/server/... ./internal/stage/...; go test -count=1 ./internal/author/...; go test -count=1 -run TestCommitterReceivesServerCapturedChangedPaths ./internal/server/. All passed. go.mod/go.sum unchanged. |
||
|
|
fae1005962 |
Allow launching runs with whole-run and per-stage model overrides
Add `pipelines run --model PROVIDER/MODEL` and repeatable `--stage-model STAGE=PROVIDER/MODEL`, resolved with precedence per-stage > whole-run > the model pinned in the pipeline version. - A shared pure resolver (`pipeline.ParseModelOverride` / `pipeline.EffectiveModels`) owns the precedence and validation: every value must be a `provider/model` pair (`stage.ValidModel`), every stage key must exist in the pinned graph, an unknown key fails naming the valid keys, and a per-stage key that names a non-model stage is refused. - The server validates against the pinned version before admission (no run row is ever inserted for an invalid request), persists `model`/`stage_models` on the run and `provider`/`model` on each step, applies the effective pair to a clone of the stage definition so the stage and its repair/formatter path share it, overlays the returned run graph with the effective pair (presentation only), and reuses the recorded overrides on retry. - The run record echoes the overrides it was launched with and each step records the provider/model it actually ran with; the TUI Evidence pane renders that effective pair. The pinned pipeline version and `show --version N` are never modified. - CLI help, `docs/cli.md` and the packaged authoring skill references document the flags and the precedence. A TUI override picker remains a documented follow-up; the CLI is the deliverable. Tests: pure precedence/validation table tests, store round-trip and request-key-mismatch plus migration, server admission/application/overlay/retry with a controlled executor, CLI flag/help/malformed-value tests, and a TUI evidence test. |
||
|
|
b3188a5c6f |
server: skip a stage that declares a skipped edge when no UI surface changed
A stage declaring a `skipped` outcome is short-circuited with no model call when the run's recorded file changes and artifacts touch no UI surface (android/, internal/tui/, a presentation extension or an image artifact). The step is recorded completed/skipped along its declared edge with a bounded one-line reason, so a backend-only run no longer pays for a ui_reviewer generation that can only answer ui_applicable=false. |
||
|
|
bd09c55c47 |
Merge lane-go: titles, verdicts, human stage, line result format, stale-client guard
Integrates nine commits of reviewed work from the lane-go branch after the automated resolver failed on this merge five times (each attempt dying on the malformed-result defect whose fix is contained in this same merge). Integration decisions: - Search: kept main's shipped design (Store.Search + SearchResponse.Results) and dropped lane-go's parallel, older attempt (an unused Message.Matches field, an absent runs.search, and two partial TUI plumbing blocks). lane-go itself did not compile: protocol.SearchMatch was undefined and model.go referenced searchNav, requestSearch and applyRunNav, which exist nowhere. Those partial TUI blocks are removed; the queued TUI omni-search item will implement the surface properly on main's design. - Both sides' additions are kept where they are independent: run titles and provenance, the human decision stage and its CLI/TUI surface, the derived stage verdict, the stale-client guard, the line-oriented result format, and main's search command. - The search key's help label is now "/ omni search"; the two tests asserting the old label are updated rather than the label reverted. - docs/cli.md and the packaged operations reference carry both sides' text. Verified: go build ./... , go vet ./... and the full go test ./... suite pass. |
||
|
|
b05277a57d |
Add line-oriented result format for model and agent stages
Introduce a short, markdown-ish result format whose first non-empty line is
the declared outcome and whose "# output" section carries the routing data,
so stage output stops carrying multi-kilobyte prose inside a JSON envelope.
- New internal/stage/line_result.go: resultFormat() gates legacy JSON ('{'
first) vs the new line format; decodeLineResult() parses outcome-from-first
line, requires a JSON-object "# output" section, stores extra sections
verbatim under normalised keys, and records a reserved result_parse status
(parsed/partial/failed). Failures return a salvage Result plus
ErrInvalidModelResult, and never route an undeclared outcome.
- internal/stage/model.go: write the raw response to raw-output.md (0600,
best-effort) before parsing, and route the line format through
decodeLineResult while leaving the JSON path (stageResultText ->
decodeModelResult -> repairModelResult) unchanged.
- internal/stage/agent.go: agentResultText returns line-format text verbatim
so the "# output" body is not stripped by the JSON-object extractor.
- internal/server/runner.go: merge a non-empty salvage Result into the error
envelope under "salvage" on failure (a no-op for the legacy JSON path).
- internal/stage/line_result_test.go: raw-text fixtures covering happy path,
missing/undeclared outcome, missing "# output", best-effort extras,
adversarial prose, legacy JSON unchanged, >=64KB responses, and
raw-output.md written before parsing.
|
||
|
|
8c1eea53a9 |
Report a stale client and refuse an unreadable newer server
Exchange a build/protocol identity so an older CLI can no longer talk to a newer native server silently: - protocol carries ProtocolVersion and every server frame (including the unsolicited welcome) is stamped with the server build and protocol, so a client that never pings still learns the server identity. - buildinfo.Compare classifies the relationship as InSync/Warn/Refuse and names both builds; the client Watch reports one warning per connection and returns a hard error when the server speaks a newer, uninterpretable protocol so the caller never acts on the frame. - client.ReceiveNotice decodes the raw frame so an unmodeled top-level field produces a one-time "unrecognized field(s)" notice instead of being dropped silently (the case that hid the artifacts field). - CLI routes warnings to stderr, keeps machine JSON stdout clean and fails non-zero on a refused mismatch; ping/connection print the client/server relationship. The TUI surfaces the same notices through the shared log. - scripts/setup-native.py now warns when an older `pipelines` earlier on PATH shadows prefix/bin/pipelines, naming both paths and versions, without installing or modifying anything else. - Docs, the packaged debugging skill reference and README no longer imply the installed CLI always matches the server. Tests cover the comparison table, unknown-field detection and caching, the server identity stamping on pong and welcome, the CLI/TUI reporting paths and the installer PATH shadowing warning. |
||
|
|
a6aff47117 |
Add a human decision stage that pauses a run for an operator's choice
A new registered stage type `human` (internal/stage/human.go) has no model,
agent, tools or configs: its required `prompt` block carries the operator
instructions and its non-error edges are the selectable options. It parses,
validates, renders, exports and imports like every other type (no parser
special-case, no config keys).
At runtime the runner parks the run instead of executing: the step and the run
persist `waiting` with self-contained {prompt, options} evidence, and the run
returns without starting later stages, capturing file diffs, scanning artifacts,
holding a goroutine/timer or applying a stage timeout. The wait is durable: a
restart leaves the waiting rows untouched (RecoverRuns excludes them) and a
rebuilt runtime over the same database still accepts the decision.
`pipelines decide <run-id> [stage] --option KEY [--feedback TEXT] [--json]` and
the TUI `c` picker resolve the decision from the pinned graph in one store
transaction, record the choice and feedback as the step's evidence and resume
along edges[KEY]. An unknown option is refused with the valid list, a stage that
is not waiting is refused, and a repeated decision for an already-decided visit
does not double-apply. Cancel and retry keep working while waiting and after a
decision; a human stage is never killed by a stage timeout or max_visits, and a
`notify` policy can fire an `on=waiting` event.
Docs: the authoring skill references (format.md, operations.md), docs/cli.md and
README describe the stage, the command and the TUI picker. Tests cover stage
validation, the parsing round-trip, the store transitions, a websocket wait/decide
integration, cancel/retry, waiting notifications and a persistence-layer rebuild
(second Open + newRunner over one database, no process restart).
|
||
|
|
c700b28e6a |
Add cross-system read-only search
One query now finds anything in the system over the wire and from the CLI. - protocol: add Search/SearchResults message types and a SearchRequest (query, optional kind filter, optional limit) / SearchResponse (typed, ranked matches plus truncation and scan-bound counters) pair, with typed payloads for pipeline, version, run, step/visit, field text and artifact. - store: add Store.Search, a read-only query over pipelines, pipeline_versions, pipeline_runs, run_steps (+step_control) and step_artifacts. Ranks exact name (3) > name substring (2) > body text (1), breaks ties by recency then a stable kind/target order, and reports the matched field. Bounds work with a 50-default/200-max result cap, a 2000-row per-kind scan limit and a 2000-step / 1 MiB field-text scan budget. - terms: a bare status word or lone date expression (today, yesterday, this week, YYYY-MM-DD) selects runs; anything else stays free text. - server: dispatch protocol.Search to stores[0].Search; read-only and unconfirmed. - cli: add `pipelines search QUERY [--type KIND] [--limit N] [--json]`, help text and a BROWSE line. - store: add OpenReadOnly so the live-database check never runs DDL. - docs: document the grammar, kinds, ranking and bounds in docs/cli.md, scope the TUI omnisearch to pipelines in docs/ui-ux-review.md, and add a search bullet to the packaged operations reference. - tests: protocol round trip, store/ranking/term-parsing/limit/scan-bound unit tests, a server end-to-end test, and an env-guarded live check. No new dependencies; go.mod/go.sum unchanged. |
||
|
|
e79ff9585f |
Derive a plain-language verdict for every completed stage visit
Each completed step now exposes a verdict: the chosen option (outcome -> target), a bounded one-line reason with model/fallback/absent provenance, and the recorded artifacts the reason refers to. The store persists only the model-generated line in a new step_verdicts table and derives the fallback at read time (no backfill); the server generates the line once at completion, best-effort; the TUI renders it in the stage progress summary. The packaged debugging reference documents it. |
||
|
|
f3e0593f4d |
Let runs carry an operator title shown in the CLI and TUI
Add a bounded, never-empty title to every run. `pipelines run --title "..."` sets it explicitly and skips model generation; otherwise the server generates it from the prompt through the shared summariser and falls back to the prompt's first line, recording title_source (model/fallback/explicit) and title_model. - protocol/store: RunSummary.Title plus Label(), a set_run_title message, title columns with an idempotent migration, CreateRunTitled, and SetRunTitle renames. - summarise: Title/GenerateTitle/DeriveTitle/ExplicitTitle bounded to TitleCap. - runner: generate the title outside the admission lock and only for a request that starts a run; a replay reuses the stored title and an explicit title skips the model entirely. - CLI: `--title` on run and a `title <run-id> "<title>"` rename command; human `runs` and `inspect` show the title beside the pipeline name and version. - TUI: the run list row, run detail header, Home run cards and last-run line, and the pipeline Results line show the title via Label(). Tests cover storage/migration/fallback, explicit-title and rename, the end-to-end CLI flow, and title rendering; CLI and operations docs updated. |
||
|
|
1818a06d2f |
Restrict chat_sessions to server-owned sessions
chat_sessions now brokers GET /api/session but filters the global list to the sessions this server recorded in run_steps, deduped by session id, and fails closed to an empty list when no ownership source is configured. A session another client holds (in-server claim, released on stream end or disconnect) or an active run is marked busy; a second client's chat_prompt for a held session is rejected before any OpenCode call. chat_create stays unbrokered. |
||
|
|
3ff5cd8838 | Poll OpenCode permission requests and forward chat_reply | ||
|
|
b28f88be66 |
Add tests proving chat_interrupt stops a running chat turn
The chat_interrupt path was already wired end to end (protocol type,
server dispatch through handleChatSimple to broker.Interrupt, and the
adapter POST /api/session/{id}/interrupt), but nothing exercised it
against a live stream. Add the missing coverage:
- internal/opencode/chat_test.go: pin the adapter's outbound call
exactly (POST /api/session/ses_1/interrupt, empty JSON body, one
request, nil on 200) plus a non-nil error on HTTP 500.
- internal/server/chat_test.go: drive the real opencode.Chat adapter
over an httptest SSE fixture that holds the turn open until the
interrupt arrives, then assert the run ends, the interrupt is
acknowledged, no error frame is sent, and the prompt/interrupt posts
and interrupt path match. Also cover invalid-session rejection,
the unconfigured-broker error, and broker-failure reporting, adding a
spare interruptErr hook to the scripted fakeChat.
Production code is unchanged: verification found no gap in the existing
interrupt transport. Tests use httptest loopback fixtures only and do
not touch the live OpenCode service.
|
||
|
|
915016df0b |
Expose OpenCode conversation history in chat_messages
decodeChatMessages only understood the native-serve {info,parts} envelope,
so records from the V2 HTTP API (GET /api/session/{id}/message) decoded
with an empty role and were silently dropped, leaving chat_messages with
empty history. Teach nativeChatMessage to read the V2 top-level shape
(id/type/agent/model/time, parts under "content", user prompt as a bare
string) while keeping the native {info,parts} path via role/id/model/agent/
created accessors and a contentParts fallback.
Add seeded-session evidence: a decoder table test covering both envelopes
(user + assistant with text, reasoning and a completed tool part carrying
input, content and status) plus handler tests that drive chat_messages
through the broker fallthrough and through the real adapter.
|
||
|
|
b8b545accd |
Stream brokered OpenCode chat replies over /connect
Add the server-side half of the brokered chat path (route B): the
opencode.Chat adapter forwards chat_prompt to POST
/api/session/{id}/prompt through the owned Connection.Call translation
and derives chat_event frames from the authenticated /api/event SSE
firehose (text and reasoning deltas, tool parts with their status
transitions, step and usage). Every pushed frame carries a stable
session and message id, with a last-message-id fallback so step/tool
events that omit the assistant message id stay attributable.
The /connect handler keeps per-session retained state (chatHub) capped at
1 MiB with least-recently-updated eviction at 128 turns, so a
reconnecting client re-syncs through chat_messages instead of replaying
deltas the OpenCode per-session log never resends. Prompt dispatch
subscribes before posting and buffers pre-ack frames to preserve order,
and cancels the stream on disconnect so no goroutine leaks. chat_sessions
and chat_create are declared but not yet brokered: they answer with an
explicit protocol error rather than the generic Pong.
Reuse the existing transport, validation and normalization; no second
OpenCode transport or credential path is introduced.
Tests: protocol golden round-trip, adapter normalization + v1/v2 Call
translation, and handler ack/stream/validation/resume/disconnect, plus a
real-adapter no-tool evidence harness.
|
||
|
|
f57457aec9 |
Make agent-stage retries safe and retain timeout failure evidence
Fix four stage failure-handling and observability defects (A2, A3, A6, A9).
- A2: add safeToReplay(def) as the single definition of retryable post-admission
work - only an explicit tools block that selects neither edit nor shell. The
agent/pi WorkStarted guards now break only when the stage is not safe to
replay, so read-only stages genuinely retry a post-admission timeout while
mutating and legacy-bundle stages still never replay admitted work.
- A6: re-emit a malformed final result on an explicit read-only stage up to the
retry budget, preserving format_repairs evidence; mutating stages keep the
immediate return and their session.
- A3: timed-out stages retain usable evidence. waitForSessionIdle records one
bounded sanitized OperationEvidence on deadline; the agent deferred cleanup
does a bounded best-effort latest-assistant readback and settles knowable V2
usage; RetainSession adds a retained_session pointer to failed step
transcripts so the session is visible beneath the BoundTranscript threshold.
- A9: a stage outcome routed to #failure now yields a non-empty run diagnostic
("stage <name> returned outcome <outcome>") while preferring an existing step
diagnostic.
Adds focused stage and server tests for retry safety, malformed-result
re-emission, timeout evidence retention and run-level diagnostics; updates the
packaged skill references and docs/cli.md, docs/model-results.md.
|
||
|
|
0f5a188c6a |
Record and serve per-stage artifacts
Every agent stage now writes into a per-run/step artifacts directory under
the data dir (<data>/artifacts/<run>/<step>), passed to Pi as
PIPELINES_ARTIFACTS_DIR, exposed on the OpenCode client env seam, and stated
in the agent prompt boilerplate so agents save images, screenshots, reports
and logs there.
After each step the server scans that directory with a bounded, symlink- and
traversal-safe inventory (os.OpenRoot + Lstat + O_NOFOLLOW) recording name,
size_bytes, content type (extension then 512-byte sniff fallback) and URL,
enforcing max files, per-file bytes and total bytes while counting skipped
and truncated entries. A missing, unreadable or oversized directory yields
only a best-effort diagnostic and never changes the step outcome.
The list is carried on protocol.RunStep as Artifacts []StageArtifact plus
skipped/truncated counters (omitempty, backwards compatible), persisted in a
new step_artifacts table keyed (run_id, sequence) alongside the
notifications/file-changes tables and loaded by GetRun.
GET /artifacts/{run}/{step}/{name} on the existing handler serves the bytes
with http.ServeContent, so Content-Type, Content-Length, HEAD and Range
(206/416) come from the stdlib; unknown runs/steps, absolute paths,
traversal, symlink escapes and non-regular files are refused.
The TUI lists each selected step's artifacts (name + human size) in the
retained-evidence section and adds an artifacts summary in the style of the
file-changes summary. Adds docs/stage-artifacts.md linked from README and an
artifacts subsection to the packaged operations skill reference.
Tests cover the scan records and caps, missing/oversized directories,
HTTP HEAD/Range/traversal/symlink/unknown refusals and protocol round-trips.
|
||
|
|
e6d00d4ee5 |
Scope agent session bookkeeping to the run for concurrent runs
The per-workspace concurrency added in
|
||
|
|
2618f0d244 |
Allow concurrent runs in different workspaces
Track each active run's resolved workspace alongside its cancel function in a single runner.active struct map. A new request key is now refused only when an already-active run is changing the same resolved workspace; the refusal names the conflicting run id and advises waiting or using a different workspace. Workspace resolution is unchanged (default cwd, absolute request workspace, EvalSymlinks), so two spellings of one directory still compare equal. The conflict check and registration stay inside the existing r.mu critical section, so concurrent starts for one workspace cannot both pass. Idempotent repeats of an active request key and the restart-recovery guard are unchanged. Add concurrency_test.go covering concurrent different-workspace runs, same-workspace refusal, symlink/default-cwd equivalence, idempotent repeats, restart recovery and retry workspace scoping; strengthen the runner_test.go refusal assertion. Update docs/cli.md, docs/first-version-testing.md and the packaged operations skill reference to describe per-workspace concurrency. Assisted-by: DeepSeek V4.1 Flash |
||
|
|
05ae530442 |
Make Pipelines Docker-only with OpenCode and Pi
Install current verified harness releases inside Docker, preserve persistent accounts and settings, and provide separate interactive and pipeline model defaults with terminal onboarding and fuzzy pickers. Add Docker-aware authoring transport, Pi stage execution, bounded workflow controls, reliable final-result readback and usage settlement, and finite short-lived CLI pipe draining. Keep dated installation evidence and packaged authoring guidance consistent. Verified with a real Docker installation, both harness stages and interactive authors, the built-in plan/implement/review starter, independent host race/vet checks, installer tests, and packaged skill tests. Generated release receipts stay ignored in clean checkouts. |
||
|
|
c42a19f7d3 |
Improve workflow UI, retain stage diffs, and add portable pipeline transfers
Make OpenCode authoring responsive and expand the managed skill references. Add conflict-aware import/export, real-provider reference workflows, and tracked V2 generation for reliable budget accounting. |