Commit Graph

47 Commits

Author SHA1 Message Date
alan 89a2bf1e01 Add operator resource pool configuration commands 2026-10-18 23:37:46 +01:00
alan a022b58c5c Wire pool resource targets through runner and repair quarantined-member handoff 2026-10-18 23:20:55 +01:00
alan a4c7149060 Merge branch 'main' into android-target-pool-allocation 2026-10-18 22:10:47 +01:00
alan 4de7cfbfe4 feat: retry selected stage with input overrides 2026-10-18 21:49:37 +01:00
alan 112e94baee feat: add pooled resource allocation contract 2026-10-18 21:11:24 +01:00
alan b8beeb250b feat: resume retries from failed stage 2026-10-18 20:38:03 +01:00
alan b79d0401f3 feat: recover inspected quarantined resources
Add local operator inspection and reconciliation for quarantined resource leases, including atomic audit-backed FIFO handoff and recovery protocol support.\n\nPreserve capability tokens in the authoritative store and reject non-loopback or ordinary client transport for recovery mutations.\n\nChecked packaged templates and packaged authoring skill references; none need updating. The task-owned docs note records shared README and skill reference follow-up for docs-fold.
2026-10-18 19:24:32 +01:00
alan 27ad7daecd test: cover inspected resource recovery boundaries
Add executable regressions for a forged forwarded loopback claim over a real WebSocket transport and for replaying an old recovery after FIFO successor handoff.\n\nChecked packaged templates and packaged authoring skill references; neither needs updating. The existing per-item documentation note records shared documentation that the manual docs-fold process must update.
2026-10-18 19:18:03 +01:00
alan 5946f1abc8 chat: separate create operation identity
Use an explicit chat_create operation_id as the durable bootstrap key while retaining envelope IDs for reply correlation. Legacy requests receive fresh server-scoped identities, so independent cli-1/android-1 requests do not collide; clients that retry retain one operation_id.

Update Android creation state and repository calls to preserve an operation identity across retries, and cover durable reconnect retries, duplicate barriers, changed input, legacy correlation reuse, and protocol serialization.

Checked packaged templates: none describe chat_create. Checked packaged authoring-skill references: the existing description is stale, but shared references and README must not be edited by an item; docs/notes/bootstrap-create-identity.md records the required docs-fold update and rollout compatibility.
2026-10-18 16:50:30 +01:00
alan 27a2e52376 Guard chat bootstrap transitions
Use conditional durable phase transitions and affected-row checks so only the request that reserves each phase can create or activate a manager chat session. Record duplicate ownership conflicts without repeating external work, and cover the reservation and restart state in pipeline and server tests.

Checked packaged templates and authoring skill references: the existing task commit already packages the dedicated manager skill and installer inventory; this transition-race fix requires no further template or reference update. Shared documentation remains untouched; the task note is docs/notes/manager-skill-bootstrap.md.
2026-10-18 15:23:07 +01:00
alan 1c89414896 Merge commit '4b66e608' into manager-skill-bootstrap
# Conflicts:
#	internal/opencode/chat_v2_test.go
2026-10-18 15:03:46 +01:00
alan ab9181eee9 Preserve manager skill bootstrap work 2026-10-18 15:03:22 +01:00
alan 9130ae8c7a Add stage-exclusive resource waiting
Add durable FIFO capacity-one resource leases for executable stages, including cleanup-confirmed release, conservative restart quarantine, queue state events, and CLI/TUI/Android waiting presentation. Add scheduler, cleanup, protocol, client, and Android coverage plus a proposed Android v3 resource draft.

Checked packaged templates and authoring skill references: neither needs updating in this item. The per-item docs note records the shared documentation that docs-fold must update; no shared README or packaged reference was edited.
2026-10-18 12:30:30 +01:00
alan 95362134aa session export: write only session.html, retire the markdown export
A stage's session was exported twice, as session.md and session.html, and both
files were presented. It is now one html file; the markdown export is retired.

- internal/stage/session_export.go: ExportSession renders and writes exactly one
  file, session.html, via writeSessionFile; renderSessionMarkdown and its helpers
  are gone; SessionExportResult carries HTML/HTMLBytes only. The source read, the
  8 MiB source / 2 MiB rendered bounds, the truncation banner, the
  never-overwrite rule and the honest empty-session handling are unchanged.
- internal/protocol/runs.go: SessionExport drops markdown/markdown_bytes (a
  record written before the change still decodes; unknown fields are ignored),
  and adds LegacySessionMarkdownArtifact with RunStep.PresentedArtifacts(), the
  one Go rule that hides the retired session.md from every artifact listing.
- internal/server/runner.go: evidence.Markdown/MarkdownBytes are no longer copied.
- internal/tui: the artifacts panel, the recorded count and the per-visit summary
  derive from PresentedArtifacts(), so the terminal never lists session.md.
- web/src/app.js: the spike's renderArtifacts applies the same filter.
- android: the Session section is the html export alone in a WebView — SessionView,
  the Markdown/HTML toggle and the session markdown path are removed;
  presentedArtifacts(step) filters the retired session.md out of the per-visit
  gallery, the run gallery and its count, the verdict/selection lists and the
  timeline indicator; data/protocol SessionExport drops the markdown fields.
- Tests pin the single html file, prove no session.md is written and that a
  pre-existing one is neither overwritten nor presented, and prove a legacy
  export record still loads and its session.md is hidden.

The legacy session.md of an older run stays on disk (dated evidence); it is only
no longer produced and no longer presented.

Checks (macOS, Go 1.24, Android Gradle 8.11.1, 18 Oct 2026): gofmt -l . prints
nothing; go build ./... && go vet ./... clean; go test ./internal/... ./cmd/...
-> 19 packages ok, 0 failures; node --check web/src/app.js ok and node --test
web/test -> 20/20; gradle --offline --no-daemon :app:assembleDebug
:app:testDebugUnitTest -> BUILD SUCCESSFUL.

Packaged references checked: internal/author/skill/references/delivery.md and
operations.md both described the session.md+session.html pair, so both are
updated here to describe the single session.html export and the retired, hidden
session.md. README.md and internal/author/skill/SKILL.md carry no session.md
reference and install.go's managed inventory is unchanged (no reference added or
removed). docs/notes/session-html-only.md records the remaining stale shared
documentation (docs/stage-artifacts.md, docs/app-integration-verification.md) for
the docs-fold writer; shared docs are not edited by this item.
2026-10-18 10:06:49 +01:00
alan 971e555ceb chat: close and reopen an operator session on the server, keeping its history
Add a server-side close for operator chats, so finishing with a chat takes it out
of the active list for every client without deleting anything. The brokered
protocol gains chat_close and chat_reopen; chat_close stops a running turn by
reusing the existing broker interrupt (its result carries an interrupted flag, so
a stream is never left running into nothing) and then sets closed_at on the
session's operator_sessions row. The session, its transcript and its history
survive, and because the closed state lives in the registry chat_sessions reports
it to every client, not just the one that asked. An id that is not a recorded
operator session, including a pipeline-owned run session, is refused before
anything is written. chat_reopen clears the state again, and simply prompting a
closed session also clears it (the store clears closed_at on touch).

The registry gains closed_at for a fresh database and an idempotent migration for
one created before it existed. The app renders closed sessions under a labelled
Closed section with a Closed chip, offers a Close action with a confirmation that
says it stops a running turn, marks the chat closed on every device and deletes
nothing, and a Reopen action on each closed row. The device-local hide is renamed
Hide on this device, with a snackbar saying the chat stays on the server, so it is
never confused with the real close. No new dependency, the installer and Docker
files are untouched, and no run is started from the app.

Checked before committing: the packaged templates carry no chat surface and need
no update, and internal/author/install.go's managed inventory needs none either
(this adds no reference file). The packaged authoring skill reference
internal/author/skill/references/operations.md and the shared docs/cli.md and
android/README.md are stale because of this change, but an item no longer edits a
shared document: what is stale and what should change there is recorded in
docs/notes/chat-close.md for the docs-fold writer, which is committed here with
the change.

Checks: gofmt clean, go build ./... and go vet ./... clean, and
go test ./internal/protocol/... ./internal/pipeline/... ./internal/server/...
./internal/author/... pass. The app side is verified with :app:assembleDebug and
:app:testDebugUnitTest (--offline --no-daemon, BUILD SUCCESSFUL); the on-device
close, second-client and reopen check is deferred to visual review.

Not deployed; deploying is the operator's step.
2026-10-18 00:16:24 +01:00
alan c1652e2bc6 Split chat/model/run files by concern (pure move)
Pure move: split the three multi-feature server files into per-concern files,
with no behaviour change - same package, same exported and unexported
identifiers, same call graph, same tests.

- internal/server/chat.go -> chat.go (broker seam, client token, dispatcher),
  chat_hub.go, chat_claims.go, chat_permissions.go, chat_sessions.go,
  chat_stream.go, chat_commands.go
- internal/stage/model.go -> model.go, model_run.go, model_failure.go,
  model_validation.go
- internal/pipeline/runs.go -> runs.go (read side), runs_schema.go,
  runs_create.go, runs_mutate.go, runs_sessions.go

Checked before committing:
- Packaged templates and internal/author/skill/references: neither refers to
  internal/server/chat.go, internal/stage/model.go or internal/pipeline/runs.go,
  so nothing there needed updating (the change is a move with no behaviour
  change).
- gofmt -l on the three packages: clean. go build ./..., go vet ./... and
  go test ./... pass. go.mod/go.sum unchanged.
- No test, docs, android/, install or Docker files touched.

Known stale path pointers deliberately left to the single docs writer (shared
docs) and out of scope (android/): docs/chat-verification.md and
android/README.md name the old file paths.
2026-10-17 21:09:08 +01:00
alan 3f5c11b7bf Record and expose per-run model usage (tokens, split, cost)
Carry the usage a model stage already measures for its budget
(stage.Budget.Snapshot()) onto the step as typed evidence and total it in
one pass, rather than a second harness read, a transcript parse or an
output-length estimate. A shared budget across stage revisits records only
the visit's increment, and a revisit that observes no new report records no
usage instead of a fabricated known zero.

- protocol: StageUsage/RunUsage/TotalUsage and UsageTotals.
- pipeline: step_usage table + migration, COALESCEd into the one step
  projection, FinishStep persistence, one json_extract history aggregate,
  retention removes the rows with their run.
- server: stageUsage maps the budget snapshot, stamping per-stage model and
  usage for every model stage (including failures).
- cli: inspect/stats human and JSON output.
- budget/pi: retain the reported in/out split, mark a total-only (Pi) report
  split-unknown, and keep an unreported cost unknown (never zero).

Evidence and docs updated in the packaged skill references and docs/cli.md;
the live database has no step_usage rows yet (all history predates this), so
per-run figures are reported as unavailable, not zero.
2026-10-17 20:40:56 +01:00
alan 1f75e4797e Broker chat from an operator's own OpenCode endpoint and adopt existing sessions
Add an opt-in way to broker chat from an operator-supplied OpenCode service
instead of the instance the server spawns for itself, and a way to adopt a
session that already exists on that endpoint so it appears in chat_sessions and
can be continued from the app.

- Endpoint selection: the `-opencode-endpoint` server flag (defaulting to
  `PIPELINES_OPENCODE_ENDPOINT`) opts in; precedence is explicit flag >
  explicit environment > the endpoint recorded on a previous run, with the
  spawned instance as the default. opencode.OpenEndpoint validates the
  reference once (http(s), host, no userinfo/query/fragment/path), detects the
  backend from its own /api/info (v2) or /global/health (v1) contract, and
  requires a streamable text/event-stream event path. It owns no process
  (Close is a no-op) and never calls startServe; every failure names the
  endpoint and disables chat rather than silently falling back. An
  authenticated endpoint takes an optional password only through the private
  file PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE, with
  PIPELINES_OPENCODE_ENDPOINT_USERNAME overriding the detected default.
- Persistence: the validated choice is recorded in an additive single-row
  server_settings table so a restart with no flag/environment reuses it; an
  unreachable recorded endpoint disables chat with a clear message and no
  fallback.
- chat_adopt: a new protocol request (ChatAdoptRequest) with handleChatAdopt in
  internal/server/chat.go. It refuses a nil broker/registry/payload and a
  blank or foreign-shaped id before any endpoint call, reads the endpoint via
  the shared Chat.Session (which reuses Sessions/decodeChatSessions), records
  through the existing RecordOperatorSession with the endpoint's directory and
  title verbatim, and echoes the adopted session plus its registry row like
  chat_create. A missing id is refused naming the id and records nothing; an
  endpoint record with an empty directory is refused rather than substituting
  the server's working directory or home.
- CLI: `pipelines chat-adopt <session-id> [--json]` (usage, notes, help
  inventory, dispatch) with docs/cli.md updated under Chat sessions.
- Autostart preserves the endpoint variables (and the password-file reference,
  never an inline secret) via serviceEnvironmentKeys.

Checks before committing:
- Package location: the change is user-facing chat behaviour, so the packaged
  authoring skill reference internal/author/skill/references/operations.md is
  updated in the same change (Brokered OpenCode chat section) with a dated
  verification-evidence entry and the not-deployed state. No new reference file
  is added, so internal/author/install.go's managed inventory is unchanged.
- Packaged templates: this repository packages no template set for this
  behaviour and none references the chat endpoint, so no template needs
  updating.
- Verification: `go build ./...`, `go vet` on the touched packages, `gofmt -l`
  clean and `go test ./...` all pass. The unit tests cover the endpoint
  validation/precedence, the adopt happy path and refusals, the recorded
  endpoint round-trip, the protocol round-trip and the CLI dispatch. The live
  scratch-server check against the operator's real endpoint was not run in this
  change.

Nothing unrelated is staged; docs left stale (README and other shared documents)
are not edited here and are for a single folding writer.
2026-10-17 20:14:11 +01:00
alan 58c6b9b7c6 server: export a finished stage's real session as session.md and session.html
Every OpenCode-harness model stage now writes its real session into its own
step artifacts directory as session.md and session.html, so the existing
bounded artifact listing, the /artifacts/<run>/<step>/<name> serving path and
the artifact scan find and serve them with no special casing, while the stage
keeps its bounded preview in the run record unchanged.

- internal/opencode/export.go adds the one shared native export invocation and
  the exact session-ID/messages validation used by both Connection.Messages and
  the new ExportSession, so the readback path does not diverge. The backend's
  published subcommand is used (v1 `export <id>`, v2 `session export <id>`),
  the source read is capped at 8 MiB, and a larger session is read as a prefix
  and reported truncated instead of failing. boundedOutput.ReadFrom keeps the
  cap effective so overflow and the truncation path are real.
- internal/stage/session_export.go renders markdown and html with a metadata
  header, every role and every part (unknown part types as raw JSON, every
  session text escaped in html), atomic temp+rename writes that never overwrite
  an existing file, a visible truncation banner at both ends when the source or
  the 2 MiB rendered cap cuts it, and an honest 0-message header for an empty
  session rather than fabricated text.
- protocol.SessionExport and RunStep.SessionExport are stored in a new
  step_session_exports table (CREATE TABLE IF NOT EXISTS, so an old database
  gains an empty table with no migration) loaded through the existing main
  SELECT alongside the other per-step evidence and deleted by run retention.
- internal/server/runner.go exports after the post-stage filediff capture and
  before scanArtifacts, with context.WithTimeout(context.WithoutCancel(ctx),
  10s); it never waits for session idleness and a failure or a Pi stage (no
  native export path) records a sanitized diagnostic and continues, never
  changing the stage status, outcome, target, timing or result validation.

Reflected where it belongs: the packaged skill references
(internal/author/skill/references/operations.md and delivery.md) carry the new
Session export rules and the run-artifacts bullet, and docs/stage-artifacts.md
documents the names, the <run>/<step> paths, the served URL, the caps, the
truncation labels, failure isolation and the Pi limitation. The packaged
templates (examples/*.md) only tell a stage to save its own artifacts and say
nothing about the server-written session export, so none needed updating; no
new reference file was added, so install.go's managed inventory is unchanged.

Shared documentation: README.md's per-step artifacts sentence is now stale - it
should name the exported session alongside images, screenshots, reports and
logs. It is deliberately not committed here; the shared documents are folded by
the single documentation writer.

Checks: gofmt -l on the touched packages produced no output; go vet ./...
passed; go test -race -count=1 ./... passed.
2026-10-17 19:40:43 +01:00
alan 4908452a74 retention: report removed runs and follow a symlinked artifacts root
Fix two honesty gaps in run-history retention that the implementer stage
left uncommitted on top of 14175eb:

- Populate RemovedRuns, RemovedRunCount, RemovedArtifactFiles/Bytes and
  Applied right after the row transaction commits, before staging cleanup.
  A cleanup failure must not report a successful row/artifact removal as
  zero removed, and staging leftovers are excluded from SpaceReclaimedBytes.
  A failed VACUUM after removal now reports the artifact bytes actually
  removed while claiming no database pages.
- Stage artifacts through os.Stat instead of os.Lstat so a symlinked
  artifacts root is followed: the report sized that directory, so apply
  removes through it too instead of counting bytes it never deleted.

Tests cover post-commit staging-cleanup failure reporting, staging through a
symlinked artifacts root (including no staging directory left in the link
target), and an honest reclaim failure after removal.

Checked documentation surfaces:
- docs/cli.md: measured example refreshed with the current before/apply/after
  output and the symlink/staging/VACUUM reporting described accurately.
- README.md: retention prose (explicit, never automatic, protected runs,
  artifacts go with the run) is unchanged and still correct.
- internal/author/skill/references/operations.md: operator-only retention
  summary describes the same behaviour; no user-facing surface changed, so
  no reference update was needed.
- Packaged templates: retention is a CLI/store capability with no template
  change, so nothing to update. internal/author/install.go inventory
  unchanged (no reference added or renamed).

Verification: gofmt clean on the touched Go files, git diff --check clean,
go vet ./internal/pipeline clean, and go test ./internal/pipeline -run
'TestRetention' -count=1 passing.
2026-10-17 16:19:39 +01:00
alan 14175eba70 retention: explicit run-history and artifact retention
Add `pipelines retention report` (strictly read-only) and
`pipelines retention apply` (explicit, approval-gated removal) with an age
and/or count policy, protected-run rules, atomic single-transaction removal
and honest database space reclaim. Nothing prunes automatically.
2026-10-17 15:45:46 +01:00
alan 18398dfa5b chat: give the operator their own sessions, separate from run sessions
chat_sessions used to list exactly the OpenCode sessions the pipelines own,
filtered through pipeline.SessionOwner (run_steps/run_session_owners), so an
operator-created chat was an accident of the run-ownership table and the
operator's working sessions were mixed with the internals of running pipelines.

Add a first-class operator session:

- internal/pipeline/operator_sessions.go: a dedicated operator_sessions table
  (session_id PK, directory, title, tag, created_at, last_used_at) with
  RecordOperatorSession (upsert preserving created_at), a one-statement
  TouchOperatorSession (no-op for a non-operator id), OperatorSessions bounded to
  128 rows most-recently-used first, and OperatorDirectories (bounded MRU,
  deduped). run_steps/run_session_owners and SessionOwner/OwnedSessions are
  untouched.
- internal/server/chat.go: chat_sessions returns operator sessions only - it
  reads the registry once, intersects it with one broker session-list call, fills
  directory/tag/created/updated from the registry, sets busy for a session held
  by another connection, keeps MRU order, and stays fail-closed (empty list, never
  the endpoint's whole list). A pipeline stage session and an untagged row can
  never be listed. chat_create resolves the directory (empty/'~'/'~/...' ->
  operator home; relative or missing rejected before any OpenCode call), calls
  broker.Create(title, resolved) and records the row with the operator tag.
  chat_directories returns home first then bounded (8) deduped recents. A
  successful prompt touches the operator session without affecting pipeline
  prompts.
- internal/opencode, internal/protocol, internal/server/websocket.go and
  message.go: unscoped session list, Create posts {"title":...,
  "location":{"directory":...}} to POST /api/session, additive protocol fields
  (omitempty) and the ChatDirectories request/response, plus the OperatorRegistry
  option defaulting to the first store. One OpenCode transport per server is
  unchanged.

Docs and references: docs/cli.md gains a "Chat sessions" section documenting the
operator-only fail-closed list, directory defaulting/rejection, chat_directories
(home first, deduped, bounded 8) and why a dedicated request was chosen.
internal/author/skill/references/operations.md's stale "Brokered OpenCode chat"
text (chat_create unbrokered, run_steps filtering, "listed the owned scratch
session" evidence) is rewritten to the operator-session registry, the fail-closed
filter, directory defaulting and the corrected live-test wording. No new
reference file, so internal/author/install.go's inventory is unchanged.

Checked for staleness elsewhere: README.md, the docs/ topic pages and the
packaged server/TUI templates carry no chat-session ownership claims, so none
needed updating.

Verified: gofmt -l clean; go vet and go build ./... clean; go test -count=1
./internal/pipeline/... ./internal/protocol/... ./internal/opencode/...
./internal/server/... and the full go test -count=1 ./... pass.
2026-10-17 11:14:25 +01:00
alan 36fad7d130 Merge lane-go: keep the lane's reviewed work
Combines lane-go's typed omni search, launch-time model overrides, TUI
artifacts view, run-state websocket push, and the "reflect the change where
it belongs" committer rule with the Templates gallery, chat/omni-search and
other reviewed work already on main. The 12 files both sides touched keep
both intents composed rather than dropping either side: the omni search keeps
the typed search contract (CLI `pipelines search` and the TUI popup with type
badges and marked snippets) alongside the run-list quick filter; cli.md and
ui-ux-review.md document both the search rework and the artifacts view;
orchestration.md, harnesses.md and operations.md keep the model-override and
reflect-where-it-belongs guidance; and the protocol/CLI/server/TUI code keeps
the new search, run-event, override and artifacts paths alongside the existing
run, title and help surfaces.

Checks before committing:
- go build ./...  -- pass
- go vet ./...  -- pass
- go test -count=1 ./internal/... ./cmd/...  -- all pass

Doc check before committing: nothing needs updating. The lane's own docs and
references (docs/cli.md, docs/ui-ux-review.md, and the orchestration,
harnesses and operations skill references) already carry the new features.
format.md and the packaged templates already describe the line-oriented result
format and the human decision stage from the earlier lane-go merge. The root
README already describes the omni search popup. Model overrides, the search
command, run events and the artifacts view are runtime/CLI/TUI surfaces that
cli.md documents; no template, README or topic-doc edit is required by this
merge.
2026-10-17 02:04:58 +01:00
alan c9b904186a Update templates and authoring skill to the current pipeline contract
Bring the packaged starters and the pipelines skill up to the guidelines the
project now works to.

Templates:
- task and three-stage now end in a committer stage that stages the union of the
  runner's captured input.server_changed_files and the producer's declared
  changed_files, runs the repository's checks, commits locally and never pushes;
  their producers no longer commit.
- Every model/agent prompt states the line-oriented result contract (outcome on the
  first non-empty line, "# output" JSON map, other "# <heading>" sections kept
  verbatim as evidence) and an "# evidence" section for exact commands and results,
  with proportionate verification and no fabricated checks.
- Stages that can write save their reports, logs and screenshots into the runtime
  artifacts directory.
- Remove the summary starter: a tool-free single-prompt text transform with no
  repository interaction, artifact or commit is a format demonstration, not a
  reusable workflow. No starter invents a pretend approval stage; a decision point
  is a type=human stage.

Skill:
- references/format.md gains a Result format section (line form, legacy JSON
  precedence, result_parse, raw-output.md, artifacts) and a Committer stages
  section with the pre-commit checklist; the role-split and envelope notes mention
  the line form and the committer.
- orchestration.md points at the checklist and records that a genuine gate is a
  human stage; operations.md and the SKILL.md index drop the removed starter and
  name the result format and committer. No new reference, so the managed inventory
  is unchanged.

Docs/tests: examples/README, README, docs/cli, docs/first-version-testing, the CLI
help example and the smoke script track the four starters; catalog, tool-role,
server, tui and pipeline tests are updated, with a new test pinning the committer
ending and server-captured change set.

Commands run: go build ./...; go vet ./...; go test ./...; python3 -m unittest
discover -s scripts -p 'test_*.py'; python3 scripts/smoke-first-install.py. All
passed. Templates inspected through an isolated CLI server (pipelines templates;
pipelines template KEY --source). No server, service or device was restarted, and
nothing was pushed.
2026-10-17 01:09:42 +01:00
alan 865993b200 Rework omni search onto a typed search contract and add a TUI artifacts view
Protocol: replace the loose Query/Kind/Matches/Limit search fields with a Search request object and a SearchResults response, and have the server answer search from Store.Search over the shared run projections (runFrom/runStamps).

CLI: add the read-only 'pipelines search QUERY [--type KIND] [--limit N] [--json]' command with usage/help, and document the typed targets, ranking and bounds in docs/cli.md.

TUI: the '/' popup keeps type-to-search, up/down, Enter and Esc, renders one ranked list with kind badges and a bracketed (non-colour) match snippet, and Enter opens the matched pipeline, run, step, artifact or field; footer text updated to match.

TUI: add a dedicated run artifacts view ('A') with open/copy of recorded files, and move artifacts out of the run-detail File changes section (artifactCaptureNotes).

Tests: add Store.OpenReadOnly for read-only inspection of a live database plus pipeline/server/TUI search tests and artifacts tests, including env-gated live checks.
2026-10-16 23:48:38 +01:00
alan fae1005962 Allow launching runs with whole-run and per-stage model overrides
Add `pipelines run --model PROVIDER/MODEL` and repeatable
`--stage-model STAGE=PROVIDER/MODEL`, resolved with precedence
per-stage > whole-run > the model pinned in the pipeline version.

- A shared pure resolver (`pipeline.ParseModelOverride` /
  `pipeline.EffectiveModels`) owns the precedence and validation: every value
  must be a `provider/model` pair (`stage.ValidModel`), every stage key must
  exist in the pinned graph, an unknown key fails naming the valid keys, and a
  per-stage key that names a non-model stage is refused.
- The server validates against the pinned version before admission (no run row
  is ever inserted for an invalid request), persists `model`/`stage_models` on
  the run and `provider`/`model` on each step, applies the effective pair to a
  clone of the stage definition so the stage and its repair/formatter path share
  it, overlays the returned run graph with the effective pair (presentation
  only), and reuses the recorded overrides on retry.
- The run record echoes the overrides it was launched with and each step records
  the provider/model it actually ran with; the TUI Evidence pane renders that
  effective pair. The pinned pipeline version and `show --version N` are never
  modified.
- CLI help, `docs/cli.md` and the packaged authoring skill references document
  the flags and the precedence. A TUI override picker remains a documented
  follow-up; the CLI is the deliverable.

Tests: pure precedence/validation table tests, store round-trip and
request-key-mismatch plus migration, server admission/application/overlay/retry
with a controlled executor, CLI flag/help/malformed-value tests, and a TUI
evidence test.
2026-10-16 23:33:53 +01:00
alan 80232a4ffe Add omni search: ranked mixed results in the TUI and CLI
Back the TUI "/" popup with the server's search command and add a matching
pipelines search CLI command.

- protocol: SearchRequest/SearchResponse and per-kind SearchResult payloads,
  one shared Label for the six-kind badge dispatch and RuneIndexFold for
  case-insensitive matching.
- pipeline: the read-only, ranked, bounded Store.Search implementation.
- cli: search QUERY [--type KIND] [--limit N] [--json], printing the shared
  label plus a text-field snippet when present.
- tui: type to search, up/down to choose, Enter to open and Esc to cancel over
  one mixed ranked list, with short type badges and a bracketed non-colour
  match marker; Enter navigates to the matched pipeline, run, step, field
  section or artifact entry.
- tests: protocol and TUI coverage, including an env-gated live test against
  the running server.
2026-10-16 23:03:17 +01:00
alan bd09c55c47 Merge lane-go: titles, verdicts, human stage, line result format, stale-client guard
Integrates nine commits of reviewed work from the lane-go branch after the
automated resolver failed on this merge five times (each attempt dying on the
malformed-result defect whose fix is contained in this same merge).

Integration decisions:
- Search: kept main's shipped design (Store.Search + SearchResponse.Results) and
  dropped lane-go's parallel, older attempt (an unused Message.Matches field, an
  absent runs.search, and two partial TUI plumbing blocks). lane-go itself did not
  compile: protocol.SearchMatch was undefined and model.go referenced searchNav,
  requestSearch and applyRunNav, which exist nowhere. Those partial TUI blocks are
  removed; the queued TUI omni-search item will implement the surface properly on
  main's design.
- Both sides' additions are kept where they are independent: run titles and
  provenance, the human decision stage and its CLI/TUI surface, the derived stage
  verdict, the stale-client guard, the line-oriented result format, and main's
  search command.
- The search key's help label is now "/ omni search"; the two tests asserting the
  old label are updated rather than the label reverted.
- docs/cli.md and the packaged operations reference carry both sides' text.

Verified: go build ./... , go vet ./... and the full go test ./... suite pass.
2026-10-16 22:21:43 +01:00
alan 98bbc2deac Report recorded decision evidence and harden the Pi artifacts env
Refinements on top of the human decision stage (a6aff47):

- `pipelines decide` now reports the feedback the server actually stored for
  the chosen option, so a repeated (idempotent) decision shows the recorded
  evidence instead of the ignored request text. A repeated decision is covered
  by a new assertion that also checks the step output does not change.
- The decision option/feedback output keys are defined once in protocol
  (DecisionOptionKey/DecisionFeedbackKey) with a DecisionEvidence reader, and
  the store and CLI use them instead of bare string literals.
- piEnvironment drops any inherited PIPELINES_ARTIFACTS_DIR before spawning Pi,
  so the artifactsDir argument is authoritative and an ambient directory (for
  example Pipelines' own when a stage runs inside another stage) never leaks
  into a child that was given none. Covered by a test with an ambient value set.
2026-10-16 17:25:48 +01:00
alan a6aff47117 Add a human decision stage that pauses a run for an operator's choice
A new registered stage type `human` (internal/stage/human.go) has no model,
agent, tools or configs: its required `prompt` block carries the operator
instructions and its non-error edges are the selectable options. It parses,
validates, renders, exports and imports like every other type (no parser
special-case, no config keys).

At runtime the runner parks the run instead of executing: the step and the run
persist `waiting` with self-contained {prompt, options} evidence, and the run
returns without starting later stages, capturing file diffs, scanning artifacts,
holding a goroutine/timer or applying a stage timeout. The wait is durable: a
restart leaves the waiting rows untouched (RecoverRuns excludes them) and a
rebuilt runtime over the same database still accepts the decision.

`pipelines decide <run-id> [stage] --option KEY [--feedback TEXT] [--json]` and
the TUI `c` picker resolve the decision from the pinned graph in one store
transaction, record the choice and feedback as the step's evidence and resume
along edges[KEY]. An unknown option is refused with the valid list, a stage that
is not waiting is refused, and a repeated decision for an already-decided visit
does not double-apply. Cancel and retry keep working while waiting and after a
decision; a human stage is never killed by a stage timeout or max_visits, and a
`notify` policy can fire an `on=waiting` event.

Docs: the authoring skill references (format.md, operations.md), docs/cli.md and
README describe the stage, the command and the TUI picker. Tests cover stage
validation, the parsing round-trip, the store transitions, a websocket wait/decide
integration, cancel/retry, waiting notifications and a persistence-layer rebuild
(second Open + newRunner over one database, no process restart).
2026-10-16 16:47:28 +01:00
alan c700b28e6a Add cross-system read-only search
One query now finds anything in the system over the wire and from the CLI.

- protocol: add Search/SearchResults message types and a SearchRequest
  (query, optional kind filter, optional limit) / SearchResponse (typed,
  ranked matches plus truncation and scan-bound counters) pair, with typed
  payloads for pipeline, version, run, step/visit, field text and artifact.
- store: add Store.Search, a read-only query over pipelines,
  pipeline_versions, pipeline_runs, run_steps (+step_control) and
  step_artifacts. Ranks exact name (3) > name substring (2) > body text (1),
  breaks ties by recency then a stable kind/target order, and reports the
  matched field. Bounds work with a 50-default/200-max result cap, a 2000-row
  per-kind scan limit and a 2000-step / 1 MiB field-text scan budget.
- terms: a bare status word or lone date expression (today, yesterday, this
  week, YYYY-MM-DD) selects runs; anything else stays free text.
- server: dispatch protocol.Search to stores[0].Search; read-only and
  unconfirmed.
- cli: add `pipelines search QUERY [--type KIND] [--limit N] [--json]`, help
  text and a BROWSE line.
- store: add OpenReadOnly so the live-database check never runs DDL.
- docs: document the grammar, kinds, ranking and bounds in docs/cli.md,
  scope the TUI omnisearch to pipelines in docs/ui-ux-review.md, and add a
  search bullet to the packaged operations reference.
- tests: protocol round trip, store/ranking/term-parsing/limit/scan-bound
  unit tests, a server end-to-end test, and an env-guarded live check.

No new dependencies; go.mod/go.sum unchanged.
2026-10-16 15:43:25 +01:00
alan e79ff9585f Derive a plain-language verdict for every completed stage visit
Each completed step now exposes a verdict: the chosen option (outcome -> target),
a bounded one-line reason with model/fallback/absent provenance, and the recorded
artifacts the reason refers to. The store persists only the model-generated line
in a new step_verdicts table and derives the fallback at read time (no backfill);
the server generates the line once at completion, best-effort; the TUI renders it
in the stage progress summary. The packaged debugging reference documents it.
2026-10-16 13:54:48 +01:00
alan f3e0593f4d Let runs carry an operator title shown in the CLI and TUI
Add a bounded, never-empty title to every run. `pipelines run --title "..."`
sets it explicitly and skips model generation; otherwise the server generates
it from the prompt through the shared summariser and falls back to the prompt's
first line, recording title_source (model/fallback/explicit) and title_model.

- protocol/store: RunSummary.Title plus Label(), a set_run_title message, title
  columns with an idempotent migration, CreateRunTitled, and SetRunTitle renames.
- summarise: Title/GenerateTitle/DeriveTitle/ExplicitTitle bounded to TitleCap.
- runner: generate the title outside the admission lock and only for a request
  that starts a run; a replay reuses the stored title and an explicit title
  skips the model entirely.
- CLI: `--title` on run and a `title <run-id> "<title>"` rename command; human
  `runs` and `inspect` show the title beside the pipeline name and version.
- TUI: the run list row, run detail header, Home run cards and last-run line,
  and the pipeline Results line show the title via Label().

Tests cover storage/migration/fallback, explicit-title and rename, the
end-to-end CLI flow, and title rendering; CLI and operations docs updated.
2026-10-16 13:36:48 +01:00
alan 1818a06d2f Restrict chat_sessions to server-owned sessions
chat_sessions now brokers GET /api/session but filters the global list to
the sessions this server recorded in run_steps, deduped by session id, and
fails closed to an empty list when no ownership source is configured. A
session another client holds (in-server claim, released on stream end or
disconnect) or an active run is marked busy; a second client's chat_prompt
for a held session is rejected before any OpenCode call. chat_create stays
unbrokered.
2026-10-16 09:21:50 +01:00
alan 0f5a188c6a Record and serve per-stage artifacts
Every agent stage now writes into a per-run/step artifacts directory under
the data dir (<data>/artifacts/<run>/<step>), passed to Pi as
PIPELINES_ARTIFACTS_DIR, exposed on the OpenCode client env seam, and stated
in the agent prompt boilerplate so agents save images, screenshots, reports
and logs there.

After each step the server scans that directory with a bounded, symlink- and
traversal-safe inventory (os.OpenRoot + Lstat + O_NOFOLLOW) recording name,
size_bytes, content type (extension then 512-byte sniff fallback) and URL,
enforcing max files, per-file bytes and total bytes while counting skipped
and truncated entries. A missing, unreadable or oversized directory yields
only a best-effort diagnostic and never changes the step outcome.

The list is carried on protocol.RunStep as Artifacts []StageArtifact plus
skipped/truncated counters (omitempty, backwards compatible), persisted in a
new step_artifacts table keyed (run_id, sequence) alongside the
notifications/file-changes tables and loaded by GetRun.

GET /artifacts/{run}/{step}/{name} on the existing handler serves the bytes
with http.ServeContent, so Content-Type, Content-Length, HEAD and Range
(206/416) come from the stdlib; unknown runs/steps, absolute paths,
traversal, symlink escapes and non-regular files are refused.

The TUI lists each selected step's artifacts (name + human size) in the
retained-evidence section and adds an artifacts summary in the style of the
file-changes summary. Adds docs/stage-artifacts.md linked from README and an
artifacts subsection to the packaged operations skill reference.

Tests cover the scan records and caps, missing/oversized directories,
HTTP HEAD/Range/traversal/symlink/unknown refusals and protocol round-trips.
2026-10-15 17:51:18 +01:00
alan e6d00d4ee5 Scope agent session bookkeeping to the run for concurrent runs
The per-workspace concurrency added in 2618f0d exposed three places that assumed a
single run, and the second concurrent run failed with 'agent quiescence not
confirmed' because of them.

- ActiveAgentSessionsForRun(runID) scopes the end-of-step quiescence check to the
  run's own sessions, so another run's legitimately active agent no longer fails
  this one. A run still fails when its own session cannot be confirmed idle.
- The startup guard now uses ActiveSessionOwners (which carries the owning run id)
  and refuses only an ORPHANED session - one whose run is not active now - which is
  what restart recovery is actually about. A session owned by a currently active
  run no longer blocks new work.
- sessionActive examines only the target session: foreign entries in the
  /api/session/active map are ignored, the target's absence is inactive, and the
  target present with an unsupported state still fails closed.

Verified: go build, go vet and go test ./... pass, including ten concurrency tests
(two workspaces run concurrently, same-workspace refusal, symlink and default-cwd
equivalence, idempotent repeats, restart recovery, retry scoping, overlapping runs
not failing quiescence, and a run's own active session still failing it).

Operator verification, which a stage cannot perform because restarting the server
would abort the run doing it: the service was rebuilt and restarted on this
commit, then two runs in separate workspaces (/tmp/ws-c and /tmp/ws-d) executed
concurrently - both reported executing at the same time, a third run in ws-c was
refused naming run 46, and both finished succeeded with no diagnostic.
2026-10-15 15:38:50 +01:00
alan 143f237268 Strengthen template planning, reuse and independent review 2026-10-12 23:10:37 +01:00
alan 05ae530442 Make Pipelines Docker-only with OpenCode and Pi
Install current verified harness releases inside Docker, preserve persistent accounts and settings, and provide separate interactive and pipeline model defaults with terminal onboarding and fuzzy pickers.

Add Docker-aware authoring transport, Pi stage execution, bounded workflow controls, reliable final-result readback and usage settlement, and finite short-lived CLI pipe draining. Keep dated installation evidence and packaged authoring guidance consistent.

Verified with a real Docker installation, both harness stages and interactive authors, the built-in plan/implement/review starter, independent host race/vet checks, installer tests, and packaged skill tests. Generated release receipts stay ignored in clean checkouts.
2026-10-11 23:00:25 +01:00
alan c42a19f7d3 Improve workflow UI, retain stage diffs, and add portable pipeline transfers
Make OpenCode authoring responsive and expand the managed skill references. Add conflict-aware import/export, real-provider reference workflows, and tracked V2 generation for reliable budget accounting.
2026-10-10 20:55:42 +01:00
alan a66d540125 Add stage execution limits and result-first workflow views 2026-10-10 03:03:46 +01:00
alan 7b10e3889e Replace niche templates with simple starters and direct template submission 2026-10-10 02:37:31 +01:00
alan e9a8eaf8ae Show full audited run graphs and scope searchable run history 2026-10-10 00:26:32 +01:00
alan 08b8647d5b Validate providers and retain complete run debugging context 2026-10-09 23:21:01 +01:00
alan a50e8648f6 Add service status and usage dashboard, refresh externally published edits 2026-10-09 20:20:44 +01:00
alan 6bd0e28ce6 Record run input directory and retry failed runs with original inputs 2026-10-09 14:23:21 +01:00
alan accbe63e39 Simplify Go installation and support standard OpenCode releases 2026-10-08 23:17:38 +01:00
alan 0f30a001c2 Initial pipelines server, CLI, TUI, and Go installation 2026-10-08 22:22:17 +01:00