Commit Graph

26 Commits

Author SHA1 Message Date
alan 36fad7d130 Merge lane-go: keep the lane's reviewed work
Combines lane-go's typed omni search, launch-time model overrides, TUI
artifacts view, run-state websocket push, and the "reflect the change where
it belongs" committer rule with the Templates gallery, chat/omni-search and
other reviewed work already on main. The 12 files both sides touched keep
both intents composed rather than dropping either side: the omni search keeps
the typed search contract (CLI `pipelines search` and the TUI popup with type
badges and marked snippets) alongside the run-list quick filter; cli.md and
ui-ux-review.md document both the search rework and the artifacts view;
orchestration.md, harnesses.md and operations.md keep the model-override and
reflect-where-it-belongs guidance; and the protocol/CLI/server/TUI code keeps
the new search, run-event, override and artifacts paths alongside the existing
run, title and help surfaces.

Checks before committing:
- go build ./...  -- pass
- go vet ./...  -- pass
- go test -count=1 ./internal/... ./cmd/...  -- all pass

Doc check before committing: nothing needs updating. The lane's own docs and
references (docs/cli.md, docs/ui-ux-review.md, and the orchestration,
harnesses and operations skill references) already carry the new features.
format.md and the packaged templates already describe the line-oriented result
format and the human decision stage from the earlier lane-go merge. The root
README already describes the omni search popup. Model overrides, the search
command, run events and the artifacts view are runtime/CLI/TUI surfaces that
cli.md documents; no template, README or topic-doc edit is required by this
merge.
2026-10-17 02:04:58 +01:00
alan 8f3036bd3b Push run state to subscribed websocket clients
Add a server-pushed run_event frame so a client no longer has to poll
once a second to notice run changes.

- protocol: RunEventFrame/RunStepDelta compact delta (run id, workspace,
  status, outcome, current stage, completed-step count, timestamps and an
  optional single step delta), plus subscribe/unsubscribe/run_event types
  and the optional RunEvent/RunIDs message fields. All fields optional so
  older peers ignore them.
- server: runEventHub with per-connection subscriptions scoped to a
  workspace (and optional explicit run ids), one bounded writer goroutine
  per subscription, slow-subscriber drop, per-client subscription cap and
  a ~220ms per-run coalescer that flushes pending deltas before a terminal
  frame.
- runner: emit deltas on created/queued, queued->running and step start,
  human waiting, step finish (outcome, target, diagnostic, artifacts and
  completed count), terminal finish, cancel of a waiting run and
  decide/resume; emitRun is a no-op without a hub.
- websocket: subscribe seeds current state before the ack, unsubscribe and
  connection teardown drop subscriptions, and pushed frames reuse the
  connection write closure.
- tests: protocol golden-JSON wire shape plus websocket coverage for the
  started-run frame sequence, subscribe current-state snapshot, silent
  unsubscribed/never-subscribed clients, workspace isolation and a
  bounded coalescing frame count.
- docs: note subscribe/unsubscribe/run_event in docs/cli.md; TUI and
  Android still poll in this release.
2026-10-17 00:43:05 +01:00
alan 865993b200 Rework omni search onto a typed search contract and add a TUI artifacts view
Protocol: replace the loose Query/Kind/Matches/Limit search fields with a Search request object and a SearchResults response, and have the server answer search from Store.Search over the shared run projections (runFrom/runStamps).

CLI: add the read-only 'pipelines search QUERY [--type KIND] [--limit N] [--json]' command with usage/help, and document the typed targets, ranking and bounds in docs/cli.md.

TUI: the '/' popup keeps type-to-search, up/down, Enter and Esc, renders one ranked list with kind badges and a bracketed (non-colour) match snippet, and Enter opens the matched pipeline, run, step, artifact or field; footer text updated to match.

TUI: add a dedicated run artifacts view ('A') with open/copy of recorded files, and move artifacts out of the run-detail File changes section (artifactCaptureNotes).

Tests: add Store.OpenReadOnly for read-only inspection of a live database plus pipeline/server/TUI search tests and artifacts tests, including env-gated live checks.
2026-10-16 23:48:38 +01:00
alan fae1005962 Allow launching runs with whole-run and per-stage model overrides
Add `pipelines run --model PROVIDER/MODEL` and repeatable
`--stage-model STAGE=PROVIDER/MODEL`, resolved with precedence
per-stage > whole-run > the model pinned in the pipeline version.

- A shared pure resolver (`pipeline.ParseModelOverride` /
  `pipeline.EffectiveModels`) owns the precedence and validation: every value
  must be a `provider/model` pair (`stage.ValidModel`), every stage key must
  exist in the pinned graph, an unknown key fails naming the valid keys, and a
  per-stage key that names a non-model stage is refused.
- The server validates against the pinned version before admission (no run row
  is ever inserted for an invalid request), persists `model`/`stage_models` on
  the run and `provider`/`model` on each step, applies the effective pair to a
  clone of the stage definition so the stage and its repair/formatter path share
  it, overlays the returned run graph with the effective pair (presentation
  only), and reuses the recorded overrides on retry.
- The run record echoes the overrides it was launched with and each step records
  the provider/model it actually ran with; the TUI Evidence pane renders that
  effective pair. The pinned pipeline version and `show --version N` are never
  modified.
- CLI help, `docs/cli.md` and the packaged authoring skill references document
  the flags and the precedence. A TUI override picker remains a documented
  follow-up; the CLI is the deliverable.

Tests: pure precedence/validation table tests, store round-trip and
request-key-mismatch plus migration, server admission/application/overlay/retry
with a controlled executor, CLI flag/help/malformed-value tests, and a TUI
evidence test.
2026-10-16 23:33:53 +01:00
alan 80232a4ffe Add omni search: ranked mixed results in the TUI and CLI
Back the TUI "/" popup with the server's search command and add a matching
pipelines search CLI command.

- protocol: SearchRequest/SearchResponse and per-kind SearchResult payloads,
  one shared Label for the six-kind badge dispatch and RuneIndexFold for
  case-insensitive matching.
- pipeline: the read-only, ranked, bounded Store.Search implementation.
- cli: search QUERY [--type KIND] [--limit N] [--json], printing the shared
  label plus a text-field snippet when present.
- tui: type to search, up/down to choose, Enter to open and Esc to cancel over
  one mixed ranked list, with short type badges and a bracketed non-colour
  match marker; Enter navigates to the matched pipeline, run, step, field
  section or artifact entry.
- tests: protocol and TUI coverage, including an env-gated live test against
  the running server.
2026-10-16 23:03:17 +01:00
alan bd09c55c47 Merge lane-go: titles, verdicts, human stage, line result format, stale-client guard
Integrates nine commits of reviewed work from the lane-go branch after the
automated resolver failed on this merge five times (each attempt dying on the
malformed-result defect whose fix is contained in this same merge).

Integration decisions:
- Search: kept main's shipped design (Store.Search + SearchResponse.Results) and
  dropped lane-go's parallel, older attempt (an unused Message.Matches field, an
  absent runs.search, and two partial TUI plumbing blocks). lane-go itself did not
  compile: protocol.SearchMatch was undefined and model.go referenced searchNav,
  requestSearch and applyRunNav, which exist nowhere. Those partial TUI blocks are
  removed; the queued TUI omni-search item will implement the surface properly on
  main's design.
- Both sides' additions are kept where they are independent: run titles and
  provenance, the human decision stage and its CLI/TUI surface, the derived stage
  verdict, the stale-client guard, the line-oriented result format, and main's
  search command.
- The search key's help label is now "/ omni search"; the two tests asserting the
  old label are updated rather than the label reverted.
- docs/cli.md and the packaged operations reference carry both sides' text.

Verified: go build ./... , go vet ./... and the full go test ./... suite pass.
2026-10-16 22:21:43 +01:00
alan 8c1eea53a9 Report a stale client and refuse an unreadable newer server
Exchange a build/protocol identity so an older CLI can no longer talk to a
newer native server silently:

- protocol carries ProtocolVersion and every server frame (including the
  unsolicited welcome) is stamped with the server build and protocol, so a
  client that never pings still learns the server identity.
- buildinfo.Compare classifies the relationship as InSync/Warn/Refuse and
  names both builds; the client Watch reports one warning per connection and
  returns a hard error when the server speaks a newer, uninterpretable
  protocol so the caller never acts on the frame.
- client.ReceiveNotice decodes the raw frame so an unmodeled top-level field
  produces a one-time "unrecognized field(s)" notice instead of being dropped
  silently (the case that hid the artifacts field).
- CLI routes warnings to stderr, keeps machine JSON stdout clean and fails
  non-zero on a refused mismatch; ping/connection print the client/server
  relationship. The TUI surfaces the same notices through the shared log.
- scripts/setup-native.py now warns when an older `pipelines` earlier on PATH
  shadows prefix/bin/pipelines, naming both paths and versions, without
  installing or modifying anything else.
- Docs, the packaged debugging skill reference and README no longer imply the
  installed CLI always matches the server.

Tests cover the comparison table, unknown-field detection and caching, the
server identity stamping on pong and welcome, the CLI/TUI reporting paths and
the installer PATH shadowing warning.
2026-10-16 18:37:01 +01:00
alan 98bbc2deac Report recorded decision evidence and harden the Pi artifacts env
Refinements on top of the human decision stage (a6aff47):

- `pipelines decide` now reports the feedback the server actually stored for
  the chosen option, so a repeated (idempotent) decision shows the recorded
  evidence instead of the ignored request text. A repeated decision is covered
  by a new assertion that also checks the step output does not change.
- The decision option/feedback output keys are defined once in protocol
  (DecisionOptionKey/DecisionFeedbackKey) with a DecisionEvidence reader, and
  the store and CLI use them instead of bare string literals.
- piEnvironment drops any inherited PIPELINES_ARTIFACTS_DIR before spawning Pi,
  so the artifactsDir argument is authoritative and an ambient directory (for
  example Pipelines' own when a stage runs inside another stage) never leaks
  into a child that was given none. Covered by a test with an ambient value set.
2026-10-16 17:25:48 +01:00
alan a6aff47117 Add a human decision stage that pauses a run for an operator's choice
A new registered stage type `human` (internal/stage/human.go) has no model,
agent, tools or configs: its required `prompt` block carries the operator
instructions and its non-error edges are the selectable options. It parses,
validates, renders, exports and imports like every other type (no parser
special-case, no config keys).

At runtime the runner parks the run instead of executing: the step and the run
persist `waiting` with self-contained {prompt, options} evidence, and the run
returns without starting later stages, capturing file diffs, scanning artifacts,
holding a goroutine/timer or applying a stage timeout. The wait is durable: a
restart leaves the waiting rows untouched (RecoverRuns excludes them) and a
rebuilt runtime over the same database still accepts the decision.

`pipelines decide <run-id> [stage] --option KEY [--feedback TEXT] [--json]` and
the TUI `c` picker resolve the decision from the pinned graph in one store
transaction, record the choice and feedback as the step's evidence and resume
along edges[KEY]. An unknown option is refused with the valid list, a stage that
is not waiting is refused, and a repeated decision for an already-decided visit
does not double-apply. Cancel and retry keep working while waiting and after a
decision; a human stage is never killed by a stage timeout or max_visits, and a
`notify` policy can fire an `on=waiting` event.

Docs: the authoring skill references (format.md, operations.md), docs/cli.md and
README describe the stage, the command and the TUI picker. Tests cover stage
validation, the parsing round-trip, the store transitions, a websocket wait/decide
integration, cancel/retry, waiting notifications and a persistence-layer rebuild
(second Open + newRunner over one database, no process restart).
2026-10-16 16:47:28 +01:00
alan c700b28e6a Add cross-system read-only search
One query now finds anything in the system over the wire and from the CLI.

- protocol: add Search/SearchResults message types and a SearchRequest
  (query, optional kind filter, optional limit) / SearchResponse (typed,
  ranked matches plus truncation and scan-bound counters) pair, with typed
  payloads for pipeline, version, run, step/visit, field text and artifact.
- store: add Store.Search, a read-only query over pipelines,
  pipeline_versions, pipeline_runs, run_steps (+step_control) and
  step_artifacts. Ranks exact name (3) > name substring (2) > body text (1),
  breaks ties by recency then a stable kind/target order, and reports the
  matched field. Bounds work with a 50-default/200-max result cap, a 2000-row
  per-kind scan limit and a 2000-step / 1 MiB field-text scan budget.
- terms: a bare status word or lone date expression (today, yesterday, this
  week, YYYY-MM-DD) selects runs; anything else stays free text.
- server: dispatch protocol.Search to stores[0].Search; read-only and
  unconfirmed.
- cli: add `pipelines search QUERY [--type KIND] [--limit N] [--json]`, help
  text and a BROWSE line.
- store: add OpenReadOnly so the live-database check never runs DDL.
- docs: document the grammar, kinds, ranking and bounds in docs/cli.md,
  scope the TUI omnisearch to pipelines in docs/ui-ux-review.md, and add a
  search bullet to the packaged operations reference.
- tests: protocol round trip, store/ranking/term-parsing/limit/scan-bound
  unit tests, a server end-to-end test, and an env-guarded live check.

No new dependencies; go.mod/go.sum unchanged.
2026-10-16 15:43:25 +01:00
alan e79ff9585f Derive a plain-language verdict for every completed stage visit
Each completed step now exposes a verdict: the chosen option (outcome -> target),
a bounded one-line reason with model/fallback/absent provenance, and the recorded
artifacts the reason refers to. The store persists only the model-generated line
in a new step_verdicts table and derives the fallback at read time (no backfill);
the server generates the line once at completion, best-effort; the TUI renders it
in the stage progress summary. The packaged debugging reference documents it.
2026-10-16 13:54:48 +01:00
alan f3e0593f4d Let runs carry an operator title shown in the CLI and TUI
Add a bounded, never-empty title to every run. `pipelines run --title "..."`
sets it explicitly and skips model generation; otherwise the server generates
it from the prompt through the shared summariser and falls back to the prompt's
first line, recording title_source (model/fallback/explicit) and title_model.

- protocol/store: RunSummary.Title plus Label(), a set_run_title message, title
  columns with an idempotent migration, CreateRunTitled, and SetRunTitle renames.
- summarise: Title/GenerateTitle/DeriveTitle/ExplicitTitle bounded to TitleCap.
- runner: generate the title outside the admission lock and only for a request
  that starts a run; a replay reuses the stored title and an explicit title
  skips the model entirely.
- CLI: `--title` on run and a `title <run-id> "<title>"` rename command; human
  `runs` and `inspect` show the title beside the pipeline name and version.
- TUI: the run list row, run detail header, Home run cards and last-run line,
  and the pipeline Results line show the title via Label().

Tests cover storage/migration/fallback, explicit-title and rename, the
end-to-end CLI flow, and title rendering; CLI and operations docs updated.
2026-10-16 13:36:48 +01:00
alan bfef465977 Fix build: define protocol.NormalizeText used by the summariser
internal/summarise called protocol.NormalizeText, which was never defined, so the tree did not compile. Add the shared normalisation helper: drop zero-width formatting runes, turn control characters into spaces and collapse whitespace runs (including the non-breaking space variants) into single spaces. Reuse it in OutputPreview instead of duplicating the mapping logic.

Cover the awkward inputs (newlines, tabs, non-breaking spaces, zero-width runes, surrounding quotes, trailing punctuation) in the protocol and summarise tests. Unrelated working-tree changes are left uncommitted.
2026-10-16 12:28:27 +01:00
alan 1818a06d2f Restrict chat_sessions to server-owned sessions
chat_sessions now brokers GET /api/session but filters the global list to
the sessions this server recorded in run_steps, deduped by session id, and
fails closed to an empty list when no ownership source is configured. A
session another client holds (in-server claim, released on stream end or
disconnect) or an active run is marked busy; a second client's chat_prompt
for a held session is rejected before any OpenCode call. chat_create stays
unbrokered.
2026-10-16 09:21:50 +01:00
alan 3ff5cd8838 Poll OpenCode permission requests and forward chat_reply 2026-10-16 08:06:39 +01:00
alan 9fa09e6ce8 Add brokered OpenCode chat wire types
Add the internal/protocol types the phone needs to talk to OpenCode chat
through the Pipelines server (route B): the chat_* request/response type
constants (sessions, create, prompt, messages, interrupt, reply) and the
server-pushed chat_event frame, plus the part/status/role/tool-state/decision
constant sets and payload structs modelled on the verified OpenCode shapes -
an assistant turn is a list of parts (text | reasoning | tool), and a tool
part carries name, executed, state.status (streaming | running | completed |
error), state.input, state.content, state.metadata, state.error and
epoch-millisecond timings.

Add eight optional omitempty chat fields to the shared Message envelope, so
older peers ignore them, and a golden round-trip test covering every new type,
an exact-JSON pin of a tool chat_event frame, envelope preservation and
omitempty. go test ./internal/protocol/... passes.
2026-10-16 03:01:55 +01:00
alan 0f5a188c6a Record and serve per-stage artifacts
Every agent stage now writes into a per-run/step artifacts directory under
the data dir (<data>/artifacts/<run>/<step>), passed to Pi as
PIPELINES_ARTIFACTS_DIR, exposed on the OpenCode client env seam, and stated
in the agent prompt boilerplate so agents save images, screenshots, reports
and logs there.

After each step the server scans that directory with a bounded, symlink- and
traversal-safe inventory (os.OpenRoot + Lstat + O_NOFOLLOW) recording name,
size_bytes, content type (extension then 512-byte sniff fallback) and URL,
enforcing max files, per-file bytes and total bytes while counting skipped
and truncated entries. A missing, unreadable or oversized directory yields
only a best-effort diagnostic and never changes the step outcome.

The list is carried on protocol.RunStep as Artifacts []StageArtifact plus
skipped/truncated counters (omitempty, backwards compatible), persisted in a
new step_artifacts table keyed (run_id, sequence) alongside the
notifications/file-changes tables and loaded by GetRun.

GET /artifacts/{run}/{step}/{name} on the existing handler serves the bytes
with http.ServeContent, so Content-Type, Content-Length, HEAD and Range
(206/416) come from the stdlib; unknown runs/steps, absolute paths,
traversal, symlink escapes and non-regular files are refused.

The TUI lists each selected step's artifacts (name + human size) in the
retained-evidence section and adds an artifacts summary in the style of the
file-changes summary. Adds docs/stage-artifacts.md linked from README and an
artifacts subsection to the packaged operations skill reference.

Tests cover the scan records and caps, missing/oversized directories,
HTTP HEAD/Range/traversal/symlink/unknown refusals and protocol round-trips.
2026-10-15 17:51:18 +01:00
alan 05ae530442 Make Pipelines Docker-only with OpenCode and Pi
Install current verified harness releases inside Docker, preserve persistent accounts and settings, and provide separate interactive and pipeline model defaults with terminal onboarding and fuzzy pickers.

Add Docker-aware authoring transport, Pi stage execution, bounded workflow controls, reliable final-result readback and usage settlement, and finite short-lived CLI pipe draining. Keep dated installation evidence and packaged authoring guidance consistent.

Verified with a real Docker installation, both harness stages and interactive authors, the built-in plan/implement/review starter, independent host race/vet checks, installer tests, and packaged skill tests. Generated release receipts stay ignored in clean checkouts.
2026-10-11 23:00:25 +01:00
alan c42a19f7d3 Improve workflow UI, retain stage diffs, and add portable pipeline transfers
Make OpenCode authoring responsive and expand the managed skill references. Add conflict-aware import/export, real-provider reference workflows, and tracked V2 generation for reliable budget accounting.
2026-10-10 20:55:42 +01:00
alan a66d540125 Add stage execution limits and result-first workflow views 2026-10-10 03:03:46 +01:00
alan e9a8eaf8ae Show full audited run graphs and scope searchable run history 2026-10-10 00:26:32 +01:00
alan 08b8647d5b Validate providers and retain complete run debugging context 2026-10-09 23:21:01 +01:00
alan a50e8648f6 Add service status and usage dashboard, refresh externally published edits 2026-10-09 20:20:44 +01:00
alan 0f80ff4e95 Add shared live run graphs, debugging, editor choice and draft cleanup 2026-10-09 19:17:43 +01:00
alan 6bd0e28ce6 Record run input directory and retry failed runs with original inputs 2026-10-09 14:23:21 +01:00
alan 0f30a001c2 Initial pipelines server, CLI, TUI, and Go installation 2026-10-08 22:22:17 +01:00