Commit Graph

34 Commits

Author SHA1 Message Date
alan bd09c55c47 Merge lane-go: titles, verdicts, human stage, line result format, stale-client guard
Integrates nine commits of reviewed work from the lane-go branch after the
automated resolver failed on this merge five times (each attempt dying on the
malformed-result defect whose fix is contained in this same merge).

Integration decisions:
- Search: kept main's shipped design (Store.Search + SearchResponse.Results) and
  dropped lane-go's parallel, older attempt (an unused Message.Matches field, an
  absent runs.search, and two partial TUI plumbing blocks). lane-go itself did not
  compile: protocol.SearchMatch was undefined and model.go referenced searchNav,
  requestSearch and applyRunNav, which exist nowhere. Those partial TUI blocks are
  removed; the queued TUI omni-search item will implement the surface properly on
  main's design.
- Both sides' additions are kept where they are independent: run titles and
  provenance, the human decision stage and its CLI/TUI surface, the derived stage
  verdict, the stale-client guard, the line-oriented result format, and main's
  search command.
- The search key's help label is now "/ omni search"; the two tests asserting the
  old label are updated rather than the label reverted.
- docs/cli.md and the packaged operations reference carry both sides' text.

Verified: go build ./... , go vet ./... and the full go test ./... suite pass.
2026-10-16 22:21:43 +01:00
alan b05277a57d Add line-oriented result format for model and agent stages
Introduce a short, markdown-ish result format whose first non-empty line is
the declared outcome and whose "# output" section carries the routing data,
so stage output stops carrying multi-kilobyte prose inside a JSON envelope.

- New internal/stage/line_result.go: resultFormat() gates legacy JSON ('{'
  first) vs the new line format; decodeLineResult() parses outcome-from-first
  line, requires a JSON-object "# output" section, stores extra sections
  verbatim under normalised keys, and records a reserved result_parse status
  (parsed/partial/failed). Failures return a salvage Result plus
  ErrInvalidModelResult, and never route an undeclared outcome.
- internal/stage/model.go: write the raw response to raw-output.md (0600,
  best-effort) before parsing, and route the line format through
  decodeLineResult while leaving the JSON path (stageResultText ->
  decodeModelResult -> repairModelResult) unchanged.
- internal/stage/agent.go: agentResultText returns line-format text verbatim
  so the "# output" body is not stripped by the JSON-object extractor.
- internal/server/runner.go: merge a non-empty salvage Result into the error
  envelope under "salvage" on failure (a no-op for the legacy JSON path).
- internal/stage/line_result_test.go: raw-text fixtures covering happy path,
  missing/undeclared outcome, missing "# output", best-effort extras,
  adversarial prose, legacy JSON unchanged, >=64KB responses, and
  raw-output.md written before parsing.
2026-10-16 21:34:33 +01:00
alan 8c1eea53a9 Report a stale client and refuse an unreadable newer server
Exchange a build/protocol identity so an older CLI can no longer talk to a
newer native server silently:

- protocol carries ProtocolVersion and every server frame (including the
  unsolicited welcome) is stamped with the server build and protocol, so a
  client that never pings still learns the server identity.
- buildinfo.Compare classifies the relationship as InSync/Warn/Refuse and
  names both builds; the client Watch reports one warning per connection and
  returns a hard error when the server speaks a newer, uninterpretable
  protocol so the caller never acts on the frame.
- client.ReceiveNotice decodes the raw frame so an unmodeled top-level field
  produces a one-time "unrecognized field(s)" notice instead of being dropped
  silently (the case that hid the artifacts field).
- CLI routes warnings to stderr, keeps machine JSON stdout clean and fails
  non-zero on a refused mismatch; ping/connection print the client/server
  relationship. The TUI surfaces the same notices through the shared log.
- scripts/setup-native.py now warns when an older `pipelines` earlier on PATH
  shadows prefix/bin/pipelines, naming both paths and versions, without
  installing or modifying anything else.
- Docs, the packaged debugging skill reference and README no longer imply the
  installed CLI always matches the server.

Tests cover the comparison table, unknown-field detection and caching, the
server identity stamping on pong and welcome, the CLI/TUI reporting paths and
the installer PATH shadowing warning.
2026-10-16 18:37:01 +01:00
alan a6aff47117 Add a human decision stage that pauses a run for an operator's choice
A new registered stage type `human` (internal/stage/human.go) has no model,
agent, tools or configs: its required `prompt` block carries the operator
instructions and its non-error edges are the selectable options. It parses,
validates, renders, exports and imports like every other type (no parser
special-case, no config keys).

At runtime the runner parks the run instead of executing: the step and the run
persist `waiting` with self-contained {prompt, options} evidence, and the run
returns without starting later stages, capturing file diffs, scanning artifacts,
holding a goroutine/timer or applying a stage timeout. The wait is durable: a
restart leaves the waiting rows untouched (RecoverRuns excludes them) and a
rebuilt runtime over the same database still accepts the decision.

`pipelines decide <run-id> [stage] --option KEY [--feedback TEXT] [--json]` and
the TUI `c` picker resolve the decision from the pinned graph in one store
transaction, record the choice and feedback as the step's evidence and resume
along edges[KEY]. An unknown option is refused with the valid list, a stage that
is not waiting is refused, and a repeated decision for an already-decided visit
does not double-apply. Cancel and retry keep working while waiting and after a
decision; a human stage is never killed by a stage timeout or max_visits, and a
`notify` policy can fire an `on=waiting` event.

Docs: the authoring skill references (format.md, operations.md), docs/cli.md and
README describe the stage, the command and the TUI picker. Tests cover stage
validation, the parsing round-trip, the store transitions, a websocket wait/decide
integration, cancel/retry, waiting notifications and a persistence-layer rebuild
(second Open + newRunner over one database, no process restart).
2026-10-16 16:47:28 +01:00
alan c700b28e6a Add cross-system read-only search
One query now finds anything in the system over the wire and from the CLI.

- protocol: add Search/SearchResults message types and a SearchRequest
  (query, optional kind filter, optional limit) / SearchResponse (typed,
  ranked matches plus truncation and scan-bound counters) pair, with typed
  payloads for pipeline, version, run, step/visit, field text and artifact.
- store: add Store.Search, a read-only query over pipelines,
  pipeline_versions, pipeline_runs, run_steps (+step_control) and
  step_artifacts. Ranks exact name (3) > name substring (2) > body text (1),
  breaks ties by recency then a stable kind/target order, and reports the
  matched field. Bounds work with a 50-default/200-max result cap, a 2000-row
  per-kind scan limit and a 2000-step / 1 MiB field-text scan budget.
- terms: a bare status word or lone date expression (today, yesterday, this
  week, YYYY-MM-DD) selects runs; anything else stays free text.
- server: dispatch protocol.Search to stores[0].Search; read-only and
  unconfirmed.
- cli: add `pipelines search QUERY [--type KIND] [--limit N] [--json]`, help
  text and a BROWSE line.
- store: add OpenReadOnly so the live-database check never runs DDL.
- docs: document the grammar, kinds, ranking and bounds in docs/cli.md,
  scope the TUI omnisearch to pipelines in docs/ui-ux-review.md, and add a
  search bullet to the packaged operations reference.
- tests: protocol round trip, store/ranking/term-parsing/limit/scan-bound
  unit tests, a server end-to-end test, and an env-guarded live check.

No new dependencies; go.mod/go.sum unchanged.
2026-10-16 15:43:25 +01:00
alan e79ff9585f Derive a plain-language verdict for every completed stage visit
Each completed step now exposes a verdict: the chosen option (outcome -> target),
a bounded one-line reason with model/fallback/absent provenance, and the recorded
artifacts the reason refers to. The store persists only the model-generated line
in a new step_verdicts table and derives the fallback at read time (no backfill);
the server generates the line once at completion, best-effort; the TUI renders it
in the stage progress summary. The packaged debugging reference documents it.
2026-10-16 13:54:48 +01:00
alan f3e0593f4d Let runs carry an operator title shown in the CLI and TUI
Add a bounded, never-empty title to every run. `pipelines run --title "..."`
sets it explicitly and skips model generation; otherwise the server generates
it from the prompt through the shared summariser and falls back to the prompt's
first line, recording title_source (model/fallback/explicit) and title_model.

- protocol/store: RunSummary.Title plus Label(), a set_run_title message, title
  columns with an idempotent migration, CreateRunTitled, and SetRunTitle renames.
- summarise: Title/GenerateTitle/DeriveTitle/ExplicitTitle bounded to TitleCap.
- runner: generate the title outside the admission lock and only for a request
  that starts a run; a replay reuses the stored title and an explicit title
  skips the model entirely.
- CLI: `--title` on run and a `title <run-id> "<title>"` rename command; human
  `runs` and `inspect` show the title beside the pipeline name and version.
- TUI: the run list row, run detail header, Home run cards and last-run line,
  and the pipeline Results line show the title via Label().

Tests cover storage/migration/fallback, explicit-title and rename, the
end-to-end CLI flow, and title rendering; CLI and operations docs updated.
2026-10-16 13:36:48 +01:00
alan 1818a06d2f Restrict chat_sessions to server-owned sessions
chat_sessions now brokers GET /api/session but filters the global list to
the sessions this server recorded in run_steps, deduped by session id, and
fails closed to an empty list when no ownership source is configured. A
session another client holds (in-server claim, released on stream end or
disconnect) or an active run is marked busy; a second client's chat_prompt
for a held session is rejected before any OpenCode call. chat_create stays
unbrokered.
2026-10-16 09:21:50 +01:00
alan 3ff5cd8838 Poll OpenCode permission requests and forward chat_reply 2026-10-16 08:06:39 +01:00
alan b28f88be66 Add tests proving chat_interrupt stops a running chat turn
The chat_interrupt path was already wired end to end (protocol type,
server dispatch through handleChatSimple to broker.Interrupt, and the
adapter POST /api/session/{id}/interrupt), but nothing exercised it
against a live stream. Add the missing coverage:

- internal/opencode/chat_test.go: pin the adapter's outbound call
  exactly (POST /api/session/ses_1/interrupt, empty JSON body, one
  request, nil on 200) plus a non-nil error on HTTP 500.
- internal/server/chat_test.go: drive the real opencode.Chat adapter
  over an httptest SSE fixture that holds the turn open until the
  interrupt arrives, then assert the run ends, the interrupt is
  acknowledged, no error frame is sent, and the prompt/interrupt posts
  and interrupt path match. Also cover invalid-session rejection,
  the unconfigured-broker error, and broker-failure reporting, adding a
  spare interruptErr hook to the scripted fakeChat.

Production code is unchanged: verification found no gap in the existing
interrupt transport. Tests use httptest loopback fixtures only and do
not touch the live OpenCode service.
2026-10-16 06:43:02 +01:00
alan 915016df0b Expose OpenCode conversation history in chat_messages
decodeChatMessages only understood the native-serve {info,parts} envelope,
so records from the V2 HTTP API (GET /api/session/{id}/message) decoded
with an empty role and were silently dropped, leaving chat_messages with
empty history. Teach nativeChatMessage to read the V2 top-level shape
(id/type/agent/model/time, parts under "content", user prompt as a bare
string) while keeping the native {info,parts} path via role/id/model/agent/
created accessors and a contentParts fallback.

Add seeded-session evidence: a decoder table test covering both envelopes
(user + assistant with text, reasoning and a completed tool part carrying
input, content and status) plus handler tests that drive chat_messages
through the broker fallthrough and through the real adapter.
2026-10-16 05:24:00 +01:00
alan b8b545accd Stream brokered OpenCode chat replies over /connect
Add the server-side half of the brokered chat path (route B): the
opencode.Chat adapter forwards chat_prompt to POST
/api/session/{id}/prompt through the owned Connection.Call translation
and derives chat_event frames from the authenticated /api/event SSE
firehose (text and reasoning deltas, tool parts with their status
transitions, step and usage). Every pushed frame carries a stable
session and message id, with a last-message-id fallback so step/tool
events that omit the assistant message id stay attributable.

The /connect handler keeps per-session retained state (chatHub) capped at
1 MiB with least-recently-updated eviction at 128 turns, so a
reconnecting client re-syncs through chat_messages instead of replaying
deltas the OpenCode per-session log never resends. Prompt dispatch
subscribes before posting and buffers pre-ack frames to preserve order,
and cancels the stream on disconnect so no goroutine leaks. chat_sessions
and chat_create are declared but not yet brokered: they answer with an
explicit protocol error rather than the generic Pong.

Reuse the existing transport, validation and normalization; no second
OpenCode transport or credential path is introduced.

Tests: protocol golden round-trip, adapter normalization + v1/v2 Call
translation, and handler ack/stream/validation/resume/disconnect, plus a
real-adapter no-tool evidence harness.
2026-10-16 04:09:14 +01:00
alan f57457aec9 Make agent-stage retries safe and retain timeout failure evidence
Fix four stage failure-handling and observability defects (A2, A3, A6, A9).

- A2: add safeToReplay(def) as the single definition of retryable post-admission
  work - only an explicit tools block that selects neither edit nor shell. The
  agent/pi WorkStarted guards now break only when the stage is not safe to
  replay, so read-only stages genuinely retry a post-admission timeout while
  mutating and legacy-bundle stages still never replay admitted work.
- A6: re-emit a malformed final result on an explicit read-only stage up to the
  retry budget, preserving format_repairs evidence; mutating stages keep the
  immediate return and their session.
- A3: timed-out stages retain usable evidence. waitForSessionIdle records one
  bounded sanitized OperationEvidence on deadline; the agent deferred cleanup
  does a bounded best-effort latest-assistant readback and settles knowable V2
  usage; RetainSession adds a retained_session pointer to failed step
  transcripts so the session is visible beneath the BoundTranscript threshold.
- A9: a stage outcome routed to #failure now yields a non-empty run diagnostic
  ("stage <name> returned outcome <outcome>") while preferring an existing step
  diagnostic.

Adds focused stage and server tests for retry safety, malformed-result
re-emission, timeout evidence retention and run-level diagnostics; updates the
packaged skill references and docs/cli.md, docs/model-results.md.
2026-10-15 18:27:22 +01:00
alan 0f5a188c6a Record and serve per-stage artifacts
Every agent stage now writes into a per-run/step artifacts directory under
the data dir (<data>/artifacts/<run>/<step>), passed to Pi as
PIPELINES_ARTIFACTS_DIR, exposed on the OpenCode client env seam, and stated
in the agent prompt boilerplate so agents save images, screenshots, reports
and logs there.

After each step the server scans that directory with a bounded, symlink- and
traversal-safe inventory (os.OpenRoot + Lstat + O_NOFOLLOW) recording name,
size_bytes, content type (extension then 512-byte sniff fallback) and URL,
enforcing max files, per-file bytes and total bytes while counting skipped
and truncated entries. A missing, unreadable or oversized directory yields
only a best-effort diagnostic and never changes the step outcome.

The list is carried on protocol.RunStep as Artifacts []StageArtifact plus
skipped/truncated counters (omitempty, backwards compatible), persisted in a
new step_artifacts table keyed (run_id, sequence) alongside the
notifications/file-changes tables and loaded by GetRun.

GET /artifacts/{run}/{step}/{name} on the existing handler serves the bytes
with http.ServeContent, so Content-Type, Content-Length, HEAD and Range
(206/416) come from the stdlib; unknown runs/steps, absolute paths,
traversal, symlink escapes and non-regular files are refused.

The TUI lists each selected step's artifacts (name + human size) in the
retained-evidence section and adds an artifacts summary in the style of the
file-changes summary. Adds docs/stage-artifacts.md linked from README and an
artifacts subsection to the packaged operations skill reference.

Tests cover the scan records and caps, missing/oversized directories,
HTTP HEAD/Range/traversal/symlink/unknown refusals and protocol round-trips.
2026-10-15 17:51:18 +01:00
alan e6d00d4ee5 Scope agent session bookkeeping to the run for concurrent runs
The per-workspace concurrency added in 2618f0d exposed three places that assumed a
single run, and the second concurrent run failed with 'agent quiescence not
confirmed' because of them.

- ActiveAgentSessionsForRun(runID) scopes the end-of-step quiescence check to the
  run's own sessions, so another run's legitimately active agent no longer fails
  this one. A run still fails when its own session cannot be confirmed idle.
- The startup guard now uses ActiveSessionOwners (which carries the owning run id)
  and refuses only an ORPHANED session - one whose run is not active now - which is
  what restart recovery is actually about. A session owned by a currently active
  run no longer blocks new work.
- sessionActive examines only the target session: foreign entries in the
  /api/session/active map are ignored, the target's absence is inactive, and the
  target present with an unsupported state still fails closed.

Verified: go build, go vet and go test ./... pass, including ten concurrency tests
(two workspaces run concurrently, same-workspace refusal, symlink and default-cwd
equivalence, idempotent repeats, restart recovery, retry scoping, overlapping runs
not failing quiescence, and a run's own active session still failing it).

Operator verification, which a stage cannot perform because restarting the server
would abort the run doing it: the service was rebuilt and restarted on this
commit, then two runs in separate workspaces (/tmp/ws-c and /tmp/ws-d) executed
concurrently - both reported executing at the same time, a third run in ws-c was
refused naming run 46, and both finished succeeded with no diagnostic.
2026-10-15 15:38:50 +01:00
alan 2618f0d244 Allow concurrent runs in different workspaces
Track each active run's resolved workspace alongside its cancel function
in a single runner.active struct map. A new request key is now refused
only when an already-active run is changing the same resolved workspace;
the refusal names the conflicting run id and advises waiting or using a
different workspace. Workspace resolution is unchanged (default cwd,
absolute request workspace, EvalSymlinks), so two spellings of one
directory still compare equal. The conflict check and registration stay
inside the existing r.mu critical section, so concurrent starts for one
workspace cannot both pass. Idempotent repeats of an active request key
and the restart-recovery guard are unchanged.

Add concurrency_test.go covering concurrent different-workspace runs,
same-workspace refusal, symlink/default-cwd equivalence, idempotent
repeats, restart recovery and retry workspace scoping; strengthen the
runner_test.go refusal assertion. Update docs/cli.md,
docs/first-version-testing.md and the packaged operations skill reference
to describe per-workspace concurrency.

Assisted-by: DeepSeek V4.1 Flash
2026-10-15 15:08:44 +01:00
alan 05ae530442 Make Pipelines Docker-only with OpenCode and Pi
Install current verified harness releases inside Docker, preserve persistent accounts and settings, and provide separate interactive and pipeline model defaults with terminal onboarding and fuzzy pickers.

Add Docker-aware authoring transport, Pi stage execution, bounded workflow controls, reliable final-result readback and usage settlement, and finite short-lived CLI pipe draining. Keep dated installation evidence and packaged authoring guidance consistent.

Verified with a real Docker installation, both harness stages and interactive authors, the built-in plan/implement/review starter, independent host race/vet checks, installer tests, and packaged skill tests. Generated release receipts stay ignored in clean checkouts.
2026-10-11 23:00:25 +01:00
alan c42a19f7d3 Improve workflow UI, retain stage diffs, and add portable pipeline transfers
Make OpenCode authoring responsive and expand the managed skill references. Add conflict-aware import/export, real-provider reference workflows, and tracked V2 generation for reliable budget accounting.
2026-10-10 20:55:42 +01:00
alan a66d540125 Add stage execution limits and result-first workflow views 2026-10-10 03:03:46 +01:00
alan 7b10e3889e Replace niche templates with simple starters and direct template submission 2026-10-10 02:37:31 +01:00
alan 68042aad5d Mount host repositories for Docker execution with host user ownership 2026-10-10 01:22:02 +01:00
alan e9a8eaf8ae Show full audited run graphs and scope searchable run history 2026-10-10 00:26:32 +01:00
alan a1c0eb4a72 Handle native structured-format incompatibility and preserve failure evidence 2026-10-10 00:01:02 +01:00
alan 08b8647d5b Validate providers and retain complete run debugging context 2026-10-09 23:21:01 +01:00
alan f700d1beb2 Configure full Docker setup and confirm consequential actions 2026-10-09 21:34:47 +01:00
alan f60599230c Retain OpenCode sessions and verify fresh retry isolation 2026-10-09 21:16:28 +01:00
alan a50e8648f6 Add service status and usage dashboard, refresh externally published edits 2026-10-09 20:20:44 +01:00
alan 67ac79765f Harden model results with lossless isolated repair and scoped permissions 2026-10-09 20:02:41 +01:00
alan 0f80ff4e95 Add shared live run graphs, debugging, editor choice and draft cleanup 2026-10-09 19:17:43 +01:00
alan 6bd0e28ce6 Record run input directory and retry failed runs with original inputs 2026-10-09 14:23:21 +01:00
alan ed93d055bf Run tasks directly with language-independent project tools 2026-10-09 01:48:46 +01:00
alan c5a5521015 Remove workspace-root configuration and use client run directories 2026-10-09 00:32:00 +01:00
alan accbe63e39 Simplify Go installation and support standard OpenCode releases 2026-10-08 23:17:38 +01:00
alan 0f30a001c2 Initial pipelines server, CLI, TUI, and Go installation 2026-10-08 22:22:17 +01:00