Combines lane-go's typed omni search, launch-time model overrides, TUI artifacts view, run-state websocket push, and the "reflect the change where it belongs" committer rule with the Templates gallery, chat/omni-search and other reviewed work already on main. The 12 files both sides touched keep both intents composed rather than dropping either side: the omni search keeps the typed search contract (CLI `pipelines search` and the TUI popup with type badges and marked snippets) alongside the run-list quick filter; cli.md and ui-ux-review.md document both the search rework and the artifacts view; orchestration.md, harnesses.md and operations.md keep the model-override and reflect-where-it-belongs guidance; and the protocol/CLI/server/TUI code keeps the new search, run-event, override and artifacts paths alongside the existing run, title and help surfaces. Checks before committing: - go build ./... -- pass - go vet ./... -- pass - go test -count=1 ./internal/... ./cmd/... -- all pass Doc check before committing: nothing needs updating. The lane's own docs and references (docs/cli.md, docs/ui-ux-review.md, and the orchestration, harnesses and operations skill references) already carry the new features. format.md and the packaged templates already describe the line-oriented result format and the human decision stage from the earlier lane-go merge. The root README already describes the omni search popup. Model overrides, the search command, run events and the artifacts view are runtime/CLI/TUI surfaces that cli.md documents; no template, README or topic-doc edit is required by this merge.
20 KiB
System, storage and run evidence
Pipelines turns a reusable process into sequential stages with explicit handoffs, outcome routing and retained evidence. A Go CLI/TUI connects to an execution server; the server owns execution and SQLite history, OpenCode owns model/tool sessions, and optional ntfy delivery supplies notifications. It is technology-neutral, not a parallel/distributed task engine or a guarantee that model reports are correct.
Lifecycle and location
- Drafts are editable Markdown plus JSON metadata. Publishing validates and creates immutable versions; drafts remain retained. Keep context create-only/base-version guards. Four optional generic starters are task, review, research and three-stage; a fresh database starts empty. Templates can be created directly with an editable prefilled name or customized first. Editing a bundled repository template changes no installed or published definition; only explicit authoring then submission creates a new immutable version.
- A run pins its exact definition version, initial input and resolved directory. The client invocation directory is the default, but files/tools execute on the server. Docker needs that directory mounted at the same absolute path. A remote client does not upload its repository or make host credentials available remotely.
- Host and Docker OpenCode credentials are separate. Preserve setup-selected model inheritance; do not select a provider just because its ID appears in an example.
- Runtime caps are 30 stage steps and five visits to a stage. Runs in different
resolved workspaces execute concurrently; only a second run in a workspace an
active run is already changing is refused, naming the conflicting run and
advising you to wait or use a different workspace (for example another
checkout). Agent-session quiescence is scoped to the run: a run fails only when
one of its own sessions cannot be confirmed idle, so one run's active agent never
fails another's concurrent run. An active session left over by a restart with no
owning active run is an orphan and still refuses new work until interrupted.
Reusing that active run's request key reattaches idempotently.
type=humandecision stages exist and pause the run (seereferences/format.md); there are no parallel branches. Optional worktrees, full backup/restore, automatic database sync and independent verifier stage types are future ideas, not existing capabilities. Do not invent them in definitions. - Services are native or Docker-based. With an existing Docker installation do not start a competing native server. Server APIs are unauthenticated: trusted networks only; never recommend public exposure. Tool scope is not a universal OS sandbox.
- Docker setup resolves the current supported OpenCode V2 release from the single
documented complete
GET .../git/matching-refs/tags/v2response (a bounded complete array with no pagination parameters; never the movinglatesttag or the registry's insertion order) and the current Pi release from npm, resolves the selected tag's immutable GHCRsha256digest, buildsFROMthat digest, and verifies the built image reports exactly those executable versions plus a Node-compatible Pi engine. It fails closed, keeping the last verified Dockerfile fallback pins, when discovery, digest resolution, completeness, build or compatibility cannot be established: never a hardcoded fallback, never a silent stale-pin substitution. A full install skips a rebuild only when the release receipt and the actual built image's release and source identity labels already match; a dirty source always invokes the Docker build and relies on layer caching. A model/login-only reconfiguration compares only the verified release image identity, so a changed source alone does not rebuild../install --opencode|--pireconfigures inside the existing container and refuses an image that does not match; it does not rebuild or restart services (an explicit ordinary./installdoes when the user asks).
Where data lives
SQLite retains pipelines/immutable versions, runs, recorded stage visits, inputs/outputs, status/errors, audit transcripts, session ownership/references, retry links, schedules/state, notification receipts and file changes. Not everything is in that database: drafts/settings are client files; complete OpenCode sessions and credentials use OpenCode storage; repository artifacts are files; ntfy has its own storage. A pipeline ZIP is not a full backup of these components. Never merge or sync live SQLite files using generic file synchronization. Do not delete databases, sessions, volumes or credentials to repair a workflow.
Inspect results first
Use the authoritative CLI_PREFIX from the prompt:
CLI_PREFIX runs --json: run history (optional--pipeline NAMEscope).CLI_PREFIX search QUERY [--type KIND] [--limit N] --json: one read-only query across pipelines, versions, runs, steps, field text and artifacts. QUERY is free text, a bare status word (queued/running/succeeded/failed/canceled/interrupted) selecting those runs, or a lone date expression (today,yesterday,this week— Monday 00:00 local — orYYYY-MM-DD) selecting the runs created in that window; anything else stays free text.--typeis the only narrowing mechanism and takes one ofpipeline,version,run,step,field,artifact; an unknown kind is an error. Ranking is exact name (3) > name substring (2) > body text (1), equal tiers by recency, and every hit reportsscore,matched_fieldandrecency. The cap defaults to 50 and clamps to 200; every kind scans at most 2000 rows and field scanning stops at 2000 steps / 1 MiB and reportstruncated/scanned_steps/scanned_bytes/cap/scan_bound_bytes. Targets are stable:pipeline:<id>,pipeline:<id>:v<version>,run:<id>,run:<id>:<sequence>,run:<id>:<sequence>:<field>, or an artifact URL.CLI_PREFIX runs --json: run history (optional--pipeline NAMEscope). Each run carries a bounded, single-linetitlesaying what it was for, set explicitly withCLI_PREFIX run ... --title "..."or generated from its prompt by the shared summariser on the configured default harness. It is never empty: when generation is unavailable or times out the server derives the prompt's first line, andtitle_source(model,fallback, orexplicit) plustitle_modelrecord how it was produced, so a diagnosis can tell them apart. Humanrunsandinspectshow the title beside the pipeline name and version, so two runs of one pipeline are distinguishable;CLI_PREFIX title RUN_ID "..."renames a run after the fact without changing its input or steps. The stored prompt is unchanged, and listing/renaming never calls a model.CLI_PREFIX inspect RUN_ID --json: selected run, visits and retained evidence. Exact stage outputs are authoritative; a transcript over the 32 KiB retained bound is a valid, credential-redacted preview envelope (with preserved stage limits, failure/format diagnostics and the retained session pointer), not the complete tool history.CLI_PREFIX decide RUN_ID [STAGE] --option KEY [--feedback TEXT] [--json]: answer a run that iswaitingon atype=humanstage. The status and the stage's instructions/options are visible inrunsandinspect --json(and the TUI picker); the stage is inferred from the run's current stage unless named. The choice and feedback are recorded as the step's evidence (its output and derived verdict; the feedback is never an error diagnostic) and the run resumes alongedges[KEY]. An unknown option is refused with the valid list; a stage that is not waiting is refused; a repeated decision for an already-decided visit does not double-apply. The wait is persisted, sodecidestill works after a server restart.cancelandretrykeep working while waiting and after a decision; a human stage is never killed by a stage timeout ormax_visits(the shared 30-step run budget still applies).CLI_PREFIX run ... --json --wait --wait-timeout DURATION:--waitpolls until terminal and--wait-timeoutbounds only the client wait after admission, independent of per-request--timeout. A reached wait limit leaves the admitted run active; inspect it read-only withinspect RUN_ID --json, or keep waiting by rerunning--waitand reusing the original--request-key. Never start a duplicate run, and never expect an automatic replay or cancel of admitted work.CLI_PREFIX show NAME --version N --json: exact pinned definition, not whichever version happens to be latest now.CLI_PREFIX status --json: connection details; candidate addresses are not proof they are reachable from another device.CLI_PREFIX ping --json:pong.build.version,build.commit(full Git revision) andbuild.dirtyidentify the compiled server, not the client or its runtime checkout. A dirty build includes-dirty;commit: "unknown"ordirty: nullmeans metadata was unavailable. Older servers omitbuild. Never infer a deployed fix from source/skill contents. Native Go builds use embedded VCS metadata; Docker setup stamps build arguments. For an explicitly requested Docker rebuild,python3 scripts/build_metadata.pyresolves the current releases and stampsdocker compose buildfrom the checkout with the immutable digest; it does not restart/deploy. A plaindocker compose buildwithout that resolved digest cannot build the image. Plainpingalso displays server identity;CLI_PREFIX version --json(alias--version) identifies the client offline. Include both in sanitized bug reports. Two dirty builds at the same commit can differ. Docker setup verifies its started server identity against the build arguments before declaring setup complete.
Pipelines is a single-user first-version preview for supervised work, not a
multi-user service. Recommend a disposable repository for initial trials. Tool
permissions and same-path home mounts do not isolate valuable files. Never collect
raw credential/session databases or publish private inspect output as diagnostics.
In a source checkout, python3 scripts/smoke-first-install.py checks a disposable
source-built test binaries, draft/publication/transfer and restart persistence without
inference, notifications or deployment. It is not a live-provider or phone test.
That is a developer test fixture, not the native installer. End-user installation
runs natively through ./install by default (scripts/setup-native.py, with
--native equivalent); the Docker server + ntfy installation is explicit with
./install --docker.
In the TUI, Home advertises / omni search and : commands. Home / opens one
popup over pipelines, versions, runs, steps/visits, artifact names and stage
input/output/evidence text; each row shows a short type badge and a text match
shows a bracketed snippet, and a bare status word or date selects runs. The same
query is available as pipelines search QUERY [--type KIND], where --type
narrows the kind. Run Tab switches only
Details ↔ Graph. n/p select recorded visits, including repeated stages; d
focuses/toggles file changes, A opens the run's grouped artifacts view (name,
content type, size and URL per entry, PgUp/PgDn scroll so a trailing visit
that recorded none stays reachable, Enter/o to open the selected URL and
y to copy it), a selects the active/latest retained
configured-harness agent session's bounded, redacted tool evidence, D/:debug
opens fresh investigation, and P opens the pinned pipeline version. :quit
exits; q does not. Results are retained reports, not independently confirmed
achievements. Distinguish partial output from full success.
Retained stage file changes
Each newly executed visit compares a bounded, read-only workspace snapshot just
before execution with one after execution/cleanup. SQLite retains unified patches
and metadata in steps[].file_changes; snapshots themselves are transient memory.
The stage's File changes section highlights added/removed lines and hunk headers.
- The baseline is stage-start content, not HEAD/index. Pre-existing edits and earlier stages' changes are not attributed to this visit. Added/deleted/modified files, modes and binary-change metadata are recorded; renames are delete/add.
- Git scope is tracked and nonignored untracked files within the run directory; outside Git, regular workspace files. Symlinks/special files, Git internals, node_modules and common environment/key/credential/database paths are excluded. This is path-based protection, not secret-content scanning. Patches may still contain sensitive text; never publish them without approval.
- Limits: 10,000 entries, 1 MiB/file, 32 MiB/snapshot, five-second snapshot context, 5,000 combined text lines, 64 KiB/patch, 128 KiB serialized file evidence/visit. Missing/skipped/truncated captures must be described as partial/unavailable.
- Failed/canceled stages retain changes when execution returns; unconfirmed agent cleanup is partial. Crashes may have no after-capture. Historical visits say not recorded—never infer no edits from absent evidence.
- These are observed workspace changes, not exclusive authorship. Concurrent edits, changes outside scope, ignored new files and changes reverted before capture can be absent or mixed in. Capture failure does not change the declared run verdict.
Do not add model output fields, new stages or extra model calls merely to obtain file diffs: current capture is server-owned. Do not promise full files/rollback.
Retained stage artifacts
Every agent visit gets a per-run/step artifacts directory under the data
directory (<data>/artifacts/<run>/<step>/); the root is configurable and
defaults next to the database. The directory is passed as
PIPELINES_ARTIFACTS_DIR to a locally spawned Pi process and exported to the
OpenCode CLI where controllable, and it is stated in the agent prompt next to
Run directory: for every harness. OpenCode V2 tools run inside the shared serve
process, so there the prompt line, not the environment, is the reliable carrier.
The instruction is general: save images, screenshots, reports and logs there.
After the visit the server makes one bounded inventory and records each file's
name, size, content type (extension, sniff fallback) and a relative url; these
appear under each step's artifacts in inspect RUN_ID --json and the TUI's
A artifacts view, which groups them by stage and visit, marks images and shows
the recorded count, capture-pending/not-recorded wording and partial-capture
notes.
GET /artifacts/<run>/<step>/<name> serves the bytes (correct Content-Type and
Content-Length, HEAD, and Range/206/416 via http.ServeContent), refusing
traversal, absolute paths, symlink escapes and unknown runs/steps.
- Limits: 200 files, 8 MiB/file, 64 MiB total per visit; over-cap files are skipped and counted; symlinks and non-regular files are excluded.
- Artifacts are read-only evidence, not a gate: absent, unreadable, symlinked or oversized directories record nothing and never change the run verdict. The database keeps the recorded list, not file copies.
- Do not invent artifact files or claim a stage produced one without the recorded evidence; the server observes the directory after the stage.
Brokered OpenCode chat (unshipped)
The protocol declares the brokered chat_* surface (chat_sessions,
chat_create, chat_prompt, chat_messages, chat_interrupt, chat_permission,
chat_reply). The server dispatches chat_sessions, chat_prompt,
chat_messages, chat_interrupt and chat_reply, answers the declared
chat_create with an explicit protocol error because it is not yet brokered, and
pushes chat_event frames derived from OpenCode's global /api/event firehose,
with retained per-session/per-message state so a reconnecting client re-syncs
instead of replaying deltas.
chat_sessions is brokered but restricted: OpenCode's GET /api/session lists
every session on the machine, so the server filters it to the sessions recorded
in run_steps as owned by this server (deduped by session id), fills any missing
directory from the owning run's workspace, and fails closed to an empty list when
no ownership source is configured rather than exposing the operator's whole
environment. A session another client currently holds (an in-server claim,
released when the stream ends or the client disconnects) or that this server
records as an active run is marked busy so the phone can show it read-only; a
second client's chat_prompt for a held session is rejected before any OpenCode
call. chat_create remains unbrokered.
While a chat_prompt subscription is open the server also polls
GET /api/permission/request on a fixed small interval (first poll immediately) and
pushes one chat_permission frame per newly pending approval request, deduped by
request id so a repeated poll never re-pushes the same request. The app answers with
chat_reply, which the server forwards as
POST /api/session/{id}/permission/{requestID}/reply with
{"decision":"once"|"always"|"reject"}. A missing session/request id or an invalid
decision is rejected before any OpenCode call, and the poller stops when the stream
ends or the client disconnects.
The permission list and reply are implemented against the V2 HTTP API only: their
field names match the service's Permission.Request/Permission.Reply schemas
(verified against the installed service's /openapi.json), and the native v1 serve
translation rejects them explicitly instead of guessing an endpoint. Because this
build enables brokered chat only when the OpenCode connection exposes a URL-bearing
SSE endpoint (the owned v1 serve process), the approval flow is not reachable on the
backend that currently enables chat; on that backend the permission poller stops
quietly rather than reporting the documented limitation as a per-prompt error. This is
an internal, unshipped capability: no client consumes it, and on a V2 CLI
(opencode api, no owned serve) that endpoint is absent, so every dispatched chat_*
request returns a protocol error naming the missing configuration. The live
end-to-end approval flow has NOT been tested against a real OpenCode service; the wire
shapes and the exact reply path/body are covered by unit tests over an httptest
fixture, not against a live pending request. Do not describe brokered chat as a
finished or user-facing feature.
Debug and retry
Debug opens a new conversation with retained failure context, exact definition and session evidence; it does not resume implementation or grant permission to edit. Inspect first, identify evidence-supported causes, and say what remains unknown. Server-side session evidence is the latest bounded, credential-redacted evidence for up to the 3 most recent session-bearing steps; OpenCode V2 selects the latest assistant message with the same bounded readback as the final result, so a small final result is visible behind large prior tool content. Inspect the retained session itself when a preview is bounded or omitted; never assume a complete tool history from a preview. Retry is allowed for a failed run after cleanup stops: a fresh run/session starts at entry with the original input/directory/pinned version/edit scope. Previous side effects remain and can happen again. Never describe retry or cancellation as rollback, or resume/replay admitted tool work just to recover formatting.
Retained model-stage failures carry a fixed sanitized category. FINAL RESULT VALIDATION names the exact duplicate-field/syntax detail and byte offset for a
strictly invalid final result; HARNESS TRANSPORT covers request, active-session
poll and final-readback failures. A validation failure never re-prompts the
admitted tool-using stage: completed edits and tool side effects are retained and
not undone, so a full retry repeats work. An explicit read-only stage is the one
exception: a malformed result there mutates nothing, so it may be re-emitted up
to retries with its format_repairs evidence retained, and a post-admission
deadline or transport failure may be retried. A stage whose tools include
edit/shell, or that has no tools block (the legacy read/glob/grep/edit/shell
bundle), is never replayed. Neither category exposes provider/model bodies or
credentials.