f5bf8ada8a
# Conflicts: # internal/cli/commands.go # internal/pipeline/store.go
281 lines
18 KiB
Markdown
281 lines
18 KiB
Markdown
# System, storage and run evidence
|
|
|
|
Pipelines turns a reusable process into sequential stages with explicit handoffs,
|
|
outcome routing and retained evidence. A Go CLI/TUI connects to an execution server;
|
|
the server owns execution and SQLite history, OpenCode owns model/tool sessions,
|
|
and optional ntfy delivery supplies notifications. It is technology-neutral, not a
|
|
parallel/distributed task engine or a guarantee that model reports are correct.
|
|
|
|
## Lifecycle and location
|
|
|
|
- Drafts are editable Markdown plus JSON metadata. Publishing validates and creates
|
|
immutable versions; drafts remain retained. Keep context create-only/base-version
|
|
guards. The four generic examples (one-stage, two-stages, feedback-loop,
|
|
human-decision) are the template set; a brand-new database is seeded with them
|
|
once, and an existing or already-seeded database never is. Templates can be created directly with
|
|
an editable prefilled name or customized first. Editing a bundled repository
|
|
template changes no installed or published definition; only explicit authoring
|
|
then submission creates a new immutable version.
|
|
- Retire an unused pipeline with `pipelines archive NAME --yes` (no deletion exists):
|
|
it leaves lists and cannot run, its history stays, its name stays reserved, and
|
|
enabled schedules must be paused first; `unarchive` and `list --archived` undo/show it.
|
|
- A run pins its exact definition version, initial input and resolved directory.
|
|
The client invocation directory is the default, but files/tools execute on the
|
|
server. Docker needs that directory mounted at the same absolute path. A remote
|
|
client does not upload its repository or make host credentials available remotely.
|
|
- Host and Docker OpenCode credentials are separate. Preserve setup-selected model
|
|
inheritance; do not select a provider just because its ID appears in an example.
|
|
- Runtime caps are 30 stage steps and five visits to a stage. Runs in different
|
|
resolved workspaces execute concurrently; only a second run in a workspace an
|
|
active run is already changing is refused, naming the conflicting run and
|
|
advising you to wait or use a different workspace (for example another
|
|
checkout). Agent-session quiescence is scoped to the run: a run fails only when
|
|
one of its own sessions cannot be confirmed idle, so one run's active agent never
|
|
fails another's concurrent run. An active session left over by a restart with no
|
|
owning active run is an orphan and still refuses new work until interrupted.
|
|
Reusing that active run's request key reattaches idempotently.
|
|
`type=human` decision stages exist and pause the run (see `references/format.md`);
|
|
there are no parallel branches. Optional worktrees, full backup/restore,
|
|
automatic database sync and independent verifier stage types are future ideas,
|
|
not existing capabilities. Do not invent them in definitions.
|
|
- Services are native or Docker-based. With an existing Docker installation do not
|
|
start a competing native server. Server APIs are unauthenticated: trusted networks
|
|
only; never recommend public exposure. Tool scope is not a universal OS sandbox.
|
|
|
|
## Where data lives
|
|
|
|
SQLite retains pipelines/immutable versions, runs, recorded stage visits,
|
|
inputs/outputs, status/errors, audit transcripts, session ownership/references,
|
|
reported per-stage model usage (tokens/cost), retry links, schedules/state,
|
|
notification receipts and file changes. Not everything
|
|
is in that database: drafts/settings are client files; complete OpenCode sessions
|
|
and credentials use OpenCode storage; repository artifacts are files; ntfy has its
|
|
own storage. A pipeline ZIP is not a full backup of these components.
|
|
Never merge or sync live SQLite files using generic file synchronization. Do not
|
|
delete databases, sessions, volumes or credentials to repair a workflow.
|
|
|
|
## Inspect results first
|
|
|
|
Use the authoritative `CLI_PREFIX` from the prompt:
|
|
|
|
- `CLI_PREFIX search QUERY [--type KIND] [--limit N] --json`: one read-only query
|
|
across pipelines, versions, runs, steps, field text and artifacts. QUERY is free
|
|
text, a bare status word (`queued`/`running`/`succeeded`/`failed`/`canceled`/
|
|
`interrupted`) selecting those runs, or a lone date expression (`today`,
|
|
`yesterday`, `this week` — Monday 00:00 local — or `YYYY-MM-DD`) selecting the
|
|
runs created in that window; anything else stays free text. `--type` is the only
|
|
narrowing mechanism and takes one of `pipeline`, `version`, `run`, `step`,
|
|
`field`, `artifact`; an unknown kind is an error. Ranking is exact name (3) >
|
|
name substring (2) > body text (1), equal tiers by recency, and every hit
|
|
reports `score`, `matched_field` and `recency`. The cap defaults to 50 and clamps
|
|
to 200; every kind scans at most 2000 rows and field scanning stops at 2000
|
|
steps / 1 MiB and reports
|
|
`truncated`/`scanned_steps`/`scanned_bytes`/`cap`/`scan_bound_bytes`. Targets are
|
|
stable: `pipeline:<id>`, `pipeline:<id>:v<version>`, `run:<id>`,
|
|
`run:<id>:<sequence>`, `run:<id>:<sequence>:<field>`, or an artifact URL.
|
|
- `CLI_PREFIX runs --json`: run history (optional `--pipeline NAME` scope). Each run
|
|
carries a bounded, single-line `title` saying what it was for, set explicitly with
|
|
`CLI_PREFIX run ... --title "..."` or generated from its prompt by the shared
|
|
summariser on the configured default harness. It is never empty: when generation is
|
|
unavailable or times out the server derives the prompt's first line, and
|
|
`title_source` (`model`, `fallback`, or `explicit`) plus `title_model` record how it
|
|
was produced, so a diagnosis can tell them apart. Human `runs` and `inspect` show the
|
|
title beside the pipeline name and version, so two runs of one pipeline are
|
|
distinguishable; `CLI_PREFIX title RUN_ID "..."` renames a run after the fact
|
|
without changing its input or steps. The stored prompt is unchanged, and
|
|
listing/renaming never calls a model.
|
|
- `CLI_PREFIX inspect RUN_ID --json`: selected run, visits and retained evidence.
|
|
Each model step carries the usage its harness reported (`usage.tokens_total`,
|
|
`tokens_in`/`tokens_out` only when `split_known`, `cost_usd` only when
|
|
`cost_known`, plus the model the step actually ran with); the run carries the
|
|
totalled `usage` over those stages, so a mixed-model run shows what each stage
|
|
cost. An unreported cost or split is shown as unavailable, never as zero and
|
|
never inferred from output length or transcript text; a historical or
|
|
pre-upgrade run simply omits `usage`.
|
|
Exact stage outputs are authoritative; a transcript over the 32 KiB retained
|
|
bound is a valid, credential-redacted preview envelope (with preserved stage
|
|
limits, failure/format diagnostics and the retained session pointer), not the
|
|
complete tool history.
|
|
- `CLI_PREFIX decide RUN_ID [STAGE] --option KEY [--feedback TEXT] [--json]`: answer
|
|
a run `waiting` on a `type=human` stage (`references/format.md`).
|
|
- `CLI_PREFIX run ... --json --wait --wait-timeout DURATION`: `--wait` polls until
|
|
terminal and `--wait-timeout` bounds only the client wait after admission,
|
|
independent of per-request `--timeout`. A reached wait limit leaves the admitted run
|
|
active; inspect it read-only with `inspect RUN_ID --json`, or keep waiting by
|
|
rerunning `--wait` and reusing the original `--request-key`. Never start a duplicate
|
|
run, and never expect an automatic replay or cancel of admitted work.
|
|
- `CLI_PREFIX show NAME --version N --json`: exact pinned definition, not whichever
|
|
version happens to be latest now.
|
|
- `CLI_PREFIX status --json`: connection details; candidate addresses are not proof
|
|
they are reachable from another device.
|
|
- `CLI_PREFIX ping --json`: `pong.build.version`, `build.commit` (full Git revision)
|
|
and `build.dirty` identify the compiled **server**, not the client or its runtime
|
|
checkout. A dirty build includes `-dirty`; `commit: "unknown"` or `dirty: null`
|
|
means metadata was unavailable. Older servers omit `build`. Never infer a deployed
|
|
fix from source/skill contents. Native Go builds use embedded VCS metadata;
|
|
Docker setup stamps build arguments. For an explicitly requested Docker rebuild,
|
|
`python3 scripts/build_metadata.py` resolves the current releases and stamps
|
|
`docker compose build` from the checkout with the immutable digest; it does not
|
|
restart/deploy. A plain `docker compose build` without that resolved digest cannot
|
|
build the image.
|
|
Plain `ping` also displays server identity; `CLI_PREFIX version --json` (alias
|
|
`--version`) identifies the client offline. Include both in sanitized bug reports.
|
|
Two dirty builds at the same commit can differ. Docker setup verifies its started
|
|
server identity against the build arguments before declaring setup complete.
|
|
|
|
Pipelines is a single-user first-version preview for supervised work, not a
|
|
multi-user service. Recommend a disposable repository for initial trials. Tool
|
|
permissions and same-path home mounts do not isolate valuable files. Never collect
|
|
raw credential/session databases or publish private inspect output as diagnostics.
|
|
In a source checkout, `python3 scripts/smoke-first-install.py` checks a disposable
|
|
source-built test binaries, draft/publication/transfer and restart persistence without
|
|
inference, notifications or deployment. It is not a live-provider or phone test.
|
|
That is a developer test fixture, not the native installer. End-user installation
|
|
runs natively through `./install` by default (`scripts/setup-native.py`, with
|
|
`--native` equivalent); the Docker server + ntfy installation is explicit with
|
|
`./install --docker`.
|
|
|
|
## Retained stage file changes
|
|
|
|
Each newly executed visit compares a bounded, read-only workspace snapshot just
|
|
before execution with one after execution/cleanup. SQLite retains unified patches
|
|
and metadata in `steps[].file_changes`; snapshots themselves are transient memory.
|
|
The stage's File changes section highlights added/removed lines and hunk headers.
|
|
|
|
- The baseline is stage-start content, **not HEAD/index**. Pre-existing edits and
|
|
earlier stages' changes are not attributed to this visit. Added/deleted/modified
|
|
files, modes and binary-change metadata are recorded; renames are delete/add.
|
|
- Git scope is tracked and nonignored untracked files within the run directory;
|
|
outside Git, regular workspace files. Symlinks/special files, Git internals,
|
|
node_modules and common environment/key/credential/database paths are excluded.
|
|
This is path-based protection, **not secret-content scanning**. Patches may still
|
|
contain sensitive text; never publish them without approval.
|
|
- Limits: 10,000 entries, 1 MiB/file and 32 MiB/snapshot of retained text,
|
|
256 MiB/hashed file, five-second snapshot context, 5,000 combined text lines,
|
|
64 KiB/patch, 128 KiB serialized file evidence/visit. Text over the 1 MiB or
|
|
32 MiB budget is still compared by hash, and in Git a path the snapshot could
|
|
not compare is listed when its `git status` entry changes; such entries carry
|
|
`no_patch: true` (and `truncated`) with no patch, and the visit is `partial`.
|
|
`no_patch_reason` says why (`binary`, `file_size`, `budget`, `lines`,
|
|
`evidence_size`, `git_status`, `baseline`). In Git only dirty/untracked files
|
|
spend the text budget; clean files' start content is read back from their
|
|
verified stage-start index blob, so stage commits do not move the baseline.
|
|
Missing/skipped/truncated captures must be described as partial/unavailable;
|
|
`inspect` adds a run-level `warnings` entry naming every partial/unavailable visit.
|
|
- Failed/canceled stages retain changes when execution returns; unconfirmed agent
|
|
cleanup is partial. Crashes may have no after-capture. Historical visits say
|
|
not recorded—**never infer no edits from absent evidence**.
|
|
- These are observed workspace changes, not exclusive authorship. Concurrent edits,
|
|
changes outside scope, ignored new files and changes reverted before capture can
|
|
be absent or mixed in. Capture failure does not change the declared run verdict.
|
|
|
|
### Server skip fallback (`skipped` edge)
|
|
|
|
A non-human stage that declares a `skipped` edge is completed by the server, with
|
|
no model call, and routed along `skipped` only when **every** earlier visit of the
|
|
run recorded `file_changes` with status `complete`, none recorded any changed file,
|
|
and none captured an image artifact. The step's verdict reason starts
|
|
"Server fallback skip (no model run):" with `reason_source: fallback`. Any changed
|
|
file (no path or extension is treated as "not UI"), any partial/unavailable or
|
|
missing capture (crash, historical row, human decision, checkpoint-retry prefix)
|
|
runs the model, which may still answer `skipped` itself. `auto_skip=false` in the
|
|
stage's `configs` disables the fallback for that stage. Never describe a fallback
|
|
skip as a model judgement, and never treat it as evidence that nothing changed.
|
|
|
|
Do not add model output fields, new stages or extra model calls merely to obtain
|
|
file diffs: current capture is server-owned. Do not promise full files/rollback.
|
|
|
|
## Retained stage artifacts
|
|
|
|
Turn files (`input.md`, `memory.md`, `history.md`, `output.md`, `private/`) and
|
|
the run-level `public/` are described in `references/format.md`. They live under
|
|
`<data>/artifacts/<run>/stages/<seq>-<stage>/`; the root is configurable and
|
|
defaults next to the database. The server owns `input.md`, `memory.md`,
|
|
`history.md` and `output.md`.
|
|
|
|
After the visit the server makes one bounded inventory of the turn and public
|
|
directories and records each file's name, size, content type (extension, sniff
|
|
fallback) and a relative url; these appear under each step's `artifacts` in
|
|
`inspect RUN_ID --json` and the TUI's `A` artifacts view, which groups them by
|
|
stage and visit, marks images and shows the recorded count,
|
|
capture-pending/not-recorded wording and partial-capture notes. Public entries
|
|
carry a `public/` prefix. To see what a late stage was given, read its
|
|
`memory.md`; `history.md` is the full record it could consult.
|
|
`GET /artifacts/<run>/<step>/<name>` serves the bytes (correct Content-Type and
|
|
Content-Length, HEAD, and Range/206/416 via `http.ServeContent`), refusing
|
|
traversal, absolute paths, symlink escapes and unknown runs/steps. A pre-turn
|
|
`<run>/<step>/` path stays readable for runs recorded before the file protocol
|
|
landed.
|
|
|
|
- Limits: 200 files, 8 MiB/file, 64 MiB total per visit; over-cap files are
|
|
skipped and counted; symlinks and non-regular files are excluded.
|
|
- Artifacts are read-only evidence, not a gate: absent, unreadable, symlinked or
|
|
oversized directories record nothing and never change the run verdict. The
|
|
database keeps the recorded list, not file copies.
|
|
- Do not invent artifact files or claim a stage produced one without the recorded
|
|
evidence; the server observes the directory after the stage.
|
|
|
|
### Session export
|
|
|
|
- When an OpenCode-harness model stage finishes, the server exports its real
|
|
session read-only (v1 `opencode export <id>`, v2 `opencode session export <id>`)
|
|
and writes exactly one file, `session.html`, into the same turn artifacts
|
|
directory; it lists and serves like any other artifact.
|
|
- The `session.md` export is retired: nothing writes it any more and it is never
|
|
presented. A run recorded before the HTML-only change still lists `session.md`
|
|
among its stored artifacts (dated evidence kept on disk, not deleted); every
|
|
listing — app, TUI, web spike — hides it rather than showing it beside the html.
|
|
- The source read is capped at 8 MiB and the rendered file at 8 MiB, matching the
|
|
ordinary per-artifact file bound; a cut source or a cut rendered file carries a
|
|
visible truncation banner at both ends and `session_export.truncated: true`, and
|
|
a truncated export is never called whole.
|
|
- The step records `session_export` (`session_id`, `backend`, `source`, the html
|
|
file name, `html_bytes`, `messages`, `truncated`, `diagnostic`) in
|
|
`inspect RUN_ID --json`. A record written before the change still carried
|
|
`markdown` and `markdown_bytes`; readers ignore those fields.
|
|
- A failed export, or a Pi stage with no native export path, records a diagnostic
|
|
(Pi: `source: "unavailable"`) and never fails the run or changes the stage result.
|
|
An existing `session.html` is never overwritten.
|
|
|
|
### Operator-only retention
|
|
|
|
Run history and its artifacts are never pruned automatically. The operator-only
|
|
`pipelines retention report` (strictly read-only) and `pipelines retention apply`
|
|
(removal after explicit approval) are the only commands that remove them; no
|
|
server startup, scheduler, `list_runs`, `stats`, `runs` or `inspect` path calls
|
|
retention, and there is no background goroutine, timer or startup hook. A policy
|
|
selects by rolling UTC age (`--older-than`), newest-N count (`--keep`), or their
|
|
intersection. Runs that are queued/running, waiting on a person, running a cleanup
|
|
agent, or referenced by a retained run's retry provenance are protected and
|
|
reported with a reason; their artifacts are kept. Removing a run removes its
|
|
`<root>/<run>` artifact directory and reclaims database space with one `VACUUM`
|
|
and one WAL checkpoint, reporting measured bytes. Pipeline definitions and their
|
|
immutable versions are never touched. The commands are local, require an existing
|
|
`--db`, and need no running server.
|
|
|
|
## Debug and retry
|
|
|
|
Debug opens a new conversation with retained failure context, exact definition and
|
|
session evidence; it does not resume implementation or grant permission to edit.
|
|
Inspect first, identify evidence-supported causes, and say what remains unknown.
|
|
Server-side session evidence is the latest bounded, credential-redacted evidence
|
|
for up to the 3 most recent session-bearing steps; OpenCode V2 selects the latest
|
|
assistant message with the same bounded readback as the final result, so a small
|
|
final result is visible behind large prior tool content. Inspect the retained
|
|
session itself when a preview is bounded or omitted; never assume a complete tool
|
|
history from a preview.
|
|
Retry is allowed for a failed run after cleanup stops: a fresh run/session starts
|
|
at entry with the original input/directory/pinned version/edit scope. Previous
|
|
side effects remain and can happen again. Never describe retry or cancellation as
|
|
rollback, or resume/replay admitted tool work just to recover formatting.
|
|
|
|
Retained model-stage failures carry a fixed sanitized category. `FINAL RESULT
|
|
VALIDATION` means `output.md` had a missing, duplicate, empty or undeclared
|
|
outcome (`references/format.md`); `HARNESS TRANSPORT` covers request, session-poll
|
|
and readback failures. Completed edits and tool side effects are retained, not
|
|
undone, so a full retry repeats work. A failure before any prompt is admitted is
|
|
retried within `retries` for any stage; a post-admission deadline or transport
|
|
failure is retried only on an explicit read-only stage; a stage with `edit`/`shell`
|
|
(or no tools block) is never replayed. Neither category exposes provider/model
|
|
bodies or credentials.
|