stage turns: replace JSON results with file-based markdown turns

Each model visit now gets <run>/stages/<seq>-<stage>/ with input.md,
memory.md, output.md, private/ and a shared run-level public/. The model
reads the two input files, follows the stage's instructions verbatim and
returns complete markdown; only the # outcome section is parsed.

- add internal/turn layout, render and output parsing
- write input.md/memory.md before a visit and output.md from the final
  message; inventory all turn files as stage artifacts
- accumulate completed turns and system-written public file additions in
  memory, including across retries and resume
- remove the JSON envelope, strict field validation, line format and the
  repair ladder for model stages
- republish pipeline authoring prompts and docs in the new style

Checks: go build ./..., go vet ./..., go test -count=1 ./..., gofmt -l .
This commit is contained in:
2026-10-19 22:27:44 +01:00
parent b2fd614ba4
commit e37b864c40
66 changed files with 2199 additions and 3028 deletions
+25
View File
@@ -564,6 +564,31 @@ appear when persisted (currently after generation). This does not attach a live
harness client or stream its messages. Supporting live attachment consistently
across the shared and privately owned harness backends is deliberately deferred.
## Stage turn files
Each model-stage visit receives a file-based turn under the run's artifact
directory. `pipelines inspect <run-id> --json` and the run inspector list these
files as that step's artifacts:
```
<artifacts>/<run>/stages/01-implementer/input.md
<artifacts>/<run>/stages/01-implementer/memory.md
<artifacts>/<run>/stages/01-implementer/output.md
<artifacts>/<run>/stages/01-implementer/private/
<artifacts>/<run>/public/
```
`input.md` is the run prompt on the first visit and the previous visit's output
body (without its outcome section) afterwards. `memory.md` is the accumulated markdown of completed turns plus the
files each turn added to the shared `public/` directory. A text-only stage (an
explicit empty `tools` block) receives both files' contents in its prompt
instead, because it has no file-reading tool. The model returns the complete
`output.md` markdown; only its `# outcome` section is interpreted by the server.
Everything else is free-form and preserved under the step's `markdown` output.
`private/` belongs to its own visit, while `public/` is shared by every stage in
the run. Artifacts are served from `/artifacts/<run>/<step>/<name>`; shared files
appear with a `public/` prefix.
## Investigating runs with a configured harness
On the runs list or inspector, press `D` or use `:debug`; CLI equivalent:
+6 -8
View File
@@ -93,14 +93,12 @@ fire, which silently makes the reviewer's `rejected` control unreachable. The ru
already counts the visits and stops at the cap, so neither loop needs new server
code and no model counts visits.
Add a fenced `memory` map such as
`{"task":"input.prompt","checks":"planner.checks"}` to select reviewer
input without previous verdicts. Implementers can select
`{"task":"input.prompt","plan":"planner.plan","feedback":"review.findings?"}`.
`?` means omit if unavailable. A memory block replaces the previous-stage input;
without it, existing previous-output handoffs are unchanged. Latest completed
outputs are shared only within this run. Read provenance and routing overrides
are retained in SQLite and exposed in run inspection/metadata.
The previous turn's output and the run's accumulated `memory.md` are handed to
each model stage as files; there is no `memory` map or field selection. Put the
facts later stages need in free-form headings. Only the `# outcome` section is
interpreted. Latest completed outputs are shared only within this run. Read
provenance and routing overrides are retained in SQLite and exposed in run
inspection/metadata.
These controls enforce counts/field selection, not semantic independence or every
prose guideline. AI-assisted guidelines validation is not implemented.
+91 -190
View File
@@ -1,207 +1,103 @@
# Model results and scoped tools
# Model turns, results and scoped tools
Every model result must be one object containing exactly `outcome` and `output`.
`outcome` must be a declared edge; `output` must be an object. Duplicate keys
(including nested output keys), missing fields, unknown fields and invalid
outcomes are rejected. Free-form `output` contract prose is not a JSON Schema.
Model stages are file-based. Each visit reads `input.md` and `memory.md` and
returns the complete markdown for `output.md`. Only the outcome section is
interpreted; the rest is free-form text for the next stage and the operator.
The native stable OpenCode adapter first checks the complete format POST/readback
contract in a separate retained, deny-all, no-inference session. Compatible native
backends request `format.type=json_schema` with this envelope and `retryCount=0`.
OpenCode 1.18.35 accepts a format but rejects its stored messages with
`Expected OutputFormatJsonSchema` (even explicit `text` fails). For that specific
incompatibility, requests omit `format`; strict local validation remains mandatory.
Other probe failures, such as authentication or network errors, are not silently
downgraded. Returned structured results remain preferred when available, and tool
transcripts are preserved. V2's published prompt/generate API does not accept a schema-output
field; that adapter retains local validation instead of sending unsupported
fields or claiming provider-side schema enforcement.
## Turn files
Output-contract prose describes the map **inside** `output`; it never overrides
the runtime `{outcome, output}` envelope. Printed DSML or other tool-call markup
is not evidence of a real tool invocation and cannot be accepted as a result.
Flattened output fields are rejected, not guessed into a new envelope. Invalid
responses are retained even when they are not eligible for syntax repair.
## Line-oriented result format
Model and agent stages also accept a line-oriented result format. The response is
a markdown-ish document whose first non-empty line is the outcome, followed by
sections introduced with `# <heading>`.
Every model visit gets its own directory under the run's artifact root:
```
<artifacts>/<run>/stages/01-implementer/input.md
<artifacts>/<run>/stages/01-implementer/memory.md
<artifacts>/<run>/stages/01-implementer/output.md
<artifacts>/<run>/stages/01-implementer/private/
<artifacts>/<run>/public/
```
- `input.md` is the run prompt on the first visit and the previous visit's body
after that, under a `# Previous turn output` heading.
- `memory.md` is one `## <stage> · visit <n>` section per completed turn: the
turn body followed by a system-written, sorted list of the files that turn
added to `public/`.
- `private/` belongs to one visit and is never handed to a later stage.
- `public/` is shared by every stage in the run.
The server writes `input.md` and `memory.md` before the model runs and
`output.md` after it. All three are inventoried as step artifacts, together with
`private/` and the files in `public/`. See
[stage artifacts](stage-artifacts.md) for capture limits and URLs.
A text-only stage (an explicit empty `tools` block) cannot read files, so the
server embeds the contents of `input.md` and `memory.md` in its prompt instead.
A stage with tools receives the paths and reads them itself.
## Output contract
`output.md` must contain exactly one outcome heading. The heading may be any ATX
level and is followed by the first non-blank line, which must be exactly one of
the stage's outgoing edge names:
```md
## outcome
success
# output
{"task": "x"}
# Result
# findings
- prose, code fences, quotes and embedded JSON need no escaping
The rest of the output is free-form markdown.
```
Only the first non-empty line and a `# output` section are required. The body of
`# output` must be a JSON object; it becomes the stage's `output` map. Every other
heading becomes an evidence field keyed by the normalised heading text (lowercased,
spaces to underscores). Extra sections are stored verbatim and are best-effort: an
empty heading, an odd heading, or a very large section must never fail the stage.
If the outcome is missing, not one of the declared edges, or `# output` is absent,
the stage fails with `FINAL RESULT VALIDATION` and still retains whatever sections
were salvaged.
Matching ignores case and surrounding whitespace and returns the canonical edge
spelling. Headings and words inside fenced code blocks are ignored, so a code
sample cannot add a false outcome. Missing, duplicate, empty or undeclared
outcomes fail the stage. The parser never guesses, repairs the text, or looks for
any other section.
The parser decides between the legacy JSON envelope and the line-oriented format by
precedence. If the response's first non-empty character is `{` after stripping an
optional surrounding markdown fence, it keeps the unchanged JSON path. If a `# `
heading appears before any `{`, it uses the line-oriented path. This preserves all
existing JSON responses, including fenced JSON or short prose prefixes, while
allowing new line-format responses to coexist.
Everything after the outcome value line is retained as the turn body under the
single `markdown` key of the internal output map. No JSON envelope, output
schema, field selection, `output_contract`, or format-repair pass exists for
model stages. Prompts should ask for free-form headings, never JSON fields.
The full raw response is written to the stage's artifacts directory as
`raw-output.md` before parsing, so the closest record of what the model produced is
always retained and appears in the step's artifact inventory automatically. The
parse status is recorded in the step's output map under the reserved key
`result_parse`: `parsed` (both required parts OK), `partial` (required parts OK but
some extra section unusable), or `failed` (a required part missing or invalid).
Human and ntfy stages are unchanged and do not use turn files or the outcome
heading.
## Format recovery is not execution retry
## Prompt shape
Parser errors retain their diagnostic and byte offset: for example, `missing
closing brace '}'`, an unterminated string, or an invalid character. A valid but
schema-invalid result is not reformatted to invent a missing field or verdict.
The server builds the prompt from the stage's own instructions plus runtime
facts:
Format recovery has a separate bounded budget, even with `retries=0`:
- the paths to `memory.md`, `input.md` and `output.md` (or their contents for a
text-only stage);
- the stage prompt verbatim;
- the worked outcome example and the complete allowed outcome list;
- the execution time and timezone;
- a granted resource target, when one exists;
- the writable `private/` and shared `public/` artifact directories, when they
exist;
- the authoritative permitted-tools line and the optional permitted shell
commands.
1. A deterministic local pass fixes only the structural error that is derivable
from the payload itself, in order: append up to eight missing closing
delimiters when the existing response consists of complete JSON tokens;
remove a single unambiguous trailing comma before a closing `}`/`]`; or
insert exactly the one comma/colon the parser expected at its reported byte.
Each candidate must pass lossless syntax validation and the full envelope
validation. No model call. An ambiguous payload - an unescaped quote or colon
inside a string value, or a stray delimiter with no single completion - is
never guessed at and falls through.
2. If the local pass cannot repair it, make at most one fresh, deny-all, retained
text-generation request with only the malformed response and precise parsing
error. The original implementing/reviewing agent is never re-prompted. The
formatter's deadline is derived from the payload size, the stage timeout and
the attempt's own remaining budget rather than a fixed short constant, so a
large or slower result is not guaranteed to time out; it stays within the
stage deadline (clamped to `remaining` minus a margin), is floored for ordinary
payloads, and keeps the 32 KiB input/output bound.
3. Accept only insertion of missing commas/colons or trailing closing delimiters,
with at most 16 insertions. Every existing string, number, literal and
structural delimiter must remain verbatim and in order. Validate the full
result again. Changed approval, outcome, evidence, reordered data, deleted
fields, incomplete strings and fabricated values are rejected.
The model returns the complete `output.md` markdown as its final message. The
server writes that file and parses only the outcome.
Every terminal path records an explicit recovery status: `repaired` (a fully
lossless local or formatter recovery), `partial` (the candidate is valid JSON but
still fails envelope validation, so it is never returned as a completed result),
or `unrecoverable` (nothing recovered, or the formatter errored or timed out).
The unrecoverable path retains the bounded original payload together with the
precise parse error and its byte offset, and the status is included in the
`FINAL RESULT VALIDATION` diagnostic so an operator can see exactly what the model
emitted.
## Failure diagnostics and retries
If recovery fails, report the original precise error and repair failure; never
rerun project edits, commands, or the entire stage to repair final formatting.
Configured execution retries remain for retryable text-generation transport
errors; an explicit read-only stage may additionally re-emit a malformed final
result and retry a post-admission deadline or transport failure up to `retries`.
A failure to reach the harness before any prompt or tool call is admitted
(connecting, or creating the session) is a distinct reach failure: nothing was
admitted, so it is retried within `retries` for any stage, mutating or not, with
a fresh session and no replayed prompt or tool call. A post-admission readback,
active-session poll or agent-execution failure is retried only on an explicit
read-only stage. A stage whose tools include `edit`/`shell`, or that has no tools
block (the legacy read/glob/grep/edit/shell bundle), never has admitted tool work
replayed. The
deterministic local pass and the fresh deny-all, tool-free formatter session never
re-prompt or replay the admitted stage session, so a shell-enabled review stage
gains malformed-result recovery without any replay of admitted work.
Repair records include the original/candidate response, parsing diagnostic,
method, recovery status, byte offset and acceptance flag in the bounded transcript
preview. `pipelines inspect <run-id> --json` and the TUI evidence section expose
these records.
Retained stage failures use two fixed, sanitized categories:
OpenCode CLI failures include the HTTP status when supplied, process exit code,
and a recognized safe error category (such as session not found or service
unavailable). Arbitrary stderr, provider messages and response bodies are not
copied into the diagnostic. A failed V2 active-session poll (or native V1 `/wait`)
is not evidence that the agent prompt failed to start, and does not trigger replay
of stage work. When the context expires while a request is still in flight, the
error keeps both the sanitized transport detail (operation/status/exit) and
`errors.Is(err, context.DeadlineExceeded)`, so a deadline that fired during a hung
transport call is not replaced by a bare context error; a deadline measured at the
attempt boundary with no transport in flight stays the stage overrun it is.
Native `StructuredOutputError` is named directly, rather than reduced to a generic
command failure, and an unrecognized provider execution-error name is reported as
a bounded, sanitized identifier plus any 4xx/5xx status instead of a blanket
summary (a name containing arbitrary bytes stays the generic message). Native HTTP
evidence records the actual HTTP operation/status; an `exit_code` of -1 there is
explicitly marked as not a subprocess exit. Cleanup failures preserve the original
generation error and active ownership.
- **FINAL RESULT VALIDATION** for an invalid turn output: missing, duplicate,
empty or undeclared outcome, or an unwritable `output.md`. Completed edits and
tool side effects are retained; a full retry repeats work.
- **HARNESS TRANSPORT** for request, active-session poll and final-readback
failures. A pre-admission reach failure (connecting or creating the session)
replays nothing and is retried on any stage. A post-admission deadline or
transport failure is retried only on an explicit read-only stage. A stage with
`edit`/`shell`, or with no tools block (the legacy read/glob/grep/edit/shell
bundle), never replays admitted tool work.
For existing native sessions with format-bearing messages, the HTTP readback
workaround is read-only CLI `opencode export`. The exported session ID must match
the requested ID. Nothing is rewritten or removed from the database/history.
Debug receives the original messages plus a note identifying that recovery path.
## Failure diagnostics and confirmed limits
Retained stage failures distinguish two fixed, sanitized categories in the step
diagnostic:
- **FINAL RESULT VALIDATION** for a strictly invalid final result, with the stage
key and the precise parser detail (for example `duplicate JSON field "score"` or
`invalid character 'z' after object key:value pair at byte N`). It states that
completed edits/tool side effects are retained and not undone, and that a full
retry repeats work.
- **HARNESS TRANSPORT** for request, V2 active-session poll and final-readback
failures, including the sanitized `malformed or incomplete` readback message.
A single classifier covers the two transport shapes: a pre-admission reach
failure (connecting or creating the session) and a post-admission failure
(readback/poll/agent execution). Both share the one `HARNESS TRANSPORT` label;
only the pre-admission shape is retried on a mutating stage, because it replays
nothing.
A structured-output validation failure is never reported as a harness transport
failure, and vice versa. Neither label includes a provider response body, prompt,
credential or arbitrary model content. Duplicate nested fields are rejected by
strict parsing rather than merged or last-wins, and only a lossless insertion of
missing commas/colons or trailing closers is repaired, so an unknown evidence or
verdict value can never become `success`, and admitted tool work on a mutating or
legacy-bundle stage is never replayed (only an explicit read-only stage may
re-emit a malformed result within `retries`).
Confirmed ordinary-endpoint failures (2026-10-11): an all-history readback returned
exactly 262144 bytes of incomplete JSON (`invalid agent messages`); a reviewer
result was rejected for a duplicate nested `performance_review` field; and an
onboarding visual-review result failed with `invalid character 'z' after object
key:value pair at byte 3968`, where unsafe lossless repair was correctly rejected.
A shell-enabled `ui_reviewer` stage (run 55, pipeline 9 v6, 2026-10-15) failed with
`invalid character ':' after object key:value pair at byte 7837`, where the
isolated formatter hit its own 30-second deadline. The malformed payload was
retrieved read-only with `pipelines inspect 55 --json` and reproduced the exact
byte offset 7837. It is an unescaped quote/colon inside a string value - an
ambiguous case - so the deterministic pass correctly reports it unrecoverable
rather than guessing, while the stage-timeout-proportional derived deadline
(floored, capped and clamped to the remaining attempt budget) removes the
fixed-30-second-deadline failure mode for larger or slower payloads. These are
source-level findings covered by mocked
`internal/stage` fixtures. An updated checkout or installed skill is **not**
evidence that a user's running server contains the fix; deploying/restarting is a
separate authorized action.
OpenCode V2's published prompt/generate API does not accept a structured-output
field, so a prose `output_contract` does **not** enforce structured output: V2
results are validated locally by strict parsing only, and no unsupported V2 format
field is sent. A review-only fresh execution that could re-derive a failed verdict
without replaying the admitted tool-using stage's edits is a future improvement and
is **not implemented**. Two items remain unresolved and must not be reported
complete: a macOS fixture path-spelling mismatch (`/var/folders` versus
`/private/var/folders` are the same real directory) in
`scripts/test_setup_harness.py`, and onboarding UI visual approval.
A validation failure is never reported as harness transport, and vice versa.
Neither diagnostic includes a provider response body, prompt, credential or
arbitrary model content.
## Optional machine-readable restrictions
@@ -230,17 +126,22 @@ the normal defaults. Null lists, unknown keys, duplicate keys/resources, path
traversal, undeclared tools and unrestricted shell alongside restricted edits
are rejected. At most 20 resources per action are accepted.
Artifact access is separate and server-derived: a visit may write only its own
`private/` directory and the run-level `public/` directory. Those are the only
paths added to the permission rules and named in the prompt. The turn directory
itself is server-owned, so the model never writes `input.md`, `memory.md` or
`output.md` directly.
OpenCode checks scanner-produced shell command resources, not necessarily the
entire submitted script. A compound script consisting only of individually
allowed commands may pass; this is not a raw-script or shell-syntax firewall.
For a deliberately Go-only stage, explicitly declare `edit:["*.go"]` and the
exact allowed verification commands. Without a machine policy, a sentence such
as “approved Go files only” is **not** an enforced permission boundary. Defaults
remain language-neutral: this does not restore a global Go-only restriction or
require users to enumerate files for ordinary runs. Existing immutable definitions
are not rewritten; publish a new version to add intentional restrictions.
as "approved Go files only" is **not** an enforced permission boundary. Defaults
remain language-neutral. Existing immutable definitions are not rewritten;
publish a new version to add intentional restrictions.
Tool permissions are not an OS sandbox. An approved command runs project code
with the host user's authority and can have filesystem/network side effects.
Do not grant shell access if stronger filesystem isolation is required.
with the host user's authority and can have filesystem/network side effects. Do
not grant shell access if stronger filesystem isolation is required.
+60 -20
View File
@@ -1,34 +1,55 @@
# Stage turns
Stage turns are file-based. Each visit gets its own directory under the run:
Stage turns are file-based. Each model-stage visit gets its own directory under
the run's artifact root:
```
<run>/stages/01-implementer/input.md
<run>/stages/01-implementer/memory.md
<run>/stages/01-implementer/output.md
<run>/stages/01-implementer/private/
<run>/public/
<artifacts>/<run>/stages/01-implementer/input.md
<artifacts>/<run>/stages/01-implementer/memory.md
<artifacts>/<run>/stages/01-implementer/output.md
<artifacts>/<run>/stages/01-implementer/private/
<artifacts>/<run>/public/
```
`internal/turn.Layout` derives those paths deterministically (the sequence is
zero-padded to two digits). `Paths.Ensure` creates the turn, private and public
directories idempotently; it never deletes or rewrites existing files.
The first turn's `input.md` is the run prompt. A later turn's input is the
previous turn's output under a `# Previous turn output` heading. Memory is
plain markdown: one `## <stage> · visit <n>` section per completed turn, with
that turn's output verbatim followed by a system-written list of files added to
`public/`. List entries are relative to the run directory, sorted, and unique.
Input, memory and output are stored per turn, and private artifacts stay in the
turn's `private/` directory while shared artifacts go to the run-level
`public/` directory.
`input.md` is the run prompt on the first visit and the previous visit's output
body (without the outcome section) under a `# Previous turn output` heading
afterwards. `memory.md` is plain
markdown: one `## <stage> · visit <n>` section per completed turn, with that
turn's body followed by a system-written list of files it added to `public/`.
List entries are relative to the run directory, sorted, and unique.
The server writes `input.md` and `memory.md` before the model runs. The model
returns the complete markdown for `output.md`; the server writes that file from
the final message and then parses it. This is deliberate rather than a model
file write: a text-only stage (an explicit empty `tools` block) has no file
tool, and every harness then shares one result path. The prompt says the server
writes the file, and the artifact permission rules keep `input.md`, `memory.md`
and `output.md` server-owned. A text-only stage also has no file-reading tool,
so the server embeds both input files in its prompt instead. The turn files are
inventoried as step artifacts alongside `private/` (visible only to its own
visit) and `public/` (shared by every stage and listed with a `public/` prefix).
A selected-stage retry appends the complete merged retry input, including the
operator's `--input-file` values, to the selected visit's `input.md` under a
`# Retry input` heading. The step row alone is not the model's contract.
The public-file list is rebuilt from each step's persisted artifact inventory on
resume, so a restarted server does not print `_(none)_` for a turn that really
added files. A retry checkpoint also records each prefix turn's exact additions;
the new run copies those files from the source attempt's `public/` into its own
artifact root before the first visit, so memory names files the retry can read.
Files written by a failed visit are not carried or attributed to a later turn.
## Outcome-only output contract
A turn's `output.md` must contain exactly one outcome section and may contain any
markdown after it. The outcome section is an ATX heading named `outcome` (any
level) followed by the first non-blank line, which must be exactly one of the
stage's allowed outcome words:
markdown before or after it. The outcome section is an ATX heading named
`outcome` (any level) followed by the first non-blank line, which must be exactly
one of the stage's allowed outcome words:
```md
## outcome
@@ -42,6 +63,25 @@ The rest of the output is the body, preserved verbatim.
Matching trims whitespace and compares case-insensitively, returning the
canonical allowed spelling. Text before the heading is ignored. Headings and
words inside fenced code blocks (triple backticks or tildes) are ignored, so a
diff or code sample cannot add a false outcome. Missing, duplicate, empty, or unknown
outcomes are explicit errors; the parser never guesses, repairs the text, or
looks for any other section.
diff or code sample cannot add a false outcome. Missing, duplicate, empty, or
unknown outcomes are explicit errors; the parser never guesses, repairs the
text, or looks for any other section.
The server does not validate any other heading. The stage's body is retained
under the single `markdown` key of the internal output map for the existing UI,
and there is no JSON envelope, schema validation, or format-repair ladder.
## Authoring prompts
A stage prompt states the task plainly and asks for the complete `output.md`
markdown. The server adds the file paths, the model's permitted tools, the
allowed outcome words, and the requirement that the only interpreted section is
`# outcome`. Prompts should not ask for JSON fields: free-form headings are for
the next stage and the operator, not a server contract.
`memory` and `output` blocks are retired for model stages. An `output` block is
tolerated and ignored so an exported old definition can be edited in place, but
it neither selects nor validates fields. A `memory` block must be removed: it is
now an unknown model-stage block and fails validation. Use `prompt`, `tools`,
`edges`, `limits`, `configs` (and the type-specific
`permissions`/`checks`/`verification` blocks) to describe a stage.
+75 -42
View File
@@ -2,36 +2,61 @@
A stage that produces images, screenshots, reports or logs - typically written
into an ignored or untracked directory - would otherwise leave nothing retrievable
per step. Every agent visit therefore gets its own artifacts directory, and the
per step. Every model visit therefore gets a file-based turn directory, and the
files it contains are recorded on the step and served over HTTP.
## Where they live
The server creates `<artifacts root>/<run>/<step>/` before executing an agent
visit. The artifacts root defaults to the directory holding the database
(`<data directory>/artifacts`) and can be pointed elsewhere with the server's
`ArtifactsRoot` option. Only model stages get a directory.
The server creates this layout under the run's artifact root:
The directory is passed to the stage:
```
<artifacts root>/<run>/stages/<seq>-<stage>/input.md
<artifacts root>/<run>/stages/<seq>-<stage>/memory.md
<artifacts root>/<run>/stages/<seq>-<stage>/output.md
<artifacts root>/<run>/stages/<seq>-<stage>/private/
<artifacts root>/<run>/public/
```
- as the `PIPELINES_ARTIFACTS_DIR` environment variable in a locally spawned Pi
process, and exported to the OpenCode CLI where the runtime controls it;
- in the agent prompt, next to `Run directory:`, for every agent harness. Because
OpenCode V2 tools execute inside the shared serve process, that prompt line is
the reliable carrier there.
The sequence is zero-padded to two digits. The artifacts root defaults to the
directory holding the database (`<data directory>/artifacts`) and can be pointed
elsewhere with the server's `ArtifactsRoot` option. Only model stages get a turn
directory.
The prompt instruction is deliberately general: save any artifacts you produce -
images, screenshots, reports, logs - into the directory; they are attached to the
stage. There are no technology, language or file-type rules.
- `input.md`, `memory.md` and `output.md` are server-written turn files.
- `private/` belongs to this visit. It is writable by the stage and is never
handed to a later stage.
- `public/` is run-level and shared by every stage. Files are writable and
readable by each visit.
The layout and ownership rules live in `internal/turn.Layout`. A retry is a new
run with its own artifact root, so it never overwrites the failed attempt's turn
files.
## What the stage is told
The prompt names the turn directory, the writable `private/` directory and the
shared `public/` directory, and says that the server owns `input.md`,
`memory.md` and `output.md`. Those are the exact paths the tool permission rules
grant. A Pi stage also receives `PIPELINES_ARTIFACTS_DIR`; OpenCode V2 tools run
inside the shared serve process, so its prompt is the reliable carrier.
Turn-local artifacts go in `private/`. Reports, logs and screenshots that later
stages should see go in `public/`. There are no technology, language or
file-type rules.
## What is recorded
After the stage finishes and before the step is stored, the server makes one
bounded inventory of the directory and records, per file, its `name`,
`size_bytes`, `content_type` (from the extension, with a content sniff of at most
the first 512 bytes as a fallback) and a server-relative `url`
(`/artifacts/<run>/<step>/<name>`). The same fields appear under each step's
`artifacts` in `pipelines inspect RUN_ID --json` and in the TUI.
bounded inventory of the turn directory and the run's public directory. It
records, per file, its `name`, `size_bytes`, `content_type` (from the extension,
with a content sniff of at most the first 512 bytes as a fallback) and a
server-relative `url`. Public entries carry a `public/` name prefix. The same
fields appear under each step's `artifacts` in `pipelines inspect RUN_ID --json`
and in the TUI.
`input.md`, `memory.md` and `output.md` are the primary artifacts of every model
visit. `private/` files are listed without a prefix; shared files are listed as
`public/...` on every visit that can see them.
- Limits: 200 files per step, 8 MiB per file and 64 MiB total. Over-cap files are
skipped and the skipped/truncated counts are recorded with the step.
@@ -44,47 +69,55 @@ the first 512 bytes as a fallback) and a server-relative `url`
## Fetching bytes
`GET /artifacts/<run>/<step>/<name>` on the server that serves `/healthz` and
`/connect` returns the file. Only files inside that run's and step's artifact root
are reachable: path traversal, absolute paths, symlink escapes and unknown
runs/steps are refused. Responses carry the correct `Content-Type` and
`Content-Length`; `HEAD` returns headers without a body; and `Range` requests
return `206 Partial Content` with the exact slice (or `416` when unsatisfiable),
so images and video can stream. `http.ServeContent` streams from the opened file,
so a request never loads the whole file into memory.
`/connect` returns the file. `public/<name>` maps to the run-level public
directory; any other name maps to that sequence's turn directory. Runs recorded
before the file-based protocol landed remain readable from the older
`<run>/<step>/` layout.
Any regular file contained in that visit's turn directory or the run's
`public/` directory is reachable, including a file added after the bounded
inventory or skipped by its size/file caps; the URL is not a lookup in the
recorded list. Path traversal, absolute paths, symlink escapes and unknown
runs/steps are refused. Responses carry the correct
`Content-Type` and `Content-Length`; `HEAD` returns headers without a body; and
`Range` requests return `206 Partial Content` with the exact slice (or `416` when
unsatisfiable), so images and video can stream. `http.ServeContent` streams from
the opened file, so a request never loads the whole file into memory.
## Session export
When a model stage that ran against an OpenCode harness finishes, the server also
exports the session the harness actually produced and writes it into the same
`<artifacts root>/<run>/<step>/` directory in markdown (`session.md`) and html
(`session.html`). They are ordinary artifacts: the bounded inventory lists them
and `GET /artifacts/<run>/<step>/session.md` (or `.html`) serves them with no
special casing.
turn directory as `session.html`. It is an ordinary artifact: the bounded
inventory lists it and `GET /artifacts/<run>/<step>/session.html` serves it with
no special casing.
- Source: the read-only native export path. The native v1 CLI exposes it as
`opencode export <id>`; the installed v2 CLI exposes it as
`opencode session export <id>`. Both reuse the same invocation helper and
session-ID/messages validation as the transcript readback. The files are the
session-ID/messages validation as the transcript readback. The file is the
harness's real session, never a reconstruction.
- Bounds: the source read is capped at 8 MiB and each rendered file at 2 MiB. When
either the source or the rendered output is cut, both files carry a visible
truncation banner near the start and the end and the step records
`truncated: true`; a truncated export is never described as whole.
- Bounds: the source read is capped at 8 MiB and the rendered file at 2 MiB. When
either is cut, the file carries a visible truncation banner near the start and
the end and the step records `truncated: true`; a truncated export is never
described as whole.
- Evidence: the step records `session_export` (`session_id`, `backend`, `source`,
`markdown`, `html`, `markdown_bytes`, `html_bytes`, `messages`, `truncated`,
`diagnostic`), so `pipelines inspect RUN_ID --json` shows what was written
without reading the files.
`html`, `html_bytes`, `messages`, `truncated`, `diagnostic`), so
`pipelines inspect RUN_ID --json` shows what was written without reading the
file.
- Isolation: exporting never changes the stage's status, outcome, target or
result validation, never waits for the session, never mutates, resumes or
interrupts it, and a failure records a sanitized diagnostic and continues. An
existing `session.md`/`session.html` is never overwritten and the skip reason is
recorded.
existing `session.html` is never overwritten and the skip reason is recorded.
- Pi limitation: a Pi stage has no native session export path, so the step records
`source: "unavailable"` with an explicit diagnostic and writes no files.
`source: "unavailable"` with an explicit diagnostic and writes no file.
A run recorded before the HTML-only change may still list `session.md` among its
stored artifacts; the dated evidence is kept on disk rather than deleted.
## Removal with run history
Run artifacts live under `<root>/<run>/<step>` and last as long as the run does.
Run artifacts live under `<root>/<run>` and last as long as the run does.
`pipelines retention report`/`apply` (see [CLI and agent authoring](cli.md#run-retention))
is the only thing that removes them, and it does so only for a run the operator
explicitly selected for removal. Removing a run removes its whole artifact