Files
pipelines/docs/stage-artifacts.md
T
alan 22801a9b06 fix(stage): align session export render cap with artifact bound
Six computed Flutter runs stopped at the export handoff gate because the
rendered session.html cap was 2 MiB while their complete sessions (well inside
the 8 MiB source read bound) rendered between 2.25 and 4.28 MiB once html
escaping and the readable projection duplicated native text.

Raise SessionExportFileLimit to 8 MiB, the server's existing per-artifact file
bound, so a complete export is still listed and served. Normal export
verification, versioned repair and the handoff gate keep using the single
shared constant. The source read bound is unchanged, and an over-cap render is
still visibly truncated and recorded partial.

Add focused regressions for a complete export above the old cap, an over-new-cap
file rejected as partial, repair above the old cap leaving original evidence
untouched, the gate accepting the larger repair, and the constant staying inside
the artifact file bound. Update the artifact, handoff-gate and operations docs
and record the diagnosis in docs/notes/session-export-render-cap.md.
2026-10-21 15:26:29 +01:00

6.8 KiB

Stage artifacts

A stage that produces images, screenshots, reports or logs - typically written into an ignored or untracked directory - would otherwise leave nothing retrievable per step. Every model visit therefore gets a file-based turn directory, and the files it contains are recorded on the step and served over HTTP.

Where they live

The server creates this layout under the run's artifact root:

<artifacts root>/<run>/stages/<seq>-<stage>/input.md
<artifacts root>/<run>/stages/<seq>-<stage>/memory.md
<artifacts root>/<run>/stages/<seq>-<stage>/output.md
<artifacts root>/<run>/stages/<seq>-<stage>/private/
<artifacts root>/<run>/public/

The sequence is zero-padded to two digits. The artifacts root defaults to the directory holding the database (<data directory>/artifacts) and can be pointed elsewhere with the server's ArtifactsRoot option. Only model stages get a turn directory.

  • input.md, memory.md and output.md are server-written turn files.
  • private/ belongs to this visit. It is writable by the stage and is never handed to a later stage.
  • public/ is run-level and shared by every stage. Files are writable and readable by each visit.

The layout and ownership rules live in internal/turn.Layout. A retry is a new run with its own artifact root, so it never overwrites the failed attempt's turn files.

What the stage is told

The prompt names the turn directory, the writable private/ directory and the shared public/ directory, and says that the server owns input.md, memory.md and output.md. Those are the exact paths the tool permission rules grant. A Pi stage also receives PIPELINES_ARTIFACTS_DIR; OpenCode V2 tools run inside the shared serve process, so its prompt is the reliable carrier.

Turn-local artifacts go in private/. Reports, logs and screenshots that later stages should see go in public/. There are no technology, language or file-type rules.

What is recorded

After the stage finishes and before the step is stored, the server makes one bounded inventory of the turn directory and the run's public directory. It records, per file, its name, size_bytes, content_type (from the extension, with a content sniff of at most the first 512 bytes as a fallback) and a server-relative url. Public entries carry a public/ name prefix. The same fields appear under each step's artifacts in pipelines inspect RUN_ID --json and in the TUI.

input.md, memory.md and output.md are the primary artifacts of every model visit. private/ files are listed without a prefix; shared files are listed as public/... on every visit that can see them.

  • Limits: 200 files per step, 8 MiB per file and 64 MiB total. Over-cap files are skipped and the skipped/truncated counts are recorded with the step.
  • Symlinks in any component and non-regular files are excluded, and the walk is bounded by the file cap so a large directory is never fully enumerated.
  • Artifacts are evidence, not a gate. A missing, unreadable, symlinked, oversized or empty directory records nothing and never fails the run or changes its outcome. The database retains the recorded list, not file copies.

Fetching bytes

GET /artifacts/<run>/<step>/<name> on the server that serves /healthz and /connect returns the file. public/<name> maps to the run-level public directory; any other name maps to that sequence's turn directory. Runs recorded before the file-based protocol landed remain readable from the older <run>/<step>/ layout.

Any regular file contained in that visit's turn directory or the run's public/ directory is reachable, including a file added after the bounded inventory or skipped by its size/file caps; the URL is not a lookup in the recorded list. Path traversal, absolute paths, symlink escapes and unknown runs/steps are refused. Responses carry the correct Content-Type and Content-Length; HEAD returns headers without a body; and Range requests return 206 Partial Content with the exact slice (or 416 when unsatisfiable), so images and video can stream. http.ServeContent streams from the opened file, so a request never loads the whole file into memory.

Session export

When a model stage that ran against an OpenCode harness finishes, the server also exports the session the harness actually produced and writes it into the same turn directory as session.html. It is an ordinary artifact: the bounded inventory lists it and GET /artifacts/<run>/<step>/session.html serves it with no special casing.

  • Source: the read-only native export path. The native v1 CLI exposes it as opencode export <id>; the installed v2 CLI exposes it as opencode session export <id>. Both reuse the same invocation helper and session-ID/messages validation as the transcript readback. The file is the harness's real session, never a reconstruction.
  • Bounds: the source read is capped at 8 MiB and the rendered file at 8 MiB, matching the ordinary per-artifact file bound. The rendered file can exceed its source because html escaping and the readable projection keep native text more than once, so the previous 2 MiB render cap cut complete sessions that were well inside the source bound. When either bound is exceeded, the file carries a visible truncation banner near the start and the end and the step records truncated: true; a truncated export is never described as whole.
  • Evidence: the step records session_export (session_id, backend, source, html, html_bytes, messages, truncated, diagnostic), so pipelines inspect RUN_ID --json shows what was written without reading the file.
  • Isolation: exporting never changes the stage's status, outcome, target or result validation, never waits for the session, never mutates, resumes or interrupts it, and a failure records a sanitized diagnostic and continues. An existing session.html is never overwritten and the skip reason is recorded.
  • Pi limitation: a Pi stage has no native session export path, so the step records source: "unavailable" with an explicit diagnostic and writes no file.

A run recorded before the HTML-only change may still list session.md among its stored artifacts; the dated evidence is kept on disk rather than deleted.

Removal with run history

Run artifacts live under <root>/<run> and last as long as the run does. pipelines retention report/apply (see CLI and agent authoring) is the only thing that removes them, and it does so only for a run the operator explicitly selected for removal. Removing a run removes its whole artifact directory; protected and retained runs keep theirs, and their recorded artifact URLs keep serving. A symlink at <root>/<run> is removed as a link without touching its target, and a regular file at that path is refused rather than deleted. Nothing removes artifacts automatically.