Files
pipelines/docs/model-results.md
T
alan e37b864c40 stage turns: replace JSON results with file-based markdown turns
Each model visit now gets <run>/stages/<seq>-<stage>/ with input.md,
memory.md, output.md, private/ and a shared run-level public/. The model
reads the two input files, follows the stage's instructions verbatim and
returns complete markdown; only the # outcome section is parsed.

- add internal/turn layout, render and output parsing
- write input.md/memory.md before a visit and output.md from the final
  message; inventory all turn files as stage artifacts
- accumulate completed turns and system-written public file additions in
  memory, including across retries and resume
- remove the JSON envelope, strict field validation, line format and the
  repair ladder for model stages
- republish pipeline authoring prompts and docs in the new style

Checks: go build ./..., go vet ./..., go test -count=1 ./..., gofmt -l .
2026-10-19 22:27:44 +01:00

148 lines
6.0 KiB
Markdown

# Model turns, results and scoped tools
Model stages are file-based. Each visit reads `input.md` and `memory.md` and
returns the complete markdown for `output.md`. Only the outcome section is
interpreted; the rest is free-form text for the next stage and the operator.
## Turn files
Every model visit gets its own directory under the run's artifact root:
```
<artifacts>/<run>/stages/01-implementer/input.md
<artifacts>/<run>/stages/01-implementer/memory.md
<artifacts>/<run>/stages/01-implementer/output.md
<artifacts>/<run>/stages/01-implementer/private/
<artifacts>/<run>/public/
```
- `input.md` is the run prompt on the first visit and the previous visit's body
after that, under a `# Previous turn output` heading.
- `memory.md` is one `## <stage> · visit <n>` section per completed turn: the
turn body followed by a system-written, sorted list of the files that turn
added to `public/`.
- `private/` belongs to one visit and is never handed to a later stage.
- `public/` is shared by every stage in the run.
The server writes `input.md` and `memory.md` before the model runs and
`output.md` after it. All three are inventoried as step artifacts, together with
`private/` and the files in `public/`. See
[stage artifacts](stage-artifacts.md) for capture limits and URLs.
A text-only stage (an explicit empty `tools` block) cannot read files, so the
server embeds the contents of `input.md` and `memory.md` in its prompt instead.
A stage with tools receives the paths and reads them itself.
## Output contract
`output.md` must contain exactly one outcome heading. The heading may be any ATX
level and is followed by the first non-blank line, which must be exactly one of
the stage's outgoing edge names:
```md
## outcome
success
# Result
The rest of the output is free-form markdown.
```
Matching ignores case and surrounding whitespace and returns the canonical edge
spelling. Headings and words inside fenced code blocks are ignored, so a code
sample cannot add a false outcome. Missing, duplicate, empty or undeclared
outcomes fail the stage. The parser never guesses, repairs the text, or looks for
any other section.
Everything after the outcome value line is retained as the turn body under the
single `markdown` key of the internal output map. No JSON envelope, output
schema, field selection, `output_contract`, or format-repair pass exists for
model stages. Prompts should ask for free-form headings, never JSON fields.
Human and ntfy stages are unchanged and do not use turn files or the outcome
heading.
## Prompt shape
The server builds the prompt from the stage's own instructions plus runtime
facts:
- the paths to `memory.md`, `input.md` and `output.md` (or their contents for a
text-only stage);
- the stage prompt verbatim;
- the worked outcome example and the complete allowed outcome list;
- the execution time and timezone;
- a granted resource target, when one exists;
- the writable `private/` and shared `public/` artifact directories, when they
exist;
- the authoritative permitted-tools line and the optional permitted shell
commands.
The model returns the complete `output.md` markdown as its final message. The
server writes that file and parses only the outcome.
## Failure diagnostics and retries
Retained stage failures use two fixed, sanitized categories:
- **FINAL RESULT VALIDATION** for an invalid turn output: missing, duplicate,
empty or undeclared outcome, or an unwritable `output.md`. Completed edits and
tool side effects are retained; a full retry repeats work.
- **HARNESS TRANSPORT** for request, active-session poll and final-readback
failures. A pre-admission reach failure (connecting or creating the session)
replays nothing and is retried on any stage. A post-admission deadline or
transport failure is retried only on an explicit read-only stage. A stage with
`edit`/`shell`, or with no tools block (the legacy read/glob/grep/edit/shell
bundle), never replays admitted tool work.
A validation failure is never reported as harness transport, and vice versa.
Neither diagnostic includes a provider response body, prompt, credential or
arbitrary model content.
## Optional machine-readable restrictions
`tools` enables actions per stage. A stage can narrow edit paths and shell
commands with an optional `permissions` block:
````markdown
```tools
read
glob
grep
edit
shell
```
```permissions
{"edit":["README.md","plugin/*.qml"],"shell":["bash -n install.sh"]}
```
````
These rules become OpenCode tool permissions, not just prompt advice. Edit
patterns are relative to the run directory and intersect any optional run-level
`--write-paths`. Shell entries are exact commands: no wildcards, chaining,
redirection or expansion. Empty lists deny an action; an omitted list retains
the normal defaults. Null lists, unknown keys, duplicate keys/resources, path
traversal, undeclared tools and unrestricted shell alongside restricted edits
are rejected. At most 20 resources per action are accepted.
Artifact access is separate and server-derived: a visit may write only its own
`private/` directory and the run-level `public/` directory. Those are the only
paths added to the permission rules and named in the prompt. The turn directory
itself is server-owned, so the model never writes `input.md`, `memory.md` or
`output.md` directly.
OpenCode checks scanner-produced shell command resources, not necessarily the
entire submitted script. A compound script consisting only of individually
allowed commands may pass; this is not a raw-script or shell-syntax firewall.
For a deliberately Go-only stage, explicitly declare `edit:["*.go"]` and the
exact allowed verification commands. Without a machine policy, a sentence such
as "approved Go files only" is **not** an enforced permission boundary. Defaults
remain language-neutral. Existing immutable definitions are not rewritten;
publish a new version to add intentional restrictions.
Tool permissions are not an OS sandbox. An approved command runs project code
with the host user's authority and can have filesystem/network side effects. Do
not grant shell access if stronger filesystem isolation is required.