Files
pipelines/docs/model-results.md
T

91 lines
4.6 KiB
Markdown

# Model results and scoped tools
Every model result must be one object containing exactly `outcome` and `output`.
`outcome` must be a declared edge; `output` must be an object. Duplicate keys
(including nested output keys), missing fields, unknown fields and invalid
outcomes are rejected. Free-form `output` contract prose is not a JSON Schema.
The native stable OpenCode adapter requests `format.type=json_schema` with this
envelope schema and `retryCount=0`, reads the returned structured result when
available, and preserves the original tool transcript. Local validation remains
mandatory. V2's published prompt/generate API does not accept a schema-output
field; that adapter retains local validation instead of sending unsupported
fields or claiming provider-side schema enforcement.
## Format recovery is not execution retry
Parser errors retain their diagnostic and byte offset: for example, `missing
closing brace '}'`, an unterminated string, or an invalid character. A valid but
schema-invalid result is not reformatted to invent a missing field or verdict.
Format recovery has a separate bounded budget, even with `retries=0`:
1. Append up to eight missing closing delimiters when the existing response
consists of complete JSON tokens. Validate the result again. No model call.
2. If necessary, make at most one fresh, deny-all, transient text-generation
request with only the malformed response and precise parsing error. The
original implementing/reviewing agent is never re-prompted. The formatter
has a 30-second deadline within the original stage deadline and a 32 KiB
input/output bound.
3. Accept only insertion of missing commas/colons or trailing closing delimiters,
with at most 16 insertions. Every existing string, number, literal and
structural delimiter must remain verbatim and in order. Validate the full
result again. Changed approval, outcome, evidence, reordered data, deleted
fields, incomplete strings and fabricated values are rejected.
If recovery fails, report the original precise error and repair failure; never
rerun project edits, commands, or the entire stage to repair final formatting.
Configured execution retries remain for retryable text-generation transport
errors; audited tool-enabled execution is never automatically replayed.
Repair records include the original/candidate response, parsing diagnostic,
method and acceptance flag in the bounded transcript preview. `pipelines
inspect <run-id> --json` and the TUI evidence section expose these records.
OpenCode CLI failures include the HTTP status when supplied, process exit code,
and a recognized safe error category (such as session not found or service
unavailable). Arbitrary stderr, provider messages and response bodies are not
copied into the diagnostic. A `/wait` command failure is not evidence that the
agent prompt failed to start, and does not trigger replay of stage work.
## Optional machine-readable restrictions
`tools` enables actions per stage. A stage can narrow edit paths and shell
commands with an optional `permissions` block:
````markdown
```tools
read
glob
grep
edit
shell
```
```permissions
{"edit":["README.md","plugin/*.qml"],"shell":["bash -n install.sh"]}
```
````
These rules become OpenCode tool permissions, not just prompt advice. Edit
patterns are relative to the run directory and intersect any optional run-level
`--write-paths`. Shell entries are exact commands: no wildcards, chaining,
redirection or expansion. Empty lists deny an action; an omitted list retains
the normal defaults. Null lists, unknown keys, duplicate keys/resources, path
traversal, undeclared tools and unrestricted shell alongside restricted edits
are rejected. At most 20 resources per action are accepted.
OpenCode checks scanner-produced shell command resources, not necessarily the
entire submitted script. A compound script consisting only of individually
allowed commands may pass; this is not a raw-script or shell-syntax firewall.
For a deliberately Go-only stage, explicitly declare `edit:["*.go"]` and the
exact allowed verification commands. Without a machine policy, a sentence such
as “approved Go files only” is **not** an enforced permission boundary. Defaults
remain language-neutral: this does not restore a global Go-only restriction or
require users to enumerate files for ordinary runs. Existing immutable definitions
are not rewritten; publish a new version to add intentional restrictions.
Tool permissions are not an OS sandbox. An approved command runs project code
with the host user's authority and can have filesystem/network side effects.
Do not grant shell access if stronger filesystem isolation is required.