Harden model results with lossless isolated repair and scoped permissions

This commit is contained in:
2026-10-09 20:02:41 +01:00
parent 0f80ff4e95
commit 67ac79765f
23 changed files with 977 additions and 58 deletions
+9
View File
@@ -76,3 +76,12 @@ investigation and live OpenCode UI attachment remain **not tested**.
Focused tests cover elapsed duration, live graph updates via run responses,
shared-renderer equivalence, loop visits, viewport resizing, and retry rejection
during execution cleanup even after a terminal status is persisted.
Result-recovery regression tests actually execute a mock audited agent that
edits a file once, returns malformed JSON, and receives a separate deny-all
formatting session. They check exact parser feedback, no agent/tool replay,
preserved approval/evidence, rejection of changed verdicts or omitted evidence,
and optional file/command permission rules. A native stable no-inference live
contract test accepted a `json_schema` message with `retryCount=0` and `noReply`.
Real model-driven schema generation, formatter behavior and tool-permission
enforcement remain **not tested**.
+84
View File
@@ -0,0 +1,84 @@
# Model results and scoped tools
Every model result must be one object containing exactly `outcome` and `output`.
`outcome` must be a declared edge; `output` must be an object. Duplicate keys
(including nested output keys), missing fields, unknown fields and invalid
outcomes are rejected. Free-form `output` contract prose is not a JSON Schema.
The native stable OpenCode adapter requests `format.type=json_schema` with this
envelope schema and `retryCount=0`, reads the returned structured result when
available, and preserves the original tool transcript. Local validation remains
mandatory. V2's published prompt/generate API does not accept a schema-output
field; that adapter retains local validation instead of sending unsupported
fields or claiming provider-side schema enforcement.
## Format recovery is not execution retry
Parser errors retain their diagnostic and byte offset: for example, `missing
closing brace '}'`, an unterminated string, or an invalid character. A valid but
schema-invalid result is not reformatted to invent a missing field or verdict.
Format recovery has a separate bounded budget, even with `retries=0`:
1. Append up to eight missing closing delimiters when the existing response
consists of complete JSON tokens. Validate the result again. No model call.
2. If necessary, make at most one fresh, deny-all, transient text-generation
request with only the malformed response and precise parsing error. The
original implementing/reviewing agent is never re-prompted. The formatter
has a 30-second deadline within the original stage deadline and a 32 KiB
input/output bound.
3. Accept only insertion of missing commas/colons or trailing closing delimiters,
with at most 16 insertions. Every existing string, number, literal and
structural delimiter must remain verbatim and in order. Validate the full
result again. Changed approval, outcome, evidence, reordered data, deleted
fields, incomplete strings and fabricated values are rejected.
If recovery fails, report the original precise error and repair failure; never
rerun project edits, commands, or the entire stage to repair final formatting.
Configured execution retries remain for retryable text-generation transport
errors; audited tool-enabled execution is never automatically replayed.
Repair records include the original/candidate response, parsing diagnostic,
method and acceptance flag in the bounded transcript preview. `pipelines
inspect <run-id> --json` and the TUI evidence section expose these records.
## Optional machine-readable restrictions
`tools` enables actions per stage. A stage can narrow edit paths and shell
commands with an optional `permissions` block:
````markdown
```tools
read
glob
grep
edit
shell
```
```permissions
{"edit":["README.md","plugin/*.qml"],"shell":["bash -n install.sh"]}
```
````
These rules become OpenCode tool permissions, not just prompt advice. Edit
patterns are relative to the run directory and intersect any optional run-level
`--write-paths`. Shell entries are exact commands: no wildcards, chaining,
redirection or expansion. Empty lists deny an action; an omitted list retains
the normal defaults. Null lists, unknown keys, duplicate keys/resources, path
traversal, undeclared tools and unrestricted shell alongside restricted edits
are rejected. At most 20 resources per action are accepted.
OpenCode checks scanner-produced shell command resources, not necessarily the
entire submitted script. A compound script consisting only of individually
allowed commands may pass; this is not a raw-script or shell-syntax firewall.
For a deliberately Go-only stage, explicitly declare `edit:["*.go"]` and the
exact allowed verification commands. Without a machine policy, a sentence such
as “approved Go files only” is **not** an enforced permission boundary. Defaults
remain language-neutral: this does not restore a global Go-only restriction or
require users to enumerate files for ordinary runs. Existing immutable definitions
are not rewritten; publish a new version to add intentional restrictions.
Tool permissions are not an OS sandbox. An approved command runs project code
with the host user's authority and can have filesystem/network side effects.
Do not grant shell access if stronger filesystem isolation is required.