From aac1be18ccb518c2fbc61cd770e3b0966eb6fbc6 Mon Sep 17 00:00:00 2001 From: Alan Silva Date: Fri, 16 Oct 2026 21:49:14 +0100 Subject: [PATCH] docs: document line-oriented model/agent result format --- docs/model-results.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/docs/model-results.md b/docs/model-results.md index 93e62ba7..576d6b65 100644 --- a/docs/model-results.md +++ b/docs/model-results.md @@ -23,6 +23,45 @@ is not evidence of a real tool invocation and cannot be accepted as a result. Flattened output fields are rejected, not guessed into a new envelope. Invalid responses are retained even when they are not eligible for syntax repair. +## Line-oriented result format + +Model and agent stages also accept a line-oriented result format. The response is +a markdown-ish document whose first non-empty line is the outcome, followed by +sections introduced with `# `. + +``` +success + +# output +{"task": "x"} + +# findings +- prose, code fences, quotes and embedded JSON need no escaping +``` + +Only the first non-empty line and a `# output` section are required. The body of +`# output` must be a JSON object; it becomes the stage's `output` map. Every other +heading becomes an evidence field keyed by the normalised heading text (lowercased, +spaces to underscores). Extra sections are stored verbatim and are best-effort: an +empty heading, an odd heading, or a very large section must never fail the stage. +If the outcome is missing, not one of the declared edges, or `# output` is absent, +the stage fails with `FINAL RESULT VALIDATION` and still retains whatever sections +were salvaged. + +The parser decides between the legacy JSON envelope and the line-oriented format by +precedence. If the response's first non-empty character is `{` after stripping an +optional surrounding markdown fence, it keeps the unchanged JSON path. If a `# ` +heading appears before any `{`, it uses the line-oriented path. This preserves all +existing JSON responses, including fenced JSON or short prose prefixes, while +allowing new line-format responses to coexist. + +The full raw response is written to the stage's artifacts directory as +`raw-output.md` before parsing, so the closest record of what the model produced is +always retained and appears in the step's artifact inventory automatically. The +parse status is recorded in the step's output map under the reserved key +`result_parse`: `parsed` (both required parts OK), `partial` (required parts OK but +some extra section unusable), or `failed` (a required part missing or invalid). + ## Format recovery is not execution retry Parser errors retain their diagnostic and byte offset: for example, `missing