stage turns: replace JSON results with file-based markdown turns

Each model visit now gets <run>/stages/<seq>-<stage>/ with input.md,
memory.md, output.md, private/ and a shared run-level public/. The model
reads the two input files, follows the stage's instructions verbatim and
returns complete markdown; only the # outcome section is parsed.

- add internal/turn layout, render and output parsing
- write input.md/memory.md before a visit and output.md from the final
  message; inventory all turn files as stage artifacts
- accumulate completed turns and system-written public file additions in
  memory, including across retries and resume
- remove the JSON envelope, strict field validation, line format and the
  repair ladder for model stages
- republish pipeline authoring prompts and docs in the new style

Checks: go build ./..., go vet ./..., go test -count=1 ./..., gofmt -l .
This commit is contained in:
2026-10-19 22:27:44 +01:00
parent b2fd614ba4
commit e37b864c40
66 changed files with 2199 additions and 3028 deletions
+7 -9
View File
@@ -12,10 +12,9 @@ pipeline automatically; all templates are manual-only and have at most four stag
| `docs-fold` | Fold accumulated item documentation notes into the shared docs, then commit locally | Fold: read/glob/grep/edit/shell; committer: read/shell |
Every workflow that changes the repository ends in a **committer** stage that
commits locally and never pushes. The committer stages the union of the runner's
server-captured changed paths (`input.server_changed_files`) and the producer's
declared `changed_files` — a declared list can be incomplete — then runs the
repository's required checks before committing. `task` and `three-stage` follow this;
commits locally and never pushes. The committer inspects `git status` and the full
diff, stages this run's change (a model-declared list can be incomplete), then runs
the repository's required checks before committing. `task` and `three-stage` follow this;
`review` and `research` change nothing and so do not commit.
`docs-fold` is the deliberate **single documentation writer**. A per-item workflow never
@@ -24,11 +23,10 @@ edits a shared README or topic doc: when its change makes one stale it records a
the manual-only `docs-fold` pipeline later folds the accumulated notes into the shared
documentation and removes them.
Model and agent stages return the line-oriented result: the first non-empty line is
the declared outcome, a `# output` section holds the JSON output map for the next
stage, and any further `# <heading>` section is retained verbatim as evidence. The
legacy `{ "outcome": ..., "output": {...} }` envelope is still accepted. See
`docs/model-results.md`.
Model and agent stages read the turn's `input.md` and `memory.md`, then return the
complete `output.md` markdown. The only required section is `# outcome` followed by
one declared edge word; every other heading is free-form evidence for the next stage
and the operator. See `docs/notes/stage-turns.md`.
Web search/fetch is conditional: stages use it only when a task genuinely needs
external knowledge such as accepted industry approaches, framework or library
+16 -31
View File
@@ -3,6 +3,7 @@ package examples_test
import (
"os"
"slices"
"strings"
"testing"
"git.a-wan.net/alan/pipelines/examples"
@@ -142,40 +143,24 @@ func TestThreeStagePlannerBlockedRoute(t *testing.T) {
}
}
func TestThreeStageReviewerMemoryKeepsPlannerAcceptance(t *testing.T) {
func TestThreeStageUsesTurnFilesWithoutRetiredBlocks(t *testing.T) {
graph := parsedCatalog(t)["three-stage"]
for _, s := range graph.Stages {
if len(s.Memory) != 0 {
t.Fatalf("%s stage %s must not use the retired memory block: %v", "three-stage", s.Key, s.Memory)
}
if s.Output != "" {
t.Fatalf("%s stage %s must not declare the retired output block", "three-stage", s.Key)
}
}
var reviewer protocol.Stage
for _, s := range graph.Stages {
if s.Key == "reviewer" {
reviewer = s
}
}
if reviewer.Key == "" {
t.Fatal("three-stage reviewer stage missing")
}
plannerOwned := map[string]string{
"plan": "planner.plan",
"design": "planner.design",
"reuse_targets": "planner.reuse_targets",
"acceptance_checks": "planner.acceptance_checks",
"research": "planner.research",
}
for alias, reference := range plannerOwned {
if got := reviewer.Memory[alias]; got != reference {
t.Fatalf("reviewer memory %s = %q, want %q", alias, got, reference)
}
}
if got := reviewer.Memory["task"]; got != "input.prompt" {
t.Fatalf("reviewer must receive the original task, got %q", got)
}
for alias, want := range map[string]string{
"changed_files": "implementer.changed_files",
"implementation_summary": "implementer.implementation_summary",
"evidence": "implementer.evidence?",
} {
if got := reviewer.Memory[alias]; got != want {
t.Fatalf("reviewer memory %s = %q, want %q", alias, got, want)
}
if reviewer.Key == "" || !strings.Contains(strings.ToLower(reviewer.Prompt), "independently review") {
t.Fatalf("three-stage reviewer must independently review the full diff: %+v", reviewer)
}
}
@@ -196,11 +181,11 @@ func TestWorkProducingStartersEndInACommitter(t *testing.T) {
if got := last.Edges["success"]; got != protocol.SuccessEnd {
t.Fatalf("%s committer success route = %q, want %q", tc.key, got, protocol.SuccessEnd)
}
if got := last.Memory["server_changed_files"]; got != "input.server_changed_files" {
t.Fatalf("%s committer must select input.server_changed_files, got %q", tc.key, got)
if len(last.Memory) != 0 || last.Output != "" {
t.Fatalf("%s committer must use the turn files, not retired blocks: %+v", tc.key, last)
}
if got := last.Memory["changed_files"]; got != tc.producer+".changed_files" {
t.Fatalf("%s committer changed_files = %q, want %q", tc.key, got, tc.producer+".changed_files")
if !strings.Contains(last.Prompt, "diff") {
t.Fatalf("%s committer prompt must inspect the actual diff", tc.key)
}
// The producer hands its work on rather than ending the run itself.
for _, s := range graph.Stages {
+5 -24
View File
@@ -1,9 +1,5 @@
# stage fold
```output
{ summary, changed_files, folded_notes }
```
```prompt
Fold the accumulated per-item documentation notes into the repository's shared
documentation. Read every file under `docs/notes/`; for each note, apply the change it
@@ -16,10 +12,7 @@ directory and the documents they name; never scan filesystem or mount roots, and
commands supported by the actual execution environment. Save any artifacts you produce
into the artifacts directory named in the runtime context. Report the notes read, the
documents changed and the notes deleted in an `# evidence` section, with the exact
commands and observed results. Return the line-oriented result: the first non-empty line
is the declared outcome, a `# output` section holds one JSON object with the output
fields named above, and any further `# <heading>` section is retained verbatim as
evidence. Return blocked if the notes directory cannot be read.
commands and observed results. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need. Return blocked if the notes directory cannot be read.
```
```tools
@@ -45,30 +38,18 @@ error -> #failure
# stage committer
```output
{ committed, commit, changed_files, pre_commit_checks }
```
```prompt
Commit this run's change locally. Never push, tag, deploy or rewrite history. Select
the run's changed paths from input.server_changed_files (the runner's captured
evidence) and the fold stage's declared changed_files, and stage their union, including
the notes the fold stage deleted (stage the removal, not only surviving paths). Inspect
Commit this run's change locally. Never push, tag, deploy or rewrite history. Inspect
`git status` and the full diff, and stage this run's change, including the notes the
fold stage deleted (stage the removal, not only surviving paths). Inspect
the staged diff before committing; never commit unrelated work, secrets or generated
artifacts. When nothing is staged - for example an empty notes directory - do not create
an empty commit; return success with a clear note. Otherwise commit with a concise
message naming this run's change and report the commit hash. If a required check fails,
do not commit; return blocked with the failing output. Return the line-oriented result:
the first non-empty line is the declared outcome, a `# output` section holds one JSON
object with the output fields named above, and any further `# <heading>` section is
retained verbatim as evidence. Never claim an unexecuted check passed or fabricate a
do not commit; return blocked with the failing output. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need. Never claim an unexecuted check passed or fabricate a
commit.
```
```memory
{"changed_files":"fold.changed_files","server_changed_files":"input.server_changed_files"}
```
```tools
read
glob
+4 -18
View File
@@ -1,11 +1,7 @@
# stage research
```output
{ question, evidence, sources, uncertainties }
```
```prompt
Research the topic in input.prompt using web search and relevant source pages.
Research the topic in the task in input.md using web search and relevant source pages.
Preserve the question and constraints. Prefer authoritative primary evidence
(official standards, upstream documentation, and versioned sources); record
supporting excerpts or facts, source URLs, and appropriate version/date/scope.
@@ -17,11 +13,7 @@ gaps and source/version applicability, and do not invent citations. Return block
when required reliable evidence cannot be obtained, including empty or unreliable
results. A single failed, blocked or anti-bot search may be replaced
by a successful official-source webfetch that fully supplies the required
evidence. Return success only with reliable evidence. Return the line-oriented
result: the first non-empty line is the declared outcome, a `# output` section
holds one JSON object with the output fields named above, and any further
`# <heading>` section is retained verbatim as evidence (for example the queries
run and source excerpts).
evidence. Return success only with reliable evidence. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need.
```
```tools
@@ -44,18 +36,12 @@ error -> #failure
# stage report
```output
{ report, sources, open_questions }
```
```prompt
Answer input.question using only the collected evidence and sources. Write a
Answer the question in input.md using only the collected evidence and sources. Write a
concise report with citations and clearly marked uncertainties. Respect the
original constraints. Do not turn missing evidence into a confident answer.
Return success for a supported report or blocked when the evidence is insufficient.
Return the line-oriented result: the first non-empty line is the declared outcome,
a `# output` section holds one JSON object with the output fields named above, and
any further `# <heading>` section is retained verbatim as evidence.
Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need.
```
```tools
+2 -10
View File
@@ -1,11 +1,7 @@
# stage review
```output
{ scope, findings, recommendations, unverified_claims }
```
```prompt
Review the repository content described by input.prompt. Read relevant files; do
Review the repository content described by the task in input.md. Read relevant files; do
not modify them or run commands. Focus on maintainability, duplication and
near-duplicate implementations, standards/library and version usage, and
consistency with repository conventions. As visible in source, assess root-cause
@@ -21,11 +17,7 @@ hypotheses. State explicitly that no commands were run and that runtime behavior
and checks are not tested; source inspection is not proof of execution. Return
success when the review is complete, even if it finds issues; return blocked if
the requested content or required official evidence cannot be obtained rather than
fabricating it. Do not claim issues have been fixed. Return the line-oriented
result: the first non-empty line is the declared outcome, a `# output` section
holds one JSON object with the output fields named above, and any further
`# <heading>` section is retained verbatim as evidence (for example the files read
and the sources checked, with their dates/versions).
fabricating it. Do not claim issues have been fixed. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need.
```
```tools
+5 -24
View File
@@ -1,11 +1,7 @@
# stage worker
```output
{ task, summary, changed_files, remaining_questions }
```
```prompt
Complete input.prompt in the run directory. Inspect only relevant context and
Complete the task in input.md in the run directory. Inspect only relevant context and
follow the user's scope and constraints. Limit repository inspection and
verification to relevant repository paths; never scan filesystem or mount roots,
and use commands supported by the actual execution environment rather than
@@ -29,10 +25,7 @@ push, deploy, or perform destructive actions. Save any artifacts you produce
(reports, logs, screenshots) into the artifacts directory named in the runtime
context; they are attached to this stage. Verify the change with the repository's
relevant checks and report the exact commands and observed results in an
`# evidence` section; never claim an unexecuted check passed. Return the
line-oriented result: the first non-empty line is the declared outcome, a
`# output` section holds one JSON object with the output fields named above, and
any further `# <heading>` section is retained verbatim as evidence. Return blocked
`# evidence` section; never claim an unexecuted check passed. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need. Return blocked
if the task or required tools are unavailable; otherwise return success with
actual results.
```
@@ -62,15 +55,10 @@ error -> #failure
# stage committer
```output
{ committed, commit, changed_files, pre_commit_checks }
```
```prompt
Commit this run's change locally. Never push, tag, deploy or rewrite history.
Select the run's changed paths from input.server_changed_files (the runner's
captured evidence) and the worker's declared changed_files, and stage their union
of paths that still exist, keeping any exclusions. Inspect the staged diff before
Inspect `git status` and the full diff, and stage this run's change, including
inherited work, restricted to paths that still exist and keeping any exclusions. Inspect the staged diff before
committing; never commit unrelated work, secrets or generated artifacts. Follow the
documentation-note step of the committer checklist in `references/format.md`: when the
change makes a shared document stale, stage the item's own documentation note
@@ -82,17 +70,10 @@ required checks and record the exact commands and observed results in an
`# evidence` section; do not commit if a required check fails, return blocked
with the failing output instead. If nothing is staged, do not create an empty
commit; return success with a clear note. Otherwise commit with a concise message
naming this run's change and report the commit hash. Return the line-oriented
result: the first non-empty line is the declared outcome, a `# output` section
holds one JSON object with the output fields named above, and any further
`# <heading>` section is retained verbatim as evidence. Never claim an unexecuted
naming this run's change and report the commit hash. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need. Never claim an unexecuted
check passed or fabricate a commit.
```
```memory
{"task":"input.prompt","changed_files":"worker.changed_files","server_changed_files":"input.server_changed_files"}
```
```tools
read
glob
+10 -43
View File
@@ -1,11 +1,7 @@
# stage planner
```output
{ task, plan, design, reuse_targets, acceptance_checks, research }
```
```prompt
Plan input.prompt without implementing it. Discover the repository's languages,
Plan the task in input.md without implementing it. Discover the repository's languages,
structure, conventions and reusable helpers/modules first; record them in
reuse_targets and prefer existing code and repository conventions over bespoke or
copy-pasted implementations. Resolve scope, acceptance, risk and design
@@ -23,10 +19,7 @@ fully supplies the required evidence, otherwise return blocked. Limit repository
inspection to relevant repository paths; never scan filesystem or mount roots,
and use commands supported by the actual execution environment rather than
guessed platform flags (a working instruction, not an OS sandbox). Preserve the
user's task and constraints; do not invent evidence. Return the line-oriented
result: the first non-empty line is the declared outcome, a `# output` section
holds one JSON object with the output fields named above, and any further
`# <heading>` section is retained verbatim as evidence.
user's task and constraints; do not invent evidence. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need.
```
```tools
@@ -52,10 +45,6 @@ error -> #failure
# stage implementer
```output
{ task, plan, acceptance_checks, changed_files, implementation_summary }
```
```prompt
Implement the supplied plan in the run directory using the repository's actual
languages and file types; create files as needed. Reuse the supplied reuse_targets
@@ -77,10 +66,7 @@ Run the relevant checks with the stage tools and record the exact commands,
observed results and any untested limits in an `# evidence` section; never
claim an unexecuted check passed. Preserve the task, plan and acceptance checks and
record actual changes in changed_files. Do not commit, push, deploy or destroy
unrelated work; the committer stage commits this run's change. Return the
line-oriented result: the first non-empty line is the declared outcome, a
`# output` section holds one JSON object with the output fields named above, and
any further `# <heading>` section is retained verbatim as evidence. Return blocked
unrelated work; the committer stage commits this run's change. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need. Return blocked
when the required edit scope or tools are unavailable.
```
@@ -107,10 +93,6 @@ error -> #failure
# stage reviewer
```output
{ summary, changed_files, verification_evidence, findings, unverified }
```
```prompt
Independently review the actual changed files and related existing code against
the supplied task, plan and acceptance checks. Inspect for duplication and
@@ -129,9 +111,9 @@ and run commands supported by the actual execution environment rather than
guessed platform flags (working instructions, not an OS sandbox guarantee). Save
any artifacts you produce (reports, logs, screenshots) into the artifacts
directory named in the runtime context; they are attached to this stage. Return
the line-oriented result: the first non-empty line is the declared outcome, a
`# output` section holds one JSON object with the output fields named above, and
any further `# <heading>` section is retained verbatim as evidence. Return success
the complete output.md markdown; the only required section is `# outcome`
followed by a declared edge word, with free-form headings for the facts later
stages need. Return success
only when the applicable acceptance checks were independently observed passing and
no unresolved substantive design, reuse, duplication, workaround or correctness
issue remains; return rejected with concrete findings for failing checks, missing
@@ -141,10 +123,6 @@ websearch/webfetch only when an official standard or documentation check is
needed. Do not edit files, commit, push or deploy.
```
```memory
{"task":"input.prompt","pwd":"input.pwd","plan":"planner.plan","design":"planner.design","reuse_targets":"planner.reuse_targets","acceptance_checks":"planner.acceptance_checks","research":"planner.research","changed_files":"implementer.changed_files","implementation_summary":"implementer.implementation_summary","evidence":"implementer.evidence?"}
```
```tools
read
glob
@@ -169,15 +147,11 @@ error -> #failure
# stage committer
```output
{ committed, commit, changed_files, pre_commit_checks }
```
```prompt
Commit this run's change locally. Never push, tag, deploy or rewrite history.
Select the run's changed paths from input.server_changed_files (the runner's
captured evidence) and the implementer's declared changed_files, and stage their
union of paths that still exist, keeping any exclusions. Inspect the staged diff
Inspect `git status` and the full diff, and stage this run's change, including
inherited work, restricted to paths that still exist and keeping any exclusions.
Inspect the staged diff
before committing; never commit unrelated work, secrets or generated artifacts.
Follow the documentation-note step of the committer checklist in
`references/format.md`: when the change makes a shared document stale, stage the item's
@@ -188,17 +162,10 @@ repository's required checks and record the exact commands and observed results
a `# evidence` section; do not commit if a required check fails, return
blocked with the failing output instead. If nothing is staged, do not create an
empty commit; return success with a clear note. Otherwise commit with a concise
message naming this run's change and report the commit hash. Return the
line-oriented result: the first non-empty line is the declared outcome, a
`# output` section holds one JSON object with the output fields named above, and
any further `# <heading>` section is retained verbatim as evidence. Never claim an
message naming this run's change and report the commit hash. Return the complete output.md markdown; the only required section is `# outcome` followed by a declared edge word, with free-form headings for the facts later stages need. Never claim an
unexecuted check passed or fabricate a commit.
```
```memory
{"task":"input.prompt","changed_files":"implementer.changed_files","server_changed_files":"input.server_changed_files"}
```
```tools
read
glob