Files
pipelines/internal/author/skill/references/control.md
T
alan e70c94e8dd control: make advisory review findings non-blocking and bound retryable blocks
Cut the review loop's repeat cost and make a blocked stage actionable, at the
definition level only, with the runner's existing visit counting and caps.

The published maintenance, quick and focused pipelines gain a reviewer
`advisory -> next stage` edge for non-blocking notes, so an advisory-only review
continues the run instead of buying another implementer visit; `rejected ->
implementer` and its unchanged {"max_visits":3,"on":["rejected"]} limit keep a
must-fix finding looping and still terminate. The implementer gains
blocked_retryable -> implementer with {"max_visits":5,"on":["blocked_retryable"]}
for a transient environment/tooling/device problem a fresh attempt could
survive, while blocked -> #failure stays terminal. The cap is at least the
reviewer's rejection budget so the implementer's own pre-execution cap cannot
preempt the reviewer's rejected control.

No production Go code changed. New tests in internal/server/review_contract_test.go
cover advisory-only never revisiting the implementer, must-fix still looping to
the reviewer limit, the implementer cap not preempting the reviewer, retryable
blocks bounded then failing closed, a retry that recovers, and a terminal block
failing immediately; concurrency/control/human tests share the new submitRunner
and startRun seams.

Documentation checked: internal/author/skill/references/control.md and
docs/harnesses-and-control.md carry the new edges and the visit-cap rule;
no packaged template defines these pipelines (they are published immutable
versions), and docs/model-results.md needs no change because the {outcome,output}
result contract is unchanged. README and other shared docs deliberately untouched.

Verified: go build ./... and go test -count=1 ./... pass.

Residual limitation: the must-fix-vs-advisory split is carried by the prompt
contract, not a machine-readable marker, because the recorded findings have none;
a model that still returns rejected for advisory-only notes would still buy a
visit. All 18 recorded implementer blocked runs classify terminal, so the
blocked_retryable edge recovers 0 of them.
2026-10-17 17:06:51 +01:00

83 lines
4.6 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Runner-owned limits and shared memory
Use these blocks for bounded feedback loops. Never ask a model to count visits,
increment revision counters, or enforce retry caps. Model verdicts concern current
work only. Counters are runner audit data, not model input. A fresh session alone
does not guarantee independent review: explicitly select its input fields.
````markdown
```limits
{"max_visits":3,"on":["rejected"]}
```
```memory
{"task":"input.prompt","checks":"planner.checks"}
```
```edges
approved -> #success
advisory -> #success
rejected -> implement
error -> #failure
```
````
These blocks accompany a normal review stage's prompt/output/tools/configs.
Pipelines counts actual visits, ignoring model-generated counters. On visit three,
`approved` can succeed; `rejected` retains its verdict but routes to `#failure`.
`rejected` is for a must-fix finding with a concrete location, evidence and remedy;
`advisory` carries non-blocking notes and continues the run without another
implementer visit, so an advisory-only review never buys a full re-implementation.
An implementer may likewise declare `blocked_retryable -> implementer` for a
transient environment, tooling or device problem a fresh attempt could survive,
keeping `blocked -> #failure` for an acceptance criterion that genuinely cannot be
met. `max_visits` caps every visit of that stage, including re-entries from the
reviewer's `rejected` edge, so an implementer that is also the rejection target must
use a cap at least as large as the reviewer's budget; the published pipelines use
`{"max_visits":5,"on":["blocked_retryable"]}`. A lower implementer cap stops the loop
before the reviewer's own cap fires, making the reviewer's `rejected` control
unreachable. Both edges need only the existing visit counting and cap; neither the
reviewer nor the implementer counts visits, and no stage invents edits or checks to
appear busy.
Visit count, limit decision and original target are retained in `inspect --json`
and run metadata. `max_visits` accepts 1–5. Every `on` outcome must declare a
nonterminal edge. Without `on`, entering beyond the cap fails before execution.
Global 30-step/five-visits-per-stage safety limits still apply.
`memory` maps input aliases to `input.FIELD` (original input) or `STAGE.FIELD`
(latest completed output of that stage **in this run**). A memory block replaces
previous-stage input with only those fields. Without it, previous output remains
input. Reads retain source and visit sequence. Missing required fields fail before
execution. Append `?` to omit an unavailable field, e.g.
`"feedback":"review.findings?"` for implement's first visit. Self-memory is
rejected. Limits: 50 bindings; 1 MiB per retained in-memory value map.
Implementers can receive task, plan and optional findings. Independent reviewers
should receive task/acceptance criteria and inspect actual artifacts—not previous
verdicts, counters, or implementation conversations. Select acceptance, design,
reuse and research references from the planning stage (for example
`"acceptance_checks":"planner.acceptance_checks"`) rather than from the
implementer, so an implementer's output cannot overwrite the agreed criteria; take
only factual implementer fields such as `changed_files`, `implementation_summary`
and `evidence` as claims. Selected content remains untrusted data; this cannot
prove semantic independence or prevent all injection.
`input.server_changed_files` is a reserved field the runner publishes: the run's
full, deduplicated, first-seen set of server-captured changed paths (observed
filediff evidence, never a model's claim). Select it with
`"server_changed_files":"input.server_changed_files"`. It is always resolvable —
an empty list when nothing was captured — and reflects every completed step,
including the persisted steps of a resumed run. Only a stage that selects it
receives it; other stages' input is unchanged. A committer should select it
alongside the implementer's declared `changed_files` and stage the union of the
captured paths that still exist and the declared paths, keeping every existing
exclusion and empty-commit rule, and record in the commit body what it
reconciled. Declared lists come from the model and are frequently incomplete;
the server-captured set is the authoritative evidence of what this run changed.
Memory is run-local; persisted completed outputs/read provenance are its audit.
There is no cross-run memory, restart/resume, or hidden replay. New runs/retries
start fresh. Arbitrary prose is not executable policy. Future AI-assisted
guidelines validation is **not implemented**; structural validation does not
recognize every unreliable counting instruction.