control: make advisory review findings non-blocking and bound retryable blocks
Cut the review loop's repeat cost and make a blocked stage actionable, at the
definition level only, with the runner's existing visit counting and caps.
The published maintenance, quick and focused pipelines gain a reviewer
`advisory -> next stage` edge for non-blocking notes, so an advisory-only review
continues the run instead of buying another implementer visit; `rejected ->
implementer` and its unchanged {"max_visits":3,"on":["rejected"]} limit keep a
must-fix finding looping and still terminate. The implementer gains
blocked_retryable -> implementer with {"max_visits":5,"on":["blocked_retryable"]}
for a transient environment/tooling/device problem a fresh attempt could
survive, while blocked -> #failure stays terminal. The cap is at least the
reviewer's rejection budget so the implementer's own pre-execution cap cannot
preempt the reviewer's rejected control.
No production Go code changed. New tests in internal/server/review_contract_test.go
cover advisory-only never revisiting the implementer, must-fix still looping to
the reviewer limit, the implementer cap not preempting the reviewer, retryable
blocks bounded then failing closed, a retry that recovers, and a terminal block
failing immediately; concurrency/control/human tests share the new submitRunner
and startRun seams.
Documentation checked: internal/author/skill/references/control.md and
docs/harnesses-and-control.md carry the new edges and the visit-cap rule;
no packaged template defines these pipelines (they are published immutable
versions), and docs/model-results.md needs no change because the {outcome,output}
result contract is unchanged. README and other shared docs deliberately untouched.
Verified: go build ./... and go test -count=1 ./... pass.
Residual limitation: the must-fix-vs-advisory split is carried by the prompt
contract, not a machine-readable marker, because the recorded findings have none;
a model that still returns rejected for advisory-only notes would still buy a
visit. All 18 recorded implementer blocked runs classify terminal, so the
blocked_retryable edge recovers 0 of them.
This commit is contained in:
@@ -75,6 +75,24 @@ For bounded review loops, add a fenced `limits` JSON object to the reviewer:
|
||||
On the third rejection the server terminates at failure regardless of model
|
||||
counters. Approval on that visit can still succeed. Limits never enter prompts.
|
||||
|
||||
A reviewer that must separate a blocking defect from advisory notes declares both
|
||||
edges: `rejected -> implement` (kept, with the `limits` above) is reserved for a
|
||||
must-fix finding that names a concrete location, evidence and remedy, and
|
||||
`advisory -> <next stage>` carries non-blocking duplication, performance or style
|
||||
notes. An advisory outcome continues the run and never buys another implementer
|
||||
visit. An implementer that can tell a transient environment, tooling or device
|
||||
problem from an acceptance criterion that genuinely cannot be met declares a
|
||||
bounded self-edge: `blocked_retryable -> implementer`, while `blocked -> #failure`
|
||||
stays terminal. `max_visits` caps every visit of that stage, including re-entries
|
||||
from the reviewer's `rejected` edge, so an implementer that is also the rejection
|
||||
target needs a cap at least as large as the reviewer's budget; the published
|
||||
maintenance, quick and focused pipelines use
|
||||
`{"max_visits":5,"on":["blocked_retryable"]}`. A smaller implementer cap stops the
|
||||
loop on the implementer's pre-execution limit before the reviewer's own cap can
|
||||
fire, which silently makes the reviewer's `rejected` control unreachable. The runner
|
||||
already counts the visits and stops at the cap, so neither loop needs new server
|
||||
code and no model counts visits.
|
||||
|
||||
Add a fenced `memory` map such as
|
||||
`{"task":"input.prompt","checks":"planner.checks"}` to select reviewer
|
||||
input without previous verdicts. Implementers can select
|
||||
|
||||
Reference in New Issue
Block a user