Make agent-stage retries safe and retain timeout failure evidence

Fix four stage failure-handling and observability defects (A2, A3, A6, A9).

- A2: add safeToReplay(def) as the single definition of retryable post-admission
  work - only an explicit tools block that selects neither edit nor shell. The
  agent/pi WorkStarted guards now break only when the stage is not safe to
  replay, so read-only stages genuinely retry a post-admission timeout while
  mutating and legacy-bundle stages still never replay admitted work.
- A6: re-emit a malformed final result on an explicit read-only stage up to the
  retry budget, preserving format_repairs evidence; mutating stages keep the
  immediate return and their session.
- A3: timed-out stages retain usable evidence. waitForSessionIdle records one
  bounded sanitized OperationEvidence on deadline; the agent deferred cleanup
  does a bounded best-effort latest-assistant readback and settles knowable V2
  usage; RetainSession adds a retained_session pointer to failed step
  transcripts so the session is visible beneath the BoundTranscript threshold.
- A9: a stage outcome routed to #failure now yields a non-empty run diagnostic
  ("stage <name> returned outcome <outcome>") while preferring an existing step
  diagnostic.

Adds focused stage and server tests for retry safety, malformed-result
re-emission, timeout evidence retention and run-level diagnostics; updates the
packaged skill references and docs/cli.md, docs/model-results.md.
This commit is contained in:
2026-10-15 18:27:22 +01:00
parent 0f5a188c6a
commit f57457aec9
13 changed files with 564 additions and 35 deletions
+9 -4
View File
@@ -56,9 +56,11 @@ authentication and inference require `pipelines provider-check` on the execution
computer/container. Optional `retries` defaults to 2 (three total attempts),
and accepts integers 0–100; explicit 0 disables execution retries. Optional
`timeout` defaults to `1h` per attempt and accepts positive durations such as
`30m` or `90s`. Automatic retries only cover retryable transport failures and
timeouts; admitted tool work is never replayed automatically. Optional
`budget_usd=1.50` and `budget_tokens=50000` independently cap reported usage.
`30m` or `90s`. Automatic retries cover retryable transport failures and
timeouts; an explicit read-only stage may additionally retry a post-admission
deadline or transport failure, while a stage with `edit`/`shell` (or no tools
block) never replays admitted work. Optional `budget_usd=1.50` and
`budget_tokens=50000` independently cap reported usage.
They accumulate across attempts, format repair sessions, and repeated visits
to this stage in the same run. Omit either budget to leave that resource unlimited.
Both budgets must be positive; tokens include input, output, reasoning and cache
@@ -104,7 +106,10 @@ permission policy. Keep ordinary workflows language-neutral and unrestricted
within their run directory unless the user asks for a narrower scope.
Malformed final JSON has bounded, syntax-only recovery even with `retries=0`;
it never replays stage tools or changes verdict/evidence. Compatible native stable
it never changes verdict/evidence. On an explicit read-only stage a rejected
malformed result may be re-emitted up to `retries` with its `format_repairs`
evidence retained; a stage with `edit`/`shell` or no tools block never replays
stage tools or re-emits the result. Compatible native stable
backends request schema output with schema retries disabled; a retained deny-all,
no-inference compatibility probe identifies known format incompatibility. Incompatible
native backends and V2 require strict local envelope validation. Do not promise
@@ -196,5 +196,10 @@ VALIDATION` names the exact duplicate-field/syntax detail and byte offset for a
strictly invalid final result; `HARNESS TRANSPORT` covers request, active-session
poll and final-readback failures. A validation failure never re-prompts the
admitted tool-using stage: completed edits and tool side effects are retained and
not undone, so a full retry repeats work. Neither category exposes provider/model
bodies or credentials.
not undone, so a full retry repeats work. An explicit read-only stage is the one
exception: a malformed result there mutates nothing, so it may be re-emitted up
to `retries` with its `format_repairs` evidence retained, and a post-admission
deadline or transport failure may be retried. A stage whose tools include
`edit`/`shell`, or that has no tools block (the legacy read/glob/grep/edit/shell
bundle), is never replayed. Neither category exposes provider/model bodies or
credentials.
@@ -94,6 +94,20 @@ field is sent. A review-only fresh execution is a future recovery idea, **not
implemented**. Onboarding UI visual approval remains unresolved and must not be
reported complete.
## Failure-handling source changes: 2026-10-15
Source-level, mocked `internal/stage` and `internal/server` regression coverage
(not live-provider evidence): an explicit read-only tools stage now actually
retries a post-admission stage deadline or harness transport failure, creating a
fresh bounded session per attempt, while a stage with `edit`/`shell` or no tools
block still never replays admitted work; a malformed final result is re-emitted
up to `retries` only on such a read-only stage with its `format_repairs` evidence
retained. A timed-out agent stage settles knowable V2 usage and retains one
bounded, redacted latest-assistant readback plus a `retained_session` pointer in
its stored transcript. A declared outcome routed to `#failure` with no step
diagnostic now yields a run diagnostic like `stage <key> returned outcome
"blocked"`; an existing step diagnostic is still used verbatim.
## Dated source-level coverage: 2026-10-11
Source-level/mocked regression coverage (not live-provider evidence):