Retry bounded budget reads after transient timeouts

This commit is contained in:
2026-11-05 15:48:14 +00:00
parent 3a9ba55277
commit 46f85556af
5 changed files with 261 additions and 10 deletions
@@ -0,0 +1,33 @@
# Budget accounting reads under CPU load
Mobile run 249 stopped before Code edits because a usage GET's two-second
context expired while its OpenCode CLI subprocess was starting under load. This
was an accounting transport deadline, not evidence that the configured token cap
had been reached. The accounting failure correctly failed closed, but one short
startup deadline was enough to reject a healthy stage.
The independent usage-read path now allows ten seconds per attempt and one
retry after 100 ms, only when the failure is a context deadline or a transport
error reporting Timeout(). Parent cancellation stops the current read or retry
backoff. Non-timeout transport failures are not retried. This is read-only
accounting retry; no session creation, generation, prompt or tool operation is
replayed. Initial, polling and final usage reads use the same policy.
A successful read still passes through the existing identity/shape/known-usage
checks and accumulated cap enforcement. Missing or malformed accounting, a
mismatched session, and exceeded caps are not retried or bypassed. Two persistent
timeouts still produce ErrBudgetAccounting and cancel work. Final accounting
retains its cancellation-independent context and must still report valid usage;
its failure is joined to the stage error. The finite transport grace can delay a
poll's accounting failure by at most approximately 20.1 seconds, or the parent's
shorter deadline; it does not change either configured cap or final totals.
Focused race tests use injected millisecond read timeouts. They cover transient
context/wrapped/network timeouts, independent retry contexts, one-retry bounds,
permanent failure and cancellation, cancelled backoff, initial/final retries,
malformed/missing/mismatched accounting, persistent polling failure, and final
cap enforcement. Existing live-agent no-replay and accounting-ledger tests are
included. No live service, run, workflow definition or budget cap was changed.
Validation: `go test -race ./internal/stage` passes (53.952s), as do the focused
race regression set, `go vet ./internal/stage`, gofmt and diff checks.