Merge lane-srv3: keep the lane's reviewed work

This commit is contained in:
2026-10-17 14:57:52 +01:00
15 changed files with 563 additions and 63 deletions
+33 -12
View File
@@ -82,10 +82,11 @@ Format recovery has a separate bounded budget, even with `retries=0`:
2. If the local pass cannot repair it, make at most one fresh, deny-all, retained 2. If the local pass cannot repair it, make at most one fresh, deny-all, retained
text-generation request with only the malformed response and precise parsing text-generation request with only the malformed response and precise parsing
error. The original implementing/reviewing agent is never re-prompted. The error. The original implementing/reviewing agent is never re-prompted. The
formatter's deadline is derived from the payload size and the attempt's own formatter's deadline is derived from the payload size, the stage timeout and
remaining budget rather than a fixed short constant, so a large result is not the attempt's own remaining budget rather than a fixed short constant, so a
guaranteed to time out; it stays within the stage deadline, is floored for large or slower result is not guaranteed to time out; it stays within the
ordinary payloads, and keeps the 32 KiB input/output bound. stage deadline (clamped to `remaining` minus a margin), is floored for ordinary
payloads, and keeps the 32 KiB input/output bound.
3. Accept only insertion of missing commas/colons or trailing closing delimiters, 3. Accept only insertion of missing commas/colons or trailing closing delimiters,
with at most 16 insertions. Every existing string, number, literal and with at most 16 insertions. Every existing string, number, literal and
structural delimiter must remain verbatim and in order. Validate the full structural delimiter must remain verbatim and in order. Validate the full
@@ -106,8 +107,14 @@ rerun project edits, commands, or the entire stage to repair final formatting.
Configured execution retries remain for retryable text-generation transport Configured execution retries remain for retryable text-generation transport
errors; an explicit read-only stage may additionally re-emit a malformed final errors; an explicit read-only stage may additionally re-emit a malformed final
result and retry a post-admission deadline or transport failure up to `retries`. result and retry a post-admission deadline or transport failure up to `retries`.
A stage whose tools include `edit`/`shell`, or that has no tools block (the legacy A failure to reach the harness before any prompt or tool call is admitted
read/glob/grep/edit/shell bundle), never has admitted tool work replayed. The (connecting, or creating the session) is a distinct reach failure: nothing was
admitted, so it is retried within `retries` for any stage, mutating or not, with
a fresh session and no replayed prompt or tool call. A post-admission readback,
active-session poll or agent-execution failure is retried only on an explicit
read-only stage. A stage whose tools include `edit`/`shell`, or that has no tools
block (the legacy read/glob/grep/edit/shell bundle), never has admitted tool work
replayed. The
deterministic local pass and the fresh deny-all, tool-free formatter session never deterministic local pass and the fresh deny-all, tool-free formatter session never
re-prompt or replay the admitted stage session, so a shell-enabled review stage re-prompt or replay the admitted stage session, so a shell-enabled review stage
gains malformed-result recovery without any replay of admitted work. gains malformed-result recovery without any replay of admitted work.
@@ -121,11 +128,18 @@ and a recognized safe error category (such as session not found or service
unavailable). Arbitrary stderr, provider messages and response bodies are not unavailable). Arbitrary stderr, provider messages and response bodies are not
copied into the diagnostic. A failed V2 active-session poll (or native V1 `/wait`) copied into the diagnostic. A failed V2 active-session poll (or native V1 `/wait`)
is not evidence that the agent prompt failed to start, and does not trigger replay is not evidence that the agent prompt failed to start, and does not trigger replay
of stage work. of stage work. When the context expires while a request is still in flight, the
error keeps both the sanitized transport detail (operation/status/exit) and
`errors.Is(err, context.DeadlineExceeded)`, so a deadline that fired during a hung
transport call is not replaced by a bare context error; a deadline measured at the
attempt boundary with no transport in flight stays the stage overrun it is.
Native `StructuredOutputError` is named directly, rather than reduced to a generic Native `StructuredOutputError` is named directly, rather than reduced to a generic
command failure. Native HTTP evidence records the actual HTTP operation/status; command failure, and an unrecognized provider execution-error name is reported as
an `exit_code` of -1 there is explicitly marked as not a subprocess exit. a bounded, sanitized identifier plus any 4xx/5xx status instead of a blanket
Cleanup failures preserve the original generation error and active ownership. summary (a name containing arbitrary bytes stays the generic message). Native HTTP
evidence records the actual HTTP operation/status; an `exit_code` of -1 there is
explicitly marked as not a subprocess exit. Cleanup failures preserve the original
generation error and active ownership.
For existing native sessions with format-bearing messages, the HTTP readback For existing native sessions with format-bearing messages, the HTTP readback
workaround is read-only CLI `opencode export`. The exported session ID must match workaround is read-only CLI `opencode export`. The exported session ID must match
@@ -144,6 +158,11 @@ diagnostic:
retry repeats work. retry repeats work.
- **HARNESS TRANSPORT** for request, V2 active-session poll and final-readback - **HARNESS TRANSPORT** for request, V2 active-session poll and final-readback
failures, including the sanitized `malformed or incomplete` readback message. failures, including the sanitized `malformed or incomplete` readback message.
A single classifier covers the two transport shapes: a pre-admission reach
failure (connecting or creating the session) and a post-admission failure
(readback/poll/agent execution). Both share the one `HARNESS TRANSPORT` label;
only the pre-admission shape is retried on a mutating stage, because it replays
nothing.
A structured-output validation failure is never reported as a harness transport A structured-output validation failure is never reported as a harness transport
failure, and vice versa. Neither label includes a provider response body, prompt, failure, and vice versa. Neither label includes a provider response body, prompt,
@@ -166,8 +185,10 @@ isolated formatter hit its own 30-second deadline. The malformed payload was
retrieved read-only with `pipelines inspect 55 --json` and reproduced the exact retrieved read-only with `pipelines inspect 55 --json` and reproduced the exact
byte offset 7837. It is an unescaped quote/colon inside a string value - an byte offset 7837. It is an unescaped quote/colon inside a string value - an
ambiguous case - so the deterministic pass correctly reports it unrecoverable ambiguous case - so the deterministic pass correctly reports it unrecoverable
rather than guessing, while the new derived deadline removes the fixed-deadline rather than guessing, while the stage-timeout-proportional derived deadline
failure mode for larger payloads. These are source-level findings covered by mocked (floored, capped and clamped to the remaining attempt budget) removes the
fixed-30-second-deadline failure mode for larger or slower payloads. These are
source-level findings covered by mocked
`internal/stage` fixtures. An updated checkout or installed skill is **not** `internal/stage` fixtures. An updated checkout or installed skill is **not**
evidence that a user's running server contains the fix; deploying/restarting is a evidence that a user's running server contains the fix; deploying/restarting is a
separate authorized action. separate authorized action.
+28
View File
@@ -428,3 +428,31 @@ blocked-class cause distribution and the transience of transport errors are
hypotheses, not measurements. Raw captures are under hypotheses, not measurements. Raw captures are under
`/Users/alan/.local/share/pipelines/data/artifacts/222/2/` and `/Users/alan/.local/share/pipelines/data/artifacts/222/2/` and
`/Users/alan/.local/share/pipelines/data/artifacts/222/4/` (outside the repo). `/Users/alan/.local/share/pipelines/data/artifacts/222/4/` (outside the repo).
## 7. Transport and repair recovery (source-level, 2026-10-17)
Read-only pass over the §1 diagnostics after the harness-transport and
format-repair change. No run was started, retried or cancelled; no server, ntfy,
lane or device was restarted or redeployed; no provider inference was performed.
Each shape is mapped to the code path that produces it. The per-run
pre-admission vs post-admission and read-only vs mutating split was **not**
re-inspected with `pipelines inspect`, so every per-run row is labelled *not
verified* and the counts are not summed into a "recovered" total.
| failure shape (§1 class, quoted diagnostic) | run ids | code path | can recover | remains unrecoverable |
|---|---|---|---|---|
| execution error at readback (`OpenCode agent reported an execution error`) | 73,74,75,77,80,149,150,152,195,202 | post-admission `completedAgentResult` / `opencode.ExecutionError` | a stage that is explicit read-only may retry within `retries`(*not verified per run*) | a mutating/legacy stage never replays admitted work; a non-transient provider error stays terminal |
| `context deadline exceeded` | 33,36,38,59,101,138 | attempt-boundary deadline, or a transport call that expired | a deadline during an in-flight request now keeps the transport cause and, when pre-admission, retries for any stage; a read-only stage retries a post-admission one | a genuine stage overrun on a mutating/legacy stage stays terminal; whether each run was a hung call or a real overrun is *not verified* |
| readback failure (`final assistant readback had no completed result`) | 161 | post-admission `completedAgentResult` | explicit read-only stage may retry | mutating/legacy stage never replays |
| native V1 `/wait` command failed (`CLI exit code 1`) | 4 | post-admission `openCodeError` on `/wait` | explicit read-only stage may retry | mutating/legacy stage never replays |
| repair timeout (`isolated format repair failed … context deadline exceeded`) | 55,81,174,179,183,221 | `repairDeadline` (payload bytes + stage timeout, floored, capped, clamped to remaining − margin) | a slow-but-repairable payload now gets a longer bounded budget | run 55's payload is the ambiguous unescaped-quote/colon class (byte offset 7837) → unrecoverable; any payload past the 32 KiB bound stays unrecoverable |
Mapping notes: the execution-error and readback shapes are read back *after* the
prompt was admitted, so they are post-admission transport failures and the
no-tool-replay rule keeps them terminal on a mutating stage. The
`session_transport_command_failed` shape (run 4) is the native V1 `/wait` call
issued after the prompt, not a session-create reach failure. The repair-timeout
shapes are `FINAL RESULT VALIDATION` (repair-stage), not transport. The new code
also reports an unknown provider execution-error name as a bounded sanitized
identifier plus any HTTP status, so future occurrences of runs 73–202 are more
precisely diagnostic even when they are not retried.
+14 -9
View File
@@ -20,14 +20,18 @@ Each playbook is *symptom → how to confirm → what to do*.
the model may have done work but no final result was read. the model may have done work but no final result was read.
- **Confirm** `inspect --json` shows the category on the failed stage. A request - **Confirm** `inspect --json` shows the category on the failed stage. A request
failure, an active-session poll failure and a final-readback failure all carry this failure, an active-session poll failure and a final-readback failure all carry this
one category. one category. A request that failed while the context was still in flight shows
both the sanitized transport detail and `context deadline exceeded`; an execution
error names the sanitized error name/tag and any HTTP status, not a provider body.
- **Do** Treat it as a transport failure, not a task failure. Distinguish a - **Do** Treat it as a transport failure, not a task failure. Distinguish a
pre-admission failure (nothing executed) from a post-admission one (tools may have pre-admission reach failure (connecting or creating the session; nothing executed)
run). An explicit read-only stage may retry; a stage whose tools include `edit`/ from a post-admission one (tools may have run). A pre-admission reach failure
`shell`, or that has no tools block, **never replays admitted work**. Do not re-run replays nothing, so it is retried automatically within `retries` for any stage; a
a full agent stage merely to recover a response's formatting. If it recurs, check post-admission failure is retried only on an explicit read-only stage. A stage
the service and harness health (`references/operations.md`); do not weaken the whose tools include `edit`/`shell`, or that has no tools block, **never replays
stage to get past it. admitted work**. Do not re-run a full agent stage merely to recover a response's
formatting. If it recurs, check the service and harness health
(`references/operations.md`); do not weaken the stage to get past it.
## Malformed model results ## Malformed model results
@@ -39,8 +43,9 @@ Each playbook is *symptom → how to confirm → what to do*.
- **Do** Recovery is bounded and work-preserving. A deterministic local pass first - **Do** Recovery is bounded and work-preserving. A deterministic local pass first
appends missing closers, drops one trailing comma, or inserts the single appends missing closers, drops one trailing comma, or inserts the single
comma/colon the parser expected; only if that is ambiguous does one deny-all comma/colon the parser expected; only if that is ambiguous does one deny-all
formatter request run, with a deadline derived from the payload size and the formatter request run, with a deadline derived from the payload size, the stage
attempt's remaining budget. Each path records a `repaired`, `partial` or timeout and the attempt's remaining budget, clamped so it never extends the
stage's own deadline. Each path records a `repaired`, `partial` or
`unrecoverable` status in `format_repairs`; a partial or unrecoverable payload `unrecoverable` status in `format_repairs`; a partial or unrecoverable payload
is never accepted as complete. On an explicit read-only stage a malformed result is never accepted as complete. On an explicit read-only stage a malformed result
may be re-emitted up to `retries`; a stage with `edit`/`shell` or no tools block may be re-emitted up to `retries`; a stage with `edit`/`shell` or no tools block
@@ -350,7 +350,11 @@ admitted tool-using stage: completed edits and tool side effects are retained an
not undone, so a full retry repeats work. An explicit read-only stage is the one not undone, so a full retry repeats work. An explicit read-only stage is the one
exception: a malformed result there mutates nothing, so it may be re-emitted up exception: a malformed result there mutates nothing, so it may be re-emitted up
to `retries` with its `format_repairs` evidence retained, and a post-admission to `retries` with its `format_repairs` evidence retained, and a post-admission
deadline or transport failure may be retried. A stage whose tools include deadline or transport failure may be retried. A failure to reach the harness
before any prompt or tool call is admitted (connecting, or creating the session)
replays nothing, so it is retried within `retries` for any stage, mutating or not.
A stage whose tools include
`edit`/`shell`, or that has no tools block (the legacy read/glob/grep/edit/shell `edit`/`shell`, or that has no tools block (the legacy read/glob/grep/edit/shell
bundle), is never replayed. Neither category exposes provider/model bodies or bundle), is never replayed. Neither category exposes provider/model bodies or
credentials. credentials. The isolated formatter's deadline is proportional to the stage
timeout and clamped so it never extends the stage's own deadline.
@@ -80,7 +80,10 @@ specific authorization.
Retained model-stage failures now carry a fixed sanitized category. A strictly Retained model-stage failures now carry a fixed sanitized category. A strictly
invalid final result (a duplicate nested field, or malformed/ambiguous JSON with a invalid final result (a duplicate nested field, or malformed/ambiguous JSON with a
byte offset) is labeled `FINAL RESULT VALIDATION`; request, V2 active-session poll byte offset) is labeled `FINAL RESULT VALIDATION`; request, V2 active-session poll
and final-readback failures are labeled `HARNESS TRANSPORT`. A validation failure and final-readback failures are labeled `HARNESS TRANSPORT`. A pre-admission
request failure replays nothing and is retried for any stage; a post-admission
failure is retried only on an explicit read-only stage (see the 2026-10-17 section
below). A validation failure
preserves the admitted session, transcript and evidence and performs no second tool preserves the admitted session, transcript and evidence and performs no second tool
execution. Confirmed ordinary-endpoint failures on 2026-10-11: an all-history execution. Confirmed ordinary-endpoint failures on 2026-10-11: an all-history
readback of exactly 262144 bytes of incomplete JSON (`invalid agent messages`), a readback of exactly 262144 bytes of incomplete JSON (`invalid agent messages`), a
@@ -100,8 +103,9 @@ Source-level, mocked `internal/stage` regression coverage (not live-provider
evidence and not a deployment): malformed final-result recovery now tries a evidence and not a deployment): malformed final-result recovery now tries a
deterministic local pass first (append missing closers, drop one trailing comma, deterministic local pass first (append missing closers, drop one trailing comma,
or insert the single comma/colon the parser expected), then at most one deny-all or insert the single comma/colon the parser expected), then at most one deny-all
formatter request whose deadline is derived from the payload size and the formatter request whose deadline is derived from the payload size, the stage
attempt's remaining budget rather than a fixed 30-second constant. Every path timeout and the attempt's remaining budget rather than a fixed 30-second
constant. Every path
records `repaired`, `partial` or `unrecoverable` in `format_repairs`; a `partial` records `repaired`, `partial` or `unrecoverable` in `format_repairs`; a `partial`
or `unrecoverable` payload is never returned as a completed result, and an or `unrecoverable` payload is never returned as a completed result, and an
ambiguous payload (an unescaped quote/colon inside a string value) is never ambiguous payload (an unescaped quote/colon inside a string value) is never
@@ -112,6 +116,26 @@ unchanged. The run-55 `ui_reviewer` payload was retrieved read-only with
ambiguous error class, so it is reported unrecoverable rather than repaired. This ambiguous error class, so it is reported unrecoverable rather than repaired. This
does not update or restart any running server. does not update or restart any running server.
## Harness transport reach retry and repair deadline: 2026-10-17
Source-level, mocked `internal/stage`, `internal/opencode` and `internal/server`
regression coverage (not live-provider evidence and not a deployment): the
harness transport path now distinguishes a pre-admission reach failure
(connecting or creating the session) from a post-admission readback/poll/agent
failure under the one `HARNESS TRANSPORT` category. A pre-admission reach failure
replays nothing, so it is retried within `retries` for any stage with a fresh
session and no replayed prompt; a post-admission failure is retried only on an
explicit read-only stage, and an `edit`/`shell`/legacy stage still never replays
admitted work. A context that expires during a request keeps both the sanitized
transport detail (operation/status/exit) and
`errors.Is(err, context.DeadlineExceeded)`; a bare attempt-boundary deadline is
not treated as a reach failure and stays terminal for a mutating stage. An
unrecognized provider execution-error name is reported as a bounded, sanitized
identifier plus any HTTP status. The isolated formatter deadline is now
proportional to the stage timeout (floored, capped, and clamped to
`remaining - margin` so it can never extend the stage/attempt deadline). These are
mocked fixtures; a deployed server is not updated by this change.
## Failure-handling source changes: 2026-10-15 ## Failure-handling source changes: 2026-10-15
Source-level, mocked `internal/stage` and `internal/server` regression coverage Source-level, mocked `internal/stage` and `internal/server` regression coverage
+29
View File
@@ -81,6 +81,15 @@ func ExecutionError(data json.RawMessage) error {
} }
safeNames := map[string]bool{"APIError": true, "ProviderAuthError": true, "ContextOverflowError": true, "MessageAbortedError": true, "OutputLengthError": true, "UnknownError": true, "ServiceUnavailableError": true} safeNames := map[string]bool{"APIError": true, "ProviderAuthError": true, "ContextOverflowError": true, "MessageAbortedError": true, "OutputLengthError": true, "UnknownError": true, "ServiceUnavailableError": true}
if !safeNames[name] { if !safeNames[name] {
// An unrecognized provider error name is still reported, but only as a
// bounded, safe identifier plus any HTTP status. No provider body,
// message, prompt or credential is copied.
if safe := sanitizedExecutionName(name); safe != "" {
if failure.Data.StatusCode >= 400 && failure.Data.StatusCode <= 599 {
return fmt.Errorf("OpenCode agent reported an execution error: %s (HTTP %d)", safe, failure.Data.StatusCode)
}
return fmt.Errorf("OpenCode agent reported an execution error: %s", safe)
}
return fmt.Errorf("OpenCode agent reported an execution error") return fmt.Errorf("OpenCode agent reported an execution error")
} }
if failure.Data.StatusCode >= 400 && failure.Data.StatusCode <= 599 { if failure.Data.StatusCode >= 400 && failure.Data.StatusCode <= 599 {
@@ -88,3 +97,23 @@ func ExecutionError(data json.RawMessage) error {
} }
return fmt.Errorf("OpenCode agent execution error: %s", name) return fmt.Errorf("OpenCode agent execution error: %s", name)
} }
// sanitizedExecutionName reduces an unknown provider error name/tag to a
// bounded, safe identifier so the diagnostic can name the failure shape without
// copying an arbitrary message, prompt or credential. A name containing any
// other byte returns empty and is reported as a generic execution error.
func sanitizedExecutionName(name string) string {
if len(name) > 64 {
name = name[:64]
}
var b strings.Builder
for _, r := range name {
switch {
case r >= 'a' && r <= 'z', r >= 'A' && r <= 'Z', r >= '0' && r <= '9', r == '_', r == '.', r == '-':
b.WriteRune(r)
default:
return ""
}
}
return b.String()
}
+23
View File
@@ -90,3 +90,26 @@ func TestNativeStructuredFailureIsClearAndDoesNotReplayGeneration(t *testing.T)
t.Fatal("wait failure discarded provider response", err, string(retained)) t.Fatal("wait failure discarded provider response", err, string(retained))
} }
} }
// TestExecutionErrorReportsSanitizedNameAndStatusWithoutBody proves an
// unrecognized provider error name is reported as a bounded, safe identifier
// plus any HTTP status instead of a blanket summary, while no provider message,
// prompt or credential is copied and known names keep their exact wording.
func TestExecutionErrorReportsSanitizedNameAndStatusWithoutBody(t *testing.T) {
err := ExecutionError(json.RawMessage(`{"name":"TotallyUnknownError","data":{"statusCode":502,"message":"secret provider body"}}`))
if err == nil || !strings.Contains(err.Error(), "TotallyUnknownError") || !strings.Contains(err.Error(), "HTTP 502") {
t.Fatalf("unknown error name/status not reported: %v", err)
}
if strings.Contains(err.Error(), "secret provider body") {
t.Fatalf("diagnostic leaked the provider body: %v", err)
}
if err := ExecutionError(json.RawMessage(`{"_tag":"SomeTagError"}`)); err == nil || !strings.Contains(err.Error(), "SomeTagError") {
t.Fatalf("unknown tag not reported: %v", err)
}
if err := ExecutionError(json.RawMessage(`{"name":"Injected Error: Bearer secret-token"}`)); err == nil || strings.Contains(err.Error(), "secret-token") || strings.Contains(err.Error(), "Bearer") || err.Error() != "OpenCode agent reported an execution error" {
t.Fatalf("unsafe name was copied into the diagnostic: %v", err)
}
if err := ExecutionError(json.RawMessage(`{"name":"APIError","data":{"statusCode":500}}`)); err == nil || !strings.Contains(err.Error(), "APIError (HTTP 500)") {
t.Fatalf("known safe name changed: %v", err)
}
}
+94
View File
@@ -247,3 +247,97 @@ func TestRetryRejectsUnavailableOriginalDirectoryAndNonfailedRuns(t *testing.T)
t.Fatal("nonexistent run accepted") t.Fatal("nonexistent run accepted")
} }
} }
// TestReadOnlyStageRetriesPostAdmissionTransportButMutatingStageDoesNot proves,
// through the real runner and CLI fixture, that a read-only stage retries a
// post-admission transport (final-readback) failure with a fresh session while
// an edit stage never replays the admitted prompt, and that the failed step's
// diagnostic retains the underlying transport cause.
func TestReadOnlyStageRetriesPostAdmissionTransportButMutatingStageDoesNot(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
dir := t.TempDir()
executable := filepath.Join(dir, "opencode")
t.Setenv("PIPELINES_OPENCODE", executable)
scriptTemplate := `#!/bin/sh
if [ "$1" = "--help" ]; then echo ' api'; exit 0; fi
printf '%s %s\n' "$2" "$3" >> "@STATE@/calls"
case "$3" in
/api/session)
n=0; if [ -f "@STATE@/sessions" ]; then n=$(cat "@STATE@/sessions"); fi
n=$((n+1)); echo "$n" > "@STATE@/sessions"
printf '{"data":{"id":"ses_bounded_%s"}}' "$n" ;;
*/prompt) echo '{}' ;;
/api/session/active) echo '{"data":{}}' ;;
*/message*)
r=0; if [ -f "@STATE@/readbacks" ]; then r=$(cat "@STATE@/readbacks"); fi
r=$((r+1)); echo "$r" > "@STATE@/readbacks"
if [ "$r" = "1" ]; then
echo '{"data":[{"finish":"stop","content":'
else
echo '{"data":[{"finish":"stop","content":[{"type":"text","text":"{\"outcome\":\"success\",\"output\":{\"summary\":\"done\"}}"}]}]}'
fi ;;
*/interrupt) echo '{}' ;;
*) exit 99 ;;
esac
`
store, err := pipeline.Open(filepath.Join(dir, "runs.db"))
if err != nil {
t.Fatal(err)
}
defer store.Close()
r, err := newRunner(ctx, store)
if err != nil {
t.Fatal(err)
}
defer func() { cancel(); r.wg.Wait() }()
toolsBlock := func(tools string) string {
return "# stage worker\n\n```output\nResult.\n```\n\n```prompt\nDo the work.\n```\n\n```tools\n" + tools + "\n```\n\n```edges\nsuccess -> #success\nerror -> #failure\n```\n\n```configs\ntype=model\nprovider=opencode-go\nmodel=deepseek-v4.1-flash\nretries=1\ntimeout=30s\n```\n"
}
runStage := func(name, tools string) (status, sessions, diagnostic string) {
t.Helper()
workspace := t.TempDir()
script := strings.ReplaceAll(scriptTemplate, "@STATE@", workspace)
if err := os.WriteFile(executable, []byte(script), 0700); err != nil {
t.Fatal(err)
}
if _, err := store.Submit(ctx, name, toolsBlock(tools)); err != nil {
t.Fatal(err)
}
items, _ := store.List(ctx)
var id int64
for _, item := range items {
if item.Name == name {
id = item.ID
}
}
started, err := r.start(protocol.Message{PipelineID: id, Version: 1, RequestKey: name, Workspace: workspace, Input: map[string]any{"prompt": "task"}})
if err != nil {
t.Fatal(err)
}
r.wg.Wait()
detail, err := store.GetRun(ctx, started.ID)
if err != nil {
t.Fatal(err)
}
if len(detail.Steps) > 0 {
diagnostic = detail.Steps[0].Diagnostic
}
if data, err := os.ReadFile(filepath.Join(workspace, "sessions")); err == nil {
sessions = strings.TrimSpace(string(data))
}
return detail.Status, sessions, diagnostic
}
status, sessions, diagnostic := runStage("readonly", "read")
if status != "succeeded" || sessions != "2" || diagnostic != "" {
t.Fatalf("read-only stage did not retry the transport readback: status=%q sessions=%q diagnostic=%q", status, sessions, diagnostic)
}
status, sessions, diagnostic = runStage("edit", "edit")
if status != "failed" || sessions != "1" {
t.Fatalf("edit stage replayed admitted work: status=%q sessions=%q diagnostic=%q", status, sessions, diagnostic)
}
if !strings.Contains(diagnostic, "HARNESS TRANSPORT") || !strings.Contains(diagnostic, "malformed or incomplete") {
t.Fatalf("step diagnostic lost the underlying transport cause: %q", diagnostic)
}
}
+5 -2
View File
@@ -55,7 +55,8 @@ func (c *AgentClient) Generate(ctx context.Context, config ModelConfig, prompt s
if c.command == nil { if c.command == nil {
connection, err := opencode.Open(ctx, opencode.Executable(), c.Directory, c.Backend) connection, err := opencode.Open(ctx, opencode.Executable(), c.Directory, c.Backend)
if err != nil { if err != nil {
return "", err // Connecting to the service failed before any session existed.
return "", harnessUnreachable(err)
} }
defer connection.Close() defer connection.Close()
c.connection = connection c.connection = connection
@@ -64,7 +65,9 @@ func (c *AgentClient) Generate(ctx context.Context, config ModelConfig, prompt s
rules := c.permissionRules() rules := c.permissionRules()
data, err := c.call(ctx, "post", "/api/session", map[string]any{"title": "Pipeline audited agent stage", "agent": "build", "location": map[string]string{"directory": c.Directory}, "model": map[string]string{"providerID": config.Provider, "id": config.Model}, "permissions": rules}) data, err := c.call(ctx, "post", "/api/session", map[string]any{"title": "Pipeline audited agent stage", "agent": "build", "location": map[string]string{"directory": c.Directory}, "model": map[string]string{"providerID": config.Provider, "id": config.Model}, "permissions": rules})
if err != nil { if err != nil {
return "", err // No prompt or tool call was admitted: a request failure here is a
// reachable retry rather than a reason to end the run.
return "", harnessUnreachable(err)
} }
var created struct { var created struct {
Data struct { Data struct {
+40 -9
View File
@@ -27,15 +27,42 @@ const (
// diagnostics. It states the retained side effects and the cost of a full retry. // diagnostics. It states the retained side effects and the cost of a full retry.
const finalResultGuidance = "completed edits and tool side effects are retained and not undone; a full retry repeats work" const finalResultGuidance = "completed edits and tool side effects are retained and not undone; a full retry repeats work"
// harnessTransportFailure is the single classifier shared by both transport
// sentinels and the typed OpenCode error: a request, poll or readback failure
// that reports as the fixed HARNESS TRANSPORT category. A bare context deadline
// is classified separately so a genuine stage overrun is never relabeled as a
// transport failure.
func harnessTransportFailure(err error) bool {
var openCode *openCodeError
return errors.Is(err, ErrHarnessUnreachable) || errors.Is(err, ErrHarnessTransport) || errors.As(err, &openCode)
}
// harnessReachFailure reports whether err is a pre-admission harness reach
// failure. It is distinct from a post-admission ErrHarnessTransport failure: no
// session or prompt was admitted, so the retry replays no work.
func harnessReachFailure(err error) bool {
return errors.Is(err, ErrHarnessUnreachable)
}
// admittedWorker is implemented by a client that can admit tool work before its
// final result is read, so the retry loop can refuse to replay it without a
// per-client type switch.
type admittedWorker interface{ workAdmitted() bool }
// workAdmitted reports whether the client admitted any prompt or tool call, so
// the retry loop may retry a transport failure on any stage only when it is
// false, and only a read-only stage otherwise.
func (c *AgentClient) workAdmitted() bool { return c.WorkStarted }
func (c *PiClient) workAdmitted() bool { return c.WorkStarted }
// modelFailureCategory classifies a stage failure using error identity only, so // modelFailureCategory classifies a stage failure using error identity only, so
// the label stays sanitized and bounded. Validation is checked first: a // the label stays sanitized and bounded. Validation is checked first: a
// structured-output failure is never reported as a transport failure and vice versa. // structured-output failure is never reported as a transport failure and vice versa.
func modelFailureCategory(err error) string { func modelFailureCategory(err error) string {
var openCode *openCodeError
switch { switch {
case errors.Is(err, ErrInvalidModelResult): case errors.Is(err, ErrInvalidModelResult):
return finalResultValidation return finalResultValidation
case errors.Is(err, ErrHarnessTransport), errors.As(err, &openCode), errors.Is(err, context.DeadlineExceeded): case harnessTransportFailure(err), errors.Is(err, context.DeadlineExceeded):
return harnessTransport return harnessTransport
default: default:
return "" return ""
@@ -294,15 +321,19 @@ func (m ModelStage) Run(ctx context.Context, def Definition, input Values) (Resu
break break
} }
replayPostAdmission = safeToReplay(def) && replayablePostAdmission(err) replayPostAdmission = safeToReplay(def) && replayablePostAdmission(err)
// Never blindly replay admitted tool work: only a read-only stage may // A reach failure observed before any work was admitted replays
// retry a post-admission deadline or transport failure. // nothing, so it is retryable within the existing budget for any
if agent, ok := client.(*AgentClient); ok && agent.WorkStarted && !replayPostAdmission { // stage. After admission, only an explicit read-only stage may
// retry; admitted tool work is never replayed.
workStarted := false
if worker, ok := client.(admittedWorker); ok {
workStarted = worker.workAdmitted()
}
if workStarted && !replayPostAdmission {
break break
} }
if pi, ok := client.(*PiClient); ok && pi.WorkStarted && !replayPostAdmission { preAdmissionReach := !workStarted && harnessReachFailure(err)
break if attempt == spec.Config.Retries || (!retryableModelError(err) && !errors.Is(err, context.DeadlineExceeded) && !replayPostAdmission && !preAdmissionReach) {
}
if attempt == spec.Config.Retries || (!retryableModelError(err) && !errors.Is(err, context.DeadlineExceeded) && !replayPostAdmission) {
break break
} }
} }
+17
View File
@@ -18,6 +18,23 @@ var ErrInvalidModelResult = errors.New("invalid model stage result")
// validation and never replay admitted tool work. // validation and never replay admitted tool work.
var ErrHarnessTransport = errors.New("harness transport failure") var ErrHarnessTransport = errors.New("harness transport failure")
// ErrHarnessUnreachable marks a request/transport failure observed before the
// harness admitted any agent work: connecting to the service, or creating the
// session. Nothing was executed or admitted, so re-attempting replays no prompt
// or tool call and is safe for any stage, mutating or not. Post-admission
// readback, poll and agent-execution failures stay ErrHarnessTransport so the
// single fixed HARNESS TRANSPORT category and the no-tool-replay rule are
// unchanged.
var ErrHarnessUnreachable = errors.New("harness unreachable")
// harnessUnreachable marks err as a pre-admission reach failure. It is the single
// wrap shared by the connect and session-create paths of both client types, so
// they cannot drift: the sentinel and the underlying cause are always both
// preserved.
func harnessUnreachable(err error) error {
return fmt.Errorf("%w: %w", ErrHarnessUnreachable, err)
}
func modelPrompt(def Definition, spec ModelSpec, input Values) (string, error) { func modelPrompt(def Definition, spec ModelSpec, input Values) (string, error) {
outcomes := make([]string, 0, len(def.Edges)) outcomes := make([]string, 0, len(def.Edges))
for name := range def.Edges { for name := range def.Edges {
+20 -9
View File
@@ -97,10 +97,14 @@ func (c OpenCodeClient) call(ctx context.Context, method, path string, body any)
evidenceStatus = c.connection.LastHTTPStatus evidenceStatus = c.connection.LastHTTPStatus
} }
c.Operations.record(OperationEvidence{Operation: method + " " + path, Backend: c.Backend, Executable: opencode.Executable(), Directory: c.Directory, ExitCode: -1, HTTPStatus: evidenceStatus, NativeOperation: c.connection.LastHTTPOperation, Diagnostic: err.Error(), Stdout: string(data)}, started) c.Operations.record(OperationEvidence{Operation: method + " " + path, Backend: c.Backend, Executable: opencode.Executable(), Directory: c.Directory, ExitCode: -1, HTTPStatus: evidenceStatus, NativeOperation: c.connection.LastHTTPOperation, Diagnostic: err.Error(), Stdout: string(data)}, started)
if ctx.Err() != nil { transport := &openCodeError{operation: method + " " + path, status: status, cause: err, diagnostic: err.Error()}
return nil, ctx.Err() if ctxErr := ctx.Err(); ctxErr != nil {
// Keep the transport shape (operation/status) with the context
// cause: a deadline that fired while the request was in flight is
// both, and neither may be discarded.
return nil, errors.Join(ctxErr, transport)
} }
return nil, &openCodeError{operation: method + " " + path, status: status, cause: err, diagnostic: err.Error()} return nil, transport
} }
return data, nil return data, nil
} }
@@ -146,14 +150,18 @@ func (c OpenCodeClient) call(ctx context.Context, method, path string, body any)
code = exit.ExitCode() code = exit.ExitCode()
} }
c.Operations.record(OperationEvidence{Operation: method + " " + path, Backend: c.Backend, Executable: cmd.Path, Directory: directory, ExitCode: code, HTTPStatus: status, Diagnostic: safeCommandDiagnostic(err, stdout.Bytes()), Stdout: stdout.String(), Stderr: stderr.String()}, started) c.Operations.record(OperationEvidence{Operation: method + " " + path, Backend: c.Backend, Executable: cmd.Path, Directory: directory, ExitCode: code, HTTPStatus: status, Diagnostic: safeCommandDiagnostic(err, stdout.Bytes()), Stdout: stdout.String(), Stderr: stderr.String()}, started)
if ctx.Err() != nil {
return nil, ctx.Err()
}
// Do not embed provider responses, prompts, credentials, or CLI logs in errors. // Do not embed provider responses, prompts, credentials, or CLI logs in errors.
// Keep the raw cause so errors.Is(err, exec.ErrWaitDelay) still holds while // Keep the raw cause so errors.Is(err, exec.ErrWaitDelay) still holds while
// Error() stays sanitized. This branch is only reached with a non-nil Run // Error() stays sanitized. This branch is only reached with a non-nil Run
// error, so valid-looking partial stdout is never returned as success. // error, so valid-looking partial stdout is never returned as success.
return nil, &openCodeError{operation: method + " " + path, status: status, cause: err, diagnostic: safeCommandDiagnostic(err, stdout.Bytes())} transport := &openCodeError{operation: method + " " + path, status: status, cause: err, diagnostic: safeCommandDiagnostic(err, stdout.Bytes())}
if ctxErr := ctx.Err(); ctxErr != nil {
// Keep the sanitized transport detail (operation/status/exit) together
// with errors.Is(err, context.DeadlineExceeded) so the true cause is
// not replaced by a bare context error.
return nil, errors.Join(ctxErr, transport)
}
return nil, transport
} }
if stdout.overflow { if stdout.overflow {
return nil, fmt.Errorf("OpenCode response exceeds 1 MiB limit") return nil, fmt.Errorf("OpenCode response exceeds 1 MiB limit")
@@ -259,7 +267,8 @@ func (c *OpenCodeClient) Generate(ctx context.Context, config ModelConfig, promp
if c.command == nil { if c.command == nil {
connection, err := opencode.Open(ctx, opencode.Executable(), c.Directory, c.Backend) connection, err := opencode.Open(ctx, opencode.Executable(), c.Directory, c.Backend)
if err != nil { if err != nil {
return "", err // Connecting to the service failed before any session existed.
return "", harnessUnreachable(err)
} }
defer connection.Close() defer connection.Close()
c.connection = connection c.connection = connection
@@ -288,7 +297,9 @@ func (c *OpenCodeClient) Generate(ctx context.Context, config ModelConfig, promp
"permissions": []map[string]string{{"action": "*", "resource": "*", "effect": "deny"}}, "permissions": []map[string]string{{"action": "*", "resource": "*", "effect": "deny"}},
}) })
if err != nil { if err != nil {
return "", err // No prompt or tool call was admitted: a request failure here is a
// reachable retry rather than a reason to end the run.
return "", harnessUnreachable(err)
} }
var session struct { var session struct {
Data struct { Data struct {
+13 -11
View File
@@ -129,14 +129,15 @@ func repairModelResult(ctx context.Context, client ModelGenerator, config ModelC
return Result{}, fmt.Errorf("%w; no isolated format repair available; recovery status %s", parseError, repairStatusUnrecoverable) return Result{}, fmt.Errorf("%w; no isolated format repair available; recovery status %s", parseError, repairStatusUnrecoverable)
} }
// One formatter request, independent of execution/transport retries. The // One formatter request, independent of execution/transport retries. The
// deadline is derived from the payload size and the attempt's own remaining // deadline is derived from the payload size, the stage timeout and the
// budget rather than a fixed short constant, so a larger result is not // attempt's own remaining budget rather than a fixed short constant, so a
// guaranteed to time out while the attempt context remains the outer bound. // larger or slower result is not guaranteed to time out while the attempt
// context remains the outer bound.
remaining := time.Hour remaining := time.Hour
if deadline, ok := ctx.Deadline(); ok { if deadline, ok := ctx.Deadline(); ok {
remaining = time.Until(deadline) remaining = time.Until(deadline)
} }
repairCtx, cancel := context.WithTimeout(ctx, repairDeadline(len(original), remaining)) repairCtx, cancel := context.WithTimeout(ctx, repairDeadline(len(original), remaining, config.Timeout))
defer cancel() defer cancel()
candidate, err := formatter.RepairFormat(repairCtx, config, original, parseError) candidate, err := formatter.RepairFormat(repairCtx, config, original, parseError)
if err := ctx.Err(); err != nil { if err := ctx.Err(); err != nil {
@@ -291,9 +292,10 @@ func lexemeOffsets(text string, tokens []string) ([]int, bool) {
} }
// Isolated formatter deadline bounds. The allowance grows by one second per KiB // Isolated formatter deadline bounds. The allowance grows by one second per KiB
// of payload up to a ceiling, is floored for ordinary payloads, and is clamped // of payload up to a ceiling and by a share of the stage timeout, is floored for
// to the attempt's remaining budget so the stage/attempt context stays the outer // ordinary payloads, and is clamped to the attempt's remaining budget so the
// bound and a large result is not guaranteed to time out. // stage/attempt context stays the outer bound and a large or slow result is not
// guaranteed to time out.
const ( const (
repairDeadlineFloor = 30 * time.Second repairDeadlineFloor = 30 * time.Second
repairDeadlineMax = 10 * time.Minute repairDeadlineMax = 10 * time.Minute
@@ -301,13 +303,13 @@ const (
) )
// repairDeadline derives the formatter deadline. It is O(1) and monotonic // repairDeadline derives the formatter deadline. It is O(1) and monotonic
// non-decreasing in payloadBytes; it never exceeds remaining and, when the // non-decreasing in payloadBytes and stageTimeout; it never exceeds remaining
// remaining budget allows, never drops below repairDeadlineFloor. // and, when the remaining budget allows, never drops below repairDeadlineFloor.
func repairDeadline(payloadBytes int, remaining time.Duration) time.Duration { func repairDeadline(payloadBytes int, remaining, stageTimeout time.Duration) time.Duration {
if remaining <= 0 { if remaining <= 0 {
return 0 return 0
} }
allowance := repairDeadlineFloor + time.Duration(payloadBytes/1024)*time.Second allowance := repairDeadlineFloor + time.Duration(payloadBytes/1024)*time.Second + stageTimeout/4
if allowance > repairDeadlineMax { if allowance > repairDeadlineMax {
allowance = repairDeadlineMax allowance = repairDeadlineMax
} }
+17 -6
View File
@@ -306,26 +306,37 @@ func TestPresentationStrippingBeforeDeterministicRepair(t *testing.T) {
// TestRepairDeadlineIsMonotonicFlooredAndClamped proves the derived deadline, not // TestRepairDeadlineIsMonotonicFlooredAndClamped proves the derived deadline, not
// a fixed constant, bounds the single formatter request. // a fixed constant, bounds the single formatter request.
func TestRepairDeadlineIsMonotonicFlooredAndClamped(t *testing.T) { func TestRepairDeadlineIsMonotonicFlooredAndClamped(t *testing.T) {
if got := repairDeadline(0, time.Hour); got != repairDeadlineFloor { if got := repairDeadline(0, time.Hour, time.Minute); got < repairDeadlineFloor || got > repairDeadlineMax {
t.Fatalf("floor not applied: %s", got) t.Fatalf("floor not applied: %s", got)
} }
if got := repairDeadline(1<<30, 24*time.Hour); got != repairDeadlineMax { if got := repairDeadline(1<<30, 24*time.Hour, time.Hour); got != repairDeadlineMax {
t.Fatalf("ceiling not applied: %s", got) t.Fatalf("ceiling not applied: %s", got)
} }
if got := repairDeadline(10, 0); got != 0 { if got := repairDeadline(10, 0, time.Hour); got != 0 {
t.Fatalf("zero remaining should yield zero: %s", got) t.Fatalf("zero remaining should yield zero: %s", got)
} }
if clamped := repairDeadline(1<<20, 5*time.Second); clamped != 5*time.Second-repairDeadlineMargin || clamped > 5*time.Second { if clamped := repairDeadline(1<<20, 5*time.Second, time.Hour); clamped != 5*time.Second-repairDeadlineMargin || clamped > 5*time.Second {
t.Fatalf("remaining budget was not the bound: %s", clamped) t.Fatalf("remaining budget was not the bound: %s", clamped)
} }
previous := time.Duration(0) previous := time.Duration(0)
for size := 0; size <= 64<<10; size += 1024 { for size := 0; size <= 64<<10; size += 1024 {
got := repairDeadline(size, time.Hour) got := repairDeadline(size, time.Hour, time.Minute)
if got < previous { if got < previous {
t.Fatalf("deadline not monotonic at %d: %s < %s", size, got, previous) t.Fatalf("deadline not monotonic in payload at %d: %s < %s", size, got, previous)
} }
previous = got previous = got
} }
previous = time.Duration(0)
for stage := time.Duration(0); stage <= 2*time.Hour; stage += time.Minute {
got := repairDeadline(64<<10, time.Hour, stage)
if got < previous {
t.Fatalf("deadline not monotonic in stage timeout at %s: %s < %s", stage, got, previous)
}
previous = got
}
if got := repairDeadline(8<<10, time.Hour, time.Hour); got < repairDeadlineFloor {
t.Fatalf("ordinary payload was not floored: %s", got)
}
} }
// TestShellStageRecoverableMalformedResultDoesNotFailRun proves a shell-enabled // TestShellStageRecoverableMalformedResultDoesNotFailRun proves a shell-enabled
+197
View File
@@ -0,0 +1,197 @@
package stage
import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"sync/atomic"
"testing"
"time"
"git.a-wan.net/alan/pipelines/internal/opencode"
)
// TestPreAdmissionReachFailureRetriesOnMutatingStage proves a failure to reach
// the harness before any prompt or tool call is admitted retries within the
// existing budget even on an edit stage, with a fresh session and no replayed
// prompt. Nothing was admitted, so no admitted work can be replayed.
func TestPreAdmissionReachFailureRetriesOnMutatingStage(t *testing.T) {
def := modelDefinition()
def.Blocks["tools"] = "edit"
def.Configs["retries"] = "1"
var sessions, prompts atomic.Int64
agent := &AgentClient{OpenCodeClient: OpenCodeClient{command: func(_ context.Context, _ string, args ...string) ([]byte, error) {
path := args[2]
switch {
case path == "/api/session":
if sessions.Add(1) == 1 {
return nil, errors.New("connection refused")
}
return []byte(`{"data":{"id":"ses_reach_2"}}`), nil
case strings.HasSuffix(path, "/prompt"):
prompts.Add(1)
return []byte(`{}`), nil
case path == "/api/session/active":
return []byte(`{"data":{}}`), nil
case strings.HasSuffix(path, finalAssistantReadback):
return []byte(`{"data":[{"finish":"stop","content":[{"type":"text","text":"{\"outcome\":\"success\",\"output\":{\"summary\":\"done\"}}"}]}]}`), nil
}
return nil, fmt.Errorf("unexpected path %s", path)
}}}
result, err := (ModelStage{Client: agent}).Run(context.Background(), def, nil)
if err != nil || result.Output["summary"] != "done" {
t.Fatalf("pre-admission reach failure was not retried to success: %v %v", result, err)
}
if sessions.Load() != 2 || prompts.Load() != 1 || agent.SessionID != "ses_reach_2" {
t.Fatalf("reach retry did not use exactly one fresh session: sessions=%d prompts=%d session=%q", sessions.Load(), prompts.Load(), agent.SessionID)
}
}
// TestPostAdmissionReadbackFailureRetriesOnlyOnReadOnlyStage proves a readback
// transport failure observed after the prompt was admitted is retried only on an
// explicit read-only stage; an edit, shell or legacy-bundle stage never replays
// the admitted prompt and ends with a HARNESS TRANSPORT failure.
func TestPostAdmissionReadbackFailureRetriesOnlyOnReadOnlyStage(t *testing.T) {
run := func(name string, setTools bool, tools string) (Result, error, int64, int64) {
t.Helper()
def := modelDefinition()
if setTools {
def.Blocks["tools"] = tools
}
def.Configs["retries"] = "1"
var sessions, prompts, readbacks atomic.Int64
agent := &AgentClient{OpenCodeClient: OpenCodeClient{command: func(_ context.Context, _ string, args ...string) ([]byte, error) {
path := args[2]
switch {
case path == "/api/session":
n := sessions.Add(1)
return []byte(fmt.Sprintf(`{"data":{"id":"ses_rb_%d"}}`, n)), nil
case strings.HasSuffix(path, "/prompt"):
prompts.Add(1)
return []byte(`{}`), nil
case path == "/api/session/active":
return []byte(`{"data":{}}`), nil
case strings.HasSuffix(path, finalAssistantReadback):
if readbacks.Add(1) == 1 {
// Truncated readback: a post-admission transport failure.
return []byte(`{"data":[{"finish":"stop","content":[{"type":"text","text":"{\"outcome\":`), nil
}
return []byte(`{"data":[{"finish":"stop","content":[{"type":"text","text":"{\"outcome\":\"success\",\"output\":{\"summary\":\"ok\"}}"}]}]}`), nil
case strings.HasSuffix(path, "/interrupt"):
return []byte(`{}`), nil
}
return nil, fmt.Errorf("unexpected path %s", path)
}}}
result, err := (ModelStage{Client: agent}).Run(context.Background(), def, nil)
return result, err, sessions.Load(), prompts.Load()
}
result, err, sessions, prompts := run("read-only", true, "read")
if err != nil || result.Output["summary"] != "ok" || sessions != 2 || prompts != 2 {
t.Fatalf("read-only readback failure was not retried: %v %v sessions=%d prompts=%d", result, err, sessions, prompts)
}
for _, tc := range []struct {
name string
setTools bool
tools string
}{
{"edit", true, "edit"},
{"shell", true, "shell"},
{"legacy", false, ""},
} {
result, err, sessions, prompts := run(tc.name, tc.setTools, tc.tools)
if !errors.Is(err, ErrHarnessTransport) || sessions != 1 || prompts != 1 {
t.Fatalf("%s stage replayed admitted work on a readback failure: err=%v sessions=%d prompts=%d", tc.name, err, sessions, prompts)
}
if result.Output != nil {
t.Fatalf("%s stage retained a partial result", tc.name)
}
if !strings.Contains(err.Error(), "HARNESS TRANSPORT") {
t.Fatalf("%s failure lost the transport category: %v", tc.name, err)
}
}
}
// TestExpiredContextDuringCLITransportKeepsTransportDetailAndDeadline proves an
// expired context during a real CLI request keeps both the sanitized transport
// detail (operation/exit) and errors.Is(err, context.DeadlineExceeded) instead
// of returning a bare context error that discards the cause.
func TestExpiredContextDuringCLITransportKeepsTransportDetailAndDeadline(t *testing.T) {
dir := t.TempDir()
executable := filepath.Join(dir, "opencode")
if err := os.WriteFile(executable, []byte("#!/bin/sh\nexec sleep 30\n"), 0700); err != nil {
t.Fatal(err)
}
t.Setenv("PIPELINES_OPENCODE", executable)
client := OpenCodeClient{Directory: dir}
ctx, cancel := context.WithTimeout(context.Background(), 40*time.Millisecond)
defer cancel()
_, err := client.call(ctx, "post", "/api/session", nil)
if err == nil || !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("expired transport call lost the context cause: %v", err)
}
var typed *openCodeError
if !errors.As(err, &typed) || typed.operation != "post /api/session" {
t.Fatalf("expired transport call discarded the typed transport error: %v", err)
}
if !strings.Contains(err.Error(), "OpenCode post /api/session") {
t.Fatalf("expired transport call discarded the operation detail: %v", err)
}
if !harnessTransportFailure(err) {
t.Fatalf("expired transport call was not classified as transport: %v", err)
}
}
// TestExpiredContextDuringNativeTransportKeepsTransportDetailAndDeadline proves
// the native V1 branch keeps both the typed transport error (the operation
// detail) and errors.Is(err, context.DeadlineExceeded) when the request failed
// under an expired context, instead of returning a bare context error that
// discards the transport shape. The native request path folds an in-flight
// deadline into ctx.Err(), so the retained transport detail is the operation and
// the parsed HTTP status is 0; the important invariant is that both shapes
// survive the join.
func TestExpiredContextDuringNativeTransportKeepsTransportDetailAndDeadline(t *testing.T) {
client := OpenCodeClient{Backend: "v1", connection: &opencode.Connection{Backend: "v1", URL: "http://127.0.0.1:1"}}
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(-time.Second))
defer cancel()
_, err := client.call(ctx, "post", "/api/session/ses_native/generate", map[string]any{"prompt": "task"})
if err == nil || !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("expired native transport call lost the context cause: %v", err)
}
var typed *openCodeError
if !errors.As(err, &typed) || typed.operation != "post /api/session/ses_native/generate" {
t.Fatalf("expired native transport call discarded the typed transport error: %v", err)
}
if !strings.Contains(err.Error(), "post /api/session/ses_native/generate") {
t.Fatalf("expired native transport call discarded the operation detail: %v", err)
}
if !harnessTransportFailure(err) {
t.Fatalf("expired native transport call was not classified as transport: %v", err)
}
}
// TestHarnessTransportSentinelsShareOneClassifierAndCategory proves the two
// transport shapes share one classifier and the single HARNESS TRANSPORT label
// while a bare stage deadline stays distinct from a pre-admission reach failure.
func TestHarnessTransportSentinelsShareOneClassifierAndCategory(t *testing.T) {
unreachable := harnessUnreachable(errors.New("connect refused"))
if !harnessTransportFailure(unreachable) || !harnessReachFailure(unreachable) {
t.Fatal("pre-admission reach failure was not classified")
}
if !strings.Contains(wrapModelStageFailure("worker", unreachable).Error(), "HARNESS TRANSPORT") {
t.Fatalf("pre-admission reach failure lost the transport category: %v", wrapModelStageFailure("worker", unreachable))
}
post := fmt.Errorf("%w: final assistant readback was malformed or incomplete", ErrHarnessTransport)
if !harnessTransportFailure(post) || harnessReachFailure(post) {
t.Fatal("post-admission transport failure was misclassified as a pre-admission reach failure")
}
if !strings.Contains(wrapModelStageFailure("worker", post).Error(), "HARNESS TRANSPORT") {
t.Fatal("post-admission transport failure lost the transport category")
}
if harnessTransportFailure(context.DeadlineExceeded) || harnessReachFailure(context.DeadlineExceeded) {
t.Fatal("a bare stage deadline must not be a transport or reach failure")
}
}