From 22801a9b06d9c465460e6b0230f34c91916f64e0 Mon Sep 17 00:00:00 2001 From: Alan Silva Date: Wed, 21 Oct 2026 15:26:29 +0100 Subject: [PATCH 1/2] fix(stage): align session export render cap with artifact bound Six computed Flutter runs stopped at the export handoff gate because the rendered session.html cap was 2 MiB while their complete sessions (well inside the 8 MiB source read bound) rendered between 2.25 and 4.28 MiB once html escaping and the readable projection duplicated native text. Raise SessionExportFileLimit to 8 MiB, the server's existing per-artifact file bound, so a complete export is still listed and served. Normal export verification, versioned repair and the handoff gate keep using the single shared constant. The source read bound is unchanged, and an over-cap render is still visibly truncated and recorded partial. Add focused regressions for a complete export above the old cap, an over-new-cap file rejected as partial, repair above the old cap leaving original evidence untouched, the gate accepting the larger repair, and the constant staying inside the artifact file bound. Update the artifact, handoff-gate and operations docs and record the diagnosis in docs/notes/session-export-render-cap.md. --- docs/notes/session-export-handoff-gate.md | 2 +- docs/notes/session-export-render-cap.md | 44 +++++++++++++ docs/stage-artifacts.md | 11 ++-- .../author/skill/references/operations.md | 7 ++- internal/server/session_export_gate_test.go | 50 +++++++++++++++ internal/server/session_export_repair_test.go | 51 +++++++++++++++ internal/server/session_export_status_test.go | 63 ++++++++++++++++++- internal/stage/session_export.go | 13 +++- internal/stage/session_export_test.go | 32 ++++++++++ 9 files changed, 261 insertions(+), 12 deletions(-) create mode 100644 docs/notes/session-export-render-cap.md diff --git a/docs/notes/session-export-handoff-gate.md b/docs/notes/session-export-handoff-gate.md index 6f09e72c..086d550a 100644 --- a/docs/notes/session-export-handoff-gate.md +++ b/docs/notes/session-export-handoff-gate.md @@ -73,7 +73,7 @@ retry and refresh of an already-open Session tab. Large (>256 KiB) API fixtures, wrong identity, partial/empty exports, restart, cancellation, workspace reservation, and exactly-once successor admission have focused regressions. A full ordinary artifact inventory no longer blocks versioned audit repair. -Repairs have a separate bounded allowance (twenty attempts, each at most 2 MiB), +Repairs have a separate bounded allowance (twenty attempts, each at most 8 MiB), so neither the ordinary inventory's file count nor its byte budget consumes the mandatory audit-recovery capacity. The ordinary scan limits remain unchanged. diff --git a/docs/notes/session-export-render-cap.md b/docs/notes/session-export-render-cap.md new file mode 100644 index 00000000..3ffd0b34 --- /dev/null +++ b/docs/notes/session-export-render-cap.md @@ -0,0 +1,44 @@ +# Session export: rendered-file cap aligned with the artifact bound + +## Problem + +Six computed Flutter runs (34, 37, 39, 42, 49, 50) reached Code visit 1 with a +successful, quiescent agent session but stopped at the export handoff gate. Each +recorded `session_export.status: partial`, `html_bytes: 2097152`, +`truncated: true`; the manual versioned repair of run 34 hit the same exact size. +The sessions had completed their computation, so a stage rerun was not the right +recovery. + +A read-only inspection of the recorded session rows showed the original native +data for those six sessions was between 1,083,391 and 2,111,470 bytes (the +complete HTTP export envelope for one of them was 2,126,449 bytes). All were +well inside the existing 8 MiB source read cap. The binding limit was the 2 MiB +rendered-file cap: html escaping and the readable projection keep native text +more than once, so an ordinary session can render larger than its source. +Re-exporting the six exact sessions read-only into scratch directories with the +new cap produced complete, untruncated files of 2,252,874 to 4,276,358 bytes. + +## Change + +`internal/stage/session_export.go` raises `SessionExportFileLimit` from 2 MiB to +8 MiB. That value is the same as the server's ordinary per-artifact file bound +(`maxArtifactFileBytes` in `internal/server/artifacts.go`), so a complete export +remains listed and served instead of being cut below the bound that the artifact +inventory already enforces. + +The source read remains capped by `SessionExportSourceLimit` (the single +`opencode.MaxSessionExport` value, 8 MiB). Normal export verification, versioned +export repair and the handoff gate all use `SessionExportFileLimit`, so they now +share one larger bound. The export still refuses to overwrite an existing file, +still marks any cut with a visible leading and trailing banner, and still reports +`truncated: true` and `status: partial`; nothing treats a partial export as +complete. + +## Checks + +Focused stage and server regressions cover a complete export above the old 2 MiB +cap but inside the new cap, a file above the new cap rejected as partial with the +verification diagnostic, repair accepting the larger complete file while the +original evidence stays unchanged, and the constant staying within the ordinary +artifact file bound. Focused stage/server suites and `go build ./...` plus +`go vet ./...` are run before landing; see the delivery report for exact results. diff --git a/docs/stage-artifacts.md b/docs/stage-artifacts.md index b29260d0..b2da6462 100644 --- a/docs/stage-artifacts.md +++ b/docs/stage-artifacts.md @@ -97,10 +97,13 @@ no special casing. `opencode session export `. Both reuse the same invocation helper and session-ID/messages validation as the transcript readback. The file is the harness's real session, never a reconstruction. -- Bounds: the source read is capped at 8 MiB and the rendered file at 2 MiB. When - either is cut, the file carries a visible truncation banner near the start and - the end and the step records `truncated: true`; a truncated export is never - described as whole. +- Bounds: the source read is capped at 8 MiB and the rendered file at 8 MiB, + matching the ordinary per-artifact file bound. The rendered file can exceed + its source because html escaping and the readable projection keep native text + more than once, so the previous 2 MiB render cap cut complete sessions that + were well inside the source bound. When either bound is exceeded, the file + carries a visible truncation banner near the start and the end and the step + records `truncated: true`; a truncated export is never described as whole. - Evidence: the step records `session_export` (`session_id`, `backend`, `source`, `html`, `html_bytes`, `messages`, `truncated`, `diagnostic`), so `pipelines inspect RUN_ID --json` shows what was written without reading the diff --git a/internal/author/skill/references/operations.md b/internal/author/skill/references/operations.md index 06a6013a..c67de23b 100644 --- a/internal/author/skill/references/operations.md +++ b/internal/author/skill/references/operations.md @@ -249,9 +249,10 @@ landed. presented. A run recorded before the HTML-only change still lists `session.md` among its stored artifacts (dated evidence kept on disk, not deleted); every listing — app, TUI, web spike — hides it rather than showing it beside the html. -- The source read is capped at 8 MiB and the rendered file at 2 MiB; a cut source or - a cut rendered file carries a visible truncation banner at both ends and - `session_export.truncated: true`, and a truncated export is never called whole. +- The source read is capped at 8 MiB and the rendered file at 8 MiB, matching the + ordinary per-artifact file bound; a cut source or a cut rendered file carries a + visible truncation banner at both ends and `session_export.truncated: true`, and + a truncated export is never called whole. - The step records `session_export` (`session_id`, `backend`, `source`, the html file name, `html_bytes`, `messages`, `truncated`, `diagnostic`) in `inspect RUN_ID --json`. A record written before the change still carried diff --git a/internal/server/session_export_gate_test.go b/internal/server/session_export_gate_test.go index a32a0943..3c9519c4 100644 --- a/internal/server/session_export_gate_test.go +++ b/internal/server/session_export_gate_test.go @@ -136,6 +136,56 @@ func TestSessionExportGateRetryOnlyAndExactlyOnce(t *testing.T) { } } +// TestSessionExportGateAcceptsRepairAboveOldRenderCap keeps the handoff gate's +// versioned-repair verification on the same larger bound as normal export. +func TestSessionExportGateAcceptsRepairAboveOldRenderCap(t *testing.T) { + h, s, id, first, second := gateRunner(t, func(context.Context, stage.SessionExportRequest) (stage.SessionExportResult, error) { + return stage.SessionExportResult{}, errors.New("export offline") + }, false) + h.runner.wg.Wait() + ctx := context.Background() + if err := s.ConfirmRecoveredSession(ctx, pipeline.SessionOwner{RunID: id, ID: "ses_exact"}); err != nil { + t.Fatal(err) + } + size := int64(2<<20) + 4096 + h.runner.exportSession = func(_ context.Context, req stage.SessionExportRequest) (stage.SessionExportResult, error) { + if err := os.MkdirAll(req.ArtifactsDir, 0700); err != nil { + return stage.SessionExportResult{}, err + } + f, err := os.Create(filepath.Join(req.ArtifactsDir, req.FileName)) + if err != nil { + return stage.SessionExportResult{}, err + } + if err := f.Truncate(size); err != nil { + f.Close() + return stage.SessionExportResult{}, err + } + if err := f.Close(); err != nil { + return stage.SessionExportResult{}, err + } + return stage.SessionExportResult{Source: "native_export", HTML: req.FileName, HTMLBytes: size, Messages: 2}, nil + } + if _, err := h.runner.repairSessionExport(ctx, id, 1); err != nil { + t.Fatal(err) + } + // The worker promotes a persisted complete repair before claiming a lease. + job, err := s.ClaimSessionExportJob(ctx, time.Now().Add(time.Second), time.Minute) + if err != nil { + t.Fatal(err) + } + if job != nil { + t.Fatalf("complete repair still claimed as a recovery job: %+v", job) + } + if err := h.runner.resumeExportGates(ctx); err != nil { + t.Fatal(err) + } + h.runner.wg.Wait() + d, _ := s.GetRun(ctx, id) + if first.Load() != 1 || second.Load() != 1 || d.Status != "succeeded" || len(d.Steps) != 2 { + t.Fatalf("large repair did not advance the gate: %+v (%d/%d)", d, first.Load(), second.Load()) + } +} + func TestSessionExportGateCleanupUnconfirmedDoesNotExportOrAdvance(t *testing.T) { var exports atomic.Int32 h, s, id, _, second := gateRunner(t, func(ctx context.Context, req stage.SessionExportRequest) (stage.SessionExportResult, error) { diff --git a/internal/server/session_export_repair_test.go b/internal/server/session_export_repair_test.go index 4c0d8252..c2aabf3a 100644 --- a/internal/server/session_export_repair_test.go +++ b/internal/server/session_export_repair_test.go @@ -111,6 +111,57 @@ func TestRuntimeSessionExportRepairPreservesOriginalAndUsesRecordedOwner(t *test } } +// TestRuntimeSessionExportRepairAcceptsAboveTheOldRenderCap proves a versioned +// repair uses the same larger bound as a normal export while leaving the +// original recorded evidence untouched. +func TestRuntimeSessionExportRepairAcceptsAboveTheOldRenderCap(t *testing.T) { + h, s, id := sessionExportRunner(t, "v2", func(stage.SessionExportRequest) (stage.SessionExportResult, error) { + return stage.SessionExportResult{}, errors.New("initial export unavailable") + }) + if err := s.ConfirmRecoveredSession(context.Background(), pipeline.SessionOwner{RunID: id, ID: "ses_run"}); err != nil { + t.Fatal(err) + } + original, err := s.GetRun(context.Background(), id) + if err != nil { + t.Fatal(err) + } + before, _ := json.Marshal(original.Steps[0].SessionExport) + size := int64(2<<20) + 4096 + h.runner.exportSession = func(_ context.Context, req stage.SessionExportRequest) (stage.SessionExportResult, error) { + if err := os.MkdirAll(req.ArtifactsDir, 0700); err != nil { + return stage.SessionExportResult{}, err + } + f, err := os.Create(filepath.Join(req.ArtifactsDir, req.FileName)) + if err != nil { + return stage.SessionExportResult{}, err + } + if err := f.Truncate(size); err != nil { + f.Close() + return stage.SessionExportResult{}, err + } + if err := f.Close(); err != nil { + return stage.SessionExportResult{}, err + } + return stage.SessionExportResult{Source: "native_export", HTML: req.FileName, HTMLBytes: size, Messages: 2}, nil + } + repaired, err := h.runner.repairSessionExport(context.Background(), id, 1) + if err != nil { + t.Fatal(err) + } + latest := repaired.Steps[0].SessionExportRepairs[0] + if latest.Status != "complete" || latest.HTMLBytes != size || latest.HTML == "" { + t.Fatalf("repair above the old cap was not complete: %+v", latest) + } + after, _ := json.Marshal(repaired.Steps[0].SessionExport) + if string(before) != string(after) { + t.Fatal("repair above the old cap overwrote original evidence") + } + info, err := os.Stat(filepath.Join(h.runner.artifactsRoot, "1", "stages", "01-"+repaired.Steps[0].Stage, latest.HTML)) + if err != nil || info.Size() != size { + t.Fatalf("repair file missing or resized: %v %+v", err, info) + } +} + func TestRuntimeRepairRejectsUnrecordedAndActiveVisits(t *testing.T) { h, _, id := sessionExportRunner(t, "v2", func(stage.SessionExportRequest) (stage.SessionExportResult, error) { return stage.SessionExportResult{}, errors.New("unavailable") diff --git a/internal/server/session_export_status_test.go b/internal/server/session_export_status_test.go index a1bae02a..10daae33 100644 --- a/internal/server/session_export_status_test.go +++ b/internal/server/session_export_status_test.go @@ -1,10 +1,13 @@ package server import ( - "git.a-wan.net/alan/pipelines/internal/protocol" "os" "path/filepath" + "strings" "testing" + + "git.a-wan.net/alan/pipelines/internal/protocol" + "git.a-wan.net/alan/pipelines/internal/stage" ) func TestSessionExportCompleteness(t *testing.T) { @@ -56,3 +59,61 @@ func TestRuntimeSessionExportFileVerificationDoesNotClaimMissingEvidence(t *test }) } } + +func sizedSessionFile(t *testing.T, size int64) string { + t.Helper() + dir := t.TempDir() + path := filepath.Join(dir, "session.html") + f, err := os.Create(path) + if err != nil { + t.Fatal(err) + } + if err := f.Truncate(size); err != nil { + f.Close() + t.Fatal(err) + } + if err := f.Close(); err != nil { + t.Fatal(err) + } + return dir +} + +// TestRuntimeSessionExportVerificationAcceptsAboveTheOldRenderCap pins the +// ordinary-session case the previous 2 MiB rendered bound cut. The file is +// larger than that old cap and inside the artifact-aligned bound, so normal +// export verification must keep it and classify the evidence complete. +func TestRuntimeSessionExportVerificationAcceptsAboveTheOldRenderCap(t *testing.T) { + size := int64(2<<20) + 4096 + dir := sizedSessionFile(t, size) + evidence := &protocol.SessionExport{Source: "native_export", Messages: 2, HTML: "session.html", HTMLBytes: size} + verifySessionExportFile(evidence, dir) + classifySessionExport(evidence) + if evidence.Status != "complete" || evidence.HTML != "session.html" || evidence.HTMLBytes != size || evidence.Diagnostic != "" { + t.Fatalf("complete export above the old cap was rejected: %+v", evidence) + } +} + +// TestRuntimeSessionExportVerificationRejectsAboveTheNewCap confirms the larger +// bound is still a hard bound: an oversized file is not presented as evidence. +func TestRuntimeSessionExportVerificationRejectsAboveTheNewCap(t *testing.T) { + size := int64(stage.SessionExportFileLimit) + 1 + dir := sizedSessionFile(t, size) + evidence := &protocol.SessionExport{Source: "native_export", Messages: 2, HTML: "session.html", HTMLBytes: size} + verifySessionExportFile(evidence, dir) + classifySessionExport(evidence) + if evidence.Status != "partial" || evidence.HTML != "" || evidence.HTMLBytes != 0 || !strings.Contains(evidence.Diagnostic, "bounded verification") { + t.Fatalf("oversized export was not rejected as partial: %+v", evidence) + } +} + +// TestSessionExportFileLimitStaysInsideArtifactBounds keeps the render cap +// principled: larger than the cap that cut ordinary sessions, and no larger +// than the ordinary per-artifact file bound that serves the file. +func TestSessionExportFileLimitStaysInsideArtifactBounds(t *testing.T) { + if stage.SessionExportFileLimit <= 2<<20 { + t.Fatalf("render cap %d did not grow beyond the old 2 MiB cutoff", stage.SessionExportFileLimit) + } + if stage.SessionExportFileLimit > maxArtifactFileBytes { + t.Fatalf("render cap %d exceeds the artifact file bound %d", stage.SessionExportFileLimit, maxArtifactFileBytes) + } +} diff --git a/internal/stage/session_export.go b/internal/stage/session_export.go index 583602dd..2c752c5c 100644 --- a/internal/stage/session_export.go +++ b/internal/stage/session_export.go @@ -20,7 +20,7 @@ import ( // artifacts directory as one html file so the existing artifact listing and // serving paths find it with no special casing. The source read is capped at // SessionExportSourceLimit and the rendered file at SessionExportFileLimit, so -// the export adds at most ~2 MiB per step, inside the artifact scan limits. +// the export stays inside the server's per-artifact file bound. const ( // SessionHTMLFile is the one session export a stage writes. SessionHTMLFile = "session.html" @@ -32,8 +32,15 @@ const ( // SessionExportSourceLimit is the native session read cap; it is the one // opencode.MaxSessionExport value, not a second copy. SessionExportSourceLimit = opencode.MaxSessionExport - SessionExportFileLimit = 2 << 20 - SessionExportTimeout = 10 * time.Second + // SessionExportFileLimit is the rendered file cap. Ordinary native sessions + // commonly render a little larger than their raw source because html escaping + // and the readable projection duplicate native text; the previous 2 MiB cap + // cut complete sessions that were well inside the source bound. This cap + // matches the server's per-artifact file bound (maxArtifactFileBytes in + // internal/server/artifacts.go), so a complete export remains listed and + // served while a genuinely oversized one is still visibly truncated. + SessionExportFileLimit = 8 << 20 + SessionExportTimeout = 10 * time.Second ) // SessionExportRequest names the stage session to export and the evidence the diff --git a/internal/stage/session_export_test.go b/internal/stage/session_export_test.go index 5db417fd..8cf2efe2 100644 --- a/internal/stage/session_export_test.go +++ b/internal/stage/session_export_test.go @@ -222,6 +222,38 @@ func TestExportSessionLargeHTTPExportWritesBoundedRealHTML(t *testing.T) { } } +// TestExportSessionCompletesAboveTheOldRenderedCap covers the ordinary case the +// previous 2 MiB render cap cut: a complete native session inside the 8 MiB +// source bound whose escaped html is larger than 2 MiB. +func TestExportSessionCompletesAboveTheOldRenderedCap(t *testing.T) { + const oldRenderCap = 2 << 20 + dir := t.TempDir() + line := "complete ordinary session line " + text := strings.Repeat(line, oldRenderCap/len(line)+1000) + body := `{"info":{"id":"ses_ok"},"messages":[{"info":{"id":"m1","role":"user"},"text":"` + text + `"}]}` + sessionCLI(t, "ses_ok", body) + result, err := ExportSession(context.Background(), sessionRequest(dir)) + if err != nil { + t.Fatal(err) + } + data, err := os.ReadFile(filepath.Join(dir, result.HTML)) + if err != nil { + t.Fatal(err) + } + if result.Truncated || result.Messages != 1 || result.HTMLBytes != int64(len(data)) { + t.Fatalf("complete export above the old cap was not recorded: %+v", result) + } + if len(data) <= oldRenderCap || len(data) > SessionExportFileLimit { + t.Fatalf("rendered size %d is not above the old cap and inside the new one", len(data)) + } + if strings.Contains(string(data), "TRUNCATED") { + t.Fatal("a complete export carried a truncation banner") + } + if !strings.Contains(string(data), "complete ordinary session line") { + t.Fatal("complete export lost its session text") + } +} + func TestExportSessionSourceTruncationIsLabelledNotWhole(t *testing.T) { session := opencode.ExportedSession{ ID: "ses_ok", From e8bde4f5787f8780d9ea6cf8770556721f35e5f8 Mon Sep 17 00:00:00 2001 From: Alan Silva Date: Wed, 21 Oct 2026 15:32:21 +0100 Subject: [PATCH 2/2] docs(app): record verified export-fix landing and safe installation gate --- app/WORK.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/app/WORK.md b/app/WORK.md index 9d4dff9a..ce3293fe 100644 --- a/app/WORK.md +++ b/app/WORK.md @@ -21,7 +21,7 @@ All work under app/. Existing applications remain running and unchanged. | R2b | Native cancel-run confirmation | run51 →52 / correction5449370 | manager reran23 action tests + analyze, pass; known refusal now immediate and dismissible while resync pending | merged main; ancestry verified | reusable component only; wiring/browser pending | | X1 | Native search and result targets | run49 / flutter/x1-search | Code computed success; WAITING truncated2MiB export | held, review not reached | pending | | I1 | App-owned socket/session + live Runs shell registration | run50 / flutter/r1-native | Code computed success; WAITING truncated2MiB export | held, review not reached | not deployed | -| E1 | Narrow authorized server export-bound fix | run53 / fix/session-export-limit | API v8, Flash; bounded export/repair regressions | pending | no service restart authorized by pipeline | +| E1 | Narrow authorized server export-bound fix | run53 / commit22801a9 | manager reproduced landing-test env failure on unchanged main; full Go suite/build/vet pass with only test-process URL env unset | merged main, ancestry verified | NOT installed; operator restart approval/window pending | | R2c | Native retry/restart confirmation | run54 / commit178f22a | manager reran17 focused tests + analyze, pass; exact modes/stable intent keys checked | main804e531, ancestry verified | reusable only; wiring/browser pending | | R2d | Native rename-run dialog | run55 / commitf2643a0 | manager reran11 focused tests + analyze, pass; clean scoped diff | main3dde288, ancestry verified | reusable only; screen/list update wiring pending | | S1a | Saved schedules repository/revision safety | run56 / commitb963965 | manager reran18 focused tests + analyze, pass; revision/unknown field preservation checked | maina290858, ancestry verified | — | @@ -65,6 +65,22 @@ flutter analyze was clean. These cover landed source/components only, not the six unreviewed export-blocked feature branches or visible screen delivery. ### Current delivery blocker — session export cap +The fix is now merged: rendered file cap8MiB, source cap8MiB unchanged, within +ordinary artifact allowance. Pipeline read-only scratch exports of all six exact +sessions were complete at2.25–4.28MiB, establishing the fix is sufficient. +Run53 automatic delivery failed in pre-existing TestHumanRunAttentionNotification +because the test inherited the running service's PIPELINES_WEB_PUBLIC_URL. Manager +reproduced that same failure on unchanged main with a harmless example URL; no +notification source/test was edited. All packages pass on candidate with only that +test-process env var unset; build/vet/diff pass. Manager landed checked source +directly (no coding-stage replay), and refreshed installed managed Pipelines skill +from current source via go run ./cmd/pipelines skill install. +Installed server is unchanged. Deployment safety inspection found57/58/59 active; +no restart or binary replacement performed. Need explicit operator approval for +installation/restart after active work settles before exact export-only repairs. +Old CLI service status cannot resolve its expected launchd label; do not guess a +service or restart unrelated processes. Inspect authoritative supervisor first. + User asked about failed runs; explicit status inspection found runs34/37/39/42/ 49/50 are waiting, not failed: each Code visit1 computed success and is quiescent, but export is partial/truncated at exactly2097152 bytes. Exact recorded native