diff --git a/docs/cli.md b/docs/cli.md index 22c6eeae..b79b70e2 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -665,6 +665,13 @@ total as input tokens. The CLI is the deliverable; a TUI picker for choosing overrides is a documented follow-up, not half-built. +The run detail (`inspect --json`, `get_run`) also carries an additive +top-level `warnings` string list, derived at read time from the steps; `inspect` +repeats each entry on stderr as `warning: ...`. Today it names every visit whose +retained file-change capture was `partial` or `unavailable`, because such a +capture is not a complete workspace diff and disables the server skip fallback +(see [stage-file-diffs.md](stage-file-diffs.md)). Older peers ignore the field. + On either the runs list or run inspector, press `P` (or use `:pipeline`) to open the selected run's exact pipeline version. Navigation does not cancel the run. diff --git a/docs/stage-file-diffs.md b/docs/stage-file-diffs.md index a075520f..9f068ac0 100644 --- a/docs/stage-file-diffs.md +++ b/docs/stage-file-diffs.md @@ -26,10 +26,21 @@ includes the same evidence under each step's `file_changes` field. `secrets/`, Git internals and `node_modules/` are excluded. This is a path-based precaution, **not secret-content scanning**; ordinary source patches may contain sensitive text and become part of retained history. -- Limits: 10,000 inventoried entries, 1 MiB per file, 32 MiB per snapshot, - five seconds per snapshot, 5,000 combined lines per textual comparison, - 64 KiB per patch and 128 KiB serialized file evidence per visit. Oversized, - unreadable, omitted or truncated evidence is explicitly flagged partial. +- Limits: 10,000 inventoried entries, 1 MiB per file and 32 MiB per snapshot of + retained text, 256 MiB per hashed file, five seconds per snapshot, 5,000 + combined lines per textual comparison, 64 KiB per patch and 128 KiB serialized + file evidence per visit. Oversized, unreadable, omitted or truncated evidence + is explicitly flagged partial. +- Over-budget content degrades to a listing, not silence: a text file over 1 MiB, + or beyond the 32 MiB snapshot budget, is still compared by SHA-256 and a change + is listed with `no_patch: true` (and `truncated`) but no patch. In a Git working + tree, a path the snapshot could not compare at all (above 256 MiB, unreadable, + or past the time/file limit) is listed the same way when its + `git status --porcelain` entry changed during the visit; a path dirty with the + same status before and after cannot be proven changed and stays in `skipped`. + Git is only read (`--no-optional-locks`); protected paths are never listed. +- `pipelines inspect RUN_ID --json` carries a top-level `warnings` list (also + printed to stderr) naming every visit whose capture was partial or unavailable. - Failed/canceled stages still retain changes when execution returns. Unconfirmed agent cleanup is flagged partial. A server/process crash cannot guarantee an after-snapshot; recovered steps without capture say “not recorded.” @@ -41,3 +52,24 @@ includes the same evidence under each step's `file_changes` field. Snapshots are transient server memory. The database retains patches/metadata, not full copies of every workspace file. No new model calls are used for capture. + +## Server skip fallback + +A non-human stage that declares a `skipped` edge (typically a UI reviewer) is +completed by the server without a model call, and routed along `skipped`, only +when every earlier visit of the run recorded a **complete** capture with **no** +changed files and no image artifact. The step's verdict reason begins +"Server fallback skip (no model run):" and has `reason_source: fallback`. + +Everything else runs the stage's model, which may still choose `skipped` itself: + +- any recorded file change, whatever its path or extension (no repository layout + or "non-UI" extension list is assumed: Go TUIs, Flutter `.dart`, Kotlin, Swift, + templates, strings and config files can all be rendered surface); +- any partial or unavailable capture, or a visit with no capture (a crash, a + historical row, a human decision during which the workspace was unobserved, or + a checkpoint-retry prefix); +- `auto_skip=false` in the stage's `configs`. + +A wrong run costs one model call; a wrong skip would ship an unreviewed change, so +absent or partial evidence is never read as "nothing changed". diff --git a/internal/author/skill/references/format.md b/internal/author/skill/references/format.md index b6a9a222..68de8830 100644 --- a/internal/author/skill/references/format.md +++ b/internal/author/skill/references/format.md @@ -82,6 +82,14 @@ and accepts integers 0–100; explicit 0 disables execution retries. Optional `timeout` defaults to `1h` per attempt and accepts positive durations such as `30m` or `90s`. +A stage that declares a `skipped` edge (for example a UI reviewer) is eligible +for the server skip fallback: when every earlier visit's file-change capture was +complete and recorded no changed files or screenshots, the server completes the +stage as `skipped` without a model call (`reason_source: fallback`). Any recorded +change or incomplete capture runs the model. Optional `auto_skip=false` (values +`true`/`false`, default `true`) always runs the model. Read +`references/operations.md` ("Server skip fallback") for the exact rule. + Optional `reasoning_effort=high` selects a model's reasoning/thinking level for this stage. Values are `default`, `none` (alias `off`), `minimal`, `low`, `medium`, `high`, `xhigh`, or `max`. Omission or `default` preserves the harness/model diff --git a/internal/author/skill/references/operations.md b/internal/author/skill/references/operations.md index 01dee382..83a26bc6 100644 --- a/internal/author/skill/references/operations.md +++ b/internal/author/skill/references/operations.md @@ -191,9 +191,14 @@ The stage's File changes section highlights added/removed lines and hunk headers node_modules and common environment/key/credential/database paths are excluded. This is path-based protection, **not secret-content scanning**. Patches may still contain sensitive text; never publish them without approval. -- Limits: 10,000 entries, 1 MiB/file, 32 MiB/snapshot, five-second snapshot context, - 5,000 combined text lines, 64 KiB/patch, 128 KiB serialized file evidence/visit. - Missing/skipped/truncated captures must be described as partial/unavailable. +- Limits: 10,000 entries, 1 MiB/file and 32 MiB/snapshot of retained text, + 256 MiB/hashed file, five-second snapshot context, 5,000 combined text lines, + 64 KiB/patch, 128 KiB serialized file evidence/visit. Text over the 1 MiB or + 32 MiB budget is still compared by hash, and in Git a path the snapshot could + not compare is listed when its `git status` entry changes; such entries carry + `no_patch: true` (and `truncated`) with no patch, and the visit is `partial`. + Missing/skipped/truncated captures must be described as partial/unavailable; + `inspect` adds a run-level `warnings` entry naming every partial/unavailable visit. - Failed/canceled stages retain changes when execution returns; unconfirmed agent cleanup is partial. Crashes may have no after-capture. Historical visits say not recorded—**never infer no edits from absent evidence**. @@ -201,6 +206,19 @@ The stage's File changes section highlights added/removed lines and hunk headers changes outside scope, ignored new files and changes reverted before capture can be absent or mixed in. Capture failure does not change the declared run verdict. +### Server skip fallback (`skipped` edge) + +A non-human stage that declares a `skipped` edge is completed by the server, with +no model call, and routed along `skipped` only when **every** earlier visit of the +run recorded `file_changes` with status `complete`, none recorded any changed file, +and none captured an image artifact. The step's verdict reason starts +"Server fallback skip (no model run):" with `reason_source: fallback`. Any changed +file (no path or extension is treated as "not UI"), any partial/unavailable or +missing capture (crash, historical row, human decision, checkpoint-retry prefix) +runs the model, which may still answer `skipped` itself. `auto_skip=false` in the +stage's `configs` disables the fallback for that stage. Never describe a fallback +skip as a model judgement, and never treat it as evidence that nothing changed. + Do not add model output fields, new stages or extra model calls merely to obtain file diffs: current capture is server-owned. Do not promise full files/rollback. diff --git a/internal/cli/app.go b/internal/cli/app.go index 7cd88615..03cadc11 100644 --- a/internal/cli/app.go +++ b/internal/cli/app.go @@ -1001,6 +1001,13 @@ func (a App) Run(ctx context.Context, args []string) error { if err != nil { return err } + if command == "inspect" && r.RunDetail != nil { + // Warnings are also in the JSON; stderr keeps them visible to a reader + // of the human stream without corrupting machine-readable stdout. + for _, warning := range r.RunDetail.Warnings { + a.notice("warning: " + warning) + } + } return a.output(r.RunDetail) case "decide": if len(pos) < 1 || len(pos) > 2 { diff --git a/internal/filediff/snapshot.go b/internal/filediff/snapshot.go index 3d8ab32d..a496bcc8 100644 --- a/internal/filediff/snapshot.go +++ b/internal/filediff/snapshot.go @@ -14,6 +14,7 @@ import ( "os/exec" "sort" "strings" + "sync/atomic" "syscall" "unicode/utf8" @@ -28,13 +29,37 @@ const ( maxPatchBytes = 128 << 10 maxFilePatchBytes = 64 << 10 maxLines = 5000 + binarySniffBytes = 8 << 10 ) +// maxHashBytes bounds one hash-only file; larger files are skipped (and, in a +// Git working tree, listed only when their Git status changes). It is a +// variable so package tests can exercise that path without huge files. +var maxHashBytes int64 = 256 << 20 + +// snapshotBudget is the retained text bytes per snapshot. Files beyond it (or +// beyond the per-file limit) are still compared by content hash and listed +// without a patch, so an over-budget workspace never hides which files changed. +var snapshotBudget atomic.Int64 + +func init() { snapshotBudget.Store(maxSnapshotBytes) } + +// SetSnapshotBudgetForTesting lowers the retained-text budget so tests can +// exercise an over-budget workspace without writing tens of megabytes. It +// returns a function restoring the previous budget. Production never calls it. +func SetSnapshotBudgetForTesting(n int64) (restore func()) { + previous := snapshotBudget.Swap(n) + return func() { snapshotBudget.Store(previous) } +} + type file struct { data []byte hash [32]byte mode fs.FileMode binary bool + // hashOnly marks a text file compared by content hash only: it exceeded the + // per-file or per-snapshot text budget, so no patch can be produced. + hashOnly bool } type Snapshot struct { @@ -44,6 +69,63 @@ type Snapshot struct { available bool diagnostic string scope string + // gitStatus maps run-relative paths to their porcelain XY status code when + // the run directory is a Git working tree and the status read succeeded + // (gitOK). Compare uses it to list changed paths the content snapshot could + // not compare (time/file limit, unreadable); it never reads HEAD/index content. + gitStatus map[string]string + gitOK bool +} + +// readGitStatus reads `git status --porcelain=v1 -z` for the run directory and +// returns its paths relative to that directory. It only reads: optional locks +// are disabled so the index is never refreshed or rewritten. +func readGitStatus(ctx context.Context, directory string) (map[string]string, bool) { + prefixOut, err := exec.CommandContext(ctx, "git", "-c", "core.fsmonitor=false", "-C", directory, "rev-parse", "--show-prefix").Output() + if err != nil { + return nil, false + } + prefix := strings.TrimSpace(string(prefixOut)) + cmd := exec.CommandContext(ctx, "git", "--no-optional-locks", "-c", "core.fsmonitor=false", "-C", directory, "status", "--porcelain=v1", "-z", "--no-renames", "--untracked-files=all", "--", ".") + out := &boundedOutput{limit: 4 << 20} + cmd.Stdout = out + if err := cmd.Run(); err != nil { + return nil, false + } + status := map[string]string{} + for _, entry := range strings.Split(out.String(), "\x00") { + if len(entry) < 4 || entry[2] != ' ' { + continue + } + name, ok := strings.CutPrefix(entry[3:], prefix) + if !ok || name == "" { + continue // outside the run directory + } + status[name] = entry[:2] + } + return status, true +} + +// hashFile streams f from its start into a content hash without retaining it. +// It returns ok=false when the file exceeds maxHashBytes or cannot be read. +func hashFile(f *os.File) (hash [32]byte, binary bool, ok bool) { + if _, err := f.Seek(0, io.SeekStart); err != nil { + return hash, false, false + } + h := sha256.New() + head := make([]byte, binarySniffBytes) + n, err := io.ReadFull(f, head) + if err != nil && !errors.Is(err, io.ErrUnexpectedEOF) && !errors.Is(err, io.EOF) { + return hash, false, false + } + head = head[:n] + h.Write(head) + copied, err := io.Copy(h, io.LimitReader(f, maxHashBytes+1-int64(n))) + if err != nil || int64(n)+copied > maxHashBytes { + return hash, false, false + } + copy(hash[:], h.Sum(nil)) + return hash, bytes.IndexByte(head, 0) >= 0, true } func protected(name string) bool { @@ -130,6 +212,9 @@ func Capture(ctx context.Context, directory string, previous ...*Snapshot) *Snap s.partial = true s.diagnostic = "Workspace inventory incomplete." } + if git { + s.gitStatus, s.gitOK = readGitStatus(ctx, directory) + } // Keep pre-stage paths even if a stage changes .gitignore/the index. known := map[string]bool{} for _, name := range names { @@ -179,7 +264,7 @@ func Capture(ctx context.Context, directory string, previous ...*Snapshot) *Snap continue } info, err := root.Lstat(name) - if err != nil || !info.Mode().IsRegular() || info.Size() > maxFileBytes { + if err != nil || !info.Mode().IsRegular() || info.Size() > maxHashBytes { s.skipped[name] = true continue } @@ -194,28 +279,43 @@ func Capture(ctx context.Context, directory string, previous ...*Snapshot) *Snap s.skipped[name] = true continue } - data, readErr := io.ReadAll(io.LimitReader(f, maxFileBytes+1)) + if opened.Size() <= maxFileBytes { + data, readErr := io.ReadAll(io.LimitReader(f, maxFileBytes+1)) + if readErr != nil { + f.Close() + s.skipped[name] = true + continue + } + if len(data) <= maxFileBytes { + f.Close() + binary := bytes.IndexByte(data, 0) >= 0 || !utf8.Valid(data) + hash := sha256.Sum256(data) + hashOnly := false + // Binary changes need only metadata and a hash, never retained patch bytes. + // Counting screenshots against the retained-text budget can otherwise + // exhaust it before any source files are reached in sorted inventory order. + // Text beyond the budget keeps only its hash, so a change to it is still + // listed (without a patch) instead of being skipped unseen. + if binary { + data = nil + } else if int64(used+len(data)) > snapshotBudget.Load() { + data = nil + hashOnly = true + } else { + used += len(data) + } + s.files[name] = file{data: data, hash: hash, mode: opened.Mode().Perm(), binary: binary, hashOnly: hashOnly} + continue + } + // The file grew past the per-file limit while being read: hash it. + } + hash, binary, ok := hashFile(f) f.Close() - if readErr != nil || len(data) > maxFileBytes { + if !ok { s.skipped[name] = true continue } - binary := bytes.IndexByte(data, 0) >= 0 || !utf8.Valid(data) - hash := sha256.Sum256(data) - // Binary changes need only metadata and a hash, never retained patch bytes. - // Counting screenshots against the retained-text budget can otherwise - // exhaust it before any source files are reached in sorted inventory order. - if binary { - data = nil - } else { - if used+len(data) > maxSnapshotBytes { - s.skipped[name] = true - s.partial = true - continue - } - used += len(data) - } - s.files[name] = file{data: data, hash: hash, mode: opened.Mode().Perm(), binary: binary} + s.files[name] = file{hash: hash, mode: opened.Mode().Perm(), binary: binary, hashOnly: !binary} } return s } @@ -258,6 +358,35 @@ func Compare(before, after *Snapshot) *protocol.StageFileChanges { sort.Strings(names) remaining := maxPatchBytes retained := 0 + add := func(change protocol.FileChange) { + if change.Truncated { + result.Status = "partial" + if !change.NoPatch { + result.Diagnostic = "Some patches were omitted/truncated by capture limits." + } + } + encoded, _ := json.Marshal(change) + if retained+len(encoded)+1 > maxPatchBytes { + change.Patch = "" + change.Truncated = true + encoded, _ = json.Marshal(change) + } + if retained+len(encoded)+1 > maxPatchBytes { + result.Omitted++ + result.Status = "partial" + result.Diagnostic = "Additional file changes omitted by the retained evidence size limit." + return + } + if change.Truncated { + result.Status = "partial" + } + retained += len(encoded) + 1 + result.Files = append(result.Files, change) + } + listed := 0 + // decided records every path the content comparison settled (changed or + // not), so the Git-status pass below only speaks for the rest. + decided := map[string]bool{} for _, name := range names { if skipped[name] { continue @@ -268,6 +397,7 @@ func Compare(before, after *Snapshot) *protocol.StageFileChanges { if !aOK && before.partial || !bOK && after.partial { continue } + decided[name] = true if aOK && bOK && a.hash == b.hash && a.mode == b.mode { continue } @@ -285,7 +415,12 @@ func Compare(before, after *Snapshot) *protocol.StageFileChanges { if bOK { change.AfterMode = fmt.Sprintf("%04o", b.mode) } - if !change.Binary && a.hash != b.hash { + if !change.Binary && a.hash != b.hash && (a.hashOnly || b.hashOnly) { + // Over-budget content was compared by hash: list it, without a patch. + change.Truncated = true + change.NoPatch = true + listed++ + } else if !change.Binary && a.hash != b.hash { if remaining == 0 || bytes.Count(a.data, []byte("\n"))+bytes.Count(b.data, []byte("\n")) > maxLines { change.Truncated = true } else { @@ -314,27 +449,46 @@ func Compare(before, after *Snapshot) *protocol.StageFileChanges { } } } - if change.Truncated { - result.Status = "partial" - result.Diagnostic = "Some patches were omitted/truncated by capture limits." + add(change) + } + // In a Git working tree, paths the content snapshot could not settle + // (skipped as unreadable/oversized, or beyond the time/file limit) are still + // listed when their Git status changed during the visit. A path dirty with + // the same status before and after cannot be proven changed and stays + // counted as skipped. Protected paths are never listed. + if before.gitOK && after.gitOK { + gitNames := map[string]bool{} + for name := range before.gitStatus { + gitNames[name] = true } - encoded, _ := json.Marshal(change) - if retained+len(encoded)+1 > maxPatchBytes { - change.Patch = "" - change.Truncated = true - encoded, _ = json.Marshal(change) + for name := range after.gitStatus { + gitNames[name] = true } - if retained+len(encoded)+1 > maxPatchBytes { - result.Omitted++ - result.Status = "partial" - result.Diagnostic = "Additional file changes omitted by the retained evidence size limit." - continue + ordered := make([]string, 0, len(gitNames)) + for name := range gitNames { + if !decided[name] && fs.ValidPath(name) && !protected(name) { + ordered = append(ordered, name) + } } - if change.Truncated { - result.Status = "partial" + sort.Strings(ordered) + for _, name := range ordered { + was, now := before.gitStatus[name], after.gitStatus[name] + if was == now { + continue + } + kind := "modified" + switch { + case now == "" && was == "??", strings.Contains(now, "D"): + kind = "deleted" + case was == "" && (now == "??" || strings.HasPrefix(now, "A")): + kind = "added" + } + add(protocol.FileChange{Path: name, Kind: kind, Truncated: true, NoPatch: true}) + listed++ } - retained += len(encoded) + 1 - result.Files = append(result.Files, change) + } + if listed > 0 && result.Status == "partial" && result.Omitted == 0 { + result.Diagnostic = "Some changed files exceeded capture limits and are listed without patches; this is not a complete workspace diff." } return result } diff --git a/internal/filediff/snapshot_test.go b/internal/filediff/snapshot_test.go index 1b4d7acf..cb1b3e83 100644 --- a/internal/filediff/snapshot_test.go +++ b/internal/filediff/snapshot_test.go @@ -132,9 +132,14 @@ func TestProtectedSymlinkAndLimits(t *testing.T) { put(t, dir, "large", "small now") put(t, dir, "normal", "safe\n") changes := Compare(before, Capture(ctx, dir, before)) - if changes.Status != "partial" || len(changes.Files) != 1 || changes.Files[0].Path != "normal" { + // The oversized file is compared by hash and listed without a patch; the + // protected file and the symlink stay excluded. + if changes.Status != "partial" || len(changes.Files) != 2 || changes.Files[0].Path != "large" || changes.Files[1].Path != "normal" { t.Fatal(changes) } + if large := changes.Files[0]; !large.NoPatch || !large.Truncated || large.Patch != "" || large.Kind != "modified" { + t.Fatalf("oversized file not listed without patch: %+v", large) + } for _, f := range changes.Files { if strings.Contains(f.Patch, "secret") { t.Fatal("secret retained") @@ -189,3 +194,92 @@ func TestBinaryFilesDoNotStarveSourceDiffs(t *testing.T) { } } } + +func gitRepo(t *testing.T) (string, func(...string)) { + t.Helper() + if _, err := exec.LookPath("git"); err != nil { + t.Skip("git unavailable") + } + dir := t.TempDir() + run := func(args ...string) { + t.Helper() + cmd := exec.Command("git", append([]string{"-C", dir, "-c", "user.email=t@example.invalid", "-c", "user.name=t", "-c", "commit.gpgsign=false"}, args...)...) + if data, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("%v: %v %s", args, err, data) + } + } + run("init", "-q") + return dir, run +} + +// TestOverBudgetChangeIsListedWithoutPatch is the run-236 regression: a +// workspace whose text exceeds the snapshot budget must still list the one +// changed file (sorted last, after the budget is exhausted) instead of +// returning files: [] with every late path skipped. +func TestOverBudgetChangeIsListedWithoutPatch(t *testing.T) { + restore := SetSnapshotBudgetForTesting(64) + defer restore() + dir, git := gitRepo(t) + if err := os.MkdirAll(filepath.Join(dir, "web/src"), 0o700); err != nil { + t.Fatal(err) + } + for i := 0; i < 20; i++ { + put(t, dir, fmt.Sprintf("a-%02d.txt", i), strings.Repeat("filler\n", 4)) + } + put(t, dir, "web/src/Inbox.test.tsx", "it('works')\n") + git("add", ".") + git("commit", "-qm", "init") + ctx := context.Background() + before := Capture(ctx, dir) + put(t, dir, "web/src/Inbox.test.tsx", "it('works')\nit('more')\n") + changes := Compare(before, Capture(ctx, dir, before)) + if len(changes.Files) != 1 || changes.Files[0].Path != "web/src/Inbox.test.tsx" { + t.Fatalf("over-budget change not listed: %+v", changes) + } + if f := changes.Files[0]; f.Kind != "modified" || !f.NoPatch || f.Patch != "" { + t.Fatalf("over-budget change should be listed without patch: %+v", f) + } + if changes.Status != "partial" || changes.Skipped != 0 { + t.Fatalf("status/skipped = %q/%d, want partial/0", changes.Status, changes.Skipped) + } + // Unchanged over-budget files are compared by hash, so a visit that edits + // nothing is a complete, empty capture rather than a partial one. + again := Capture(ctx, dir) + if empty := Compare(again, Capture(ctx, dir, again)); empty.Status != "complete" || len(empty.Files) != 0 { + t.Fatalf("no-op over-budget visit = %+v, want complete and empty", empty) + } +} + +// TestGitStatusListsUncomparableChanges proves a file the content snapshot +// skips (here: above the hash bound) is still listed when its Git status +// changes during the visit, while a protected path never is. +func TestGitStatusListsUncomparableChanges(t *testing.T) { + previous := maxHashBytes + maxHashBytes = 16 + defer func() { maxHashBytes = previous }() + dir, git := gitRepo(t) + put(t, dir, "big.bin", strings.Repeat("x", 64)) + put(t, dir, ".env", "A=1\n") + put(t, dir, "small.txt", "one\n") + git("add", "big.bin", "small.txt") + git("commit", "-qm", "init") + ctx := context.Background() + before := Capture(ctx, dir) + put(t, dir, "big.bin", strings.Repeat("y", 64)) + put(t, dir, "fresh.bin", strings.Repeat("z", 64)) + put(t, dir, ".env", "A=2\n") + changes := Compare(before, Capture(ctx, dir, before)) + got := map[string]string{} + for _, f := range changes.Files { + if !f.NoPatch { + t.Fatalf("git-listed change carries a patch: %+v", f) + } + got[f.Path] = f.Kind + } + if len(got) != 2 || got["big.bin"] != "modified" || got["fresh.bin"] != "added" { + t.Fatalf("git status listing = %v (%+v)", got, changes) + } + if changes.Status != "partial" || !strings.Contains(changes.Diagnostic, "without patches") { + t.Fatalf("status/diagnostic = %q/%q", changes.Status, changes.Diagnostic) + } +} diff --git a/internal/pipeline/runs.go b/internal/pipeline/runs.go index d0f1a8e9..c461863c 100644 --- a/internal/pipeline/runs.go +++ b/internal/pipeline/runs.go @@ -303,6 +303,7 @@ strftime('%Y-%m-%dT%H:%M:%SZ',started_at),COALESCE(strftime('%Y-%m-%dT%H:%M:%SZ' // The run total is one pass over the steps already loaded above: it adds no // query and no transcript parse. r.Usage = protocol.TotalUsage(r.Steps) + r.Warnings = protocol.RunWarnings(r.Steps) alerts, err := tx.QueryContext(ctx, `SELECT sequence,event,channel,status,attempts,diagnostic,message_id FROM run_notifications WHERE run_id=? ORDER BY sequence,event,channel`, id) if err != nil { return nil, err diff --git a/internal/protocol/runs.go b/internal/protocol/runs.go index 49a91844..0869e695 100644 --- a/internal/protocol/runs.go +++ b/internal/protocol/runs.go @@ -1,6 +1,9 @@ package protocol -import "strings" +import ( + "fmt" + "strings" +) const ( RunPipeline = "run_pipeline" @@ -363,6 +366,32 @@ type RunUsage struct { CacheKnownStages int `json:"cache_known_stages,omitempty"` } +// RunWarnings derives the run-level warnings from its steps: every visit whose +// retained file-change capture was partial or unavailable is named, because +// such evidence is not a complete workspace diff and must never be read as "no +// edits". Steps with no capture (historical, human or server-skip visits) are +// not flagged here; their File changes section already says "not recorded". +func RunWarnings(steps []RunStep) []string { + var visits []string + for _, step := range steps { + if step.FileChanges == nil || step.FileChanges.Status == "complete" { + continue + } + visits = append(visits, fmt.Sprintf("%s #%d (%s)", step.Stage, step.Sequence, step.FileChanges.Status)) + } + if len(visits) == 0 { + return nil + } + const shown = 8 + list := visits + extra := "" + if len(list) > shown { + extra = fmt.Sprintf(" and %d more", len(list)-shown) + list = list[:shown] + } + return []string{fmt.Sprintf("File-change capture was incomplete for %d visit(s): %s%s. Recorded file changes are not a complete workspace diff, and no stage is auto-skipped after such a visit.", len(visits), strings.Join(list, ", "), extra)} +} + // TotalUsage sums the per-stage usage of a run's steps into one run total. It is // the one definition of the run total, so per-run inspection cannot drift from // the history-wide aggregate over the same rows. @@ -432,11 +461,15 @@ type StageFileChanges struct { } type FileChange struct { - Path string `json:"path"` - Kind string `json:"kind"` // added, modified, deleted - Patch string `json:"patch,omitempty"` - Binary bool `json:"binary,omitempty"` - Truncated bool `json:"truncated,omitempty"` + Path string `json:"path"` + Kind string `json:"kind"` // added, modified, deleted + Patch string `json:"patch,omitempty"` + Binary bool `json:"binary,omitempty"` + Truncated bool `json:"truncated,omitempty"` + // NoPatch marks a change listed without any patch: the content exceeded the + // capture budget (compared by hash) or the change was seen only in Git + // status. Truncated is also set so older readers show "patch omitted". + NoPatch bool `json:"no_patch,omitempty"` BeforeMode string `json:"before_mode,omitempty"` AfterMode string `json:"after_mode,omitempty"` } @@ -465,6 +498,10 @@ type RunDetail struct { Input map[string]any `json:"input"` Output map[string]any `json:"output"` Steps []RunStep `json:"steps"` + // Warnings are run-level, read-time notes an operator should see before + // trusting the evidence, e.g. that a visit's file-change capture was partial. + // Derived from Steps by RunWarnings; absent when there is nothing to flag. + Warnings []string `json:"warnings,omitempty"` // TitleSource is "model" when Title came from the shared summariser or // "fallback" when it was derived from the prompt; TitleModel names the model // that produced a model title. They are retained so a diagnosis can tell the diff --git a/internal/server/runner.go b/internal/server/runner.go index 3e3f7292..c00ad88a 100644 --- a/internal/server/runner.go +++ b/internal/server/runner.go @@ -951,11 +951,14 @@ type runState struct { input stage.Values // changedPaths retains a bounded sample of the run's recorded changed paths // and changedCount the full total, so the skip guard reads only in-memory - // evidence. uiSurface is latched once any recorded change touched a UI - // surface. Both are folded in by recordChanges (ui_surface.go). - changedPaths []string - changedCount int - uiSurface bool + // evidence. uiSurface is latched once any visit captured a rendered (image) + // artifact. captureIncomplete counts visits whose file-change capture was + // missing, partial or unavailable; any such visit forbids the server skip + // fallback. All are folded in by recordChanges (ui_surface.go). + changedPaths []string + changedCount int + uiSurface bool + captureIncomplete int // capturedPaths is the full, deduplicated, first-seen-ordered set of the // run's server-captured changed paths, distinct from the bounded // changedPaths sample used only by the skip reason. capturedSeen is the @@ -1020,7 +1023,7 @@ func rebuildRunState(detail *protocol.RunDetail, decided protocol.RunStep) *runS } // Seed the UI-surface accumulator from the persisted evidence so a // resumed run reaches the same skip decision with no extra query. - state.recordChanges(step.FileChanges, step.Artifacts) + state.recordPersistedStep(step) // Only completed, recorded visits are eligible for a later stage's // review capability. The path is reconstructed by the runner below. if step.Status == "completed" { @@ -1062,7 +1065,7 @@ func rebuildRetryState(detail *protocol.RunDetail, failed protocol.RunStep, hand state.turns = append(state.turns, turn.MemoryEntry{Stage: step.Stage, Visit: state.visits[step.Stage], Output: turnOutput(step.Output), PublicFiles: added}) state.lastOutput = turnOutput(step.Output) } - state.recordChanges(step.FileChanges, step.Artifacts) + state.recordPersistedStep(step) if step.Status == "completed" { state.artifactDirs = append(state.artifactDirs, filepath.Join("", strconv.Itoa(step.Sequence))) } @@ -1073,6 +1076,9 @@ func rebuildRetryState(detail *protocol.RunDetail, failed protocol.RunStep, hand // retry source's immediate steps (which is what made retry-of-retry fail // closed with "memory source ... has not completed"). if prefix, ok := handoff["prefix"].(map[string]any); ok { + // The checkpoint carries outputs, not file-change captures, so an inherited + // prefix is unobserved evidence: the server skip fallback must not apply. + state.captureIncomplete++ for _, record := range retryPrefixRecords(handoff["prefix_provenance"]) { output, exists := prefix[record.stage].(map[string]any) if !exists { @@ -1595,7 +1601,7 @@ func (r *runner) runLoop(ctx context.Context, id int64, graph protocol.Graph, na } } _, declaresSkip := def.Edges["skipped"] - skip := !isHuman && declaresSkip && !state.uiSurface + skip := !isHuman && declaresSkip && autoSkipEligible(def.Configs, state) state.nextSequence = sequence + 1 state.visits[key]++ // Publish the run's server-captured changed paths into the input memory @@ -1687,13 +1693,15 @@ func (r *runner) runLoop(ctx context.Context, id int64, graph protocol.Graph, na r.deliverRunNotificationBatch(time.Now(), 1) return } - // A stage that declares a `skipped` route is short-circuited when the run - // has recorded no UI surface: the stage would answer exactly that, so skip - // the model call and its capture work, record the step as completed and - // route along the declared edge. Only a declared route is honoured, and no - // other stage or outcome routing changes. + // Server skip fallback: a stage that declares a `skipped` route is + // short-circuited only when the run's evidence is complete and empty + // (autoSkipEligible, ui_surface.go): every visit recorded a complete + // capture with no file changes and no rendered artifact. Then skip the + // model call and its capture work, record the step as completed with a + // verdict that names the server fallback, and route along the declared + // edge. Partial, unavailable or missing evidence always runs the model. if skip { - reason := skipReasonLine(state) + reason := skipReasonLine() step := protocol.RunStep{ Sequence: sequence, Stage: key, @@ -1702,6 +1710,7 @@ func (r *runner) runLoop(ctx context.Context, id int64, graph protocol.Graph, na Output: stage.Values{"summary": reason}, Target: def.Edges["skipped"], Control: &protocol.ControlEvidence{Visit: state.visits[key], MaxVisits: maxVisits, Memory: reads}, + Verdict: &protocol.StageVerdict{Reason: reason, ReasonSource: protocol.VerdictReasonFallback}, } status, outcome, diagnostic, terminal, finishErr := r.finishStage(id, name, workspace, selected, state, step) if finishErr != nil { diff --git a/internal/server/ui_skip_test.go b/internal/server/ui_skip_test.go index 2658707b..7b333ad7 100644 --- a/internal/server/ui_skip_test.go +++ b/internal/server/ui_skip_test.go @@ -2,12 +2,15 @@ package server import ( "context" + "fmt" "os" + "os/exec" "path/filepath" "strings" "sync" "testing" + "git.a-wan.net/alan/pipelines/internal/filediff" "git.a-wan.net/alan/pipelines/internal/pipeline" "git.a-wan.net/alan/pipelines/internal/protocol" "git.a-wan.net/alan/pipelines/internal/stage" @@ -30,6 +33,12 @@ func uiReviewerSource(withSkip bool) string { // then returns the persisted detail, the visited keys and the number of verdict // (model) calls the runner made. func runUISkip(t *testing.T, source string, files map[string]string) (*protocol.RunDetail, []string, int) { + t.Helper() + return runUISkipIn(t, source, files, t.TempDir()) +} + +// runUISkipIn is runUISkip against a prepared workspace directory. +func runUISkipIn(t *testing.T, source string, files map[string]string, dir string) (*protocol.RunDetail, []string, int) { t.Helper() ctx, cancel := context.WithCancel(context.Background()) defer cancel() @@ -50,7 +59,6 @@ func runUISkip(t *testing.T, source string, files map[string]string) (*protocol. t.Fatal(err) } defer r.wg.Wait() - dir := t.TempDir() var mu sync.Mutex var visited []string verdictCalls := 0 @@ -106,37 +114,28 @@ func executed(visited []string, key string) bool { return false } -// TestUISkipShortCircuitsWhenNoSurfaceChanged proves a non-UI change skips the -// ui_reviewer stage with no model call and routes along its declared skip edge. -func TestUISkipShortCircuitsWhenNoSurfaceChanged(t *testing.T) { - detail, visited, verdictCalls := runUISkip(t, uiReviewerSource(true), map[string]string{"implement": "internal/server/fake.go"}) - if executed(visited, "ui_reviewer") { - t.Fatalf("ui_reviewer executed a model call; visited=%v", visited) +// assertModelRan fails unless ui_reviewer executed its model and routed through +// its own outcome rather than the server fallback. +func assertModelRan(t *testing.T, detail *protocol.RunDetail, visited []string, verdictCalls int) { + t.Helper() + if !executed(visited, "ui_reviewer") { + t.Fatalf("ui_reviewer was auto-skipped; visited=%v steps=%+v", visited, detail.Steps) } - if verdictCalls != 1 { - t.Fatalf("verdict (model) calls = %d, want only implement's 1", verdictCalls) + if verdictCalls != 2 { + t.Fatalf("verdict (model) calls = %d, want 2 for both executed stages", verdictCalls) } if detail.Status != "succeeded" { t.Fatalf("run status = %q, want succeeded: %+v", detail.Status, detail.RunSummary) } - if len(detail.Steps) != 2 { - t.Fatalf("steps = %d, want 2: %+v", len(detail.Steps), detail.Steps) - } - step := detail.Steps[1] - if step.Stage != "ui_reviewer" || step.Status != "completed" || step.Outcome != "skipped" || step.Target != protocol.SuccessEnd { - t.Fatalf("skip step wrong: %+v", step) - } - if step.FileChanges != nil && len(step.FileChanges.Files) != 0 { - t.Fatalf("skip step recorded file changes: %+v", step.FileChanges) - } - reason, _ := step.Output["summary"].(string) - if !strings.Contains(reason, "internal/server/fake.go") { - t.Fatalf("skip reason does not name the checked path: %q", reason) + step := detail.Steps[len(detail.Steps)-1] + if step.Stage != "ui_reviewer" || step.Outcome != "approved" || step.Target != protocol.SuccessEnd { + t.Fatalf("executed step did not route through approved: %+v", step) } } -// TestUISkipZeroChangesStillSkips proves the guard also short-circuits when the -// run recorded no file changes at all, with a reason that says so. +// TestUISkipZeroChangesStillSkips proves the fallback short-circuits when every +// visit recorded a complete capture with no file changes, and that the verdict +// says it was a server fallback with no model run. func TestUISkipZeroChangesStillSkips(t *testing.T) { detail, visited, verdictCalls := runUISkip(t, uiReviewerSource(true), nil) if executed(visited, "ui_reviewer") { @@ -148,48 +147,127 @@ func TestUISkipZeroChangesStillSkips(t *testing.T) { if detail.Status != "succeeded" || len(detail.Steps) != 2 { t.Fatalf("run = %q steps=%d, want succeeded/2", detail.Status, len(detail.Steps)) } + if first := detail.Steps[0].FileChanges; first == nil || first.Status != "complete" { + t.Fatalf("implement capture = %+v, want complete", first) + } step := detail.Steps[1] - if step.Outcome != "skipped" { - t.Fatalf("step outcome = %q, want skipped: %+v", step.Outcome, step) + if step.Stage != "ui_reviewer" || step.Status != "completed" || step.Outcome != "skipped" || step.Target != protocol.SuccessEnd { + t.Fatalf("skip step wrong: %+v", step) + } + if step.FileChanges != nil { + t.Fatalf("skip step recorded a capture: %+v", step.FileChanges) + } + if step.Verdict == nil || step.Verdict.ReasonSource != protocol.VerdictReasonFallback || !strings.HasPrefix(step.Verdict.Reason, serverSkipReasonPrefix) { + t.Fatalf("skip verdict does not name the server fallback: %+v", step.Verdict) } reason, _ := step.Output["summary"].(string) - if !strings.Contains(reason, "no recorded file changes") { + if !strings.Contains(reason, "recorded no file changes") { t.Fatalf("zero-change reason does not state that: %q", reason) } + if len(detail.Warnings) != 0 { + t.Fatalf("complete run carries warnings: %v", detail.Warnings) + } } -// TestUISkipRunsStageOnSurfaceChange proves a run that touched a UI surface -// still executes the stage and routes through its normal outcome. -func TestUISkipRunsStageOnSurfaceChange(t *testing.T) { +// TestUISkipRunsModelOnAnyRecordedChange proves no path is classified as "not +// UI": a backend file, a Flutter .dart file, a Kotlin file, a Go TUI file and a +// web .tsx file all make the stage run its model. The fallback never encodes a +// repository layout. +func TestUISkipRunsModelOnAnyRecordedChange(t *testing.T) { cases := map[string]string{ - "android": "android/app/src/main/kotlin/Main.kt", - "internal tui": "internal/tui/view.go", - "presentation": "web/app.tsx", + "go backend": "internal/server/fake.go", + "flutter dart": "app/lib/features/home/home_page.dart", + "kotlin": "android/app/src/main/kotlin/Main.kt", + "go tui": "internal/tui/view.go", + "web tsx": "web/app.tsx", + "docs": "docs/readme.md", } for name, path := range cases { t.Run(name, func(t *testing.T) { detail, visited, verdictCalls := runUISkip(t, uiReviewerSource(true), map[string]string{"implement": path}) - if !executed(visited, "ui_reviewer") { - t.Fatalf("ui_reviewer did not execute on %s; visited=%v", path, visited) - } - if verdictCalls != 2 { - t.Fatalf("verdict (model) calls = %d, want 2 for both executed stages", verdictCalls) - } - if detail.Status != "succeeded" { - t.Fatalf("run status = %q, want succeeded: %+v", detail.Status, detail.RunSummary) - } - step := detail.Steps[len(detail.Steps)-1] - if step.Stage != "ui_reviewer" || step.Outcome != "approved" || step.Target != protocol.SuccessEnd { - t.Fatalf("executed step did not route through approved: %+v", step) - } + assertModelRan(t, detail, visited, verdictCalls) }) } } +// TestUISkipPartialCaptureNeverSkips proves a visit whose capture was partial +// (here a protected .env file forces skipped>0) blocks the fallback even when +// it recorded no file changes, and that inspect carries a run-level warning. +func TestUISkipPartialCaptureNeverSkips(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, ".env"), []byte("A=1\n"), 0o600); err != nil { + t.Fatal(err) + } + detail, visited, verdictCalls := runUISkipIn(t, uiReviewerSource(true), nil, dir) + if first := detail.Steps[0].FileChanges; first == nil || first.Status != "partial" || len(first.Files) != 0 { + t.Fatalf("fixture capture = %+v, want partial with no files", first) + } + assertModelRan(t, detail, visited, verdictCalls) + if len(detail.Warnings) != 1 || !strings.Contains(detail.Warnings[0], "implement #1 (partial)") { + t.Fatalf("partial capture warning missing: %v", detail.Warnings) + } +} + +// TestUISkipOverBudgetGitWorkspaceRunsModel is the run-236 regression: in a Git +// workspace whose text exceeds the snapshot budget, a stage edits one .tsx file +// sorted after the budget is exhausted. The capture must list that file +// (without a patch) and the next stage, which declares `skipped`, must run its +// model rather than be auto-routed with "no recorded file changes". +func TestUISkipOverBudgetGitWorkspaceRunsModel(t *testing.T) { + if _, err := exec.LookPath("git"); err != nil { + t.Skip("git unavailable") + } + restore := filediff.SetSnapshotBudgetForTesting(256) + defer restore() + dir := t.TempDir() + git := func(args ...string) { + t.Helper() + cmd := exec.Command("git", append([]string{"-C", dir, "-c", "user.email=t@example.invalid", "-c", "user.name=t", "-c", "commit.gpgsign=false"}, args...)...) + if data, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("%v: %v %s", args, err, data) + } + } + git("init", "-q") + target := "web/src/pages/inbox/PortfolioIntelligenceInbox.test.tsx" + if err := os.MkdirAll(filepath.Join(dir, filepath.Dir(target)), 0o700); err != nil { + t.Fatal(err) + } + for i := 0; i < 40; i++ { + if err := os.WriteFile(filepath.Join(dir, fmt.Sprintf("a-%02d.txt", i)), []byte(strings.Repeat("filler line\n", 4)), 0o600); err != nil { + t.Fatal(err) + } + } + if err := os.WriteFile(filepath.Join(dir, filepath.FromSlash(target)), []byte(strings.Repeat("original test line\n", 8)), 0o600); err != nil { + t.Fatal(err) + } + git("add", ".") + git("commit", "-qm", "fixture") + detail, visited, verdictCalls := runUISkipIn(t, uiReviewerSource(true), map[string]string{"implement": target}, dir) + capture := detail.Steps[0].FileChanges + if capture == nil || len(capture.Files) != 1 || capture.Files[0].Path != target || !capture.Files[0].NoPatch { + t.Fatalf("over-budget capture did not list the edited file: %+v", capture) + } + assertModelRan(t, detail, visited, verdictCalls) + if len(detail.Warnings) == 0 { + t.Fatal("partial capture produced no run-level warning") + } +} + +// TestUISkipOptOut proves auto_skip=false makes a `skipped` stage run its model +// even on complete, empty evidence. +func TestUISkipOptOut(t *testing.T) { + source := uiReviewerSource(true) + at := strings.LastIndex(source, "timeout=1h\n") + source = source[:at] + "timeout=1h\nauto_skip=false\n" + source[at+len("timeout=1h\n"):] + detail, visited, verdictCalls := runUISkip(t, source, nil) + assertModelRan(t, detail, visited, verdictCalls) +} + // TestUISkipNeedsDeclaredEdge proves the guard never invents a skip for a stage -// whose definition declares no `skipped` route: it always executes. +// whose definition declares no `skipped` route: it always executes, even on +// complete, empty evidence. func TestUISkipNeedsDeclaredEdge(t *testing.T) { - detail, visited, verdictCalls := runUISkip(t, uiReviewerSource(false), map[string]string{"implement": "internal/server/fake.go"}) + detail, visited, verdictCalls := runUISkip(t, uiReviewerSource(false), nil) if !executed(visited, "ui_reviewer") { t.Fatalf("ui_reviewer without a skipped edge did not execute; visited=%v", visited) } @@ -205,33 +283,37 @@ func TestUISkipNeedsDeclaredEdge(t *testing.T) { } } -func TestUISurfacePathClassification(t *testing.T) { - cases := []struct { - path string - want bool - }{ - {"android/app/src/main/kotlin/Main.kt", true}, - {"android/README.md", true}, - {"internal/tui/view.go", true}, - {"internal/tui", true}, - {"web/index.html", true}, - {"web/app.css", true}, - {"web/app.js", true}, - {"web/app.ts", true}, - {"web/app.jsx", true}, - {"web/app.tsx", true}, - {"web/Index.HTML", true}, - {"internal/server/fake.go", false}, - {"androidx/foo.go", false}, - {"cmd/pipelines/main.go", false}, - {"docs/readme.md", false}, - {"README.md", false}, +// TestUISkipPersistedEvidence proves the resume/retry seed: a stored server +// fallback step neither adds nor breaks evidence, while a stored visit with no +// capture (a human decision, a crash, a historical row) or a partial one makes +// the evidence incomplete. +func TestUISkipPersistedEvidence(t *testing.T) { + complete := &protocol.StageFileChanges{Status: "complete", Files: []protocol.FileChange{}} + reason := skipReasonLine() + serverSkip := protocol.RunStep{Stage: "ui", Outcome: "skipped", Verdict: &protocol.StageVerdict{Reason: reason, ReasonSource: protocol.VerdictReasonFallback}} + s := &runState{} + s.recordPersistedStep(protocol.RunStep{Stage: "implement", FileChanges: complete}) + s.recordPersistedStep(serverSkip) + if !autoSkipEligible(nil, s) { + t.Fatalf("complete evidence plus a server skip is not eligible: %+v", s) } - for _, tc := range cases { - if got := uiSurfacePath(tc.path); got != tc.want { - t.Errorf("uiSurfacePath(%q) = %v, want %v", tc.path, got, tc.want) + legacy := serverSkip + legacy.Verdict = &protocol.StageVerdict{Reason: "no UI surface changed (no recorded file changes)", ReasonSource: protocol.VerdictReasonFallback} + for name, step := range map[string]protocol.RunStep{ + "no capture": {Stage: "gate", Outcome: "approve"}, + "partial": {Stage: "implement", FileChanges: &protocol.StageFileChanges{Status: "partial", Files: []protocol.FileChange{}}}, + "unavailable": {Stage: "implement", FileChanges: &protocol.StageFileChanges{Status: "unavailable"}}, + "legacy skip row": legacy, + } { + s := &runState{} + s.recordPersistedStep(step) + if autoSkipEligible(nil, s) { + t.Errorf("%s: incomplete evidence is eligible for the fallback", name) } } + if autoSkipEligible(map[string]string{"auto_skip": "false"}, &runState{}) { + t.Error("auto_skip=false did not opt out") + } } func TestUISurfaceArtifactClassification(t *testing.T) { diff --git a/internal/server/ui_surface.go b/internal/server/ui_surface.go index 3cd7a0c9..26e73297 100644 --- a/internal/server/ui_surface.go +++ b/internal/server/ui_surface.go @@ -3,36 +3,35 @@ package server import ( "fmt" "path" - "path/filepath" "strings" "git.a-wan.net/alan/pipelines/internal/protocol" ) -// UI-surface classification is the single source of truth for the deterministic -// pre-check that lets a stage declaring a `skipped` route short-circuit when the -// run changed no rendered surface. The path/extension lists live only here; the -// skip guard, its reason and the resumed-run seed all read them. +// The server skip fallback lets a stage that declares a `skipped` route +// short-circuit, without a model call, only when the run's evidence proves there +// is nothing to review: every visit so far recorded a complete file-change +// capture, none of them recorded any change, and no visit captured a rendered +// (image) artifact. Anything else runs the stage's model, which may still choose +// `skipped` itself. +// +// The rule deliberately classifies no path as "not UI". A path/extension list +// encodes one repository's layout (Go TUIs, Flutter .dart, Kotlin, SwiftUI, +// templates, strings files and config can all be rendered surface), and absent +// or partial evidence must never be read as "no edits". A wrong run costs one +// model call; a wrong skip ships an unreviewed UI change. // maxRetainedChangedPaths bounds how many recorded changed paths a runState -// retains for the skip reason. The full count is kept separately, so the reason -// stays informative without unbounded memory. +// retains as a sample. The full count is kept separately. const maxRetainedChangedPaths = 32 -// uiSurfaceDirs are the workspace prefixes whose changes are a user-facing -// surface. Matching is on a path boundary, so "androidx/" is not "android/". -var uiSurfaceDirs = []string{"android", "internal/tui"} +// serverSkipReasonPrefix starts every server fallback skip reason, so the +// verdict says plainly that no model ran, and so a resumed run recognises its +// own earlier fallback steps (which have no capture of their own). +const serverSkipReasonPrefix = "Server fallback skip (no model run):" -// presentationExtensions are the rendered presentation file types a change can -// touch without living under a surface directory. -var presentationExtensions = map[string]bool{ - ".html": true, - ".css": true, - ".js": true, - ".ts": true, - ".jsx": true, - ".tsx": true, -} +// autoSkipConfig is the stage config that opts a stage out of the fallback. +const autoSkipConfig = "auto_skip" // renderedArtifactExtensions are the image types a captured artifact may carry // when its content type was not recorded or sniffed as an image. @@ -44,22 +43,6 @@ var renderedArtifactExtensions = map[string]bool{ ".webp": true, } -// uiSurfacePath reports whether a run-relative, forward-slash workspace path -// changed a user-facing surface: a file under android/ or internal/tui/, or a -// file with a presentation extension. It is O(len(path)) with no I/O. -func uiSurfacePath(p string) bool { - clean := strings.TrimPrefix(path.Clean(filepath.ToSlash(p)), "/") - if clean == "" || clean == "." { - return false - } - for _, dir := range uiSurfaceDirs { - if clean == dir || strings.HasPrefix(clean, dir+"/") { - return true - } - } - return presentationExtensions[strings.ToLower(path.Ext(clean))] -} - // uiSurfaceArtifact reports whether a captured artifact is a rendered surface: // an image content type, or an image file extension when no type was recorded. func uiSurfaceArtifact(a protocol.StageArtifact) bool { @@ -69,19 +52,20 @@ func uiSurfaceArtifact(a protocol.StageArtifact) bool { return renderedArtifactExtensions[strings.ToLower(path.Ext(a.Name))] } -// recordChanges folds one completed step's observed changes into the run's -// UI-surface decision. It retains a bounded sample of paths; the count is -// unbounded so the reason can state the true total. It is O(changes) with no I/O. +// recordChanges folds one executed visit's observed changes into the run's +// skip evidence. A nil, partial or unavailable capture marks the run's +// evidence incomplete for the rest of the run. It retains a bounded sample of +// paths; the count is unbounded. It is O(changes) with no I/O. func (s *runState) recordChanges(changes *protocol.StageFileChanges, artifacts []protocol.StageArtifact) { + if changes == nil || changes.Status != "complete" { + s.captureIncomplete++ + } if changes != nil { for _, file := range changes.Files { s.changedCount++ if len(s.changedPaths) < maxRetainedChangedPaths { s.changedPaths = append(s.changedPaths, file.Path) } - if !s.uiSurface && uiSurfacePath(file.Path) { - s.uiSurface = true - } // The full captured set is a separate, unbounded dedupe of the actual // evidence, unlike the bounded changedPaths sample above. if file.Path == "" || s.capturedSeen[file.Path] { @@ -101,16 +85,38 @@ func (s *runState) recordChanges(changes *protocol.StageFileChanges, artifacts [ } } -// skipReasonLine states, as one bounded line, why the guard skipped a stage: no -// recorded change touched a UI surface. It names the retained checked paths. -func skipReasonLine(state *runState) string { - if state.changedCount == 0 { - return "no UI surface changed (no recorded file changes)" +// recordPersistedStep seeds the skip evidence from a stored step when a run is +// resumed or retried. A server fallback skip executed nothing and carries no +// capture, so it neither adds evidence nor makes it incomplete; every other +// stored visit (including a human decision, during which the workspace was not +// observed) is folded in like a live visit. +func (s *runState) recordPersistedStep(step protocol.RunStep) { + if serverSkipStep(step) { + return } - line := fmt.Sprintf("no UI surface changed in %d recorded file change(s): %s", state.changedCount, strings.Join(state.changedPaths, ", ")) - if extra := state.changedCount - len(state.changedPaths); extra > 0 { - line = fmt.Sprintf("%s (+%d more)", line, extra) + s.recordChanges(step.FileChanges, step.Artifacts) +} + +// serverSkipStep reports whether a stored step is a server fallback skip. +func serverSkipStep(step protocol.RunStep) bool { + return step.Outcome == "skipped" && step.FileChanges == nil && step.Verdict != nil && + step.Verdict.ReasonSource == protocol.VerdictReasonFallback && + strings.HasPrefix(step.Verdict.Reason, serverSkipReasonPrefix) +} + +// autoSkipEligible reports whether the server may skip a stage declaring a +// `skipped` route: the stage has not opted out, every visit so far recorded a +// complete capture, no change was recorded and no rendered artifact captured. +func autoSkipEligible(configs map[string]string, state *runState) bool { + if configs[autoSkipConfig] == "false" { + return false } + return state.captureIncomplete == 0 && state.changedCount == 0 && !state.uiSurface +} + +// skipReasonLine states, as one bounded line, why the server skipped a stage. +func skipReasonLine() string { + line := fmt.Sprintf("%s every earlier visit's file-change capture was complete and recorded no file changes or screenshots.", serverSkipReasonPrefix) runes := []rune(line) if len(runes) > protocol.VerdictReasonCap { return string(runes[:protocol.VerdictReasonCap]) diff --git a/internal/stage/auto_skip_test.go b/internal/stage/auto_skip_test.go new file mode 100644 index 00000000..a339238d --- /dev/null +++ b/internal/stage/auto_skip_test.go @@ -0,0 +1,12 @@ +package stage + +import "testing" + +func TestAutoSkipConfig(t *testing.T) { + for value, ok := range map[string]bool{"": true, "true": true, "false": true, "no": false, "FALSE": false, "0": false} { + def := Definition{Type: ModelType, Blocks: map[string]string{"prompt": "task", "configs": "config"}, Configs: map[string]string{"provider": "fixture", "model": "model", "auto_skip": value}} + if _, err := (ModelStage{}).Spec(def); (err == nil) != ok { + t.Errorf("auto_skip=%q: err=%v, want ok=%v", value, err, ok) + } + } +} diff --git a/internal/stage/model.go b/internal/stage/model.go index a74c39e1..5c111ac0 100644 --- a/internal/stage/model.go +++ b/internal/stage/model.go @@ -79,7 +79,7 @@ func (ModelStage) Spec(def Definition) (ModelSpec, error) { } for key := range def.Configs { switch key { - case "provider", "model", "reasoning_effort", "retries", "timeout", "budget_usd", "budget_tokens", "harness", "delivery": + case "provider", "model", "reasoning_effort", "retries", "timeout", "budget_usd", "budget_tokens", "harness", "delivery", "auto_skip": default: return ModelSpec{}, fmt.Errorf("unknown config %q", key) } @@ -101,6 +101,11 @@ func (ModelStage) Spec(def Definition) (ModelSpec, error) { if delivery := def.Configs["delivery"]; delivery != "" && delivery != "true" { return ModelSpec{}, fmt.Errorf("delivery must be true when set") } + // auto_skip=false opts a stage with a `skipped` edge out of the server skip + // fallback (the runner reads it); true is the default and may be explicit. + if value := def.Configs["auto_skip"]; value != "" && value != "true" && value != "false" { + return ModelSpec{}, fmt.Errorf("auto_skip must be true or false") + } if def.Configs["delivery"] == "true" && backend != "opencode" { return ModelSpec{}, fmt.Errorf("delivery is supported only by the opencode harness") }