Strengthen template planning, reuse and independent review

This commit is contained in:
2026-10-12 23:10:37 +01:00
parent 03ed4e02e4
commit 143f237268
14 changed files with 424 additions and 46 deletions
+7 -2
View File
@@ -39,8 +39,13 @@ rejected. Limits: 50 bindings; 1 MiB per retained in-memory value map.
Implementers can receive task, plan and optional findings. Independent reviewers
should receive task/acceptance criteria and inspect actual artifacts—not previous
verdicts, counters, or implementation conversations. Selected content remains
untrusted data; this cannot prove semantic independence or prevent all injection.
verdicts, counters, or implementation conversations. Select acceptance, design,
reuse and research references from the planning stage (for example
`"acceptance_checks":"planner.acceptance_checks"`) rather than from the
implementer, so an implementer's output cannot overwrite the agreed criteria; take
only factual implementer fields such as `changed_files`, `implementation_summary`
and `evidence` as claims. Selected content remains untrusted data; this cannot
prove semantic independence or prevent all injection.
Memory is run-local; persisted completed outputs/read provenance are its audit.
There is no cross-run memory, restart/resume, or hidden replay. New runs/retries
+10 -2
View File
@@ -82,8 +82,16 @@ run directory; no file list is required. `shell` supports commands appropriate t
the repository's language and tooling, not a language-specific allowlist. A missing
tools block uses local read/glob/grep/edit/shell tools; prefer explicit stage roles.
Planning stages must not label a task blocked simply because they defer edits to
an implementer stage. Keep examples and prompts language-neutral unless the task
itself asks for a specific language.
an implementer stage. Bundled starters show the intended role split: a task stage
may read/edit/shell and conditionally use `websearch`/`webfetch`; a planner may
read plus conditional web search/fetch; a three-stage implementer has no web tools
and works from the planner's research with local read/glob/grep/edit/shell; a
reviewer may read plus conditional web; a source-only review stage has no
edit/shell; a research stage uses web search/fetch. Never force web access for
trivial, fully understood local edits, and return blocked only when required
reliable evidence cannot be obtained rather than whenever one search tool fails.
Keep examples and prompts language-neutral unless the task itself asks for a
specific language.
Optional `permissions` is a JSON object with `edit` (run-directory-relative
patterns) and `shell` (exact commands) arrays. It narrows declared tools and any
@@ -12,7 +12,9 @@ parallel/distributed task engine or a guarantee that model reports are correct.
immutable versions; drafts remain retained. Keep context create-only/base-version
guards. Five optional generic starters are task, summary, review, research and
three-stage; a fresh database starts empty. Templates can be created directly with
an editable prefilled name or customized first.
an editable prefilled name or customized first. Editing a bundled repository
template changes no installed or published definition; only explicit authoring
then submission creates a new immutable version.
- A run pins its exact definition version, initial input and resolved directory.
The client invocation directory is the default, but files/tools execute on the
server. Docker needs that directory mounted at the same absolute path. A remote
@@ -20,6 +22,9 @@ parallel/distributed task engine or a guarantee that model reports are correct.
- Host and Docker OpenCode credentials are separate. Preserve setup-selected model
inheritance; do not select a provider just because its ID appears in an example.
- Runtime caps are 30 stage steps and five visits to a stage. There are no parallel
runs on one server: admission rejects new work while another run is active;
reusing that active run's request key reattaches idempotently. Inspect and wait
for the existing run rather than starting duplicates. There are no parallel
branches or built-in human approval stages. Optional worktrees, full backup/restore,
automatic database sync and independent verifier stage types are future ideas,
not existing capabilities. Do not invent them in definitions.
@@ -13,7 +13,10 @@ can use appropriate permitted commands; do not accept a preceding stage's succes
summary as independent proof. Actual file bytes, command exit status, retained tool
parts and server-captured diffs are stronger evidence than prose. A read failure
can prove a deliberately missing input, but is not a successful read. Tool names
mentioned in text do not prove tools executed.
mentioned in text do not prove tools executed. A reviewer stage must inspect the
actual files and run its own checks; a source-only review runs no commands and
must state that runtime behavior was not tested. Structural, mocked or
authoring-time checks are not deployed or live-provider evidence.
Do not add technology-specific checks unless the task/repository requires them.
Keep runtime verification separate from authoring: ask before running project code,
@@ -420,3 +423,43 @@ container; do not copy host credentials or publish tests to the normal database.
`TestReferenceArtifactsReview` rechecks copied run/fixture evidence without
inference using `PIPELINES_REFERENCE_REVIEW_DIR`; select only cases present in
that artifact directory. Missing cases skip and must not be reported as passed.
## 2026-10-12 run22 success, run23 live CSV fixture and run24 interruption
Source/structural and live-fixture evidence stays separate from runtime
deployment; nothing here is a deployment claim.
- **run22 (structural/source checks, success):** the retained run22 verification
passed its structural/source checks of the scoped template and prompt changes.
- **run23 (live Pi CSV fixture):** a real exact-template Pi CSV run fetched the
official Python `csv` documentation through the conditional official-source
webfetch and implemented CSV parsing with the standard library's `csv.reader`
through an existing shared helper, adding no duplicate preview parser; both the
implementer and reviewer checks actually passed. Root independently verified the
full retained native tool results and protected-file hashes against its saved
baseline, rather than relying on model-written reports.
- **run24 (interrupted, not a pass):** the run24 implementation completed, but its
reviewer ignored the scope instruction and ran a prohibited filesystem-root scan
(`find /`), leaving a process in uninterruptible D-state and stalling Docker
mount/snapshot reads. Root canceled the run, made a consistent SQLite backup
(`integrity ok`), recovered Docker with the supported stop/start path, and
restored the existing ntfy/Pipelines services with the original definitions, run
history, credentials and settings verified unchanged. run24 is retained as
**interrupted**, not a success.
- **run25 (live Pi source-only review):** the reviewer identified duplicated
implementation, a hardcoded special case, suppressed errors and test gaps.
Root inspected retained native calls: only local read/search tools ran, with no
shell/edit/network calls. Original fixture hashes were unchanged; runtime
behavior was explicitly marked untested.
- **run26 (final maintenance review, success):** the scoped plan/implement/review
workflow passed package tests, vet, build and formatting checks. Root inspected
the actual retained outputs and changes. No rendered UI change was claimed.
The scoped prompt guidance asks stages to keep inspection/verification to relevant
repository paths and supported commands; it is a working instruction, **not** an OS
sandbox, and neither fixes Docker VM/disk bugs nor universally enforces tool
boundaries. At these recorded checkpoints the source changes had not been deployed.
Catalog deployment requires a separate check of the running server and its actual
template contents; these observations do not establish publication or deployment.
Other providers and phone delivery remain **not tested** by this record.
+7
View File
@@ -3,6 +3,7 @@ package pipeline
import (
"os"
"path/filepath"
"strings"
"testing"
"git.a-wan.net/alan/pipelines/internal/protocol"
@@ -24,6 +25,9 @@ func TestShowcaseDefinitions(t *testing.T) {
if stage.FallbackError {
t.Fatalf("%s should declare its error route explicitly", stage.Key)
}
if strings.TrimSpace(stage.Output) == "" || strings.TrimSpace(stage.Prompt) == "" {
t.Fatalf("%s stage %s needs a nonempty output and prompt", name, stage.Key)
}
for _, target := range stage.Edges {
if target == protocol.SuccessEnd {
success = true
@@ -36,6 +40,9 @@ func TestShowcaseDefinitions(t *testing.T) {
if !success || !failure {
t.Fatal("showcase must have both terminal outcomes")
}
if graph.Settings != nil {
t.Fatalf("%s starters must stay manual-only", name)
}
})
}
}