From 01d5dad9b9e0a7102f4571fea8f3221bc15be30a Mon Sep 17 00:00:00 2001 From: Alan Silva Date: Sun, 18 Oct 2026 01:50:55 +0100 Subject: [PATCH] docs: describe the agent-driven workflow, drop the scripted driver model MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewrite the workflow documentation to describe how the system actually works: the agent (the session) is the brain and advances the queue by hand through the `pipelines` CLI. There is no orchestration script, no driver, no polling loop and no lane machinery, and none may be written. - references/work.md: rewritten end to end. The queue-driver script (drivers/workd.sh), the claim-file ledger with atomic renames, the PAUSED marker and the poll loop are gone. State is now the server (authoritative for run state), the queue file (authoritative for intent) and the conversation. The agent reads the queue, picks the item, creates its worktree/branch and launches it with `pipelines run --json --yes`; progress is event-driven via a background `pipelines wait [...]` (never polled); on completion the agent reconciles with `pipelines inspect --json`, verifies it landed and starts the next item. - references/queue.md: the landing gate runs the command directly (`git merge-base --is-ancestor main`, exit 0 = landed) instead of a `land-check.sh` helper; the never-delete-branch / never-remove-worktree rules are kept, and the waiting section states the agent advances the queue by hand. - references/orchestration.md: intro and "Running and watching" state the agent-driven, event-driven model; "Pipeline choice" names the current set (Android, TUI, Server, Generic, Plan and land, UI sweep, Probe) by area and records the retired names. - references/debugging.md: "Duplicated work after a driver restart" became "…after a session restart", resuming from the queue file not a cursor file. - SKILL.md: the routing row no longer says "queue driver". - AGENTS.md: the maintenance workflow names the current pipeline set instead of the retired `Pipelines maintenance - plan implement review`. - internal/tui/AGENTS.md: the restart line no longer names lanes. - docs/notes/agent-driven-work-model.md: the docs-fold note recording that docs/cli.md needs its "Scripted runs"/`--wait` framing aligned to the event-driven model, and that the dated evidence docs stay as they are. Checked before committing: no packaged template directory exists in this repo, so none needed updating; internal/author/install.go's managed file inventory needs no change (no packaged reference was added or removed — the 13 listed files all match internal/author/skill/references/); no stale `queue driver`/`workd.sh`/ `claim-file` references remain anywhere under internal/author. Genuine device tooling (android/scripts/capture_png.sh-style) and server-side polling prose were left alone. Checks: `go test ./internal/author -count=1` ok; `go build ./...` clean; `go vet ./...` clean; `gofmt -l .` clean. --- AGENTS.md | 18 +- docs/notes/agent-driven-work-model.md | 56 ++++ internal/author/skill/SKILL.md | 2 +- internal/author/skill/references/debugging.md | 19 +- .../author/skill/references/orchestration.md | 40 ++- internal/author/skill/references/queue.md | 8 +- internal/author/skill/references/work.md | 294 ++++++------------ internal/tui/AGENTS.md | 2 +- 8 files changed, 207 insertions(+), 232 deletions(-) create mode 100644 docs/notes/agent-driven-work-model.md diff --git a/AGENTS.md b/AGENTS.md index cc40fd17..4f100447 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,10 +31,14 @@ the single `docs-fold` writer rather than editing them per item: leave a ## Maintenance workflow -Top-level agents use the configured `Pipelines maintenance - plan implement review` -maintenance workflow for code changes. Inspect the retained plan/implement/review -outputs and diffs, execute the checks independently, and iterate on concrete -failures rather than trusting stage summaries. An executing stage performs its own -role directly and never recursively launches a workflow. If the workflow is -unavailable, ask instead of silently bypassing it. Never hardcode a disposable -server URL as a project default; the configured endpoint is authoritative. +Top-level agents advance work through the `pipelines` CLI; **the agent is the brain** +and there is no orchestration script, driver, polling loop or lane machinery, and none +may be written. Choose the pipeline by the area the work is in — `Android`, `TUI`, +`Server`, `Generic`, `Plan and land`, `UI sweep`, `Probe` — each reading the project's +own map (`android/AGENTS.md`, `internal/tui/AGENTS.md`, `internal/server/AGENTS.md`) +for how that technology is built, tested, run and captured. Inspect the retained +plan/implement/review outputs and diffs, execute the checks independently, and iterate +on concrete failures rather than trusting stage summaries. An executing stage performs +its own role directly and never recursively launches a workflow. If a pipeline is +unavailable, ask instead of silently bypassing it. Never hardcode a disposable server +URL as a project default; the configured endpoint is authoritative. diff --git a/docs/notes/agent-driven-work-model.md b/docs/notes/agent-driven-work-model.md new file mode 100644 index 00000000..90b412df --- /dev/null +++ b/docs/notes/agent-driven-work-model.md @@ -0,0 +1,56 @@ +# Work documentation now describes the agent-driven model + +## What this item changed + +The workflow documentation was rewritten to describe how the system actually works: +the agent (the session) is the brain and advances the queue by hand through the +`pipelines` CLI. There is **no orchestration script, no driver, no polling loop and no +lane machinery, and none may be written.** + +- `internal/author/skill/references/work.md` — rewritten end to end. The queue-driver + script (`drivers/workd.sh`), the claim-file ledger with atomic renames, the `PAUSED` + marker and the poll loop are gone. The ledger is now the per-project markdown + `queue.md` (authoritative for intent) plus `notes.md`; run state is read from the + server (`pipelines inspect --json`); the agent launches each run itself with + `pipelines run --json --yes` from the item's worktree and is woken + by a background `pipelines wait`. +- `internal/author/skill/references/queue.md` — the landing gate now runs the command + directly (`git merge-base --is-ancestor main`, exit 0 = landed) instead of a + `land-check.sh` helper; the waiting section states the agent advances the queue by + hand and there is no driver. +- `internal/author/skill/references/orchestration.md` — the intro and "Running and + watching" state the agent-driven, event-driven model; "Pipeline choice" now names the + current set (`Android`, `TUI`, `Server`, `Generic`, `Plan and land`, `UI sweep`, + `Probe`), chosen by area, and records the retired names. +- `internal/author/skill/references/debugging.md` — "Duplicated work after a driver + restart" became "…after a session restart", resuming from the queue file rather than a + cursor file. +- `internal/author/skill/SKILL.md` — the routing table row no longer says "queue driver". +- `AGENTS.md` — the maintenance workflow section names the current pipeline set instead + of the retired `Pipelines maintenance - plan implement review`. +- `internal/tui/AGENTS.md` — the "do not restart … the lanes …" line no longer names + lanes. + +`delivery.md`, `operations.md` and `verification.md` were swept and needed no change: +they contain no driver, poll loop, cursor file or lane guidance. Genuine device tooling +(`android/scripts/capture_png.sh`) is left alone. + +## Shared documents this change makes stale + +No README or topic doc still teaches the scripted driver model. One topic doc needs a +wording alignment for the event-driven model; the `docs-fold` writer should apply it +when it next folds notes: + +- `docs/cli.md` — its "Scripted runs" section frames runs as scripted automation, and + its `--wait` paragraph is the only follow-a-run guidance. It should state that the + agent (not a script) advances work, and that `pipelines wait [run-id ...]` + run as a **background task** is the event-driven wait. The `wait` command's own flag + text is already recorded separately in `docs/notes/wait-command.md`. + +## Do not change + +- The dated evidence documents (`docs/pipeline-performance-review.md`, + `docs/chat-verification.md`, `docs/app-integration-verification.md`) describe the + scripted era as it was and stay as they are. +- `internal/author/install.go`'s managed file inventory needs no change: no packaged + reference file was added or removed. diff --git a/internal/author/skill/SKILL.md b/internal/author/skill/SKILL.md index 68ec01d2..93942ffc 100644 --- a/internal/author/skill/SKILL.md +++ b/internal/author/skill/SKILL.md @@ -59,7 +59,7 @@ and `references/format.md` (in parallel when possible); for notifications also r | Task | Packaged reference | | --- | --- | | Briefs, work items, pipeline choice, running and watching | `references/orchestration.md` | -| Across projects: work ledger, start/pause/resume/status, queue driver | `references/work.md` | +| Across projects: work ledger, start/pause/resume/status, the queue | `references/work.md` | | The work queue, parallel items and landing | `references/queue.md` | | Diagnose a wedged, blocked or failed run | `references/debugging.md` | | Publish artefacts, LAN URLs, install boundaries | `references/delivery.md` | diff --git a/internal/author/skill/references/debugging.md b/internal/author/skill/references/debugging.md index 5f8c0da0..54e82afd 100644 --- a/internal/author/skill/references/debugging.md +++ b/internal/author/skill/references/debugging.md @@ -130,13 +130,16 @@ Each playbook is *symptom → how to confirm → what to do*. approval. Put the capture helper in the repository so every UI run reuses one path. See the example in `references/delivery.md`. -## Duplicated work after a driver restart +## Duplicated work after a session restart -- **Symptom** Items already completed run again after a queue driver - restarts: the same brief is re-invoked, side effects repeat, or a commit is made +- **Symptom** Items already completed run again after the session that was advancing the + queue restarts: the same brief is re-invoked, side effects repeat, or a commit is made twice. -- **Confirm** Compare the queue's items against their recorded state. A list alone - has no resume point. -- **Do** Record each item's state in the queue itself (`references/queue.md`) and resume from it. Record an item - only once it has landed, and skip recorded items on startup. Never relaunch a - duplicate of a run that is still active — reattach with its request key instead. +- **Confirm** Compare the queue's items against their recorded state and the server's run + history (`inspect RUN_ID --json`). A list alone has no resume point. +- **Do** Record each item's state in the queue itself (`references/queue.md`) and resume + from it: read the queue top to bottom, skip items already marked done, and settle the + ones marked running by inspecting their run id. Record an item only once it has landed. + Never relaunch a duplicate of a run that is still active — reattach with its request key + instead. Do not build a driver or cursor file to remember position; the queue file is + the resume point. diff --git a/internal/author/skill/references/orchestration.md b/internal/author/skill/references/orchestration.md index c36ddce7..aadfbc80 100644 --- a/internal/author/skill/references/orchestration.md +++ b/internal/author/skill/references/orchestration.md @@ -5,6 +5,11 @@ briefs and a set of managed runs, not the change itself. The implementer stage m the change; the reviewer stages check it; the committer stage commits it. Route all project work through Pipelines runs. +The agent advances this **by hand through the `pipelines` CLI**: it launches each run, +is woken when it finishes, reconciles against the server, and starts the next item. +There is no orchestration script, driver, polling loop or lane machinery, and none may +be written. + ## Stance - **Reconnoitre first.** Read enough of the repository, its notes and recent history @@ -141,6 +146,24 @@ pipelines run "Docs fold" ## Pipeline choice +Choose by the **area the work is in**; each area pipeline reads the project's own map +for its build, test, run and capture commands: + +- **`Android`** — `android/`. +- **`TUI`** — `internal/tui/`. +- **`Server`** — `internal/server/`, `internal/stage/`, `internal/pipeline/` and + `internal/protocol/`. +- **`Generic`** — only when a change genuinely spans areas. +- **`Plan and land`** — when the change needs a plan before implementation + (contract-level work). +- **`UI sweep`** — a full-screen review pass. +- **`Probe`** — a harness smoke check. + +The retired names (`Pipelines quick`/`maintenance`/`critical`, `Focused tweak`, `Quick +tweak`, `Plan, implement, review`, `Harness probe`) stay in the registry because +definitions are immutable, but nothing should start them. If a pipeline is unavailable, +ask instead of silently bypassing it. + - Use a pipeline that keeps **implementer + reviewer + ui_reviewer + committer** for anything that matters. The planner is the only stage a leaner variant may drop. - **Never remove a review stage to go faster.** The reviewer stages are the @@ -162,9 +185,10 @@ pipelines run "Docs fold" - A run cannot approve its own work. When an item genuinely needs an operator gate before it continues or commits, insert a `type=human` decision stage (`references/format.md`); never invent a model stage that pretends to approve. -- Choose per item: a logic, server or format change usually wants the planner - (a plan/implement/review pipeline); a small, fully understood tweak may use the - leaner implementer-first variant. Both keep all review stages. +- Pick the pipeline whose area matches the change (above); `Generic` only when a change + genuinely spans areas. A logic, server or format change usually wants the planner + (`Plan and land`); a small, fully understood tweak may use the leaner area pipeline. + Both keep all review stages. ## Running and watching @@ -179,9 +203,13 @@ pipelines run "Docs fold" and are recorded on the run; `retry` reuses them. See `references/harnesses.md` for how a stage's model and its repair path both use the effective model. - Watch to a **terminal state** (`succeeded`, `failed`, `canceled`, `interrupted`). - Poll `CLI_PREFIX inspect RUN_ID --json` (or use `--wait`) until terminal. A reached - client wait limit is not a run failure: the run stays active; keep waiting or - reattach with the original request key, never start a duplicate run. + Progress is **event-driven, never polled**: `CLI_PREFIX wait RUN_ID […]` blocks until + the runs are terminal and is run as a **background task**, so the session is woken + exactly when the batch finishes; `CLI_PREFIX inspect RUN_ID --json` then reads the + retained outcome. A reached client wait limit is not a run failure: the run stays + active; keep waiting or reattach with the original request key, never start a + duplicate run. Do not build a polling loop, a cursor file or a driver script to + answer "is it done yet". - Read the **outcomes**, not just the status: the option each stage returned, the evidence it recorded, and any run or step diagnostic. Interpret diagnostics by category (`references/debugging.md`). diff --git a/internal/author/skill/references/queue.md b/internal/author/skill/references/queue.md index c4b51d0b..246d4f7f 100644 --- a/internal/author/skill/references/queue.md +++ b/internal/author/skill/references/queue.md @@ -49,8 +49,9 @@ Serialize only when: ## The landing gate, and what must never happen -- After every item, and **before any cleanup or before starting the next item**, run - `land-check.sh `. It must report the branch as an **ancestor of main**. +- After every item, and **before any cleanup or before starting the next item**, verify + the branch is an **ancestor of main** directly, with no helper script: + `git merge-base --is-ancestor main` (exit 0 = landed). - If it refuses: do not remove the worktree, do not touch the branch, do not start the next item. Report the landing failure and the commits that are not in main. - **Never delete a branch.** Deleting a branch whose commits are not yet in main leaves them @@ -60,6 +61,9 @@ Serialize only when: ## Waiting without a scripting harness +- The agent advances the queue **by hand**: read it, create the item's worktree, and launch + `pipelines run --json --yes` from that worktree. Nothing else advances it — + there is no driver, and none may be written. - `pipelines wait […​]` blocks until the runs finish and exits; run it as a **background task** so the session is woken exactly when the batch is done. A run that failed is still a finished run: it exits 0 and says what happened. diff --git a/internal/author/skill/references/work.md b/internal/author/skill/references/work.md index 03dfeb69..5308c137 100644 --- a/internal/author/skill/references/work.md +++ b/internal/author/skill/references/work.md @@ -1,256 +1,136 @@ # Project work ledger and sessions Work in a project is not a chat. It is a durable ledger: what is queued, what is -running, what has landed, and what is blocked. A session — this conversation — starts, -pauses, resumes and reports that ledger. Nothing important lives only in the -conversation: running the skill again, in any project on this machine, reads the -ledger and continues. This is the multi-project lifecycle on top of +running, what has landed, and what is blocked. A session — this conversation — reads +the ledger, launches the runs, is woken when they finish, and reports. + +**The agent is the brain.** It advances the queue by hand through the `pipelines` +CLI: it reads the queue, picks the item, launches the run(s) itself, is woken when they +finish, reconciles against the server, and starts the next item. There is **no +orchestration script, no driver, no polling loop and no lane machinery, and none may be +written.** Where a check is needed, run the command directly. + +State lives in exactly three places: + +- the **server** is authoritative for run state — `pipelines inspect --json`; +- the **queue file** is authoritative for intent — what is queued, in what order, and + what is finished; +- the **conversation** holds this session's working memory. + +Never keep progress in a script's own files. A session that dies and comes back reads +the queue file and continues. This is the multi-project lifecycle on top of `references/orchestration.md` (work items, briefs, pipeline choice) and -`references/queue.md` (parallel items). +`references/queue.md` (parallel items, one worktree per item, landing). ## Where the ledger lives -Keep state **outside repositories**, under one work root, so it never collides with a +Keep state **outside repositories**, under one work root, so it never collides with an item's worktree or a commit: ${XDG_STATE_HOME:-$HOME/.local/state}/pipelines/work/ - registry.json every project this machine knows + registry.json every project this machine knows projects// - state.json project, root, endpoint, pipeline, workspace - notes.md recon, decisions, open questions, blockers - queue/.brief pending briefs, read in filename order - active/.brief, .claim claimed: the brief plus its run id and key - done/ landed: run id, status, finished - failed/ and .brief failed twice: run id, status, diagnostic - log/driver.log - drivers/workd.sh this project's queue driver - PAUSED marker: stop dispatching new items + queue.md the project's markdown queue: items, briefs, state + notes.md recon, decisions, open questions, blockers - `` is `-<8 hex of sha256(absolute project root)>`. **Resolve a - worktree to its main repository** so every item shares the project's one ledger. + worktree to its main repository** so every item shares the project's one queue. - `PIPELINES_WORK_DIR` overrides the work root. `registry.json` maps projects to roots; add a project the first time it is worked on and keep it current. -- `state.json` holds `project`, `root`, `endpoint` (the CLI's `--url`), `pipeline` - (the default pipeline) and `workspace` (the default run directory). -- The ledger is the operator's record: keep `notes.md` truthful about decisions taken - and left open, never narrate it as a diary. +- `queue.md` is the resume point (`references/queue.md`): read it top to bottom and + take the first unfinished item. `notes.md` is the operator's record: keep it truthful + about decisions taken and left open, never narrate it as a diary. +- The CLI's configured endpoint and the area map's default pipeline come from the CLI + and the project's own map, not from a script. Do not write a per-project state file + for a script to read. ## Operations Map one operator sentence to one operation. Do the operation, then answer with the -status block — nothing else. +status block — nothing else. **You launch the runs yourself**; nothing else does it. | Operator says | Do | | --- | --- | -| start / work on X | Reconnoitre, register the project, write small briefs into `queue/`, then run the driver. | -| status / how is it going | Print the status block. | -| queue "…" / add … | Reconnoitre, split into small items, append briefs; the driver picks them up. | -| pause / stop for now | Write `PAUSED`. The item in flight finishes; no new one starts. | -| resume / continue | Reconcile, remove `PAUSED`, run the driver. | -| drop item N | Move its brief to `failed/` with a reason. Never silently delete work. | -| done / stop everything | Pause and report what landed and what remains. | +| start / work on X | Reconnoitre, register the project, write the queue file, then launch the first item's run yourself. | +| status / how is it going | Read the server and the queue, then print the status block. | +| queue "…" / add … | Reconnoitre, split into small items, append them to the queue file, launch what is ready. | +| pause / stop for now | Launch nothing new. The runs already in flight finish; reconcile them when they do. | +| resume / continue | Reconcile against the server, then launch the next item. | +| drop item N | Mark its item failed in the queue file with a reason. Never silently delete work. | +| done / stop everything | Stop launching, wait for what is in flight, and report what landed and what remains. | **Reconcile before acting, always.** The server is authoritative for run state; the -ledger is authoritative for intent. On every invocation: read `registry.json` and the -project's ledger; for each `active/` claim, `CLI_PREFIX inspect --json` and -settle it (running → keep waiting; succeeded → `done/`; failed/interrupted → -`failed/`); only then do the requested operation. Never trust a claim file for a -status, and never start a second run for an item whose request key is already -admitted — that key is what makes a redispatch attach instead of duplicate. +queue file is authoritative for intent. On every invocation: read the project's +`queue.md`; for each item marked running, `pipelines inspect --json` and settle +it (running → keep waiting; succeeded → verify it landed and mark it done; +failed/interrupted → mark it failed with the diagnostic). Only then do the requested +operation. Never trust your memory of a run's status, and never start a second run for +an item whose request key is already admitted — reuse the original `--request-key` so a +redispatch attaches instead of duplicating. Status block (these lines, no prose around them): pipelines — active queue: 4 pending · 1 running (run 185 · reviewer · 6m) · 12 done · 2 failed - next: 0005-graph-arrows on "Focused tweak - implement review commit" + next: 0005-graph-arrows on "TUI" blocked: none ## The brief, and per-item directives -Write every brief exactly as `references/orchestration.md` describes. A brief file in -`queue/` may open with up to three directive lines, stripped before the prompt is sent: +Write every brief exactly as `references/orchestration.md` describes. An item in +`queue.md` may open its brief with up to three directive lines, stripped before the +prompt is sent: - # pipeline: Focused tweak - implement review commit + # pipeline: TUI # workspace: /path/to/item-worktree # device: emulator-5556 Goal: … -`pipeline` overrides the default for that item; `workspace` routes an item to its own -checkout (a driver only claims items for its own workspace); `device` is for the -brief's own text. Number briefs `0001-…`, `0002-…` so order is explicit. +`pipeline` overrides the default for that item; `workspace` names the item's own +worktree (create it with `git worktree add -b main`, `references/queue.md`); +`device` is for the brief's own text. Number items `0001-…`, `0002-…` so order is +explicit. -## The driver +## Advancing the queue by hand -A queue is advanced by a small driver, so work continues while the operator sleeps. -Write the driver once per project to `drivers/workd.sh` and launch it; it is -restart-safe (it resumes from `done/`, reattaches to `active/` runs by id, and never -starts a duplicate), and it stops on the first item that fails twice. Run **one -item at a time per worktree**; several items in their own worktrees advance in -parallel because an item is claimed by an atomic rename. +There is no driver. You advance the queue yourself, through the CLI: -```sh -#!/bin/sh -# workd.sh - advance one project's Pipelines work ledger. -# -# sh workd.sh [PROJECT_ROOT] [WORKSPACE] -# -# PROJECT_ROOT identifies the ledger (default: the current directory). -# WORKSPACE is the checkout this driver may run in (default: the ledger's -# workspace). One item per worktree lets several items advance against one shared -# queue: an item is claimed by an atomic rename, so it runs exactly once. -# -# Restart-safe. It resumes from done/, reattaches to active/ runs by id and -# never starts a duplicate. PIPELINES_WORK_POLL sets the poll gap in seconds -# (default 20); PIPELINES_CLI uses another client binary. -set -u +1. **Read the queue top to bottom** and take the first unfinished item. +2. **Choose the pipeline by area** (`references/orchestration.md`): `Android`, `TUI`, + `Server`, `Generic`, `Plan and land`, `UI sweep`, `Probe`. Each area pipeline reads + the project's own map for its build, test, run and capture commands. +3. **Create the item's worktree and branch**: `git worktree add -b main`. + One worktree per item is what lets independent items run at once without the + workspace guard becoming a bottleneck (`references/queue.md`). +4. **Launch the run from that worktree**: + `pipelines run --json --yes`. Keep the returned run id. +5. **Be woken when it finishes — do not poll.** `pipelines wait […]` blocks + until the runs are terminal and is run as a **background task**, so the session is + woken exactly when the batch finishes — or arm a timer and check on the tick. No + polling loop, no cursor file, no driver script, ever. +6. **Reconcile.** `pipelines inspect --json`; read the outcomes and evidence; + verify the branch landed with `git merge-base --is-ancestor main` (exit 0 = + landed); mark the item in `queue.md`; then start the next item — or several + independent items at once. -CLI=${PIPELINES_CLI:-pipelines} -POLL=${PIPELINES_WORK_POLL:-20} -ROOT=$(cd "${1:-$(pwd)}" && pwd) -WORK_ROOT=${PIPELINES_WORK_DIR:-${XDG_STATE_HOME:-$HOME/.local/state}/pipelines/work} -SLUG=$(basename "$ROOT")-$(printf '%s' "$ROOT" | shasum -a 256 | cut -c1-8) -D=$WORK_ROOT/projects/$SLUG -LOG=$D/log/driver.log -mkdir -p "$D/queue" "$D/active" "$D/done" "$D/failed" "$D/log" 2>/dev/null +**Parallelism is your judgement, bounded by real conflicts only**: two items editing +the same files at once, the one capture device, or the provider's spend cap. Launch as +many independent runs as the machine can carry without overload; serialise only for a +real conflict and say which limit is binding. Never present serialisation as the +operator's decision. -say() { printf '%s %s\n' "$(date -u '+%Y-%m-%dT%H:%M:%SZ')" "$*" | tee -a "$LOG"; } -jfield() { python3 -c 'import json,sys -try: d=json.load(open(sys.argv[1])) -except Exception: d={} -print(d.get(sys.argv[2]) or "")' "$1" "$2"; } -dstatus() { "$CLI" inspect "$1" --json 2>/dev/null | python3 -c 'import json,sys -try: d=json.load(sys.stdin) -except Exception: print("unknown"); raise SystemExit -print(d.get("status") or "unknown")'; } -item_field() { sed -n "s/^# *$2 *: *//p" "$1" 2>/dev/null | head -1; } # from a brief -claim_field() { sed -n "s/^$2=//p" "$1" 2>/dev/null | head -1; } # from a claim file -prompt_of() { python3 -c ' -import re,sys -lines=open(sys.argv[1],encoding="utf-8",errors="replace").read().splitlines() -while lines and re.match(r"^#\s*(pipeline|workspace|device)\s*:", lines[0]): - lines.pop(0) -while lines and lines[0].strip()=="": - lines.pop(0) -print("\n".join(lines))' "$1"; } - -[ -f "$D/state.json" ] || { echo "no ledger: $D/state.json" >&2; exit 2; } -PIPELINE=$(jfield "$D/state.json" pipeline) -ENDPOINT=$(jfield "$D/state.json" endpoint) -WS=${2:-$(jfield "$D/state.json" workspace)} -[ -n "$WS" ] || WS=$ROOT -[ -n "$PIPELINE" ] || { echo "state.json has no pipeline" >&2; exit 2; } - -wait_terminal() { # $1 run id -> terminal status (or "unknown" if unreadable) - bad=0 - while :; do - st=$(dstatus "$1") - case "$st" in - succeeded|failed|canceled|interrupted) echo "$st"; return 0;; - unknown) bad=$((bad+1)); [ "$bad" -ge 5 ] && { echo "unknown"; return 0; };; - *) bad=0;; - esac - sleep "$POLL" - done -} - -dispatch() { # $1 brief file $2 pipeline $3 key -> run id (empty on failure) - if [ -n "$ENDPOINT" ]; then - out=$("$CLI" run "$2" "$(prompt_of "$1")" --json --yes --workspace "$WS" --request-key "$3" --url "$ENDPOINT" 2>&1) - else - out=$("$CLI" run "$2" "$(prompt_of "$1")" --json --yes --workspace "$WS" --request-key "$3" 2>&1) - fi - printf '%s' "$out" | python3 -c 'import json,sys -try: d=json.load(sys.stdin) -except Exception: print("") -else: print((d.get("run_detail") or {}).get("id") or "")' -} - -record() { # $1 id $2 status $3 run id - where=done; [ "$2" = succeeded ] || where=failed - printf 'run_id=%s\nstatus=%s\nfinished=%s\n' "$3" "$2" "$(date -u '+%Y-%m-%dT%H:%M:%SZ')" > "$D/$where/$1" - rm -f "$D/active/$1.brief" "$D/active/$1.claim" -} - -run_item() { # $1 id $2 brief $3 pipeline $4 key -> 0 succeeded / 1 failed - rid=$(dispatch "$2" "$3" "$4") - if [ -z "$rid" ]; then printf 'status=dispatch-failed\n' > "$D/active/$1.claim"; say "dispatch failed: $1"; return 1; fi - printf 'run_id=%s\n' "$rid" > "$D/active/$1.claim" - st=$(wait_terminal "$rid") - if [ "$st" = succeeded ]; then record "$1" succeeded "$rid"; say "done $1 (run $rid)"; return 0; fi - printf 'run_id=%s\nstatus=%s\n' "$rid" "$st" > "$D/active/$1.claim" - say "failed $1 (run $rid, $st)" - return 1 -} - -reconcile() { # settle anything the previous process left claimed - for a in "$D"/active/*; do - [ -e "$a" ] || continue - id=$(basename "$a"); id=${id%.brief}; id=${id%.claim} - rid=$(claim_field "$D/active/$id.claim" run_id) - if [ -z "$rid" ]; then - say "requeueing undispatched $id" - [ -e "$D/active/$id.brief" ] && mv "$D/active/$id.brief" "$D/queue/$id.brief" - rm -f "$D/active/$id.claim" - continue - fi - st=$(wait_terminal "$rid") - record "$id" "$st" "$rid" - say "reconciled $id as $st (run $rid)" - done -} - -reconcile - -while :; do - if [ -e "$D/PAUSED" ]; then sleep "$POLL"; continue; fi - next="" - for q in $(ls -1 "$D"/queue/*.brief 2>/dev/null | sort); do - id=$(basename "$q" .brief) - [ -e "$D/done/$id" ] && continue - [ -e "$D/failed/$id" ] && continue - iws=$(item_field "$q" workspace) - [ -n "$iws" ] && [ "$iws" != "$WS" ] && continue - next=$q; break - done - if [ -z "$next" ]; then say "idle (no claimable items for $WS)"; exit 0; fi - id=$(basename "$next" .brief) - mv "$next" "$D/active/$id.brief" 2>/dev/null || continue - ipipe=$(item_field "$D/active/$id.brief" pipeline) - [ -n "$ipipe" ] || ipipe=$PIPELINE - key="work-$SLUG-$id" - if run_item "$id" "$D/active/$id.brief" "$ipipe" "$key"; then continue; fi - if [ "$(claim_field "$D/active/$id.claim" status)" = "dispatch-failed" ]; then - say "STOP: could not dispatch $id" - mv "$D/active/$id.brief" "$D/failed/$id.brief" 2>/dev/null - exit 1 - fi - say "retrying $id once" - if run_item "$id" "$D/active/$id.brief" "$ipipe" "$key-r1"; then continue; fi - rid=$(claim_field "$D/active/$id.claim" run_id) - st=$(claim_field "$D/active/$id.claim" status) - printf 'run_id=%s\nstatus=%s\nattempts=2\n' "$rid" "${st:-failed}" > "$D/failed/$id" - mv "$D/active/$id.brief" "$D/failed/$id.brief" 2>/dev/null - rm -f "$D/active/$id.claim" - say "STOP: $id failed twice; recorded under failed/" - exit 1 -done -``` - -Launch it detached and keep its log: - - nohup sh /projects//drivers/workd.sh >/dev/null 2>&1 & - -The driver only *advances* a prepared queue; it never writes briefs. Preparing a good -queue and settling a failure is the agent's work. +**Landing stays the pipeline's job**: the committer stage merges and lands its own +work. There is no merger, no resolver and nothing to hand to a script. You verify the +landing gate (`references/queue.md`); you do not perform the merge. ## Sleeping and waking -- Before the operator sleeps: reconcile, make sure the queue is full enough to cover - the night, and confirm the driver is running. A drained queue is fine; it idles. -- Configure a **failure notification** (`references/notifications.md`) so a stopped - driver is not silent. On an overnight failure the driver stops after one retry; that - is by design — do not add retry loops to make it look busy. +- Before the operator sleeps: reconcile, make sure the queue has enough items to cover + the night, and launch what is ready with a background `pipelines wait`. +- Configure a **failure notification** (`references/notifications.md`) so a failed run + is not silent. A run that fails is recorded and the next item starts; do not add + retry loops to make it look busy. - On waking, reconcile first, then answer with the status block: what landed, what failed and why, what is next. @@ -262,7 +142,7 @@ The operator wants the result, not the shift report. two sentences; never answer a narrow question with a sectioned report. - Answer the question that was asked first. Put any extra fact that changes a decision in a single clause, not in a new section. -- Answer with the status block for status; do not narrate polling or re-print the +- Answer with the status block for status; do not narrate waiting or re-print the queue when nothing changed. - Report a run by its outcome and evidence (`references/verification.md`), not the steps you took to watch it. diff --git a/internal/tui/AGENTS.md b/internal/tui/AGENTS.md index 09886d5b..06f7275a 100644 --- a/internal/tui/AGENTS.md +++ b/internal/tui/AGENTS.md @@ -42,5 +42,5 @@ say why is a review finding. you checked. - Terminal width matters: verify at a narrow width as well as a wide one, and never assume colour support (`plain-nocolor-*` exists for a reason). -- Do not restart the deployed server, the lanes or ntfy to test a TUI change: run the binary against a +- Do not restart the deployed server or ntfy to test a TUI change: run the binary against a scratch server or use the fixtures.