Teach the pipelines skill to manage runs instead of implementing
Rewrite SKILL.md to state the stance (reconnoitre, then route all work through Pipelines runs; manage and debug, never implement), keep the existing authoring/operations index, and route to four new references. Add references/orchestration.md (stance, work items, brief template, hard constraints, pipeline choice, running and watching), references/lanes.md (worktrees, devices, sync/merge, restart-replay cursor fix), references/debugging.md (eight symptom/confirm/do playbooks) and references/delivery.md (artifacts, LAN URLs, WIP naming, install boundaries), each with a clearly labelled project example where Android/device specifics apply. Append orchestration verification habits to references/verification.md and add the four new files to install.go's managed inventory.
This commit is contained in:
@@ -0,0 +1,119 @@
|
||||
# Debugging wedged and failed runs
|
||||
|
||||
Debug from **retained evidence**, never from an agent's summary. Open the run with
|
||||
`CLI_PREFIX inspect RUN_ID --json` and read the exact stage outputs, outcomes,
|
||||
diagnostics, file changes and artifacts. Identify evidence-supported causes and say
|
||||
what remains unknown. Never fix a run's symptom by editing the project by hand: fix
|
||||
the pipeline definition or the Pipelines/harness service, or record the failure and
|
||||
continue.
|
||||
|
||||
Each playbook is *symptom → how to confirm → what to do*.
|
||||
|
||||
## HARNESS TRANSPORT errors
|
||||
|
||||
- **Symptom** A stage or run fails with the sanitized category `HARNESS TRANSPORT`;
|
||||
the model may have done work but no final result was read.
|
||||
- **Confirm** `inspect --json` shows the category on the failed stage. A request
|
||||
failure, an active-session poll failure and a final-readback failure all carry this
|
||||
one category.
|
||||
- **Do** Treat it as a transport failure, not a task failure. Distinguish a
|
||||
pre-admission failure (nothing executed) from a post-admission one (tools may have
|
||||
run). An explicit read-only stage may retry; a stage whose tools include `edit`/
|
||||
`shell`, or that has no tools block, **never replays admitted work**. Do not re-run
|
||||
a full agent stage merely to recover a response's formatting. If it recurs, check
|
||||
the service and harness health (`references/operations.md`); do not weaken the
|
||||
stage to get past it.
|
||||
|
||||
## Malformed model results
|
||||
|
||||
- **Symptom** The category `FINAL RESULT VALIDATION` names a duplicate nested field
|
||||
or malformed/ambiguous JSON with a byte offset.
|
||||
- **Confirm** Read the diagnostic and the retained final result: the stage's result
|
||||
does not match its `output` contract. A retained transcript can be a valid bounded
|
||||
preview envelope, not the complete history.
|
||||
- **Do** A malformed result on an explicit read-only stage may be re-emitted up to
|
||||
`retries` with its `format_repairs` evidence retained; a stage with `edit`/`shell`
|
||||
or no tools block is never replayed. Never accept prose, flattened output or a
|
||||
tool-call envelope as success. If one stage keeps producing invalid JSON, tighten
|
||||
its `output`/`prompt` in the **pipeline definition** (`references/format.md`) — a
|
||||
pipeline fix, not a project edit.
|
||||
|
||||
## A reviewer or visual reviewer that blocks, rejects repeatedly, or hangs
|
||||
|
||||
- **Symptom** The reviewer returns `rejected` on every visit, returns `blocked`, or
|
||||
the run stalls inside a reviewer stage.
|
||||
- **Confirm** Inspect every visit and read what the reviewer actually ran and saw. A
|
||||
rejection loop is bounded by the stage `limits` (`max_visits`); at the cap the
|
||||
rejection is retained and routes to `#failure`. `blocked` means the reviewer could
|
||||
not obtain the evidence it needed.
|
||||
- **Do**
|
||||
- A visual reviewer that returns `blocked` with no screenshots is right to refuse.
|
||||
Fix the capture path (below) rather than removing it.
|
||||
- Rejections that repeat the same finding mean either the implementer is not fixing
|
||||
it or the acceptance criteria contradict the brief. Read the retained findings,
|
||||
correct the brief or the pipeline's criteria, and re-run.
|
||||
- **Never** remove a review stage, relax an acceptance check, or approve the review
|
||||
yourself to get past it (`references/orchestration.md`).
|
||||
- A reviewer that hangs on an unbounded command (for example a filesystem-root
|
||||
scan) can leave a process in uninterruptible sleep and stall the server or
|
||||
Docker. Cancel the run, stop the offending command, and add a scope instruction
|
||||
to the brief or pipeline: bounded, relevant paths only.
|
||||
|
||||
## Workspace-guard refusals
|
||||
|
||||
- **Symptom** A run is refused with "run N is already active in workspace ...".
|
||||
- **Confirm** The message names the conflicting run and its workspace; the guard is
|
||||
working as designed — one active run per resolved workspace.
|
||||
- **Do** Wait for the conflicting run, or start this one in a different workspace
|
||||
(another worktree/checkout, which is what a lane is for — `references/lanes.md`).
|
||||
Do not delete the active run, the database or sessions to clear it. An **orphan**
|
||||
active session left by a restart with no owning active run still refuses new work
|
||||
until it is interrupted; reusing the active run's original request key reattaches
|
||||
idempotently.
|
||||
|
||||
## A client older than the server dropping fields
|
||||
|
||||
- **Symptom** `inspect --json` or `ping --json` is missing fields this skill
|
||||
describes, or a flag is rejected as unknown.
|
||||
- **Confirm** `ping --json` for the server build identity and `version --json` for the
|
||||
client. An older server omits `build`; an older client may not send newer fields.
|
||||
- **Do** Treat it as a version mismatch, not a run failure. State both versions and
|
||||
the mismatch. **Do not infer** that a source fix or an updated skill is deployed.
|
||||
Use only flags the installed help shows; do not restart or reinstall a service to
|
||||
"fix" it without the operator (`references/operations.md`).
|
||||
|
||||
## A server restart killing in-flight runs
|
||||
|
||||
- **Symptom** Runs that were active when the server restarted are now `interrupted`,
|
||||
with every step's `agent_active=false`.
|
||||
- **Confirm** The status is `interrupted`, not `failed`, and there is **no automatic
|
||||
replay**.
|
||||
- **Do** Recovery is explicit. Inspect what completed; retry only when the server is
|
||||
healthy and the previous attempt's cleanup has stopped. Never restart a server that
|
||||
has runs in flight — quiesce first (`references/lanes.md`). Keep the interrupted run
|
||||
as history; never describe a restart, retry or cancellation as rollback.
|
||||
|
||||
## Screenshot capture that silently produces non-images
|
||||
|
||||
- **Symptom** A run's "screenshot" artifact, or an image cited in an approval, is not
|
||||
an image at all — for example 100+ KB of text beginning with a
|
||||
`[Warning] Multiple displays were found` line — yet the capture command exited 0.
|
||||
- **Confirm** Check the **bytes** (magic bytes or the reported content type), not the
|
||||
`.png` name. A file that is not a real image is not evidence.
|
||||
- **Do** Capture on the device to a file and pull it — never write `exec-out` or a
|
||||
shell redirect straight to a `.png`; name the display explicitly when more than one
|
||||
exists; validate the pulled bytes before citing them. A capture that fails
|
||||
validation is deleted and the stage returns `blocked`; never cite it to support an
|
||||
approval. Put the capture helper in the repository so every UI run reuses one path.
|
||||
See the example in `references/delivery.md`.
|
||||
|
||||
## Duplicated work after a lane or chain restart
|
||||
|
||||
- **Symptom** Items already completed run again after a lane or chain script
|
||||
restarts: the same brief is re-invoked, side effects repeat, or a commit is made
|
||||
twice.
|
||||
- **Confirm** Compare the lane's list against its completed-items record. A list alone
|
||||
has no resume point.
|
||||
- **Do** Persist a cursor and resume from it (`references/lanes.md`). Record an item
|
||||
only once it has landed, and skip recorded items on startup. Never relaunch a
|
||||
duplicate of a run that is still active — reattach with its request key instead.
|
||||
Reference in New Issue
Block a user