web: exercise the human decision sheet live (R11)

Add an opt-in --decide smoke mode that reuses or submits a human-only pipeline, starts two runs in throwaway workspaces, and presses the real card, sheet and Confirm for the feedback and no-feedback shapes. Record R11 as done with the live captures.
This commit is contained in:
2026-10-19 16:16:05 +01:00
parent 0d3093ac27
commit 2deaaea05a
2 changed files with 450 additions and 12 deletions
+5 -5
View File
@@ -41,7 +41,7 @@ Status legend: `todo` · `partial` (some of it works) · `done` (working + captu
| R8 | Artifacts gallery + full-screen viewer (images inline, others download/open) | `GET /artifacts/…` | done (captured 2026-10-19: `captures/04-run-430-artifacts.png`, `captures/05-artifact-viewer.png`) |
| R9 | Session export link (read-only `session.html`) | `GET /artifacts/…` | done (captured 2026-10-19: `captures/06-session-export-link.png`) |
| R10 | Live run updates: subscribe/unsubscribe + event fold | `subscribe`, `unsubscribe`, `run_event` | done — one app-wide watch owned by the connection, never by a screen: becoming connected best-effort `unsubscribe`s the previous watch, sends `{type:'subscribe',workspace:''}` send-only (no `run_ids`; `{type:'unsubscribe'}` drops it), then takes one authoritative resync (`list_runs`, plus one `get_run` when a run detail is open); a socket drop clears the watch and no screen change ever unsubscribes. `src/runEvents.js` is the pure fold ported from `ui/RunEvents.kt` (in-place list update preserving order, insert-at-head from the frame's pipeline identity, `needsResync` without identity, detail merge with explicit `resource_wait` set/clear, one step matched by `sequence` and inserted in sequence order, the monotonic run/step ladders and the full-sync decision) and is unit-tested in `test/runEvents.test.mjs`; the send-only path (`src/client.js#sendOnly`), the exact watch frames (`src/repository.js`) and the `run_event`/unsolicited-frame dispatch are unit-tested in `test/client.test.mjs` and `test/repository.test.mjs`. The 1 s run-detail poll is removed — pushed frames drive the detail and a settled/terminal frame triggers one coalesced `get_run` (at most one in flight per kind plus one queued follow-up, retried at 500 ms × attempt at most 3 times before the badge drops to `reconnecting`); the only surviving intervals are render-only clocks. The badge is the explicit stream state (`live` / `reconnecting` / `not live`, LiveBadge.kt words), independent of whether a run is executing, on the Runs list and the run detail. Captured 2026-10-19 against the live server with a **real pushed run** started by the smoke in a throwaway workspace: `captures/75-run-live-list.png` (the new run's row inserted from its own frame, badge `live`), `captures/76-run-live-detail.png` (the open detail folded to `running` from pushes, badge `live`) and `captures/77-run-live-terminal.png` (the pushed terminal fold to `canceled`, no poll), plus the reachable badge states `captures/28-connected-banner.png` (live), `captures/29-disconnected-banner.png` (not live while idle; reconnecting while the socket retries) and `captures/30-reconnect-recovered.png` (live again). `test/ui-smoke.mjs` asserts the reconnect wire order (unsubscribe → subscribe blank-workspace → list_runs) and that at least one genuine `run_event` frame for the started run arrived. The R10 step itself passes on every run; the full smoke can still fail later in unrelated, pre-existing steps on the shared live server (observed after the main merge: a flaky L1 stale page-level error on one run, a C5 chat fold `expected at least one folded agent turn, got 0` on another, and a C8 composer assertion while the operator's live chat session was streaming, so the composer correctly showed Stop; the focused `--notify` run is `UI SMOKE PASS`), which is not an R10 defect. |
| R11 | Human decision sheet: prompt, options, feedback | `decide_run` | partial — the pure model (`src/actions.js`: `pendingDecision`, `lastWaitingStep`, `waitingPrompt`/`waitingOptions`, `humanOptions`, `decisionTargetLabel`, `DecisionDraft`) and the card + sheet are implemented with the Android copy verbatim: card title `Waiting for your decision`, meta `{stage} · run {id} · {runLabel}`, action `Decide`; sheet title `Decide {stage}`, meta `run {id} · {runLabel}`, the recorded instructions verbatim when non-blank, one selectable row per option in author order showing the option and `→ {target}` only when the declared edge resolves, the `Feedback (optional)` field, and `Cancel` / `Confirm decision` (reading `Sending…` while in flight). Selecting a row sends nothing and Confirm is disabled until a non-blank option is chosen; a refusal keeps the sheet open with the server's exact reason and re-reads the run. The request shape is unit-tested (`decide_run` = `{type, run_id, option, stage?, feedback?}`, stage/feedback omitted when blank, feedback trimmed). **No browser capture yet**: the live server had no `waiting` run when `ui-smoke.mjs` ran (only executing/failed/finished runs), so this row stays `partial`; `test/ui-smoke.mjs` now captures the card, the sheet and the selected state automatically (`captures/41-run-<id>-decision-card.png`, `captures/42-run-<id>-decision-sheet.png`, `captures/43-run-<id>-decision-selected.png`) the moment a waiting run exists |
| R11 | Human decision sheet: prompt, options, feedback | `decide_run` | done — the pure model (`src/actions.js`: `pendingDecision`, `lastWaitingStep`, `waitingPrompt`/`waitingOptions`, `humanOptions`, `decisionTargetLabel`, `DecisionDraft`) and the card + sheet are implemented with the Android copy verbatim: card title `Waiting for your decision`, meta `{stage} · run {id} · {runLabel}`, action `Decide`; sheet title `Decide {stage}`, meta `run {id} · {runLabel}`, the recorded instructions verbatim when non-blank, one selectable row per option in author order showing the option and `→ {target}` only when the declared edge resolves, the `Feedback (optional)` field, and `Cancel` / `Confirm decision` (reading `Sending…` while in flight). Selecting a row sends nothing and Confirm is disabled until a non-blank option is chosen; a refusal keeps the sheet open with the server's exact reason and re-reads the run. The request shape is unit-tested (`decide_run` = `{type, run_id, option, stage?, feedback?}`, stage/feedback omitted when blank, feedback trimmed). **Live, end to end (2026-10-19)**: `test/ui-smoke.mjs` gained an explicit `--decide` mode (the only mode allowed to submit). It reused the entry-stage human-only pipeline submitted create-only as `Web R11 decision 2026-10-19T14-59-18-785Z` (id 30: one `type=human` `gate` stage, `approve -> #success`, `hold -> #success`, `error -> #failure`, so no model stage runs) and started two runs in separate fresh temp workspaces. Run 513 parked in `waiting`; the browser asserted the card (`Waiting for your decision`, `gate · run 513 · Web R11 decision 1 (feedback)`, `Decide`), opened the sheet (`Decide gate`, `run 513 · …`, the recorded instructions `R11 live decision: review the run and choose approve or hold.`, `approve`/`hold` in author order each with `→ SUCCESS`, `Feedback (optional)`, Confirm disabled), selected `approve` (still no `decide_run`, and the run was still `waiting` via `pipelines inspect`), typed feedback, and pressed `Confirm decision`: exactly one frame `{"type":"decide_run","run_id":513,"option":"approve","stage":"gate","feedback":"Approved in the live browser decision sheet."}` was sent, the run left `waiting`, resumed and reached `succeeded`. Run 514 proved the no-feedback shape: selecting `hold` sent nothing, Confirm sent `{"type":"decide_run","run_id":514,"option":"hold","stage":"gate"}` with **no** `feedback` key, the run reached `succeeded`, its recorded feedback is blank and its verdict carries no reason. Captured: `captures/41-run-513-decision-card.png` (waiting card), `captures/42-run-513-decision-sheet.png` (open sheet with the options), `captures/43-run-513-decision-selected.png` (selected + feedback, Confirm enabled), `captures/44-run-513-decision-confirmed.png` (resumed detail, `succeeded`), `captures/45-run-513-decision-verdict.png` (`gate · visit 1` Verdict chip `approve → success` and the exact reason), and the no-feedback path `captures/46-run-514-decision-card.png`, `captures/47-run-514-decision-sheet.png`, `captures/48-run-514-decision-selected.png`, `captures/49-run-514-decision-confirmed.png`, `captures/50-run-514-decision-verdict.png` (`hold → success`, no reason). Command: `cd web && node test/ui-smoke.mjs --decide --out "$PWD/captures"` → `UI SMOKE PASS` (the reused pipeline means no new submit on the capture run); `cd web && node --test test/` stays green. Both runs are terminal, no R11 run remains active and both temp workspaces were removed. The submitted definition is intentionally retained and named here; no operator pipeline, version, run, chat or repository was modified |
| R12 | Cancel run | `cancel_run` | done — `src/actions.js` `canCancel` offers a `Cancel run` action only while the run is `executing`; the confirm dialog is the Android copy verbatim (title `Cancel run {id}?`, `This asks the server to stop the active run.`, confirm `Cancel run`, dismiss `Keep`), and the request is exactly `{type:'cancel_run',run_id}` followed by a silent `get_run` refresh (unit-tested in `test/repository.test.mjs`). Captured `captures/40-run-459-cancel-confirm.png` on the executing run present at capture time. **Shown, not executed**: the only candidate was another live pipeline's executing run, so the dialog was shown and dismissed with `Keep` — no `cancel_run` was sent (a real cancel is only safe against a run started in a scratch workspace, which this stage did not do) |
| R13 | Retry failed stage (resume) / restart from start | `retry_run` | done — `canRetry` is true only for a loaded, non-executing, `failed` run; the Retry confirm is the Android copy verbatim (title `Retry run {id}?`, `Only failed runs can be retried. The server starts a new run from the same input.`, confirm `Retry`, dismiss `Keep`) and the default send carries only `run_id` + a fresh `request_key` (**no** `retry_mode`, so the server's `resume` default applies exactly like the Android client's mode-less call; unit-tested). Captured `captures/35-run-440-run-actions.png` (failed run 440 offers Retry and no Cancel) and `captures/36-run-440-retry-confirm.png`. **Divergence from the Android UI, stated here**: Android exposes resume only while the protocol has both modes, so the browser adds a clearly-labelled secondary `Restart from start` confirm that states it `replays the whole pipeline from the start instead of resuming the failed stage` and sends `retry_mode:"restart"` (`captures/37-run-440-restart-confirm.png`). Both dialogs were shown and dismissed with `Keep`; no run was started |
| R14 | Rename a run | `set_run_title` | done — an inline field on the Summary title row (pre-filled with the current title, `Rename` / `Save` / `Cancel`) sending exactly `{type:'set_run_title',run_id,title}` and then refreshing the run and the runs-list row (unit-tested). Captured `captures/38-run-440-rename-field.png`. The one shared `Action failed` dialog was exercised by a real refused rename: `captures/39-run-440-action-failed.png` shows the live server's exact reason `a title with usable text is required` (a blank title mutates nothing). **Divergence from the Android UI, stated here**: Android does not surface `set_run_title` at all (its README records the deliberate omission), so this control is browser-only |
@@ -66,12 +66,12 @@ R11–R14 share one `Action failed` dialog (title `Action failed`, the server's
message, a single `OK`), matching `RunDetailScreen.kt`; every server string is set
with `textContent`, never `innerHTML`. Where a live action was unsafe this stage
says so rather than mutating shared state: the R12 cancel and the R13 retry/restart
sends were shown and dismissed with `Keep` (never executed), and R11 has no live
capture because the server had no waiting run. The request shapes for all four are
sends were shown and dismissed with `Keep` (never executed); R11's live Confirm
**was** executed end to end against a run it started in its own throwaway
workspace (see the row above). The request shapes for all four are
unit-tested in `test/repository.test.mjs` and the model in `test/actions.test.mjs`;
`test/ui-smoke.mjs` captures the rendered dialogs, the rename field, the refused
rename's exact server reason and — when a waiting run exists — the decision
card/sheet.
rename's exact server reason and, under `--decide`, the full live decision flow.
## Search
| # | Feature | Protocol | Status |
+445 -7
View File
@@ -14,8 +14,14 @@
// against a scratch server and skips the live suite, which needs the operator's
// real runs and chat session. The default run never sends a `submit`: the live
// server's pipeline list is never mutated.
//
// `--decide` drives ONLY R11's live human-decision flow: it reuses an existing
// entry-stage human pipeline or submits one uniquely named create-only
// definition, starts two throwaway runs in fresh temp workspaces, and presses
// the real card/sheet/Confirm decision for both the feedback and no-feedback
// paths. It is deliberately opt-in because it is the only mode that submits.
import { spawn } from "node:child_process";
import { mkdir, mkdtemp, rm } from "node:fs/promises";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import net from "node:net";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
@@ -74,6 +80,8 @@ const out = resolve(
const scratch = process.argv.includes("--scratch");
// Focused notification evidence only; skips the unrelated live-suite steps.
const notifyOnly = process.argv.includes("--notify");
// Focused R11 evidence only; the one explicit mode allowed to submit a pipeline.
const decideOnly = process.argv.includes("--decide");
// The page must connect through its own proxy (the server rejects a cross-origin
// upgrade), but serve.mjs only proxies /connect — not /artifacts. Point the URL
@@ -81,6 +89,7 @@ const notifyOnly = process.argv.includes("--notify");
// viewer resolve to real files; that is exactly the URL a correctly configured
// page would carry.
const artifactServerUrl = `ws://${pipelines}/connect`;
const pipelineUrl = artifactServerUrl;
const shots = [];
let serve = null;
@@ -94,6 +103,14 @@ let throwawaySessionId = null;
// workspace and the workspace itself, both cleaned up in `finally`.
let liveRunId = null;
let liveRunScratch = null;
// R11 evidence: the human-stage pipeline used (reused or uniquely submitted),
// every throwaway run started for the decision path, and their temp workspaces.
// Cleanup cancels only these runs while they are still active and removes only
// these directories; the submitted pipeline definition is intentionally retained.
let decidePipelineName = null;
let decidePipelineReused = false;
const decideRunIds = [];
const decideScratchDirs = [];
// C2's one real session. It is closed in its own step; kept here only so a
// failure before that step can still close it rather than leave it in the list.
let createdSessionId = null;
@@ -231,6 +248,387 @@ async function cancelLiveRun(id) {
}
}
// --- R11: live human-decision evidence -------------------------------------
// A human stage parks a run without starting a model, so it is safe to run on
// the live server in throwaway workspaces. This path is reached only with the
// explicit `--decide` flag: the default suite still never submits a pipeline.
const DECIDE_PROMPT = "R11 live decision: review the run and choose approve or hold.";
const DECIDE_OPTIONS = ["approve", "hold"];
function cliJson(result, what) {
if (result.code !== 0) {
throw new Error(`${what} exited ${result.code}: ${result.stderr.trim().slice(-400)}`);
}
try {
return JSON.parse(result.stdout);
} catch {
throw new Error(`${what} returned no JSON: ${result.stdout.slice(0, 300)}`);
}
}
async function inspectRun(id) {
return cliJson(
await runCli(["inspect", String(id), "--json", "--url", pipelineUrl], tmpdir()),
`pipelines inspect ${id}`,
);
}
async function waitForRun(id, predicate, what, timeout = 60000) {
const deadline = Date.now() + timeout;
let detail = null;
while (Date.now() < deadline) {
detail = await inspectRun(id);
if (predicate(detail)) return detail;
await new Promise((resolvePromise) => setTimeout(resolvePromise, 250));
}
throw new Error(`run ${id} did not ${what}; last status ${detail?.status ?? "unknown"}`);
}
/**
* The one pipeline R11 may use without submitting: an existing definition whose
* entry stage is already `type=human`, so starting it runs no model stage. A
* human stage later in a model pipeline is deliberately not reused.
*/
async function findEntryHumanPipeline() {
const list = cliJson(
await runCli(["list", "--json", "--url", pipelineUrl], tmpdir()),
"pipelines list",
);
for (const item of list.pipelines ?? []) {
const shown = await runCli(["show", String(item.id), "--json", "--url", pipelineUrl], tmpdir());
if (shown.code !== 0) continue;
let detail = null;
try {
detail = JSON.parse(shown.stdout);
} catch {
continue;
}
if (detail?.graph?.stages?.[0]?.type === "human") {
return { id: Number(item.id), name: String(detail.name ?? item.name) };
}
}
return null;
}
/** The tiny create-only definition: one human stage, both options succeed. */
function humanOnlySource() {
return [
"# stage gate",
"",
"```output",
"Decision evidence.",
"```",
"",
"```prompt",
DECIDE_PROMPT,
"```",
"",
"```edges",
"approve -> #success",
"hold -> #success",
"error -> #failure",
"```",
"",
"```configs",
"type=human",
"```",
"",
].join("\n");
}
/**
* Use an existing entry-human pipeline, or submit exactly one uniquely named
* create-only definition. Then start two runs in separate fresh workspaces (the
* server permits one run per workspace) and wait for both to park in `waiting`.
* Every run id and workspace is tracked for the `finally` cleanup.
*/
async function prepareHumanDecisionRuns() {
const existing = await findEntryHumanPipeline();
if (existing) {
decidePipelineName = existing.name;
decidePipelineReused = true;
console.log(` reusing entry-human pipeline ${existing.name} (id ${existing.id}); no submit`);
} else {
const definitionDir = await mkdtemp(join(tmpdir(), "pipelines-web-r11-def-"));
decideScratchDirs.push(definitionDir);
const definition = join(definitionDir, "human.md");
await writeFile(definition, humanOnlySource(), "utf8");
const stamp = new Date().toISOString().replace(/[:.]/g, "-");
const requested = `Web R11 decision ${stamp}`;
const submitted = cliJson(
await runCli(
[
"submit",
definition,
"--name",
requested,
"--create-only",
"--yes",
"--json",
"--url",
pipelineUrl,
],
definitionDir,
),
"pipelines submit",
);
decidePipelineName = String(submitted.name ?? requested).trim() || requested;
console.log(` submitted create-only pipeline ${JSON.stringify(decidePipelineName)}`);
}
const prepared = [];
for (const [index, suffix] of [
[1, "feedback"],
[2, "no-feedback"],
]) {
const workspace = await mkdtemp(join(tmpdir(), "pipelines-web-r11-"));
decideScratchDirs.push(workspace);
const title = `Web R11 decision ${index} (${suffix})`;
const started = cliJson(
await runCli(
[
"run",
decidePipelineName,
"R11 web decision smoke",
"--workspace",
workspace,
"--title",
title,
"--json",
"--yes",
"--url",
pipelineUrl,
],
workspace,
),
`pipelines run ${decidePipelineName}`,
);
const id = Number(started.run_detail?.id);
if (!(id > 0)) throw new Error(`pipelines run returned no run id: ${started ? JSON.stringify(started).slice(0, 200) : "no output"}`);
decideRunIds.push(id);
prepared.push({ id, title, suffix, workspace });
}
for (const run of prepared) {
const detail = await waitForRun(
run.id,
(d) => d.status === "waiting" && d.current_stage === "gate",
"reach waiting on the gate stage",
);
const waitingStep = (detail.steps ?? []).find((step_) => step_.status === "waiting");
const options = waitingStep?.output?.options ?? [];
if (JSON.stringify(options) !== JSON.stringify(DECIDE_OPTIONS)) {
throw new Error(`run ${run.id} waiting options: ${JSON.stringify(options)}`);
}
if (waitingStep?.output?.prompt !== DECIDE_PROMPT) {
throw new Error(`run ${run.id} waiting instructions: ${JSON.stringify(waitingStep?.output?.prompt)}`);
}
console.log(` run ${run.id} waiting on gate in ${run.workspace} (options ${options.join(", ")})`);
}
return prepared;
}
/**
* One complete live decision: card, sheet, deliberate selection, one Confirm,
* the resumed run and the recorded verdict. Feedback is optional; the same flow
* proves both wire shapes.
*/
async function exerciseDecisionRun(run, { option, feedback, labels }) {
const { id, title } = run;
const expectedMeta = `gate · run ${id} · ${title}`;
console.log(`• R11: ${labels.label} (run ${id})`);
await ensureConnected();
await eventually(
() => page.evaluate(`!!document.querySelector('.list.runs .run-row-meta[data-run-id="${id}"]')`),
{ timeout: 30000, what: `run ${id}'s waiting row to appear` },
);
await openRunDetail(id);
await page.waitForSelector(".run-detail .decision-card", { timeout: 20000 });
const card = await page.evaluate(`(() => ({
title: document.querySelector(".decision-card .decision-card-title")?.textContent ?? "",
meta: document.querySelector(".decision-card .decision-card-meta")?.textContent ?? "",
action: document.querySelector(".decision-card .decision-open")?.textContent ?? "",
}))()`);
if (card.title !== "Waiting for your decision") throw new Error(`card title: ${card.title}`);
if (card.meta !== expectedMeta) throw new Error(`card meta: ${card.meta}`);
if (card.action !== "Decide") throw new Error(`card action: ${card.action}`);
console.log(` card: ${card.title} — ${card.meta}`);
shots.push(await page.screenshot(`${out}/${labels.card}`));
await page.click(".decision-card .decision-open");
await page.waitForSelector(".decision-sheet .decision-option", { timeout: 20000 });
const sheet = await page.evaluate(`(() => ({
title: document.querySelector(".decision-sheet-title")?.textContent ?? "",
meta: document.querySelector(".decision-sheet-meta")?.textContent ?? "",
instructions: document.querySelector(".decision-instructions")?.textContent ?? "",
options: [...document.querySelectorAll(".decision-sheet .decision-option")].map((n) => ({
key: n.querySelector(".decision-option-key")?.textContent ?? "",
target: n.querySelector(".decision-option-target")?.textContent ?? "",
})),
feedbackLabel: document.querySelector(".decision-field-label")?.textContent ?? "",
confirm: document.querySelector(".decision-confirm")?.textContent ?? "",
confirmDisabled: document.querySelector(".decision-confirm")?.disabled ?? null,
}))()`);
if (sheet.title !== "Decide gate") throw new Error(`sheet title: ${sheet.title}`);
if (sheet.meta !== `run ${id} · ${title}`) throw new Error(`sheet meta: ${sheet.meta}`);
if (sheet.instructions !== DECIDE_PROMPT) {
throw new Error(`sheet instructions: ${JSON.stringify(sheet.instructions)}`);
}
const optionKeys = sheet.options.map((row) => row.key);
if (JSON.stringify(optionKeys) !== JSON.stringify(DECIDE_OPTIONS)) {
throw new Error(`sheet options: ${JSON.stringify(optionKeys)}`);
}
if (sheet.options.some((row) => row.target !== "\u2192 SUCCESS")) {
throw new Error(`sheet targets: ${JSON.stringify(sheet.options)}`);
}
if (sheet.feedbackLabel !== "Feedback (optional)") {
throw new Error(`sheet feedback label: ${sheet.feedbackLabel}`);
}
if (sheet.confirm !== "Confirm decision") throw new Error(`sheet confirm: ${sheet.confirm}`);
if (sheet.confirmDisabled !== true) throw new Error("Confirm decision must start disabled");
console.log(
` sheet: ${sheet.title} — ${sheet.meta}; ${sheet.options
.map((row) => `${row.key} ${row.target}`)
.join(", ")}; ${sheet.feedbackLabel}; Confirm disabled`,
);
shots.push(await page.screenshot(`${out}/${labels.sheet}`));
await page.click(`.decision-sheet .decision-option[data-option="${option}"]`);
await eventually(() => page.evaluate(`document.querySelector(".decision-confirm")?.disabled === false`), {
timeout: 5000,
what: "Confirm to enable after selecting an option",
});
const selected = await page.evaluate(`(() => ({
checked: document.querySelector('.decision-sheet .decision-option[data-option="${option}"]')?.getAttribute("aria-checked") ?? "",
decideBodies: (window.__decideBodies ?? []).filter((body) => Number(body.run_id) === ${id}),
}))()`);
if (selected.checked !== "true") throw new Error(`option ${option} did not render selected`);
if (selected.decideBodies.length !== 0) {
throw new Error(`selecting an option sent decide_run: ${JSON.stringify(selected.decideBodies)}`);
}
const stillWaiting = await inspectRun(id);
if (stillWaiting.status !== "waiting") {
throw new Error(`run ${id} left waiting on selection: ${stillWaiting.status}`);
}
console.log(" selecting sent no decide_run; the run is still waiting");
if (feedback) await page.fill(".decision-sheet .decision-feedback", feedback);
shots.push(await page.screenshot(`${out}/${labels.selected}`));
await page.click(".decision-sheet .decision-confirm");
await eventually(
() =>
page.evaluate(
`!document.querySelector(".decision-sheet") && !document.querySelector(".decision-card")`,
),
{ timeout: 20000, what: "the sheet and card to close after Confirm" },
);
const final = await waitForRun(
id,
(d) => d.status === "succeeded",
"succeed after the decision",
);
const bodies = await page.evaluate(
`(window.__decideBodies ?? []).filter((body) => Number(body.run_id) === ${id})`,
);
if (bodies.length !== 1) throw new Error(`expected exactly one decide_run, got ${JSON.stringify(bodies)}`);
const body = bodies[0];
if (body.type !== "decide_run" || Number(body.run_id) !== id || body.stage !== "gate" || body.option !== option) {
throw new Error(`decide_run body: ${JSON.stringify(body)}`);
}
if (feedback) {
if (body.feedback !== feedback) throw new Error(`decide_run feedback: ${JSON.stringify(body.feedback)}`);
} else if (Object.hasOwn(body, "feedback")) {
throw new Error(`no-feedback decide_run carried feedback: ${JSON.stringify(body)}`);
}
const gate = (final.steps ?? [])[0] ?? {};
if (gate.outcome !== option || gate.output?.option !== option) {
throw new Error(`recorded decision: ${JSON.stringify(gate.output)} outcome ${gate.outcome}`);
}
const recordedFeedback = String(gate.output?.feedback ?? "").trim();
if (recordedFeedback !== feedback) {
throw new Error(`recorded feedback: ${JSON.stringify(gate.output?.feedback)}`);
}
const expectedVerdict = `${option} \u2192 success`;
if (gate.verdict?.option !== expectedVerdict) {
throw new Error(`recorded verdict option: ${JSON.stringify(gate.verdict?.option)}`);
}
if (String(gate.verdict?.reason ?? "") !== feedback) {
throw new Error(`recorded verdict reason: ${JSON.stringify(gate.verdict?.reason)}`);
}
console.log(
` one decide_run ${JSON.stringify(body)}; run succeeded with ${expectedVerdict}` +
(feedback ? ` and the exact reason` : " and no reason"),
);
await eventually(
() =>
page.evaluate(
`["succeeded","Succeeded"].includes(document.querySelector(".run-detail .run-chip-row .chip")?.textContent ?? "")`,
),
{ timeout: 20000, what: `run ${id}'s detail to show succeeded` },
);
shots.push(await page.screenshot(`${out}/${labels.confirmed}`));
await page.clickText(".detail .detail-tabs .detail-tab", "Timeline");
await page.waitForSelector(`.detail .timeline-row[data-stage-key="gate"]`, { timeout: 20000 });
await page.click(`.detail .timeline-row[data-stage-key="gate"]`);
await page.waitForSelector(".stage-pane .verdict", { timeout: 20000 });
const verdict = await page.evaluate(`(() => ({
title: document.querySelector(".stage-pane .verdict-title")?.textContent ?? "",
option: document.querySelector(".stage-pane .verdict .outcome-chip")?.textContent ?? "",
reason: document.querySelector(".stage-pane .verdict-reason")?.textContent ?? "",
}))()`);
if (verdict.title !== "Verdict") throw new Error(`verdict heading: ${verdict.title}`);
if (verdict.option !== expectedVerdict) throw new Error(`verdict chip: ${verdict.option}`);
if (verdict.reason !== feedback) throw new Error(`verdict reason: ${JSON.stringify(verdict.reason)}`);
console.log(` verdict: ${verdict.option}${feedback ? ` — ${verdict.reason}` : " — no reason"}`);
shots.push(await page.screenshot(`${out}/${labels.verdict}`));
await page.evaluate(
`document.dispatchEvent(new KeyboardEvent("keydown", { key: "Escape", bubbles: true }))`,
);
await eventually(() => page.evaluate(`!document.querySelector(".stage-pane")`), {
timeout: 5000,
what: "the stage pane to close",
});
}
/** The focused R11 suite: two real waiting runs, both decided in the browser. */
async function runDecisionChecks() {
const prepared = await prepareHumanDecisionRuns();
await ensureConnected();
await page.clickText("nav.tabs button.tab", "Runs");
await page.waitForSelector(".list.runs .run-row", { timeout: 25000 });
await exerciseDecisionRun(prepared[0], {
option: "approve",
feedback: "Approved in the live browser decision sheet.",
labels: {
label: "decision with feedback",
card: `41-run-${prepared[0].id}-decision-card.png`,
sheet: `42-run-${prepared[0].id}-decision-sheet.png`,
selected: `43-run-${prepared[0].id}-decision-selected.png`,
confirmed: `44-run-${prepared[0].id}-decision-confirmed.png`,
verdict: `45-run-${prepared[0].id}-decision-verdict.png`,
},
});
await exerciseDecisionRun(prepared[1], {
option: "hold",
feedback: "",
labels: {
label: "decision without feedback",
card: `46-run-${prepared[1].id}-decision-card.png`,
sheet: `47-run-${prepared[1].id}-decision-sheet.png`,
selected: `48-run-${prepared[1].id}-decision-selected.png`,
confirmed: `49-run-${prepared[1].id}-decision-confirmed.png`,
verdict: `50-run-${prepared[1].id}-decision-verdict.png`,
},
});
console.log(
` pipeline ${JSON.stringify(decidePipelineName)} (${decidePipelineReused ? "reused" : "submitted create-only"}), runs ${decideRunIds.join(", ")} both succeeded`,
);
}
// Open one chat row's Chat actions menu (the row's management items live behind
// it since C13: Rename… / Close chat… or Reopen / Hide on this device).
async function openChatRowMenu(sessionId) {
@@ -719,6 +1117,10 @@ try {
// ack, so the card's refusal path (reopen with the reason) is exercised.
window.__failChatReplyIds = [];
window.__chatReplyCounts = { suppressed: 0, forwarded: 0, failed: 0 };
// R11: the exact decide_run bodies the page actually sends, retained (not
// suppressed) so the smoke can prove a selection sends nothing and Confirm
// sends exactly one body with the chosen option and optional feedback.
window.__decideBodies = [];
function Shim(url, protocols) {
const ws = protocols === undefined ? new Native(url) : new Native(url, protocols);
const nativeSend = ws.send.bind(ws);
@@ -779,6 +1181,8 @@ try {
frame.chat_prompt?.text ?? "",
);
}
// R11: retain the real decision frames without altering them.
if (frame.type === "decide_run") window.__decideBodies.push(frame);
// C2: the create bodies this page actually sent, so the smoke can prove
// the picker's chosen value reached chat_create.
if (frame.type === "chat_create") {
@@ -852,7 +1256,11 @@ try {
for (const shot of shots) console.log(` ${shot}`);
}
if (notifyOnly) {
if (decideOnly) {
await runDecisionChecks();
console.log("\nUI SMOKE PASS");
for (const shot of shots) console.log(` ${shot}`);
} else if (notifyOnly) {
await runNotificationChecks();
console.log("\nUI SMOKE PASS");
for (const shot of shots) console.log(` ${shot}`);
@@ -2814,10 +3222,11 @@ try {
}
}
// R11: the decision card/sheet, only when the live server actually has a run
// waiting on a person with recorded options. There is normally none (a human
// stage blocks a run until a person answers), so this stays a best-effort
// capture rather than a failure; the model is unit-tested in actions.test.mjs.
// R11: the decision card/sheet, only when the live server already has a run
// waiting on a person with recorded options. This default suite never submits
// and never confirms: it captures the waiting card, the sheet and the selected
// state and then dismisses with Cancel. The full live Confirm (both feedback
// paths) is exercised by the opt-in `--decide` mode above.
step("R11: the human-decision card/sheet (only when a waiting run is present)");
const waitingRunId = await page.evaluate(`(() => {
const row = [...document.querySelectorAll(".list.runs .run-row")].find((n) => n.querySelector(".chip-waiting"));
@@ -2825,7 +3234,7 @@ try {
return meta ? Number(meta.getAttribute("data-run-id")) : null;
})()`);
if (waitingRunId == null) {
console.log(" -- no waiting run available; skipped the decision card/sheet capture (R11 stays partial)");
console.log(" -- no waiting run available; skipped the decision card/sheet capture (full live path covered by --decide)");
} else {
await openRunDetail(waitingRunId);
await page.waitForSelector(".run-detail .decision-card", { timeout: 15000 });
@@ -6063,6 +6472,35 @@ try {
// leave the scratch directory rather than failing cleanup
}
}
// Best-effort cleanup of the R11 decision runs: cancel only these tracked runs
// while they are still active (never another run), then remove only the temp
// workspaces/definition directories this smoke created. The uniquely submitted
// human-only pipeline definition is intentionally retained and named in PARITY.
for (const id of decideRunIds) {
try {
const detail = await inspectRun(id);
if (["queued", "running", "waiting"].includes(detail.status)) {
const result = await runCli(
["cancel", String(id), "--yes", "--json", "--url", pipelineUrl],
tmpdir(),
);
console.log(
result.code === 0
? ` cleanup: canceled the R11 decision run ${id} (${detail.status})`
: ` cleanup: cancel ${id} exited ${result.code}`,
);
}
} catch {
// the server/CLI may be gone; the failure above is the real signal
}
}
for (const dir of decideScratchDirs) {
try {
await rm(dir, { recursive: true, force: true });
} catch {
// leave the scratch directory rather than failing cleanup
}
}
// Best-effort cleanup of the throwaway session this run created: if the run
// failed before its `/close` step, close it now so it is never left open. It
// closes only the throwaway: if the open conversation is something else (the