Files
pipelines/docs/app-integration-verification.md
alan db313336b2 docs: record the four end-to-end verification gap checks (run 340)
Add a dated follow-up section to docs/app-integration-verification.md that
closes the four claims the first pass left unproven, each with the command or
capture that proved it against the deployed server (0d1ddc7) on the Pixel 9 Pro
XL 47241FDAS00DLK:

- Chat reply arrives: a throwaway session showed the assistant reply `Done.`
  rendering live, and the close-a-chat flow left the session `closed=true`
  server-side.
- Live pushed run events: disproven on this build. Runs 341/342/343 never
  appeared or advanced without a manual `Refresh runs`; only a local elapsed
  timer ticked. Consistent with the run-event push-hub defect fixed on main by
  cff6f4d, absent from the deployed 0d1ddc7. Recorded as a defect with exact
  reproduction, not an app defect.
- Disconnected state: the closed-port state and the restore to
  ws://10.0.0.75:8080/connect were both captured and Settings verified after.
- Wide windows: the covered-pane region is a constant 553 px at depths 1-3 and
  at 2000x2244 and 2076x2152; `wm size reset` left no override.

Checked for staleness from this change: it touches no source or resource file,
so no packaged template or packaged authoring skill reference
(internal/author/skill/references/) depends on it - none references this
document and none needs updating.

Report-only: the untracked scratch directory .gaps-tmp/ is deliberately left
uncommitted.
2026-10-18 02:46:46 +01:00

16 KiB
Raw Permalink Blame History

App integration verification — real server, on the phone (2026-10-18)

Verification only: nothing was fixed, and no app, server, installer or Docker file was edited. This change adds this document alone.

Revision under test

Piece Revision How established
Server v0.0.0-20261018001257-0d1ddc73a444 (commit 0d1ddc73a444fa24110ba5223646856c8f053d29) pipelines ping against ws://127.0.0.1:8080/connect
Phone app 0d1ddc7 deploy-app.sh main rebuilt main, installed it, and matched installed md5 722f6a139a8aea6d15b09c89360dbea0 against the built APK

The phone ended on main's build (0d1ddc7). Endpoint under test: ws://10.0.0.75:8080/connect, workspace /Users/alan/repos/pipelines (shown in Settings, unchanged throughout — no stub, no deviation).

Device: operator's Pixel 9 Pro XL, adb serial 47241FDAS00DLK, window 1008x2244. Captures: adb shell screencap -p /sdcard/shot.png then adb pull; UI text read back with uiautomator dump. Screen text quoted below is what the dump actually returned, so it is evidence of rendered data, not just a screen appearing.

Capture directory (all rows below): /Users/alan/.local/share/pipelines/data/artifacts/332/1/

Flows

Flow Result Evidence Notes
Pipelines list Pass 01-pipelines-list.png, 01-ui.xml Real pipelines and last runs: Android id 20 v1 4 stages, running, run 332 · 4m 56s; Complete a Task id 1 v1 1 stage succeeded run 112; Docker harness verification id 5 v2 "no runs recorded". Matches pipelines list / pipelines runs.
Pipelines filters Pass 02-pipelines-filter.png, 03-*, 04-pipelines-filter-canceled.png Filter sheet offers Name, Runs (Any/Has runs/No runs) and Last run status (queued, running, waiting, succeeded, failed, canceled, interrupted). Selecting canceled gave header Filter pipelines, 1 filter active and exactly two rows: Focused tweak - implement review commit (id 11, run 318, canceled 3h ago) and Quick tweak - implement and commit (id 10, run 219, canceled 1d ago). Correct.
Pipelines sort Pass 05-pipelines-sort-moststages.png, 05-sort-sheet-ui.xml Sort sheet: Server order, Name A–Z, Name Z–A, Most recent run, Fewest stages, Most stages, Reset. "Most stages" reordered the list (4-stage pipelines first); "Server order" restored it.
Pipeline detail + tabs Pass 06-pipeline-detail.png, 07-pipeline-graph.png, 08-pipeline-source.png Android detail: content hash 219b8915…4312, latest version v1, created/updated 17 Oct 22:57 — matches the server. Tabs Summary / Graph / Source: Graph draws 4 stages (implementer, reviewer, ui_reviewer, committer) with labelled edges and Terminal Success/Failure nodes; Source shows Raw + the four collapsible stage sections.
Runs list Pass 09-runs-list.png, 09-runs-list-ui.xml Real aggregate header: 246 Total, average 18m 44s, 135 Succeeded, 99 Failed, 11 Canceled, 0 Interrupted, 1 Active, with a live marker. Rows carry title, pipeline, version, status and visit counts: run 332 running "stage implementer"; run 331 canceled (context canceled); run 330 succeeded, 4 completed visits. Matches pipelines runs.
Run detail Pass 10-run-detail.png, 10-run-detail-ui.xml Run 330: succeeded, outcome success, 4 recorded visits · 4 completed visits, Pipeline id 22 v1, Stage committer, workspace path, content hash, Started 18 Oct 00:56, Finished 18 Oct 01:13, Duration 17m 9s. Tabs Summary / Graph / Input / Output.
Run detail graph Pass 11-run-graph.png Executed vs not-executed stages rendered with the legend.
Stage sheet with visits Partial 12-stage-sheet.png Tapping a stage node in the run graph did not open a stage sheet (screen unchanged). The visits are reachable from the run header line, not reproduced here — see "Not verified".
Timeline Not verified — Not reached; the run detail tabs observed are Summary/Graph/Input/Output.
Artifacts gallery Pass 13-artifacts-gallery.png, 13-artifacts-gallery-ui.xml Run 330 Artifacts · 6 recorded, expanded, grouped by stage/visit with real names and sizes: implementer visit 1 wait-live-demo.log 3.8 KiB; reviewer visit 1 scratch-server.log 12.7 KiB, wait-batch-streaming.log 98 B, wait-blocking.log 55 B; committer visit 1 session.html 89.1 KiB, session.md 81.7 KiB.
Full-screen viewer / next-prev Partial 14-artifact-viewer.png Tapping the artifact row opened the artifact in an external Chrome Web View (http://10.0.0.75:8080/artifacts/330/2/wait-blocking.log) rather than an in-app viewer; next/previous was therefore not exercised.
Omni search Not verified — The pill's search entry could not be opened reliably from the run detail by coordinate or dump-matched tap within the time budget; no query was run.
Settings Pass 16-settings.png, 16-settings-ui.xml Connection ws://10.0.0.75:8080/connect, Connected, Server build v0.0.0-20261018001257-0d1ddc73a444 (0d1ddc73a444…), server time 2026-10-18T00:24:47Z, workspace /Users/alan/repos/pipelines with "Saved workspace" confirmation.
Templates gallery Pass 17-templates-gallery.png, 17-templates-gallery-ui.xml Present in this revision, reached from Settings → Browse templates: Complete a task (2 stages), Plan, implement, review (4 stages), Review a repository (1 stage), Research a topic (2 stages), Fold documentation notes (2 stages).
Chat Partial 18-chat.png, 19-chat-new.png Server does broker chat in this revision: the list shows real sessions ("Pipeline audited agent stage", "Checking progress on Android pipeline application", "Pipelines — continue the work"), each Closed / Idle with its workspace; opening one renders a live transcript (reasoning + tool-call chips) and a composer ("Message the agent… or / for commands"). No prompt was sent, so a reply arriving is not confirmed — see below.
Live run updates Not verified — No run other than this verification run (332) was running on the server during the window, and a run may not be started from the app. See below.

Not verified, and why

  • Stage sheet visits and the timeline. The stage node tap in the run graph did not open a stage sheet, and the run-detail tabs observed were Summary/Graph/Input/Output only; the time budget ran out before locating the visits/timeline entry points. The visits are visible as a per-stage grouping inside the artifacts section (see gallery row).
  • Artifact full-screen viewer, next/previous. The artifact row opened the file in an external browser, so the in-app viewer and its next/previous controls were not exercised.
  • Omni search (one query plus every filter and sort option). Not reached.
  • Chat reply. The server brokers chat and transcripts render from real session data, but no prompt was sent, so "a reply arrives" is unproven. Sending would have injected a turn into the operator's own live OpenCode session; with the time budget exhausted I did not risk it. This is a gap in coverage, not a defect.
  • Live pushed run events. There was no genuinely running run to watch other than run 332 itself (pipelines runs showed only 332 running; every other run succeeded/canceled/failed), and this brief forbids starting a run from the app. Observing it would require a concurrent run on the server — for example a lane whose pipeline is executing while the app sits on the Runs list.
  • Disconnected-state capture. Not performed. The sequence (point at a closed local port, capture, then restore ws://10.0.0.75:8080/connect, force-stop, relaunch, confirm reconnect) was cut for safety: overrunning the stage budget mid-sequence risks being killed with the phone left pointed at a dead port. The endpoint is therefore not deviated from — the app is on the real endpoint as captured in Settings.
  • Wide-window checks (wm size 2000x2244 and the fold shape 2076x2152). Not run; every capture here is the phone's own 1008x2244 window. wm size was left at its default (no override set), so nothing needs resetting.

Nothing failed in a way that points at an app defect: every flow that was reached rendered the server's real data correctly. The unverified items are coverage gaps from the time budget, not observed failures.

Device lock

device.lock was already present and stale: it was created at 01:14 by run 331 (the same brief), which the server reports as canceled; no build, capture or adb process was live and the Pixel's last capture predated my run. I removed and recreated the lock at 01:23 to take it, and released it (rmdir) when the captures were done. No other lane's work was running on the device.

Follow-up: the four coverage gaps (2026-10-18, run 340)

This section closes the four items the first pass left open. Same server (0d1ddc7, confirmed with pipelines ping), same endpoint ws://10.0.0.75:8080/connect with workspace /Users/alan/repos/pipelines (no stub was used and none was left behind), same device (Pixel 9 Pro XL 47241FDAS00DLK, 360 dpi, window 1008x2244). The phone app was not reinstalled, so it is the same main build the first pass ended on. Capture directory for every row below: /Users/alan/.local/share/pipelines/data/artifacts/340/1/.

1. A chat reply arrives — verified

A throwaway session was created in the app (New chat → Home (~)); never an operator or run session. Server-side it is ses_eb3671fa1ffeX7BdCHKfvnE9Kk (/Users/alan, created 2026-10-18T01:20:49Z). The prompt reply with one short line was typed and sent; the composer cleared, the user turn rendered, and then the assistant reply Done. rendered in the transcript with no refresh, no re-navigation and no second send. Evidence: 03b-typed.png (prompt in the composer), 04a-reply.png (user turn + Done. in the transcript); server confirmation in evidence-chat-messages.json — chat_messages returns the user turn reply with one short line and the assistant turn Done.

Close-a-chat: Chat actions → Close chat… → Close chat (06-menu.png, 07-close-dialog.png). The dialog states the chat is marked closed on every device and kept, not deleted. After it the server reports the session closed=true, closed_at=1792286575309 (2026-10-18T01:22:55Z) — evidence-chat-sessions.json — so the close is real server-side, not a local hide, and reopening the row renders the same answered turn from history (60-closed-chat-transcript.png).

One deviation from the brief's wording, reported as a decision rather than a failure: the closed chat does not disappear from the list. It leaves the active list and moves under a Closed header carrying a Closed chip (08-closed-list.png); the app deliberately keeps closed chats as history. The active section no longer contains it and the closed state is server-side, which is the substance the brief asked for.

2. Live pushed run events — disproven on this build (defect)

A real 2-stage run was started from the shell and watched on the phone. The app's run views do not update from pushed frames on the deployed server; every update needed a manual Refresh runs.

Reproduction (runs 341/342/343, all created from the shell after 41b-runs-before-new.png):

  • A new run does not appear in the Runs list. Run 342 started 01:29:13Z; at 01:29:24Z the list still lacked it (41c-list-no-refresh.xml); one Refresh runs tap added it (41d-list-after-refresh.xml). The same held for 341 and 343.
  • A run's view does not advance. Run 343 (Research a topic) was opened in its run view at 01:30:34Z while stage research was running (42-run-view-t0, 51-runview-t0-research-running). The server recorded it succeeded, stage report, 2 completed visits, finished 01:30:41Z (pipelines inspect 343). The app kept showing running · stage research · 0 completed visits · 1 recorded visit for the whole ~5-minute watch (52-runview-latest, 53-runview-still, taken 01:35:56Z) — no stage advance, no terminal transition, no interaction.
  • The Runs list row was stale the same way. At 01:36:13Z run 343's row read running · 0 completed visits · stage research · run 343 · 6m 13s (54-list-before-refresh.png); one Refresh runs flipped it to succeeded · 2 completed visits · stage report · run 343 · 18 Oct 02:30 · 20s (55-list-after-refresh.png).
  • The only "live" value is the elapsed clock, and it is a local timer, not a pushed update: run 340 counted 14m 41s → 15m 56s → 23m 42s with no interaction, while run 343's clock reached 6m 13s although the server had finished it at 20s.

Cross-checks that rule out a dead socket: chat streamed a reply live at 01:21 in the same app session (item 1), and Settings read Connected (32-settings-endpoint-restored). The server log shows the app's single subscribe (android-5, 01:26:47Z), then only list_runs on each manual refresh and get_run on each open — no further subscription traffic. This is consistent with the known run-event push-hub defect the in-flight run 339 fixes ("an oversized subscribe snapshot overflow"), which is not in the deployed 0d1ddc7 build; it is not an app defect, and it is exactly what the brief's item 2 tested for. Not verified, and disproven as stated: run state does not arrive by push on this deployment; a manual refresh is required.

3. Disconnected state — verified

One command sequence deviated and restored (.gaps-tmp/disconnected.py, run 340's scratch): set the URL to the closed local port ws://127.0.0.1:9/connect, tapped Connect, captured, then set the real URL back, tapped Connect, force-stopped and relaunched. The disconnected state read Disconnected: Failed to connect to /127.0.0.1:9 · retrying with backoff (30-disconnected.png/.xml). After the restore the app showed live server data again — run 340 running 14m 10s (31-restored.png/.xml) — and Settings confirmed ws://10.0.0.75:8080/connect, Connected, build 0d1ddc7, workspace /Users/alan/repos/pipelines (32-settings-endpoint-restored). The phone was never left on a dead endpoint.

4. Wide-window checks — verified

adb … shell wm size 2000x2244 and wm size 2076x2152 (the fold's shape), then wm size reset. At 360 dpi the covered region (the peek strip, i.e. the active pane's left edge) was read from the covered pane's own tap-target bounds in uiautomator dump:

Depth Covered edge bounds (px) Covered region
1 (pipeline detail) [67,176,553,…] 553 px
2 (stage pane) [67,176,175,…] + [175,176,553,…] 553 px
3 (stage section) [67,176,175,…] + [175,176,553,…] 553 px

At 2076x2152 the region is the same 553 px at depths 1 and 3. So depth adds no width: the region is identical at depth 1, 2 and 3 and at both window sizes. 553 px = 245.8 dp = the measured compact-identity width (216 dp = MIN_COVERED_PANE_WIDTH_DP 112 + PEEK_WIDTH_DP 56 + one 48 dp touch target — the full-statistics budget) plus the 30 dp leading gesture inset. The older edge is 175 px = one touch target after the inset, so no covered edge is squeezed below the tappable minimum; the measured width sits on the intended budget, inside the allowed band. Evidence: 21-wide-pipelines-list.png, 22-wide-depth1.png, 24-wide-depth2.png, 25-wide-depth3.png, 26-fold-depth3.png, 27-fold-depth1.png. After the reset wm size reports only Physical size: 1008x2244 and dumpsys window displays reports cur=1008x2244 — no override left (28-after-reset).

Second client / fold

The brief allowed the fold's own view only if it cost no more than a capture; it was not opened. The shared-state proof is instead a different client: the CLI, querying the same server, sees the created session closed=true (evidence-chat-sessions.json) — the registry state the protocol says every device shares.

Still unverified

The fold's own client view (not opened) and the chat streaming's intermediate frames (the reply was captured complete, not frame by frame). Everything else across the four gaps is settled above.

Device lock: taken at 02:13 local (mkdir …/device.lock) and released with rmdir when the captures were done; no other lane's device work was running.