Files
pipelines/docs/app-integration-verification.md
alan db313336b2 docs: record the four end-to-end verification gap checks (run 340)
Add a dated follow-up section to docs/app-integration-verification.md that
closes the four claims the first pass left unproven, each with the command or
capture that proved it against the deployed server (0d1ddc7) on the Pixel 9 Pro
XL 47241FDAS00DLK:

- Chat reply arrives: a throwaway session showed the assistant reply `Done.`
  rendering live, and the close-a-chat flow left the session `closed=true`
  server-side.
- Live pushed run events: disproven on this build. Runs 341/342/343 never
  appeared or advanced without a manual `Refresh runs`; only a local elapsed
  timer ticked. Consistent with the run-event push-hub defect fixed on main by
  cff6f4d, absent from the deployed 0d1ddc7. Recorded as a defect with exact
  reproduction, not an app defect.
- Disconnected state: the closed-port state and the restore to
  ws://10.0.0.75:8080/connect were both captured and Settings verified after.
- Wide windows: the covered-pane region is a constant 553 px at depths 1-3 and
  at 2000x2244 and 2076x2152; `wm size reset` left no override.

Checked for staleness from this change: it touches no source or resource file,
so no packaged template or packaged authoring skill reference
(internal/author/skill/references/) depends on it - none references this
document and none needs updating.

Report-only: the untracked scratch directory .gaps-tmp/ is deliberately left
uncommitted.
2026-10-18 02:46:46 +01:00

198 lines
16 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# App integration verification — real server, on the phone (2026-10-18)
Verification only: nothing was fixed, and no app, server, installer or Docker file was edited.
This change adds this document alone.
## Revision under test
| Piece | Revision | How established |
| --- | --- | --- |
| Server | `v0.0.0-20261018001257-0d1ddc73a444` (commit `0d1ddc73a444fa24110ba5223646856c8f053d29`) | `pipelines ping` against `ws://127.0.0.1:8080/connect` |
| Phone app | `0d1ddc7` | `deploy-app.sh main` rebuilt main, installed it, and matched installed md5 `722f6a139a8aea6d15b09c89360dbea0` against the built APK |
The phone ended on **main's build (`0d1ddc7`)**.
Endpoint under test: `ws://10.0.0.75:8080/connect`, workspace `/Users/alan/repos/pipelines`
(shown in Settings, unchanged throughout — no stub, no deviation).
Device: operator's Pixel 9 Pro XL, `adb` serial `47241FDAS00DLK`, window 1008x2244.
Captures: `adb shell screencap -p /sdcard/shot.png` then `adb pull`; UI text read back with
`uiautomator dump`. Screen text quoted below is what the dump actually returned, so it is
evidence of rendered data, not just a screen appearing.
Capture directory (all rows below):
`/Users/alan/.local/share/pipelines/data/artifacts/332/1/`
## Flows
| Flow | Result | Evidence | Notes |
| --- | --- | --- | --- |
| Pipelines list | Pass | `01-pipelines-list.png`, `01-ui.xml` | Real pipelines and last runs: Android id 20 v1 4 stages, **running**, `run 332 · 4m 56s`; Complete a Task id 1 v1 1 stage succeeded run 112; Docker harness verification id 5 v2 "no runs recorded". Matches `pipelines list` / `pipelines runs`. |
| Pipelines filters | Pass | `02-pipelines-filter.png`, `03-*`, `04-pipelines-filter-canceled.png` | Filter sheet offers Name, Runs (Any/Has runs/No runs) and Last run status (queued, running, waiting, succeeded, failed, canceled, interrupted). Selecting **canceled** gave header `Filter pipelines, 1 filter active` and exactly two rows: Focused tweak - implement review commit (id 11, run 318, canceled 3h ago) and Quick tweak - implement and commit (id 10, run 219, canceled 1d ago). Correct. |
| Pipelines sort | Pass | `05-pipelines-sort-moststages.png`, `05-sort-sheet-ui.xml` | Sort sheet: Server order, Name A–Z, Name Z–A, Most recent run, Fewest stages, Most stages, Reset. "Most stages" reordered the list (4-stage pipelines first); "Server order" restored it. |
| Pipeline detail + tabs | Pass | `06-pipeline-detail.png`, `07-pipeline-graph.png`, `08-pipeline-source.png` | Android detail: content hash `219b8915…4312`, latest version v1, created/updated 17 Oct 22:57 — matches the server. Tabs Summary / Graph / Source: Graph draws 4 stages (implementer, reviewer, ui_reviewer, committer) with labelled edges and Terminal Success/Failure nodes; Source shows Raw + the four collapsible stage sections. |
| Runs list | Pass | `09-runs-list.png`, `09-runs-list-ui.xml` | Real aggregate header: **246 Total, average 18m 44s, 135 Succeeded, 99 Failed, 11 Canceled, 0 Interrupted, 1 Active**, with a `live` marker. Rows carry title, pipeline, version, status and visit counts: run 332 running "stage implementer"; run 331 canceled (context canceled); run 330 succeeded, 4 completed visits. Matches `pipelines runs`. |
| Run detail | Pass | `10-run-detail.png`, `10-run-detail-ui.xml` | Run 330: succeeded, outcome success, `4 recorded visits · 4 completed visits`, Pipeline id 22 v1, Stage committer, workspace path, content hash, Started 18 Oct 00:56, Finished 18 Oct 01:13, Duration 17m 9s. Tabs Summary / Graph / Input / Output. |
| Run detail graph | Pass | `11-run-graph.png` | Executed vs not-executed stages rendered with the legend. |
| Stage sheet with visits | Partial | `12-stage-sheet.png` | Tapping a stage node in the run graph did not open a stage sheet (screen unchanged). The visits are reachable from the run header line, not reproduced here — see "Not verified". |
| Timeline | Not verified | — | Not reached; the run detail tabs observed are Summary/Graph/Input/Output. |
| Artifacts gallery | Pass | `13-artifacts-gallery.png`, `13-artifacts-gallery-ui.xml` | Run 330 `Artifacts · 6 recorded, expanded`, grouped by stage/visit with real names and sizes: implementer visit 1 `wait-live-demo.log` 3.8 KiB; reviewer visit 1 `scratch-server.log` 12.7 KiB, `wait-batch-streaming.log` 98 B, `wait-blocking.log` 55 B; committer visit 1 `session.html` 89.1 KiB, `session.md` 81.7 KiB. |
| Full-screen viewer / next-prev | Partial | `14-artifact-viewer.png` | Tapping the artifact row opened the artifact in an external Chrome Web View (`http://10.0.0.75:8080/artifacts/330/2/wait-blocking.log`) rather than an in-app viewer; next/previous was therefore not exercised. |
| Omni search | Not verified | — | The pill's search entry could not be opened reliably from the run detail by coordinate or dump-matched tap within the time budget; no query was run. |
| Settings | Pass | `16-settings.png`, `16-settings-ui.xml` | Connection `ws://10.0.0.75:8080/connect`, **Connected**, `Server build v0.0.0-20261018001257-0d1ddc73a444 (0d1ddc73a444…)`, server time `2026-10-18T00:24:47Z`, workspace `/Users/alan/repos/pipelines` with "Saved workspace" confirmation. |
| Templates gallery | Pass | `17-templates-gallery.png`, `17-templates-gallery-ui.xml` | Present in this revision, reached from Settings → Browse templates: Complete a task (2 stages), Plan, implement, review (4 stages), Review a repository (1 stage), Research a topic (2 stages), Fold documentation notes (2 stages). |
| Chat | Partial | `18-chat.png`, `19-chat-new.png` | Server **does** broker chat in this revision: the list shows real sessions ("Pipeline audited agent stage", "Checking progress on Android pipeline application", "Pipelines — continue the work"), each `Closed` / `Idle` with its workspace; opening one renders a live transcript (reasoning + tool-call chips) and a composer ("Message the agent… or / for commands"). No prompt was sent, so a reply arriving is **not** confirmed — see below. |
| Live run updates | Not verified | — | No run other than this verification run (332) was running on the server during the window, and a run may not be started from the app. See below. |
## Not verified, and why
- **Stage sheet visits and the timeline.** The stage node tap in the run graph did not open a
stage sheet, and the run-detail tabs observed were Summary/Graph/Input/Output only; the time
budget ran out before locating the visits/timeline entry points. The visits are visible as a
per-stage grouping inside the artifacts section (see gallery row).
- **Artifact full-screen viewer, next/previous.** The artifact row opened the file in an external
browser, so the in-app viewer and its next/previous controls were not exercised.
- **Omni search** (one query plus every filter and sort option). Not reached.
- **Chat reply.** The server brokers chat and transcripts render from real session data, but no
prompt was sent, so "a reply arrives" is unproven. Sending would have injected a turn into the
operator's own live OpenCode session; with the time budget exhausted I did not risk it. This is
a gap in coverage, not a defect.
- **Live pushed run events.** There was no genuinely running run to watch other than run 332
itself (`pipelines runs` showed only 332 running; every other run succeeded/canceled/failed),
and this brief forbids starting a run from the app. Observing it would require a concurrent run
on the server — for example a lane whose pipeline is executing while the app sits on the Runs
list.
- **Disconnected-state capture.** Not performed. The sequence (point at a closed local port,
capture, then restore `ws://10.0.0.75:8080/connect`, force-stop, relaunch, confirm reconnect)
was cut for safety: overrunning the stage budget mid-sequence risks being killed with the phone
left pointed at a dead port. The endpoint is therefore **not** deviated from — the app is on the
real endpoint as captured in Settings.
- **Wide-window checks** (`wm size 2000x2244` and the fold shape `2076x2152`). Not run; every
capture here is the phone's own 1008x2244 window. `wm size` was left at its default (no
override set), so nothing needs resetting.
Nothing failed in a way that points at an app defect: every flow that was reached rendered the
server's real data correctly. The unverified items are coverage gaps from the time budget, not
observed failures.
## Device lock
`device.lock` was already present and **stale**: it was created at 01:14 by run 331 (the same
brief), which the server reports as `canceled`; no build, capture or `adb` process was live and the
Pixel's last capture predated my run. I removed and recreated the lock at 01:23 to take it, and
released it (`rmdir`) when the captures were done. No other lane's work was running on the device.
## Follow-up: the four coverage gaps (2026-10-18, run 340)
This section closes the four items the first pass left open. Same server (`0d1ddc7`, confirmed with
`pipelines ping`), same endpoint `ws://10.0.0.75:8080/connect` with workspace
`/Users/alan/repos/pipelines` (no stub was used and none was left behind), same device
(Pixel 9 Pro XL `47241FDAS00DLK`, 360 dpi, window 1008x2244). The phone app was **not** reinstalled,
so it is the same main build the first pass ended on. Capture directory for every row below:
`/Users/alan/.local/share/pipelines/data/artifacts/340/1/`.
### 1. A chat reply arrives — verified
A throwaway session was created in the app (New chat → Home (~)); never an operator or run session.
Server-side it is `ses_eb3671fa1ffeX7BdCHKfvnE9Kk` (`/Users/alan`, created 2026-10-18T01:20:49Z).
The prompt `reply with one short line` was typed and sent; the composer cleared, the user turn
rendered, and then the assistant reply **`Done.`** rendered in the transcript with no refresh, no
re-navigation and no second send. Evidence: `03b-typed.png` (prompt in the composer),
`04a-reply.png` (user turn + `Done.` in the transcript); server confirmation in
`evidence-chat-messages.json` — `chat_messages` returns the `user` turn `reply with one short line`
and the `assistant` turn `Done.`
Close-a-chat: Chat actions → `Close chat…` → `Close chat` (`06-menu.png`, `07-close-dialog.png`).
The dialog states the chat is marked closed on every device and kept, not deleted. After it the
server reports the session `closed=true`, `closed_at=1792286575309` (2026-10-18T01:22:55Z) —
`evidence-chat-sessions.json` — so the close is real server-side, not a local hide, and reopening the
row renders the same answered turn from history (`60-closed-chat-transcript.png`).
One deviation from the brief's wording, reported as a decision rather than a failure: the closed
chat does **not** disappear from the list. It leaves the active list and moves under a `Closed`
header carrying a `Closed` chip (`08-closed-list.png`); the app deliberately keeps closed chats as
history. The active section no longer contains it and the closed state is server-side, which is the
substance the brief asked for.
### 2. Live pushed run events — disproven on this build (defect)
A real 2-stage run was started from the shell and watched on the phone. The app's run views do
**not** update from pushed frames on the deployed server; every update needed a manual `Refresh runs`.
Reproduction (runs 341/342/343, all created from the shell after `41b-runs-before-new.png`):
- **A new run does not appear in the Runs list.** Run 342 started 01:29:13Z; at 01:29:24Z the list
still lacked it (`41c-list-no-refresh.xml`); one `Refresh runs` tap added it
(`41d-list-after-refresh.xml`). The same held for 341 and 343.
- **A run's view does not advance.** Run 343 (Research a topic) was opened in its run view at
01:30:34Z while stage `research` was running (`42-run-view-t0`,
`51-runview-t0-research-running`). The server recorded it **succeeded, stage `report`, 2 completed
visits, finished 01:30:41Z** (`pipelines inspect 343`). The app kept showing
`running · stage research · 0 completed visits · 1 recorded visit` for the whole ~5-minute watch
(`52-runview-latest`, `53-runview-still`, taken 01:35:56Z) — no stage advance, no terminal
transition, no interaction.
- **The Runs list row was stale the same way.** At 01:36:13Z run 343's row read
`running · 0 completed visits · stage research · run 343 · 6m 13s` (`54-list-before-refresh.png`);
one `Refresh runs` flipped it to `succeeded · 2 completed visits · stage report · run 343 · 18 Oct
02:30 · 20s` (`55-list-after-refresh.png`).
- **The only "live" value is the elapsed clock**, and it is a local timer, not a pushed update:
run 340 counted `14m 41s → 15m 56s → 23m 42s` with no interaction, while run 343's clock reached
`6m 13s` although the server had finished it at `20s`.
Cross-checks that rule out a dead socket: chat streamed a reply live at 01:21 in the same app session
(item 1), and Settings read `Connected` (`32-settings-endpoint-restored`). The server log shows the
app's single `subscribe` (android-5, 01:26:47Z), then only `list_runs` on each manual refresh and
`get_run` on each open — no further subscription traffic. This is consistent with the known
run-event push-hub defect the in-flight run 339 fixes ("an oversized subscribe snapshot overflow"),
which is not in the deployed `0d1ddc7` build; it is not an app defect, and it is exactly what the
brief's item 2 tested for. **Not verified, and disproven as stated:** run state does not arrive by
push on this deployment; a manual refresh is required.
### 3. Disconnected state — verified
One command sequence deviated and restored (`.gaps-tmp/disconnected.py`, run 340's scratch): set the
URL to the closed local port `ws://127.0.0.1:9/connect`, tapped Connect, captured, then set the real
URL back, tapped Connect, force-stopped and relaunched. The disconnected state read
`Disconnected: Failed to connect to /127.0.0.1:9 · retrying with backoff`
(`30-disconnected.png`/`.xml`). After the restore the app showed live server data again — run 340
`running 14m 10s` (`31-restored.png`/`.xml`) — and Settings confirmed
`ws://10.0.0.75:8080/connect`, `Connected`, build `0d1ddc7`, workspace `/Users/alan/repos/pipelines`
(`32-settings-endpoint-restored`). The phone was never left on a dead endpoint.
### 4. Wide-window checks — verified
`adb … shell wm size 2000x2244` and `wm size 2076x2152` (the fold's shape), then `wm size reset`. At
360 dpi the covered region (the peek strip, i.e. the active pane's left edge) was read from the
covered pane's own tap-target bounds in `uiautomator dump`:
| Depth | Covered edge bounds (px) | Covered region |
| --- | --- | --- |
| 1 (pipeline detail) | `[67,176,553,…]` | 553 px |
| 2 (stage pane) | `[67,176,175,…]` + `[175,176,553,…]` | 553 px |
| 3 (stage section) | `[67,176,175,…]` + `[175,176,553,…]` | 553 px |
At 2076x2152 the region is the same 553 px at depths 1 and 3. So **depth adds no width**: the region
is identical at depth 1, 2 and 3 and at both window sizes. 553 px = 245.8 dp = the measured
compact-identity width (216 dp = `MIN_COVERED_PANE_WIDTH_DP` 112 + `PEEK_WIDTH_DP` 56 + one 48 dp
touch target — the full-statistics budget) plus the 30 dp leading gesture inset. The older edge is
175 px = one touch target after the inset, so no covered edge is squeezed below the tappable
minimum; the measured width sits on the intended budget, inside the allowed band. Evidence:
`21-wide-pipelines-list.png`, `22-wide-depth1.png`, `24-wide-depth2.png`, `25-wide-depth3.png`,
`26-fold-depth3.png`, `27-fold-depth1.png`. After the reset `wm size` reports only
`Physical size: 1008x2244` and `dumpsys window displays` reports `cur=1008x2244` — no override left
(`28-after-reset`).
### Second client / fold
The brief allowed the fold's own view only if it cost no more than a capture; it was not opened. The
shared-state proof is instead a different client: the CLI, querying the same server, sees the created
session `closed=true` (`evidence-chat-sessions.json`) — the registry state the protocol says every
device shares.
### Still unverified
The fold's own client view (not opened) and the chat streaming's intermediate frames (the reply was
captured complete, not frame by frame). Everything else across the four gaps is settled above.
Device lock: taken at 02:13 local (`mkdir …/device.lock`) and released with `rmdir` when the captures
were done; no other lane's device work was running.