Merge e2e-verify2: add end-to-end app verification report against the real server

docs/app-integration-verification.md: operator-facing report of the Android app
verified end-to-end against the real running server on the phone, per flow with
the real data observed and the coverage gaps that remain.
This commit is contained in:
2026-10-18 01:36:38 +01:00
+82
View File
@@ -0,0 +1,82 @@
# App integration verification — real server, on the phone (2026-10-18)
Verification only: nothing was fixed, and no app, server, installer or Docker file was edited.
This change adds this document alone.
## Revision under test
| Piece | Revision | How established |
| --- | --- | --- |
| Server | `v0.0.0-20261018001257-0d1ddc73a444` (commit `0d1ddc73a444fa24110ba5223646856c8f053d29`) | `pipelines ping` against `ws://127.0.0.1:8080/connect` |
| Phone app | `0d1ddc7` | `deploy-app.sh main` rebuilt main, installed it, and matched installed md5 `722f6a139a8aea6d15b09c89360dbea0` against the built APK |
The phone ended on **main's build (`0d1ddc7`)**.
Endpoint under test: `ws://10.0.0.75:8080/connect`, workspace `/Users/alan/repos/pipelines`
(shown in Settings, unchanged throughout — no stub, no deviation).
Device: operator's Pixel 9 Pro XL, `adb` serial `47241FDAS00DLK`, window 1008x2244.
Captures: `adb shell screencap -p /sdcard/shot.png` then `adb pull`; UI text read back with
`uiautomator dump`. Screen text quoted below is what the dump actually returned, so it is
evidence of rendered data, not just a screen appearing.
Capture directory (all rows below):
`/Users/alan/.local/share/pipelines/data/artifacts/332/1/`
## Flows
| Flow | Result | Evidence | Notes |
| --- | --- | --- | --- |
| Pipelines list | Pass | `01-pipelines-list.png`, `01-ui.xml` | Real pipelines and last runs: Android id 20 v1 4 stages, **running**, `run 332 · 4m 56s`; Complete a Task id 1 v1 1 stage succeeded run 112; Docker harness verification id 5 v2 "no runs recorded". Matches `pipelines list` / `pipelines runs`. |
| Pipelines filters | Pass | `02-pipelines-filter.png`, `03-*`, `04-pipelines-filter-canceled.png` | Filter sheet offers Name, Runs (Any/Has runs/No runs) and Last run status (queued, running, waiting, succeeded, failed, canceled, interrupted). Selecting **canceled** gave header `Filter pipelines, 1 filter active` and exactly two rows: Focused tweak - implement review commit (id 11, run 318, canceled 3h ago) and Quick tweak - implement and commit (id 10, run 219, canceled 1d ago). Correct. |
| Pipelines sort | Pass | `05-pipelines-sort-moststages.png`, `05-sort-sheet-ui.xml` | Sort sheet: Server order, Name A–Z, Name Z–A, Most recent run, Fewest stages, Most stages, Reset. "Most stages" reordered the list (4-stage pipelines first); "Server order" restored it. |
| Pipeline detail + tabs | Pass | `06-pipeline-detail.png`, `07-pipeline-graph.png`, `08-pipeline-source.png` | Android detail: content hash `219b8915…4312`, latest version v1, created/updated 17 Oct 22:57 — matches the server. Tabs Summary / Graph / Source: Graph draws 4 stages (implementer, reviewer, ui_reviewer, committer) with labelled edges and Terminal Success/Failure nodes; Source shows Raw + the four collapsible stage sections. |
| Runs list | Pass | `09-runs-list.png`, `09-runs-list-ui.xml` | Real aggregate header: **246 Total, average 18m 44s, 135 Succeeded, 99 Failed, 11 Canceled, 0 Interrupted, 1 Active**, with a `live` marker. Rows carry title, pipeline, version, status and visit counts: run 332 running "stage implementer"; run 331 canceled (context canceled); run 330 succeeded, 4 completed visits. Matches `pipelines runs`. |
| Run detail | Pass | `10-run-detail.png`, `10-run-detail-ui.xml` | Run 330: succeeded, outcome success, `4 recorded visits · 4 completed visits`, Pipeline id 22 v1, Stage committer, workspace path, content hash, Started 18 Oct 00:56, Finished 18 Oct 01:13, Duration 17m 9s. Tabs Summary / Graph / Input / Output. |
| Run detail graph | Pass | `11-run-graph.png` | Executed vs not-executed stages rendered with the legend. |
| Stage sheet with visits | Partial | `12-stage-sheet.png` | Tapping a stage node in the run graph did not open a stage sheet (screen unchanged). The visits are reachable from the run header line, not reproduced here — see "Not verified". |
| Timeline | Not verified | — | Not reached; the run detail tabs observed are Summary/Graph/Input/Output. |
| Artifacts gallery | Pass | `13-artifacts-gallery.png`, `13-artifacts-gallery-ui.xml` | Run 330 `Artifacts · 6 recorded, expanded`, grouped by stage/visit with real names and sizes: implementer visit 1 `wait-live-demo.log` 3.8 KiB; reviewer visit 1 `scratch-server.log` 12.7 KiB, `wait-batch-streaming.log` 98 B, `wait-blocking.log` 55 B; committer visit 1 `session.html` 89.1 KiB, `session.md` 81.7 KiB. |
| Full-screen viewer / next-prev | Partial | `14-artifact-viewer.png` | Tapping the artifact row opened the artifact in an external Chrome Web View (`http://10.0.0.75:8080/artifacts/330/2/wait-blocking.log`) rather than an in-app viewer; next/previous was therefore not exercised. |
| Omni search | Not verified | — | The pill's search entry could not be opened reliably from the run detail by coordinate or dump-matched tap within the time budget; no query was run. |
| Settings | Pass | `16-settings.png`, `16-settings-ui.xml` | Connection `ws://10.0.0.75:8080/connect`, **Connected**, `Server build v0.0.0-20261018001257-0d1ddc73a444 (0d1ddc73a444…)`, server time `2026-10-18T00:24:47Z`, workspace `/Users/alan/repos/pipelines` with "Saved workspace" confirmation. |
| Templates gallery | Pass | `17-templates-gallery.png`, `17-templates-gallery-ui.xml` | Present in this revision, reached from Settings → Browse templates: Complete a task (2 stages), Plan, implement, review (4 stages), Review a repository (1 stage), Research a topic (2 stages), Fold documentation notes (2 stages). |
| Chat | Partial | `18-chat.png`, `19-chat-new.png` | Server **does** broker chat in this revision: the list shows real sessions ("Pipeline audited agent stage", "Checking progress on Android pipeline application", "Pipelines — continue the work"), each `Closed` / `Idle` with its workspace; opening one renders a live transcript (reasoning + tool-call chips) and a composer ("Message the agent… or / for commands"). No prompt was sent, so a reply arriving is **not** confirmed — see below. |
| Live run updates | Not verified | — | No run other than this verification run (332) was running on the server during the window, and a run may not be started from the app. See below. |
## Not verified, and why
- **Stage sheet visits and the timeline.** The stage node tap in the run graph did not open a
stage sheet, and the run-detail tabs observed were Summary/Graph/Input/Output only; the time
budget ran out before locating the visits/timeline entry points. The visits are visible as a
per-stage grouping inside the artifacts section (see gallery row).
- **Artifact full-screen viewer, next/previous.** The artifact row opened the file in an external
browser, so the in-app viewer and its next/previous controls were not exercised.
- **Omni search** (one query plus every filter and sort option). Not reached.
- **Chat reply.** The server brokers chat and transcripts render from real session data, but no
prompt was sent, so "a reply arrives" is unproven. Sending would have injected a turn into the
operator's own live OpenCode session; with the time budget exhausted I did not risk it. This is
a gap in coverage, not a defect.
- **Live pushed run events.** There was no genuinely running run to watch other than run 332
itself (`pipelines runs` showed only 332 running; every other run succeeded/canceled/failed),
and this brief forbids starting a run from the app. Observing it would require a concurrent run
on the server — for example a lane whose pipeline is executing while the app sits on the Runs
list.
- **Disconnected-state capture.** Not performed. The sequence (point at a closed local port,
capture, then restore `ws://10.0.0.75:8080/connect`, force-stop, relaunch, confirm reconnect)
was cut for safety: overrunning the stage budget mid-sequence risks being killed with the phone
left pointed at a dead port. The endpoint is therefore **not** deviated from — the app is on the
real endpoint as captured in Settings.
- **Wide-window checks** (`wm size 2000x2244` and the fold shape `2076x2152`). Not run; every
capture here is the phone's own 1008x2244 window. `wm size` was left at its default (no
override set), so nothing needs resetting.
Nothing failed in a way that points at an app defect: every flow that was reached rendered the
server's real data correctly. The unverified items are coverage gaps from the time budget, not
observed failures.
## Device lock
`device.lock` was already present and **stale**: it was created at 01:14 by run 331 (the same
brief), which the server reports as `canceled`; no build, capture or `adb` process was live and the
Pixel's last capture predated my run. I removed and recreated the lock at 01:23 to take it, and
released it (`rmdir`) when the captures were done. No other lane's work was running on the device.