Files
pipelines/docs/app-integration-verification.md
T
alan 8c3efa4143 docs: add end-to-end app verification report against the real server
Adds docs/app-integration-verification.md, an operator-facing report of an
end-to-end verification of the Android app against the real running Pipelines
server on the phone. Report only: no app, server, installer or Docker file was
changed, and nothing was fixed.

The report records, per flow, the result, capture path and the real server data
observed: the pipelines list and its filters/sort, pipeline detail tabs, runs
list aggregate (246 total) and run detail, the stage sheet/timeline and
full-screen artifact viewer gaps, artifacts gallery, omni search, Settings
(endpoint ws://10.0.0.75:8080/connect, server build 0d1ddc73a4), the templates
gallery, chat sessions, live run updates and the disconnected state that were
not verified and why.

Revision tested: phone app at main's build (0d1ddc7, installed md5
722f6a139a8aea6d15b09c89360dbea0), server build
v0.0.0-20261018001257-0d1ddc73a444. The phone was left on main's build.

Checked for staleness: no packaged template or packaged authoring skill
reference (internal/author/skill/references/*) describes or indexes this
document, and the change adds no runtime behaviour, so none of them needed
updating. Per the maintenance convention the shared docs were not edited; the
verification findings live in this document alone.
2026-10-18 01:36:22 +01:00

83 lines
8.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# App integration verification — real server, on the phone (2026-10-18)
Verification only: nothing was fixed, and no app, server, installer or Docker file was edited.
This change adds this document alone.
## Revision under test
| Piece | Revision | How established |
| --- | --- | --- |
| Server | `v0.0.0-20261018001257-0d1ddc73a444` (commit `0d1ddc73a444fa24110ba5223646856c8f053d29`) | `pipelines ping` against `ws://127.0.0.1:8080/connect` |
| Phone app | `0d1ddc7` | `deploy-app.sh main` rebuilt main, installed it, and matched installed md5 `722f6a139a8aea6d15b09c89360dbea0` against the built APK |
The phone ended on **main's build (`0d1ddc7`)**.
Endpoint under test: `ws://10.0.0.75:8080/connect`, workspace `/Users/alan/repos/pipelines`
(shown in Settings, unchanged throughout — no stub, no deviation).
Device: operator's Pixel 9 Pro XL, `adb` serial `47241FDAS00DLK`, window 1008x2244.
Captures: `adb shell screencap -p /sdcard/shot.png` then `adb pull`; UI text read back with
`uiautomator dump`. Screen text quoted below is what the dump actually returned, so it is
evidence of rendered data, not just a screen appearing.
Capture directory (all rows below):
`/Users/alan/.local/share/pipelines/data/artifacts/332/1/`
## Flows
| Flow | Result | Evidence | Notes |
| --- | --- | --- | --- |
| Pipelines list | Pass | `01-pipelines-list.png`, `01-ui.xml` | Real pipelines and last runs: Android id 20 v1 4 stages, **running**, `run 332 · 4m 56s`; Complete a Task id 1 v1 1 stage succeeded run 112; Docker harness verification id 5 v2 "no runs recorded". Matches `pipelines list` / `pipelines runs`. |
| Pipelines filters | Pass | `02-pipelines-filter.png`, `03-*`, `04-pipelines-filter-canceled.png` | Filter sheet offers Name, Runs (Any/Has runs/No runs) and Last run status (queued, running, waiting, succeeded, failed, canceled, interrupted). Selecting **canceled** gave header `Filter pipelines, 1 filter active` and exactly two rows: Focused tweak - implement review commit (id 11, run 318, canceled 3h ago) and Quick tweak - implement and commit (id 10, run 219, canceled 1d ago). Correct. |
| Pipelines sort | Pass | `05-pipelines-sort-moststages.png`, `05-sort-sheet-ui.xml` | Sort sheet: Server order, Name A–Z, Name Z–A, Most recent run, Fewest stages, Most stages, Reset. "Most stages" reordered the list (4-stage pipelines first); "Server order" restored it. |
| Pipeline detail + tabs | Pass | `06-pipeline-detail.png`, `07-pipeline-graph.png`, `08-pipeline-source.png` | Android detail: content hash `219b8915…4312`, latest version v1, created/updated 17 Oct 22:57 — matches the server. Tabs Summary / Graph / Source: Graph draws 4 stages (implementer, reviewer, ui_reviewer, committer) with labelled edges and Terminal Success/Failure nodes; Source shows Raw + the four collapsible stage sections. |
| Runs list | Pass | `09-runs-list.png`, `09-runs-list-ui.xml` | Real aggregate header: **246 Total, average 18m 44s, 135 Succeeded, 99 Failed, 11 Canceled, 0 Interrupted, 1 Active**, with a `live` marker. Rows carry title, pipeline, version, status and visit counts: run 332 running "stage implementer"; run 331 canceled (context canceled); run 330 succeeded, 4 completed visits. Matches `pipelines runs`. |
| Run detail | Pass | `10-run-detail.png`, `10-run-detail-ui.xml` | Run 330: succeeded, outcome success, `4 recorded visits · 4 completed visits`, Pipeline id 22 v1, Stage committer, workspace path, content hash, Started 18 Oct 00:56, Finished 18 Oct 01:13, Duration 17m 9s. Tabs Summary / Graph / Input / Output. |
| Run detail graph | Pass | `11-run-graph.png` | Executed vs not-executed stages rendered with the legend. |
| Stage sheet with visits | Partial | `12-stage-sheet.png` | Tapping a stage node in the run graph did not open a stage sheet (screen unchanged). The visits are reachable from the run header line, not reproduced here — see "Not verified". |
| Timeline | Not verified | — | Not reached; the run detail tabs observed are Summary/Graph/Input/Output. |
| Artifacts gallery | Pass | `13-artifacts-gallery.png`, `13-artifacts-gallery-ui.xml` | Run 330 `Artifacts · 6 recorded, expanded`, grouped by stage/visit with real names and sizes: implementer visit 1 `wait-live-demo.log` 3.8 KiB; reviewer visit 1 `scratch-server.log` 12.7 KiB, `wait-batch-streaming.log` 98 B, `wait-blocking.log` 55 B; committer visit 1 `session.html` 89.1 KiB, `session.md` 81.7 KiB. |
| Full-screen viewer / next-prev | Partial | `14-artifact-viewer.png` | Tapping the artifact row opened the artifact in an external Chrome Web View (`http://10.0.0.75:8080/artifacts/330/2/wait-blocking.log`) rather than an in-app viewer; next/previous was therefore not exercised. |
| Omni search | Not verified | — | The pill's search entry could not be opened reliably from the run detail by coordinate or dump-matched tap within the time budget; no query was run. |
| Settings | Pass | `16-settings.png`, `16-settings-ui.xml` | Connection `ws://10.0.0.75:8080/connect`, **Connected**, `Server build v0.0.0-20261018001257-0d1ddc73a444 (0d1ddc73a444…)`, server time `2026-10-18T00:24:47Z`, workspace `/Users/alan/repos/pipelines` with "Saved workspace" confirmation. |
| Templates gallery | Pass | `17-templates-gallery.png`, `17-templates-gallery-ui.xml` | Present in this revision, reached from Settings → Browse templates: Complete a task (2 stages), Plan, implement, review (4 stages), Review a repository (1 stage), Research a topic (2 stages), Fold documentation notes (2 stages). |
| Chat | Partial | `18-chat.png`, `19-chat-new.png` | Server **does** broker chat in this revision: the list shows real sessions ("Pipeline audited agent stage", "Checking progress on Android pipeline application", "Pipelines — continue the work"), each `Closed` / `Idle` with its workspace; opening one renders a live transcript (reasoning + tool-call chips) and a composer ("Message the agent… or / for commands"). No prompt was sent, so a reply arriving is **not** confirmed — see below. |
| Live run updates | Not verified | — | No run other than this verification run (332) was running on the server during the window, and a run may not be started from the app. See below. |
## Not verified, and why
- **Stage sheet visits and the timeline.** The stage node tap in the run graph did not open a
stage sheet, and the run-detail tabs observed were Summary/Graph/Input/Output only; the time
budget ran out before locating the visits/timeline entry points. The visits are visible as a
per-stage grouping inside the artifacts section (see gallery row).
- **Artifact full-screen viewer, next/previous.** The artifact row opened the file in an external
browser, so the in-app viewer and its next/previous controls were not exercised.
- **Omni search** (one query plus every filter and sort option). Not reached.
- **Chat reply.** The server brokers chat and transcripts render from real session data, but no
prompt was sent, so "a reply arrives" is unproven. Sending would have injected a turn into the
operator's own live OpenCode session; with the time budget exhausted I did not risk it. This is
a gap in coverage, not a defect.
- **Live pushed run events.** There was no genuinely running run to watch other than run 332
itself (`pipelines runs` showed only 332 running; every other run succeeded/canceled/failed),
and this brief forbids starting a run from the app. Observing it would require a concurrent run
on the server — for example a lane whose pipeline is executing while the app sits on the Runs
list.
- **Disconnected-state capture.** Not performed. The sequence (point at a closed local port,
capture, then restore `ws://10.0.0.75:8080/connect`, force-stop, relaunch, confirm reconnect)
was cut for safety: overrunning the stage budget mid-sequence risks being killed with the phone
left pointed at a dead port. The endpoint is therefore **not** deviated from — the app is on the
real endpoint as captured in Settings.
- **Wide-window checks** (`wm size 2000x2244` and the fold shape `2076x2152`). Not run; every
capture here is the phone's own 1008x2244 window. `wm size` was left at its default (no
override set), so nothing needs resetting.
Nothing failed in a way that points at an app defect: every flow that was reached rendered the
server's real data correctly. The unverified items are coverage gaps from the time budget, not
observed failures.
## Device lock
`device.lock` was already present and **stale**: it was created at 01:14 by run 331 (the same
brief), which the server reports as `canceled`; no build, capture or `adb` process was live and the
Pixel's last capture predated my run. I removed and recreated the lock at 01:23 to take it, and
released it (`rmdir`) when the captures were done. No other lane's work was running on the device.