From 4f0d2aab6ffc88002b3bf1ca17d652f2720f42d7 Mon Sep 17 00:00:00 2001 From: Alan Silva Date: Thu, 22 Oct 2026 18:40:35 +0100 Subject: [PATCH] test(app): browser probe for search result selection --- app/notes/search-selection-probe.md | 140 +++++ app/tool/search_selection_probe.mjs | 646 +++++++++++++++++++++ app/tool/search_selection_probe_entry.dart | 469 +++++++++++++++ 3 files changed, 1255 insertions(+) create mode 100644 app/notes/search-selection-probe.md create mode 100644 app/tool/search_selection_probe.mjs create mode 100644 app/tool/search_selection_probe_entry.dart diff --git a/app/notes/search-selection-probe.md b/app/notes/search-selection-probe.md new file mode 100644 index 00000000..2d219e84 --- /dev/null +++ b/app/notes/search-selection-probe.md @@ -0,0 +1,140 @@ +# Search result-target selection: browser probe + +Real-Chrome evidence for the PARITY `Search` row's missing capture: a field hit +and an artifact hit select the recorded run Summary section or visit, and a +retired field / unknown visit falls back to the plain Summary. + +## Files + +- `app/tool/search_selection_probe_entry.dart` — the normal `PipelinesApp` + composition over the shared `PipelinesClient`, with the socket replaced by an + in-process channel that answers `search`, `get_run`, `list_runs`, + `list_pipelines`, `get_pipeline` and `ping` with server-shaped fixtures. + `subscribe`/`unsubscribe` are silent like the real server. A `search` body + that is not exactly `{query, kind, limit: 0}` is answered with an error + envelope, so a wrong request shape fails the captures instead of passing. + No live server; nothing is contacted over the network. +- `app/tool/search_selection_probe.mjs` — CDP driver (same shape as + `changes_probe.mjs`): loopback static server with index fallback for route + paths, headless Chrome, Flutter semantics enabled, exact-label clicks, PNGs. + +Run from `app/`: + +``` +flutter build web --target tool/search_selection_probe_entry.dart --output build/probe-search-selection +node tool/search_selection_probe.mjs --root build/probe-search-selection --out build/search-selection-probe +``` + +Exit code 0 means every assertion held and no defect was recorded; exit code 2 +means the assertions held but at least one recorded state differs from the +expected one (listed under `defects` in the JSON report). A thrown assertion +also writes `failure.png` next to the captures. + +## Fixture + +Run 7 "Release train run" (`name: release-train`), `input {prompt: "ship it"}`, +`output {result: "ok"}`, steps in recorded order plan seq 5, build seq 2 +(artifact `release-report.txt`), plan seq 1 — the sequences are deliberately +not the row indexes, as in `test/features/runs/ui/run_search_selection_test.dart`. + +Search replies (`truncated: false`, `scanned_steps 3`, `scanned_bytes 2048`, +`cap 50`, `scan_bound_bytes 1048576`): + +| query | hits | +| --- | --- | +| `release` | `pipeline:24`, `pipeline:22:v6`, `run:7`, `run:7:5` (plan visit 2), `run:7:5:output`, `/artifacts/7/2/release-report.txt` | +| `retired` | `run:7:5:evidence` (retired field) | +| `missing` | `/artifacts/7/9/missing.log` (sequence 9 is not recorded) | +| `first` | `run:7:1` (plan visit 1) | + +## Captures (`app/build/search-selection-probe/`) + +Light scheme is pinned through `Emulation.setEmulatedMedia`; one dark pass is +included. Phone is 390×844, desktop 1280×800. + +| capture | state | +| --- | --- | +| `search-selection-material-phone-results.png` | Search panel, `release`, one row per kind | +| `search-selection-material-phone-artifact-visit.png` | artifact hit → Timeline, `build · visit 2` pane | +| `search-selection-material-phone-retired-summary.png` | retired field after the open visit → plain Summary | +| `search-selection-material-phone-unknown-visit-summary.png` | unknown-visit artifact after an open visit → plain Summary | +| `search-selection-material-phone-field-summary.png` | field hit (`output`) → Summary, Output expanded, Input collapsed | +| `search-selection-material-phone-retired-after-field-summary.png` | retired field after that field hit (defect 1 evidence) | +| `search-selection-material-phone-step-visit.png` | step hit `plan visit 1` (defect 2 evidence) | +| `search-selection-material-desktop-results.png` | desktop results | +| `search-selection-material-desktop-artifact-visit.png` | desktop artifact visit | +| `search-selection-material-desktop-retired-summary.png` | desktop plain Summary | +| `search-selection-material-desktop-field-summary.png` | desktop field selection | +| `search-selection-material-phone-dark-results.png` | dark results | +| `search-selection-material-phone-dark-field-summary.png` | dark field selection | +| `search-selection-cupertino-phone-results.png` | Cupertino results (`Kind: All kinds` control) | +| `search-selection-cupertino-phone-artifact-visit.png` | Cupertino artifact visit | +| `search-selection-cupertino-phone-retired-summary.png` | Cupertino plain Summary after retired field | +| `search-selection-cupertino-phone-unknown-visit-summary.png` | Cupertino plain Summary after unknown visit | +| `search-selection-cupertino-phone-field-summary.png` | Cupertino field selection | +| `search-selection-cupertino-phone-retired-after-field-summary.png` | Cupertino retired-after-field (defect 1 evidence) | +| `search-selection-cupertino-desktop-results.png` | Cupertino desktop results | +| `search-selection-cupertino-desktop-artifact-visit.png` | Cupertino desktop artifact visit | +| `search-selection-cupertino-desktop-field-summary.png` | Cupertino desktop field selection | + +## Asserted strings (Flutter semantics, exact) + +- Results, every host/width: row titles `Release train`, `pipeline 22 · v6`, + `run #7 · Release train`, `run #7 · plan visit 2`, `run #7 · step 5 · output`, + `release-report.txt`; row meta `Pipeline`, `Pipeline version`, `Run`, `Step`, + `Field`, `Artifact`; snippet `ship the release`; bounds + `6 results · cap 50 · scanned 3 steps · 2.0 KiB scanned · scan limit 1.0 MiB`; + Cupertino also `Kind: All kinds`. +- Every hit: `location.pathname === '/runs/7'`, title `Release train run`, + and the Search page's text gone. +- Field hit: Summary tile `Output` announces `Expanded` and `Input` announces + `Collapsed` (first line of the tile's `aria-description`); no + `Back to timeline`, no `build · visit 2`. +- Artifact hit: `build · visit 2` and `Back to timeline` present; + `Original prompt` (Summary-only text) absent. +- Retired field / unknown visit: `Original prompt` present, `Input` and + `Output` both `Collapsed`, no visit pane text. +- One-shot: after an artifact hit, `Back to list` → row `Release train run` + reopens `/runs/7` on the Summary with no `build · visit 2`. +- Fixture-side: a `search` body other than `{query, kind, limit: 0}` and a + `get_run` for any run but 7 reply with an error envelope; both would have + failed the label assertions above (the JSON report has `pageErrors: []`). + +Semantics limitation: Flutter web renders a read-only `SelectableText` / +read-only `CupertinoTextField` as a `