Files
pipelines/app/tool/capture/README.md
T
alan d02d5ea11d Chat transcript: wheel and trackpad scrolling always reaches the transcript
Root causes found in release web captures (1280x800 and 560x900):
- Follow mode undid small wheel/trackpad ticks at the end: each tick is its
  own scroll, re-judged as following within 32 px and jumped back. A user
  scroll toward older rows now releases the end however small; the jump
  control appears once streamed content moves the end beyond the threshold.
- The end-of-scroll jump ran inside the position's activity switch, so a
  touch or scrollbar drag released near the end recursed without end (stack
  overflow during the gesture). The jump now runs after the notification.
- Horizontally scrolling code blocks and tables claimed diagonal trackpad
  gestures whole. HorizontalScrollView gives mostly vertical gestures to the
  enclosing list; sideways and Shift + wheel still scroll the block.
- The wheel over the floating chat bar, composer and jump control moved
  nothing. VerticalWheelForwarder scrolls the transcript from them unless
  something under the pointer claims the event.

Adds widget tests per content kind and scroll mode, follow/touch/Load older
tests, a capture-kit check (tool/capture/examples/chat-wheel-scroll.json)
with eval/expect steps, wheel dx/repeat, --params and eval: targets, a
transcript scroll probe and fixture options, and fixes the kit ignoring a
script's size. Notes in notes/chat-wheel-scroll.md.
2026-11-02 20:37:36 +00:00

125 lines
5.6 KiB
Markdown

# Capture kit
One generic browser driver for UI review. It works on the whole app, not one
feature, so a renamed label never makes it go stale. Use it instead of writing
new capture code, and instead of a per-feature `tool/*_probe.mjs` driver that
no longer matches the app.
- `capture.mjs`: the CLI. It serves a build on an OS-assigned loopback port
(`--build`) or takes `--url`. It starts private headless Chrome (fresh
profile, debugging port chosen by the OS) or attaches to one with
`--cdp http://127.0.0.1:<port>`. Then it runs the steps and writes
`NN-<name>.png` and `report.json` to `--out`. On exit, Ctrl-C or failure it
removes Chrome, the profile and the server.
- It needs no npm packages: Node 22+ (global `WebSocket` and `fetch`) and
Google Chrome (`CHROME_PATH` overrides the macOS default).
## The fixture app
`tool/search_selection_probe_entry.dart` is the real `PipelinesApp` running on
an in-process fixture channel. It never contacts a server. It has run 7 (three
visits plus the artifact `release-report.txt`), run 6, pipelines 24 and 22, and
search hits for `release`. Build it once per change, from `app/`:
```sh
/Users/alan/development/flutter/bin/flutter build web \
--target tool/search_selection_probe_entry.dart --output build/probe-app
```
URL parameters (the kit sets them from `open=` and the options):
`route=<app route>`, `q=<search>`, `platform=material|cupertino`,
`theme=light|dark`; `--params` (or a step's or script's `params`) adds fixture
options such as `chats=2&rich=1`. Without `theme`, the app follows the browser's colour
scheme. `--theme` also sets that colour scheme through Chrome emulation.
## Steps
You can pass steps on the command line as `verb=value`. With `--script`, they
go in a JSON array or in `{ "size", "platform", "theme", "q", "steps": [...] }`.
In JSON, a step can also set `wait` (ms), `timeout`, `name`, `ms` and `frames`.
| step | effect |
| --- | --- |
| `open=/runs/7[@WxH]` | load a route at a size (default `--size`, 880x1000) and turn on Flutter semantics |
| `shot=name` | take a screenshot |
| `click=<target>` | click the centre of the target, then wait `--wait` ms (default 1000) |
| `waitfor=<target>` | wait up to 10 s for the target to appear |
| `type=text` | type into the focused field (or the first visible one) |
| `press=Enter` | press a key: Enter, Escape, Tab, Backspace, ArrowUp or ArrowDown |
| `scroll=600` | scroll with the mouse wheel at the centre (JSON `at` sets a target, `dx` a horizontal delta, `repeat` and `gap` a run of ticks) |
| `size=WxH` | resize without reloading |
| `labels=name` | write the visible semantics labels, keys and rects to JSON |
| `film=<target>` | screencast a click for `--film-ms` (900) and write the frames plus a contact sheet |
| `wait=ms` | sleep |
| `eval=<js>` | evaluate an expression in the page and log/record its value |
| `expect=<js>` | fail the run (status 1) unless the expression is truthy; JSON `name` labels the check |
Targets:
- A bare `Artifacts` or `label:Artifacts` is a semantics label (`aria-label` or
the node's text). An exact match beats a prefix match, and then the smallest
visible node wins.
- `key:list-detail-body` is a `Semantics(identifier:)`. A Flutter `Key` is not
visible in the DOM.
- `120,340` or `xy:120,340` is a point in CSS pixels.
- `eval:<js>` is a point an expression in the page yields as `[x, y]`, waited
for while it yields null (e.g. `eval:JSON.parse(chatScrollTargets()).table`).
When a target is missing, the step fails. The kit lists the visible labels,
saves `NN-FAILED.png` and exits with status 1.
## Recipes (run from `app/` after building `build/probe-app`)
Capture a page at phone and fold widths, in Material and in Cupertino. Use 560
for phone: headless Chrome misrenders Flutter below about 500 px.
```sh
for platform in material cupertino; do
for size in 560x1000 880x1000; do
node tool/capture/capture.mjs --build build/probe-app \
--out /tmp/review/runs-$platform-$size --platform $platform --size $size \
open=/runs/7 shot=run7
done
done
```
Add `--theme dark` (or `light`) to pin the theme.
Click through to a state:
```sh
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/artifact \
open=/runs/7@880x1000 click=Artifacts waitfor=release-report.txt shot=artifacts \
click=release-report.txt shot=artifact-open labels=artifact-open
```
To run the same flow as a script, use `--script tool/capture/examples/run7-artifacts.json`.
Search with `open=/search` then `type=release`, or use `--q release`.
Film a transition. This writes the frames (`fNNN_<ms>ms.png`, relative to the
click) to `<out>/film-label-Pipelines/` and the sheet to
`NN-film-label-Pipelines-sheet.png`:
```sh
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/film \
--film-ms 900 open=/runs@560x1000 film=Pipelines
```
In a JSON script, `{ "film": "Pipelines", "name": "tab-switch", "ms": 900,
"frames": 6 }` sets the name, the duration and how many frames go on the sheet.
Find targets: run `labels=now` after any step and read `NN-now.json`.
Look at every PNG you cite. A capture that exits 0 is not a review.
Check a chat scroll: the full-app fixture exposes `chatScrollMetrics()` (the
open transcript's `{pixels, min, max, viewport}` as JSON;
`tool/perf/chat_scroll_probe.dart`), and `?chats=N` with `?msgs=`, `?rich=1`,
`?older=1` and `?live=N` (`tool/perf/chat_scroll_fixture.dart`) shapes the
transcripts. `tool/capture/examples/chat-wheel-scroll.json` wheels over each
content kind and expects the offset to move:
```sh
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/chat-wheel \
--script tool/capture/examples/chat-wheel-scroll.json
```