e0e8ca68f3
The app builds one Material UI for every platform. Cupertino code, tests and captures are being removed, so the agreement, the capture recipe and the floating-windows plan no longer ask for them.
167 lines
8.6 KiB
Markdown
167 lines
8.6 KiB
Markdown
# Capture kit
|
|
|
|
One generic browser driver for UI review. It works on the whole app, not one
|
|
feature, so a renamed label never makes it go stale. Use it instead of writing
|
|
new capture code, and instead of a per-feature `tool/*_probe.mjs` driver that
|
|
no longer matches the app.
|
|
|
|
- `capture.mjs`: the CLI. It serves a build on an OS-assigned loopback port
|
|
(`--build`) or takes `--url`. It starts private headless Chrome (fresh
|
|
profile, debugging port chosen by the OS) or attaches to one with
|
|
`--cdp http://127.0.0.1:<port>`. Then it runs the steps and writes
|
|
`NN-<name>.png` and `report.json` to `--out`. On exit, Ctrl-C or failure it
|
|
removes Chrome, the profile and the server.
|
|
- It needs no npm packages: Node 22+ (global `WebSocket` and `fetch`) and
|
|
Google Chrome (`CHROME_PATH` overrides the macOS default).
|
|
|
|
## The fixture app
|
|
|
|
`tool/search_selection_probe_entry.dart` is the real `PipelinesApp` running on
|
|
an in-process fixture channel. It never contacts a server. It has run 7 (three
|
|
visits plus the artifact `release-report.txt`), run 6, pipelines 24 and 22, and
|
|
search hits for `release`. It also answers `server_stats`, so the Settings route
|
|
can be captured before the server command is available. Build it once per
|
|
change, from `app/`:
|
|
|
|
```sh
|
|
/Users/alan/development/flutter/bin/flutter build web \
|
|
--target tool/search_selection_probe_entry.dart --output build/probe-app
|
|
```
|
|
|
|
URL parameters (the kit sets them from `open=` and the options):
|
|
`route=<app route>`, `q=<search>`, `platform=material|cupertino`,
|
|
`theme=light|dark`; `--params` (or a step's or script's `params`) adds fixture
|
|
options such as `chats=2&rich=1`. Use `route=/settings` to open Settings and then
|
|
click `Server health`; the fixture channel returns `server_stats` automatically.
|
|
Without `theme`, the app follows the browser's colour
|
|
scheme. `--theme` also sets that colour scheme through Chrome emulation.
|
|
|
|
## Steps
|
|
|
|
You can pass steps on the command line as `verb=value`. With `--script`, they
|
|
go in a JSON array or in `{ "size", "platform", "theme", "q", "steps": [...] }`.
|
|
In JSON, a step can also set `wait` (ms), `timeout`, `name`, `ms` and `frames`.
|
|
|
|
| step | effect |
|
|
| --- | --- |
|
|
| `open=/runs/7[@WxH]` | load a route at a size (default `--size`, 880x1000) and turn on Flutter semantics |
|
|
| `shot=name` | take a screenshot |
|
|
| `click=<target>` | click the centre of the target, then wait `--wait` ms (default 1000) |
|
|
| `waitfor=<target>` | wait up to 10 s for the target to appear |
|
|
| `type=text` | type into the focused field (or the first visible one) |
|
|
| `press=Enter` | press a key: Enter, Escape, Tab, Backspace, ArrowUp, ArrowDown, a letter or a digit, optionally after `Control+`, `Shift+`, `Alt+` or `Meta+` (`press=Control+2`) |
|
|
| `scroll=600` | scroll with the mouse wheel at the centre (JSON `at` sets a target, `dx` a horizontal delta, `repeat` and `gap` a run of ticks) |
|
|
| `hover=<target>` | move the mouse onto the target without pressing (hover states) |
|
|
| `drag=<target>>dx,dy` | press on the target, move by dx,dy in `steps` (12) moves `gap` (16) ms apart, release (JSON `dx`, `dy`, `steps`, `gap`; `hold: true` keeps the button down to capture mid-drag) |
|
|
| `size=WxH` | resize without reloading |
|
|
| `labels=name` | write the visible semantics labels, keys and rects to JSON |
|
|
| `film=<target>` | screencast a click for `--film-ms` (900) and write the frames plus a contact sheet; `film=press:Alt+t` films a key chord (as `press=`) instead |
|
|
| `wait=ms` | sleep |
|
|
| `files=a.ttf[,b.ttf]` | answer the next file chooser the page opens with these files (paths relative to the working directory); put it before the click that opens the picker |
|
|
| `swipe=<target>>dx,dy` | one-finger touch swipe (needs `--touch true`, which emulates a touch screen once semantics is on) |
|
|
| `pinch=<target>>scale` | two-finger touch pinch, spreading (>1) or closing (<1) (needs `--touch true`) |
|
|
| `tabs=name` | write the open tabs (`{url, type}`) to JSON, e.g. to see a link open a new tab |
|
|
| `eval=<js>` | evaluate an expression in the page and log/record its value |
|
|
| `expect=<js>` | fail the run (status 1) unless the expression is truthy; JSON `name` labels the check |
|
|
|
|
Targets:
|
|
- A bare `Artifacts` or `label:Artifacts` is a semantics label (`aria-label` or
|
|
the node's text). An exact match beats a prefix match, and then the smallest
|
|
visible node wins.
|
|
- `key:list-detail-body` is a `Semantics(identifier:)`. A Flutter `Key` is not
|
|
visible in the DOM.
|
|
- `120,340` or `xy:120,340` is a point in CSS pixels.
|
|
- `eval:<js>` is a point an expression in the page yields as `[x, y]`, waited
|
|
for while it yields null (e.g. `eval:JSON.parse(chatScrollTargets()).table`).
|
|
|
|
When a target is missing, the step fails. The kit lists the visible labels,
|
|
saves `NN-FAILED.png` and exits with status 1.
|
|
|
|
An HTML artifact: `--params html=1` adds `release-dashboard.html` to run 7.
|
|
Its bytes are served from the build, so copy the fixture in after building:
|
|
`mkdir -p build/probe-app/artifacts/7/2 && cp tool/capture/fixtures/release-dashboard.html build/probe-app/artifacts/7/2/`.
|
|
|
|
## Recipes (run from `app/` after building `build/probe-app`)
|
|
|
|
Capture a page at phone and fold widths. Use 560 for phone: headless Chrome
|
|
misrenders Flutter below about 500 px. The app has one Material UI; do not
|
|
capture `platform=cupertino`.
|
|
|
|
```sh
|
|
for size in 560x1000 880x1000; do
|
|
node tool/capture/capture.mjs --build build/probe-app \
|
|
--out /tmp/review/runs-$size --size $size open=/runs/7 shot=run7
|
|
done
|
|
```
|
|
|
|
Add `--theme dark` (or `light`) to pin the theme.
|
|
|
|
Click through to a state:
|
|
|
|
```sh
|
|
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/artifact \
|
|
open=/runs/7@880x1000 click=Artifacts waitfor=release-report.txt shot=artifacts \
|
|
click=release-report.txt shot=artifact-open labels=artifact-open
|
|
```
|
|
|
|
To run the same flow as a script, use `--script tool/capture/examples/run7-artifacts.json`.
|
|
Search with `open=/search` then `type=release`, or use `--q release`.
|
|
|
|
Film a transition. This writes the frames (`fNNN_<ms>ms.png`, relative to the
|
|
click) to `<out>/film-label-Pipelines/` and the sheet to
|
|
`NN-film-label-Pipelines-sheet.png`:
|
|
|
|
```sh
|
|
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/film \
|
|
--film-ms 900 open=/runs@560x1000 film=Pipelines
|
|
```
|
|
|
|
In a JSON script, `{ "film": "Pipelines", "name": "tab-switch", "ms": 900,
|
|
"frames": 6 }` sets the name, the duration and how many frames go on the sheet.
|
|
|
|
Find targets: run `labels=now` after any step and read `NN-now.json`.
|
|
|
|
Look at every PNG you cite. A capture that exits 0 is not a review.
|
|
|
|
Check a chat scroll: the full-app fixture exposes `chatScrollMetrics()` (the
|
|
open transcript's `{pixels, min, max, viewport}` as JSON;
|
|
`tool/perf/chat_scroll_probe.dart`), and `?chats=N` with `?msgs=`, `?rich=1`,
|
|
`?older=1`, `?live=N` and `?tasks=N` (`tool/perf/chat_scroll_fixture.dart`) shape the
|
|
transcripts. `tool/capture/examples/chat-wheel-scroll.json` wheels over each
|
|
content kind and expects the offset to move:
|
|
|
|
```sh
|
|
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/chat-wheel \
|
|
--script tool/capture/examples/chat-wheel-scroll.json
|
|
```
|
|
|
|
For a prompt delivered to the running reply, use `--params chats=2&steered=1`,
|
|
open `/chat/perf-chat-0`, then type and send a prompt. Without `steered=1`
|
|
the fixture retains the older-server Sent acknowledgement.
|
|
|
|
## Chat file explorer
|
|
|
|
Use `--params 'chats=2&files=1' open=/chat/perf-chat-0 click=Files` to open a
|
|
populated explorer (the fixture always answers `list_directory`). `files=1`
|
|
redirects view/download URLs to same-origin static fixture files; production
|
|
continues to use `/files`. After building the probe, copy the fixtures:
|
|
|
|
```sh
|
|
mkdir -p build/probe-app/capture-files
|
|
cp tool/capture/fixtures/files/* build/probe-app/capture-files/
|
|
```
|
|
|
|
The tree has an assets subfolder, a hidden folder, PNG/JPEG miniatures, text,
|
|
JSON, CSV, PDF, archive, log and video entries. Image/text/table/archive files
|
|
have bundled bytes; PDF and video entries demonstrate type labels and menus
|
|
only. Use the real server for their external handlers and playback acceptance.
|
|
|
|
For mobile peek regression captures, `chatPaneScrollMetrics()` returns every
|
|
retained transcript's `{id,pixels,min,max,viewport,top,height}` as a JSON array.
|
|
Open a chat, use Add chat pane, then pick the other fixture chat. A wheel/swipe
|
|
over the inactive pane must change its own offset without activating it;
|
|
at its bottom `pixels` reaches `max` and its `viewport` equals the small
|
|
visible height. Use `film=key:chat-peek-<pane-id>` to record the swap.
|
|
|
|
Chat header grouping: `?chats=2&header=1` supplies a model, permission mode and thinking levels (answering `chat_effort`, tool/perf/chat_effort_fixture.dart) to the fixture chat so the selector island and Files/More island can both be reviewed.
|