69ae0519bd
Keep capture and probe tooling on the app's single Material surface by removing obsolete platform switches, hosts, and documentation. Preserve the existing Material routes and fixture behavior while retaining compatibility APIs outside this lane.
167 lines
8.6 KiB
Markdown
167 lines
8.6 KiB
Markdown
# Capture kit
|
|
|
|
One generic browser driver for UI review. It works on the whole app, not one
|
|
feature, so a renamed label never makes it go stale. Use it instead of writing
|
|
new capture code, and instead of a per-feature `tool/*_probe.mjs` driver that
|
|
no longer matches the app.
|
|
|
|
- `capture.mjs`: the CLI. It serves a build on an OS-assigned loopback port
|
|
(`--build`) or takes `--url`. It starts private headless Chrome (fresh
|
|
profile, debugging port chosen by the OS) or attaches to one with
|
|
`--cdp http://127.0.0.1:<port>`. Then it runs the steps and writes
|
|
`NN-<name>.png` and `report.json` to `--out`. On exit, Ctrl-C or failure it
|
|
removes Chrome, the profile and the server.
|
|
- It needs no npm packages: Node 22+ (global `WebSocket` and `fetch`) and
|
|
Google Chrome (`CHROME_PATH` overrides the macOS default).
|
|
|
|
## The fixture app
|
|
|
|
`tool/search_selection_probe_entry.dart` is the real `PipelinesApp` running on
|
|
an in-process fixture channel. It never contacts a server. It has run 7 (three
|
|
visits plus the artifact `release-report.txt`), run 6, pipelines 24 and 22, and
|
|
search hits for `release`. It also answers `server_stats`, so the Settings route
|
|
can be captured before the server command is available. Build it once per
|
|
change, from `app/`:
|
|
|
|
```sh
|
|
/Users/alan/development/flutter/bin/flutter build web \
|
|
--target tool/search_selection_probe_entry.dart --output build/probe-app
|
|
```
|
|
|
|
URL parameters (the kit sets them from `open=` and the options):
|
|
`route=<app route>`, `q=<search>`, `theme=light|dark`; `--params` (or a step's
|
|
or script's `params`) adds fixture
|
|
options such as `chats=2&rich=1`. Use `route=/settings` to open Settings and then
|
|
click `Server health`; the fixture channel returns `server_stats` automatically.
|
|
Without `theme`, the app follows the browser's colour
|
|
scheme. `--theme` also sets that colour scheme through Chrome emulation.
|
|
|
|
## Steps
|
|
|
|
You can pass steps on the command line as `verb=value`. With `--script`, they
|
|
go in a JSON array or in `{ "size", "theme", "q", "steps": [...] }`.
|
|
In JSON, a step can also set `wait` (ms), `timeout`, `name`, `ms` and `frames`.
|
|
|
|
| step | effect |
|
|
| --- | --- |
|
|
| `open=/runs/7[@WxH]` | load a route at a size (default `--size`, 880x1000) and turn on Flutter semantics |
|
|
| `shot=name` | take a screenshot |
|
|
| `click=<target>` | click the centre of the target, then wait `--wait` ms (default 1000) |
|
|
| `waitfor=<target>` | wait up to 10 s for the target to appear |
|
|
| `type=text` | type into the focused field (or the first visible one) |
|
|
| `press=Enter` | press a key: Enter, Escape, Tab, Backspace, ArrowUp, ArrowDown, a letter or a digit, optionally after `Control+`, `Shift+`, `Alt+` or `Meta+` (`press=Control+2`) |
|
|
| `scroll=600` | scroll with the mouse wheel at the centre (JSON `at` sets a target, `dx` a horizontal delta, `repeat` and `gap` a run of ticks) |
|
|
| `hover=<target>` | move the mouse onto the target without pressing (hover states) |
|
|
| `drag=<target>>dx,dy` | press on the target, move by dx,dy in `steps` (12) moves `gap` (16) ms apart, release (JSON `dx`, `dy`, `steps`, `gap`; `hold: true` keeps the button down to capture mid-drag) |
|
|
| `size=WxH` | resize without reloading |
|
|
| `labels=name` | write the visible semantics labels, keys and rects to JSON |
|
|
| `film=<target>` | screencast a click for `--film-ms` (900) and write the frames plus a contact sheet; `film=press:Alt+t` films a key chord (as `press=`) instead |
|
|
| `wait=ms` | sleep |
|
|
| `files=a.ttf[,b.ttf]` | answer the next file chooser the page opens with these files (paths relative to the working directory); put it before the click that opens the picker |
|
|
| `swipe=<target>>dx,dy` | one-finger touch swipe (needs `--touch true`, which emulates a touch screen once semantics is on) |
|
|
| `pinch=<target>>scale` | two-finger touch pinch, spreading (>1) or closing (<1) (needs `--touch true`) |
|
|
| `tabs=name` | write the open tabs (`{url, type}`) to JSON, e.g. to see a link open a new tab |
|
|
| `eval=<js>` | evaluate an expression in the page and log/record its value |
|
|
| `expect=<js>` | fail the run (status 1) unless the expression is truthy; JSON `name` labels the check |
|
|
|
|
Targets:
|
|
- A bare `Artifacts` or `label:Artifacts` is a semantics label (`aria-label` or
|
|
the node's text). An exact match beats a prefix match, and then the smallest
|
|
visible node wins.
|
|
- `key:list-detail-body` is a `Semantics(identifier:)`. A Flutter `Key` is not
|
|
visible in the DOM.
|
|
- `120,340` or `xy:120,340` is a point in CSS pixels.
|
|
- `eval:<js>` is a point an expression in the page yields as `[x, y]`, waited
|
|
for while it yields null (e.g. `eval:JSON.parse(chatScrollTargets()).table`).
|
|
|
|
When a target is missing, the step fails. The kit lists the visible labels,
|
|
saves `NN-FAILED.png` and exits with status 1.
|
|
|
|
An HTML artifact: `--params html=1` adds `release-dashboard.html` to run 7.
|
|
Its bytes are served from the build, so copy the fixture in after building:
|
|
`mkdir -p build/probe-app/artifacts/7/2 && cp tool/capture/fixtures/release-dashboard.html build/probe-app/artifacts/7/2/`.
|
|
|
|
## Recipes (run from `app/` after building `build/probe-app`)
|
|
|
|
Capture a page at phone and fold widths. Use 560 for phone: headless Chrome
|
|
misrenders Flutter below about 500 px. The app has one Material UI. The fixture
|
|
is Material-only on every platform.
|
|
|
|
```sh
|
|
for size in 560x1000 880x1000; do
|
|
node tool/capture/capture.mjs --build build/probe-app \
|
|
--out /tmp/review/runs-$size --size $size open=/runs/7 shot=run7
|
|
done
|
|
```
|
|
|
|
Add `--theme dark` (or `light`) to pin the theme.
|
|
|
|
Click through to a state:
|
|
|
|
```sh
|
|
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/artifact \
|
|
open=/runs/7@880x1000 click=Artifacts waitfor=release-report.txt shot=artifacts \
|
|
click=release-report.txt shot=artifact-open labels=artifact-open
|
|
```
|
|
|
|
To run the same flow as a script, use `--script tool/capture/examples/run7-artifacts.json`.
|
|
Search with `open=/search` then `type=release`, or use `--q release`.
|
|
|
|
Film a transition. This writes the frames (`fNNN_<ms>ms.png`, relative to the
|
|
click) to `<out>/film-label-Pipelines/` and the sheet to
|
|
`NN-film-label-Pipelines-sheet.png`:
|
|
|
|
```sh
|
|
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/film \
|
|
--film-ms 900 open=/runs@560x1000 film=Pipelines
|
|
```
|
|
|
|
In a JSON script, `{ "film": "Pipelines", "name": "tab-switch", "ms": 900,
|
|
"frames": 6 }` sets the name, the duration and how many frames go on the sheet.
|
|
|
|
Find targets: run `labels=now` after any step and read `NN-now.json`.
|
|
|
|
Look at every PNG you cite. A capture that exits 0 is not a review.
|
|
|
|
Check a chat scroll: the full-app fixture exposes `chatScrollMetrics()` (the
|
|
open transcript's `{pixels, min, max, viewport}` as JSON;
|
|
`tool/perf/chat_scroll_probe.dart`), and `?chats=N` with `?msgs=`, `?rich=1`,
|
|
`?older=1`, `?live=N` and `?tasks=N` (`tool/perf/chat_scroll_fixture.dart`) shape the
|
|
transcripts. `tool/capture/examples/chat-wheel-scroll.json` wheels over each
|
|
content kind and expects the offset to move:
|
|
|
|
```sh
|
|
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/chat-wheel \
|
|
--script tool/capture/examples/chat-wheel-scroll.json
|
|
```
|
|
|
|
For a prompt delivered to the running reply, use `--params chats=2&steered=1`,
|
|
open `/chat/perf-chat-0`, then type and send a prompt. Without `steered=1`
|
|
the fixture retains the older-server Sent acknowledgement.
|
|
|
|
## Chat file explorer
|
|
|
|
Use `--params 'chats=2&files=1' open=/chat/perf-chat-0 click=Files` to open a
|
|
populated explorer (the fixture always answers `list_directory`). `files=1`
|
|
redirects view/download URLs to same-origin static fixture files; production
|
|
continues to use `/files`. After building the probe, copy the fixtures:
|
|
|
|
```sh
|
|
mkdir -p build/probe-app/capture-files
|
|
cp tool/capture/fixtures/files/* build/probe-app/capture-files/
|
|
```
|
|
|
|
The tree has an assets subfolder, a hidden folder, PNG/JPEG miniatures, text,
|
|
JSON, CSV, PDF, archive, log and video entries. Image/text/table/archive files
|
|
have bundled bytes; PDF and video entries demonstrate type labels and menus
|
|
only. Use the real server for their external handlers and playback acceptance.
|
|
|
|
For mobile peek regression captures, `chatPaneScrollMetrics()` returns every
|
|
retained transcript's `{id,pixels,min,max,viewport,top,height}` as a JSON array.
|
|
Open a chat, use Add chat pane, then pick the other fixture chat. A wheel/swipe
|
|
over the inactive pane must change its own offset without activating it;
|
|
at its bottom `pixels` reaches `max` and its `viewport` equals the small
|
|
visible height. Use `film=key:chat-peek-<pane-id>` to record the swap.
|
|
|
|
Chat header grouping: `?chats=2&header=1` supplies a model, permission mode and thinking levels (answering `chat_effort`, tool/perf/chat_effort_fixture.dart) to the fixture chat so the selector island and Files/More island can both be reviewed.
|