Capture kit
One generic browser driver for UI review. It works on the whole app, not one
feature, so a renamed label never makes it go stale. Use it instead of writing
new capture code, and instead of a per-feature tool/*_probe.mjs driver that
no longer matches the app.
capture.mjs: the CLI. It serves a build on an OS-assigned loopback port (--build) or takes--url. It starts private headless Chrome (fresh profile, debugging port chosen by the OS) or attaches to one with--cdp http://127.0.0.1:<port>. Then it runs the steps and writesNN-<name>.pngandreport.jsonto--out. On exit, Ctrl-C or failure it removes Chrome, the profile and the server.- It needs no npm packages: Node 22+ (global
WebSocketandfetch) and Google Chrome (CHROME_PATHoverrides the macOS default).
The fixture app
tool/search_selection_probe_entry.dart is the real PipelinesApp running on
an in-process fixture channel. It never contacts a server. It has run 7 (three
visits plus the artifact release-report.txt), run 6, pipelines 24 and 22, and
search hits for release. Build it once per change, from app/:
/Users/alan/development/flutter/bin/flutter build web \
--target tool/search_selection_probe_entry.dart --output build/probe-app
URL parameters (the kit sets them from open= and the options):
route=<app route>, q=<search>, platform=material|cupertino,
theme=light|dark; --params (or a step's or script's params) adds fixture
options such as chats=2&rich=1. Without theme, the app follows the browser's colour
scheme. --theme also sets that colour scheme through Chrome emulation.
Steps
You can pass steps on the command line as verb=value. With --script, they
go in a JSON array or in { "size", "platform", "theme", "q", "steps": [...] }.
In JSON, a step can also set wait (ms), timeout, name, ms and frames.
| step | effect |
|---|---|
open=/runs/7[@WxH] |
load a route at a size (default --size, 880x1000) and turn on Flutter semantics |
shot=name |
take a screenshot |
click=<target> |
click the centre of the target, then wait --wait ms (default 1000) |
waitfor=<target> |
wait up to 10 s for the target to appear |
type=text |
type into the focused field (or the first visible one) |
press=Enter |
press a key: Enter, Escape, Tab, Backspace, ArrowUp, ArrowDown, a letter or a digit, optionally after Control+, Shift+, Alt+ or Meta+ (press=Control+2) |
scroll=600 |
scroll with the mouse wheel at the centre (JSON at sets a target, dx a horizontal delta, repeat and gap a run of ticks) |
hover=<target> |
move the mouse onto the target without pressing (hover states) |
drag=<target>>dx,dy |
press on the target, move by dx,dy in steps (12) moves gap (16) ms apart, release (JSON dx, dy, steps, gap; hold: true keeps the button down to capture mid-drag) |
size=WxH |
resize without reloading |
labels=name |
write the visible semantics labels, keys and rects to JSON |
film=<target> |
screencast a click for --film-ms (900) and write the frames plus a contact sheet |
wait=ms |
sleep |
swipe=<target>>dx,dy |
one-finger touch swipe (needs --touch true, which emulates a touch screen once semantics is on) |
pinch=<target>>scale |
two-finger touch pinch, spreading (>1) or closing (<1) (needs --touch true) |
tabs=name |
write the open tabs ({url, type}) to JSON, e.g. to see a link open a new tab |
eval=<js> |
evaluate an expression in the page and log/record its value |
expect=<js> |
fail the run (status 1) unless the expression is truthy; JSON name labels the check |
Targets:
- A bare
Artifactsorlabel:Artifactsis a semantics label (aria-labelor the node's text). An exact match beats a prefix match, and then the smallest visible node wins. key:list-detail-bodyis aSemantics(identifier:). A FlutterKeyis not visible in the DOM.120,340orxy:120,340is a point in CSS pixels.eval:<js>is a point an expression in the page yields as[x, y], waited for while it yields null (e.g.eval:JSON.parse(chatScrollTargets()).table).
When a target is missing, the step fails. The kit lists the visible labels,
saves NN-FAILED.png and exits with status 1.
An HTML artifact: --params html=1 adds release-dashboard.html to run 7.
Its bytes are served from the build, so copy the fixture in after building:
mkdir -p build/probe-app/artifacts/7/2 && cp tool/capture/fixtures/release-dashboard.html build/probe-app/artifacts/7/2/.
Recipes (run from app/ after building build/probe-app)
Capture a page at phone and fold widths, in Material and in Cupertino. Use 560 for phone: headless Chrome misrenders Flutter below about 500 px.
for platform in material cupertino; do
for size in 560x1000 880x1000; do
node tool/capture/capture.mjs --build build/probe-app \
--out /tmp/review/runs-$platform-$size --platform $platform --size $size \
open=/runs/7 shot=run7
done
done
Add --theme dark (or light) to pin the theme.
Click through to a state:
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/artifact \
open=/runs/7@880x1000 click=Artifacts waitfor=release-report.txt shot=artifacts \
click=release-report.txt shot=artifact-open labels=artifact-open
To run the same flow as a script, use --script tool/capture/examples/run7-artifacts.json.
Search with open=/search then type=release, or use --q release.
Film a transition. This writes the frames (fNNN_<ms>ms.png, relative to the
click) to <out>/film-label-Pipelines/ and the sheet to
NN-film-label-Pipelines-sheet.png:
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/film \
--film-ms 900 open=/runs@560x1000 film=Pipelines
In a JSON script, { "film": "Pipelines", "name": "tab-switch", "ms": 900, "frames": 6 } sets the name, the duration and how many frames go on the sheet.
Find targets: run labels=now after any step and read NN-now.json.
Look at every PNG you cite. A capture that exits 0 is not a review.
Check a chat scroll: the full-app fixture exposes chatScrollMetrics() (the
open transcript's {pixels, min, max, viewport} as JSON;
tool/perf/chat_scroll_probe.dart), and ?chats=N with ?msgs=, ?rich=1,
?older=1, ?live=N and ?tasks=N (tool/perf/chat_scroll_fixture.dart) shape the
transcripts. tool/capture/examples/chat-wheel-scroll.json wheels over each
content kind and expects the offset to move:
node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/chat-wheel \
--script tool/capture/examples/chat-wheel-scroll.json