Files
pipelines/app/tool/capture

Capture kit

One generic browser driver for UI review. It works on the whole app, not one feature, so a renamed label never makes it go stale. Use it instead of writing new capture code, and instead of a per-feature tool/*_probe.mjs driver that no longer matches the app.

  • capture.mjs: the CLI. It serves a build on an OS-assigned loopback port (--build) or takes --url. It starts private headless Chrome (fresh profile, debugging port chosen by the OS) or attaches to one with --cdp http://127.0.0.1:<port>. Then it runs the steps and writes NN-<name>.png and report.json to --out. On exit, Ctrl-C or failure it removes Chrome, the profile and the server.
  • It needs no npm packages: Node 22+ (global WebSocket and fetch) and Google Chrome (CHROME_PATH overrides the macOS default).

The fixture app

tool/search_selection_probe_entry.dart is the real PipelinesApp running on an in-process fixture channel. It never contacts a server. It has run 7 (three visits plus the artifact release-report.txt), run 6, pipelines 24 and 22, and search hits for release. Build it once per change, from app/:

/Users/alan/development/flutter/bin/flutter build web \
  --target tool/search_selection_probe_entry.dart --output build/probe-app

URL parameters (the kit sets them from open= and the options): route=<app route>, q=<search>, platform=material|cupertino, theme=light|dark; --params (or a step's or script's params) adds fixture options such as chats=2&rich=1. Without theme, the app follows the browser's colour scheme. --theme also sets that colour scheme through Chrome emulation.

Steps

You can pass steps on the command line as verb=value. With --script, they go in a JSON array or in { "size", "platform", "theme", "q", "steps": [...] }. In JSON, a step can also set wait (ms), timeout, name, ms and frames.

step effect
open=/runs/7[@WxH] load a route at a size (default --size, 880x1000) and turn on Flutter semantics
shot=name take a screenshot
click=<target> click the centre of the target, then wait --wait ms (default 1000)
waitfor=<target> wait up to 10 s for the target to appear
type=text type into the focused field (or the first visible one)
press=Enter press a key: Enter, Escape, Tab, Backspace, ArrowUp, ArrowDown, a letter or a digit, optionally after Control+, Shift+, Alt+ or Meta+ (press=Control+2)
scroll=600 scroll with the mouse wheel at the centre (JSON at sets a target, dx a horizontal delta, repeat and gap a run of ticks)
hover=<target> move the mouse onto the target without pressing (hover states)
drag=<target>>dx,dy press on the target, move by dx,dy in steps (12) moves gap (16) ms apart, release (JSON dx, dy, steps, gap; hold: true keeps the button down to capture mid-drag)
size=WxH resize without reloading
labels=name write the visible semantics labels, keys and rects to JSON
film=<target> screencast a click for --film-ms (900) and write the frames plus a contact sheet
wait=ms sleep
eval=<js> evaluate an expression in the page and log/record its value
expect=<js> fail the run (status 1) unless the expression is truthy; JSON name labels the check

Targets:

  • A bare Artifacts or label:Artifacts is a semantics label (aria-label or the node's text). An exact match beats a prefix match, and then the smallest visible node wins.
  • key:list-detail-body is a Semantics(identifier:). A Flutter Key is not visible in the DOM.
  • 120,340 or xy:120,340 is a point in CSS pixels.
  • eval:<js> is a point an expression in the page yields as [x, y], waited for while it yields null (e.g. eval:JSON.parse(chatScrollTargets()).table).

When a target is missing, the step fails. The kit lists the visible labels, saves NN-FAILED.png and exits with status 1.

Recipes (run from app/ after building build/probe-app)

Capture a page at phone and fold widths, in Material and in Cupertino. Use 560 for phone: headless Chrome misrenders Flutter below about 500 px.

for platform in material cupertino; do
  for size in 560x1000 880x1000; do
    node tool/capture/capture.mjs --build build/probe-app \
      --out /tmp/review/runs-$platform-$size --platform $platform --size $size \
      open=/runs/7 shot=run7
  done
done

Add --theme dark (or light) to pin the theme.

Click through to a state:

node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/artifact \
  open=/runs/7@880x1000 click=Artifacts waitfor=release-report.txt shot=artifacts \
  click=release-report.txt shot=artifact-open labels=artifact-open

To run the same flow as a script, use --script tool/capture/examples/run7-artifacts.json. Search with open=/search then type=release, or use --q release.

Film a transition. This writes the frames (fNNN_<ms>ms.png, relative to the click) to <out>/film-label-Pipelines/ and the sheet to NN-film-label-Pipelines-sheet.png:

node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/film \
  --film-ms 900 open=/runs@560x1000 film=Pipelines

In a JSON script, { "film": "Pipelines", "name": "tab-switch", "ms": 900, "frames": 6 } sets the name, the duration and how many frames go on the sheet.

Find targets: run labels=now after any step and read NN-now.json.

Look at every PNG you cite. A capture that exits 0 is not a review.

Check a chat scroll: the full-app fixture exposes chatScrollMetrics() (the open transcript's {pixels, min, max, viewport} as JSON; tool/perf/chat_scroll_probe.dart), and ?chats=N with ?msgs=, ?rich=1, ?older=1, ?live=N and ?tasks=N (tool/perf/chat_scroll_fixture.dart) shape the transcripts. tool/capture/examples/chat-wheel-scroll.json wheels over each content kind and expects the offset to move:

node tool/capture/capture.mjs --build build/probe-app --out /tmp/review/chat-wheel \
  --script tool/capture/examples/chat-wheel-scroll.json