Files
pipelines/app/notes/perf-4k-chat.md
T
alan 9be31b8ba0 Re-derive only the live turn's transcript rows per streamed delta
ChatTranscriptRows keeps each message run's rows while the run holds the
identical message objects; applyChatEvent and the controller keep untouched
messages' identity. Shown-item filtering, task-notice groups and row-key ids
are held per row object, and the view reads message ids once per list.
Tests check the incremental path equals a full derivation across streaming,
appends, shown items, cards, turn diffs and prepended pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-22 18:24:02 +00:00

4.2 KiB

Chat performance at 4K: baseline and hotspots (2026-11-22)

tool/perf/chat_4k_bench.mjs drives the full-app fixture (profile build with source maps, no server) at 3840x2160 CSS px, device scale 1, with the GPU (--gpu: ANGLE on Vulkan, RTX 2080 Ti; tool/capture/browser.mjs now picks Vulkan on Linux and Metal on macOS). Data: 12 chats x 150 rich messages (code, wide tables, images, agent reports, links, turn activity, shown items, turn diffs). Semantics stays off in the measured loads.

flutter build web --profile --source-maps \
  --target tool/search_selection_probe_entry.dart --output build/probe-profile
node tool/perf/chat_4k_bench.mjs --build build/probe-profile --out /tmp/chat4k --gpu --repeat 3

Baseline (median of 3; frame = Flutter FrameTiming)

Scenario Frames Build p50 / p90 / max ms >16 ms >33 ms Worst frame
open (first frame 1.4 s) 67 1.9 / 9.5 / 117 7 5 152 ms
scroll 60 wheel ticks up, 60 down 458 2.3 / 6.2 / 29 38 2 38 ms
stream 200 deltas (60 ms apart) 911 2.9 / 6.4 / 9 1 0 17 ms
switch through 6 chats 291 2.1 / 6.3 / 65 19 12 90 ms
type 74 characters 100 5.2 / 5.9 / 11 1 0 19 ms

Raster is cheap with a GPU (p90 under 6 ms); the UI thread is the cost. printString (the PERF_FRAME lines) and _reportTaskEvent (profile-build timeline) in the self-time lists are measurement overhead.

Hotspots, by size

  1. Running-turn composer border (composer_running_border.dart:116, composer_running_arc.dart:135): 602 ms inclusive over the 15 s stream, the largest app cost. Every animation frame rebuilds the pill's outline path, runs computeMetrics() and extracts the highlight; CanvasKit's path measuring and its wire conversions (fromWireType, 301 ms self) dominate. The outline and its metrics only change with size and shape: cache them and extract per frame. Runs for every running turn, on every device.
  2. Transcript selection ordering while scrolling: 241 ms in one scroll. MultiSelectableSelectionContainerDelegate._flushAdditions sorts every selectable text piece of the transcript-wide selection area by screen position (_compareScreenOrder -> getTransformTo) as rows scroll in. Ordering by transcript row (a delegate that compares row indices, or one selection container per message so the top level sorts messages, not text runs) avoids the matrix work. Main cause of the 38 slow scroll frames.
  3. Table column measuring (chat_table_column_width.dart, added 2026-11-19 for wrapping tables): 142 ms (max + min intrinsic) in the scroll, 61 ms switching chats, 21 ms opening. Each cell's paragraph is laid out for its max width, its min width and then its real width. Caching per cell text, or skipping the min pass when the max widths already fit, cuts it.
  4. Per-delta transcript rebuild: transcriptRows 41 ms, withChatShownItems 28 ms (_rowTime 15 ms), ChatRowKeys.forRows 16 ms, sliver_animated_extent layout 250 ms inclusive over the stream. Every delta re-derives all rows of the transcript; only the live message changed. Memoise the settled rows and recompute the tail. Change (not yet re-benchmarked): ChatTranscriptRows (chat_transcript_rows.dart) keeps each message run's rows while it holds the identical message objects, applyChatEvent keeps untouched messages' identity, and the shown-item filter, task-notice groups and ChatRowKeys ids are held per row object. Equality with a full derivation is covered by test/features/chat/conversation/chat_transcript_rows_memo_test.dart.
  5. Spinner during streaming: CircularProgressIndicator's controller getter looks up an inherited widget every frame (89 ms over the stream) on top of the border animation; two 60 fps animations for one running turn.
  6. Switching chats: 12 frames over 33 ms per 6 switches, worst 90 ms; includes a first-use shader compile (getShaderParameter 27 ms) and the new transcript's table measuring (item 3).
  7. Typing: a LayoutBuilder subtree above the composer rebuilds on every keystroke (166 ms over 74 keys, about 2 ms each), and the draft is written to preferences per keystroke (3 ms total). Low priority.