Files
pipelines/app/notes/perf-4k-chat.md
T
alan 9be31b8ba0 Re-derive only the live turn's transcript rows per streamed delta
ChatTranscriptRows keeps each message run's rows while the run holds the
identical message objects; applyChatEvent and the controller keep untouched
messages' identity. Shown-item filtering, task-notice groups and row-key ids
are held per row object, and the view reads message ids once per list.
Tests check the incremental path equals a full derivation across streaming,
appends, shown items, cards, turn diffs and prepended pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-22 18:24:02 +00:00

73 lines
4.2 KiB
Markdown

# Chat performance at 4K: baseline and hotspots (2026-11-22)
`tool/perf/chat_4k_bench.mjs` drives the full-app fixture (profile build with
source maps, no server) at 3840x2160 CSS px, device scale 1, with the GPU
(`--gpu`: ANGLE on Vulkan, RTX 2080 Ti; `tool/capture/browser.mjs` now picks
Vulkan on Linux and Metal on macOS). Data: 12 chats x 150 rich messages
(code, wide tables, images, agent reports, links, turn activity, shown items,
turn diffs). Semantics stays off in the measured loads.
```sh
flutter build web --profile --source-maps \
--target tool/search_selection_probe_entry.dart --output build/probe-profile
node tool/perf/chat_4k_bench.mjs --build build/probe-profile --out /tmp/chat4k --gpu --repeat 3
```
## Baseline (median of 3; frame = Flutter FrameTiming)
| Scenario | Frames | Build p50 / p90 / max ms | >16 ms | >33 ms | Worst frame |
| --- | --- | --- | --- | --- | --- |
| open (first frame 1.4 s) | 67 | 1.9 / 9.5 / 117 | 7 | 5 | 152 ms |
| scroll 60 wheel ticks up, 60 down | 458 | 2.3 / 6.2 / 29 | 38 | 2 | 38 ms |
| stream 200 deltas (60 ms apart) | 911 | 2.9 / 6.4 / 9 | 1 | 0 | 17 ms |
| switch through 6 chats | 291 | 2.1 / 6.3 / 65 | 19 | 12 | 90 ms |
| type 74 characters | 100 | 5.2 / 5.9 / 11 | 1 | 0 | 19 ms |
Raster is cheap with a GPU (p90 under 6 ms); the UI thread is the cost.
`printString` (the `PERF_FRAME` lines) and `_reportTaskEvent` (profile-build
timeline) in the self-time lists are measurement overhead.
## Hotspots, by size
1. **Running-turn composer border** (`composer_running_border.dart:116`,
`composer_running_arc.dart:135`): 602 ms inclusive over the 15 s stream,
the largest app cost. Every animation frame rebuilds the pill's outline
path, runs `computeMetrics()` and extracts the highlight; CanvasKit's
path measuring and its wire conversions (`fromWireType`, 301 ms self)
dominate. The outline and its metrics only change with size and shape:
cache them and extract per frame. Runs for every running turn, on every
device.
2. **Transcript selection ordering while scrolling**: 241 ms in one scroll.
`MultiSelectableSelectionContainerDelegate._flushAdditions` sorts every
selectable text piece of the transcript-wide selection area by screen
position (`_compareScreenOrder` -> `getTransformTo`) as rows scroll in.
Ordering by transcript row (a delegate that compares row indices, or one
selection container per message so the top level sorts messages, not text
runs) avoids the matrix work. Main cause of the 38 slow scroll frames.
3. **Table column measuring** (`chat_table_column_width.dart`, added
2026-11-19 for wrapping tables): 142 ms (max + min intrinsic) in the
scroll, 61 ms switching chats, 21 ms opening. Each cell's paragraph is laid
out for its max width, its min width and then its real width. Caching per
cell text, or skipping the min pass when the max widths already fit, cuts
it.
4. **Per-delta transcript rebuild**: `transcriptRows` 41 ms,
`withChatShownItems` 28 ms (`_rowTime` 15 ms), `ChatRowKeys.forRows`
16 ms, `sliver_animated_extent` layout 250 ms inclusive over the stream.
Every delta re-derives all rows of the transcript; only the live message
changed. Memoise the settled rows and recompute the tail.
Change (not yet re-benchmarked): `ChatTranscriptRows`
(`chat_transcript_rows.dart`) keeps each message run's rows while it holds
the identical message objects, `applyChatEvent` keeps untouched messages'
identity, and the shown-item filter, task-notice groups and
`ChatRowKeys` ids are held per row object. Equality with a full derivation
is covered by `test/features/chat/conversation/chat_transcript_rows_memo_test.dart`.
5. **Spinner during streaming**: `CircularProgressIndicator`'s controller
getter looks up an inherited widget every frame (89 ms over the stream) on
top of the border animation; two 60 fps animations for one running turn.
6. **Switching chats**: 12 frames over 33 ms per 6 switches, worst 90 ms;
includes a first-use shader compile (`getShaderParameter` 27 ms) and the
new transcript's table measuring (item 3).
7. **Typing**: a `LayoutBuilder` subtree above the composer rebuilds on every
keystroke (166 ms over 74 keys, about 2 ms each), and the draft is written
to preferences per keystroke (3 ms total). Low priority.