Files
pipelines/app/notes/chat-stream-perf.md
T
alan 20bf6202b3 Rebuild only the live chat turn while replies stream
Reuse unchanged transcript rows, the composer bar and the header across
streamed deltas, and share parsed markdown by text so rows scrolled back
into view are not parsed again. Adds a rebuild/parse harness and a
release-web frame-timing probe with a long-transcript fixture.
2026-11-01 23:59:40 +00:00

59 lines
2.6 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Chat thread streaming performance
Measurement-led round on the chat thread while replies stream and while long
transcripts scroll. No behaviour change.
## Changes
- `ChatRowWidgets` (`lib/features/chat/conversation/chat_row_widgets.dart`)
hands back a row's previous `ChatMessageView` while the row deep-equals the
one it was built from (and delivery state, platform style, remove action,
images, controller and composer are unchanged). Snapshots are copy-on-write,
so Flutter skips every finished row's subtree. Rows with permission chips
always rebuild, because those objects compare by identity.
- The composer bar and the floating header are reused the same way while the
thread widget, running state and latest reply model hold.
- `ChatMarkdownCache` (`lib/features/chat/ui/transcript/chat_markdown_cache.dart`)
shares parsed blocks by text, bounded to 400 entries and 1M characters, so a
row scrolled back into view is not parsed again.
Already in place and left unchanged: `ListView(children:)` lays out lazily
(SliverChildListDelegate) with a `RepaintBoundary` and a stable key for each row.
Thumbnails decode at display size (`ResizeImage`). Activity timers run only on
the streaming turn.
## Numbers
Widget harness `test/features/chat/perf/chat_stream_rebuild_perf_test.dart`
streams 200 deltas into a 300-message transcript (debug VM, so compare the
numbers with each other only):
| | before | after |
| --- | --- | --- |
| ChatMessageView builds | 856 | 200 |
| all widget builds | 83,888 | 20,018 |
| markdown parses while streaming | 200 | 200 (live turn only) |
| frame time per delta (mean / p90) | 7.3 / 10.8 ms | 4.8 / 5.7 ms |
| markdown parses, 80 scroll drags | 43 | 0 |
Release web probe `node tool/chat_perf_probe.mjs <build> 300 200` (built from
`tool/chat_composer_probe_entry.dart`; headless Chrome renders in software).
Three runs of each, Flutter FrameTiming:
| | before | after |
| --- | --- | --- |
| streaming build mean | 9.1–10.5 ms | 6.0–6.4 ms |
| streaming build p90 | 14.6–17.4 ms | 7.2–8.5 ms |
| streaming total frame mean | 67–72 ms | 45–50 ms |
| scrolling build mean | 2.5–2.8 ms | 1.6–2.8 ms (within noise) |
## Findings left open
- Opening a 300-message transcript builds every row once: jumping to the end of
a top-anchored `SliverList` has to lay out each row in between (~150 parses,
about 0.9 s in the debug harness). A bottom-anchored (reversed) list would
avoid that, but it changes the scroll-restore and follow logic, so it needs a
separate change.
- The fixture logs a `_PlatformViewPlaceholderBox.getTransformTo` null-check
error on web with a long transcript. It also happens before this change.