Files
pipelines/app/notes/chat-stream-perf.md
alan 20bf6202b3 Rebuild only the live chat turn while replies stream
Reuse unchanged transcript rows, the composer bar and the header across
streamed deltas, and share parsed markdown by text so rows scrolled back
into view are not parsed again. Adds a rebuild/parse harness and a
release-web frame-timing probe with a long-transcript fixture.
2026-11-01 23:59:40 +00:00

2.6 KiB
Raw Permalink Blame History

Chat thread streaming performance

Measurement-led round on the chat thread while replies stream and while long transcripts scroll. No behaviour change.

Changes

  • ChatRowWidgets (lib/features/chat/conversation/chat_row_widgets.dart) hands back a row's previous ChatMessageView while the row deep-equals the one it was built from (and delivery state, platform style, remove action, images, controller and composer are unchanged). Snapshots are copy-on-write, so Flutter skips every finished row's subtree. Rows with permission chips always rebuild, because those objects compare by identity.
  • The composer bar and the floating header are reused the same way while the thread widget, running state and latest reply model hold.
  • ChatMarkdownCache (lib/features/chat/ui/transcript/chat_markdown_cache.dart) shares parsed blocks by text, bounded to 400 entries and 1M characters, so a row scrolled back into view is not parsed again.

Already in place and left unchanged: ListView(children:) lays out lazily (SliverChildListDelegate) with a RepaintBoundary and a stable key for each row. Thumbnails decode at display size (ResizeImage). Activity timers run only on the streaming turn.

Numbers

Widget harness test/features/chat/perf/chat_stream_rebuild_perf_test.dart streams 200 deltas into a 300-message transcript (debug VM, so compare the numbers with each other only):

before after
ChatMessageView builds 856 200
all widget builds 83,888 20,018
markdown parses while streaming 200 200 (live turn only)
frame time per delta (mean / p90) 7.3 / 10.8 ms 4.8 / 5.7 ms
markdown parses, 80 scroll drags 43 0

Release web probe node tool/chat_perf_probe.mjs <build> 300 200 (built from tool/chat_composer_probe_entry.dart; headless Chrome renders in software). Three runs of each, Flutter FrameTiming:

before after
streaming build mean 9.1–10.5 ms 6.0–6.4 ms
streaming build p90 14.6–17.4 ms 7.2–8.5 ms
streaming total frame mean 67–72 ms 45–50 ms
scrolling build mean 2.5–2.8 ms 1.6–2.8 ms (within noise)

Findings left open

  • Opening a 300-message transcript builds every row once: jumping to the end of a top-anchored SliverList has to lay out each row in between (~150 parses, about 0.9 s in the debug harness). A bottom-anchored (reversed) list would avoid that, but it changes the scroll-restore and follow logic, so it needs a separate change.
  • The fixture logs a _PlatformViewPlaceholderBox.getTransformTo null-check error on web with a long transcript. It also happens before this change.