ChatScreen keeps a bounded in-memory cache of recently left transcripts (16 chats / 4000 messages, LRU). A reopened chat is seeded from it with no loading state, still re-subscribes and re-reads, and the newest page replaces the cached copy while live frames received during the read are kept. /close removes the entry; only transcripts backed by a successful read on the current connection are written. Adds tool/chat_reopen_probe.mjs and an optional history delay in the loopback fixture. Measurement shows a second cost the cache does not remove: opening at the end of the top-anchored transcript lays out every row (notes).
4.3 KiB
Chat transcript cache
Reported problem: going list -> chat -> Back -> same or another chat reloads the whole conversation for a few seconds each time.
What it does
ChatScreen owns a ChatTranscriptCache (lib/features/chat/conversation/chat_transcript_cache.dart)
next to its scroll store. It is in memory only and lasts as long as the
destination. It is bounded to 16 conversations and 4,000 messages in total.
The least recently used entry is evicted first, and a transcript too large for
the whole bound is not kept.
- Leaving a conversation (phone page pop, wide pane switch, destination teardown)
writes the controller's transcript, cursor and
has_more. It is written only if a read on the current connection succeeded. After a socket gap that has not been repaired, or after a refused read, the earlier entry is kept unchanged. - Reopening seeds
ChatThreadControllerwith that entry, so there is no loading state and the saved scroll position is applied to it immediately. The controller still runschat_subscribeand thenchat_messages. refreshCachedChatHistorymerges the newest page. The page decides which cached messages still exist and supplies its copies and cursor. Messages that a live frame or local send touched during the read are kept after the page, so no frame is dropped. Until a read or frame replaces the cached transcript, the view treats it like a pre-gap transcript: it shows no running turn or compaction state drawn from it./closeremoves the entry and the closed view never writes it back. A gateway change clears the cache. Rename and model switch do not affect the transcript: their row data stays in_openedSessionsand_acceptedModels. A socket replacement needs no invalidation, because every visit re-reads.
Measured cause of the delay
tool/chat_reopen_probe.mjs serves the built app against the loopback fixture
(100 user/assistant pairs, 390x844, Android UA). HISTORY_DELAY_MS holds every
chat_messages reply to stand in for a slow server read. These are controlled
samples, not device timings.
| 1.5 s read delay | Reopen, before | Reopen, after |
|---|---|---|
| 1x CPU: transcript in semantics (visits 2/3) | 1,661 / 1,641 ms | 550 / 414 ms |
| 4x CPU: frame 250 ms after tap | loading spinner | cached transcript |
Two costs add up:
- The history read gates the first paint. Every visit waits for it, and the real server projects the native transcript with tool state. The cache removes this cost from reopen.
- Rendering is itself slow for a long transcript. Opening at the end of the
top-anchored lazy
ListViewbuilds and lays out every row above the end (101 first builds of the markdown view for 100 turns, with only 2 kept). The probe recorded one main-thread long task per open of about 250-390 ms at 1x CPU and about 1-3 s at 4x CPU. Markdown parsing is not the cost: parsing 100 answers takes about 5 ms. The cache does not remove this cost. With the cache it happens when the page is pushed instead of when the reply arrives.
The real fix for (2) is a bottom-anchored transcript, for example a
CustomScrollView with a center sliver at the newest row. Opening would then
lay out only the visible tail. That rewrites follow, restore, older-page and
jump-control geometry, which about 66 chat geometry assertions cover, and it
changes how a short chat is aligned. It is left as a separate item rather than
mixed into this one.
Evidence
- Probe:
HISTORY_DELAY_MS=1500 PROBE_PLATFORM=android CPU_RATE=4 node tool/chat_reopen_probe.mjs <web build> <out>.runs_probe_server.mjsgained an optionalchatHistoryDelayMs/onChatHistory. Its default replies immediately, as before. - Captures:
build/reopen-before-d1500/visit-3-early.png(spinner) andbuild/reopen-after-d1500/visit-3-early.png(cached transcript), plusresult.jsonin both directories and inbuild/reopen-{before,after}-cpu1/. - Tests:
test/features/chat/conversation/chat_transcript_cache_test.dart(bounds, merge, seeded controller, live frame during the read, refused read, gap),test/features/chat/chat_transcript_reopen_test.dart(phone page and wide pane), a cached restore inchat_thread_scroll_test.dart, and/closeinchat_command_session_test.dart. - Not verified on a device. No APK, no deployment.