0a19ef229f
ChatScreen keeps a bounded in-memory cache of recently left transcripts (16 chats / 4000 messages, LRU). A reopened chat is seeded from it with no loading state, still re-subscribes and re-reads, and the newest page replaces the cached copy while live frames received during the read are kept. /close removes the entry; only transcripts backed by a successful read on the current connection are written. Adds tool/chat_reopen_probe.mjs and an optional history delay in the loopback fixture. Measurement shows a second cost the cache does not remove: opening at the end of the top-anchored transcript lays out every row (notes).
79 lines
4.3 KiB
Markdown
79 lines
4.3 KiB
Markdown
# Chat transcript cache
|
|
|
|
Reported problem: going list -> chat -> Back -> same or another chat reloads the
|
|
whole conversation for a few seconds each time.
|
|
|
|
## What it does
|
|
|
|
`ChatScreen` owns a `ChatTranscriptCache` (`lib/features/chat/conversation/chat_transcript_cache.dart`)
|
|
next to its scroll store. It is in memory only and lasts as long as the
|
|
destination. It is bounded to 16 conversations and 4,000 messages in total.
|
|
The least recently used entry is evicted first, and a transcript too large for
|
|
the whole bound is not kept.
|
|
|
|
- Leaving a conversation (phone page pop, wide pane switch, destination teardown)
|
|
writes the controller's transcript, cursor and `has_more`. It is written only
|
|
if a read on the current connection succeeded. After a socket gap that has not
|
|
been repaired, or after a refused read, the earlier entry is kept unchanged.
|
|
- Reopening seeds `ChatThreadController` with that entry, so there is no
|
|
loading state and the saved scroll position is applied to it immediately. The
|
|
controller still runs `chat_subscribe` and then `chat_messages`.
|
|
- `refreshCachedChatHistory` merges the newest page. The page decides which
|
|
cached messages still exist and supplies its copies and cursor. Messages that
|
|
a live frame or local send touched during the read are kept after the page,
|
|
so no frame is dropped. Until a read or frame replaces the cached transcript,
|
|
the view treats it like a pre-gap transcript: it shows no running turn or
|
|
compaction state drawn from it.
|
|
- `/close` removes the entry and the closed view never writes it back. A
|
|
gateway change clears the cache. Rename and model switch do not affect the
|
|
transcript: their row data stays in `_openedSessions` and `_acceptedModels`.
|
|
A socket replacement needs no invalidation, because every visit re-reads.
|
|
|
|
## Measured cause of the delay
|
|
|
|
`tool/chat_reopen_probe.mjs` serves the built app against the loopback fixture
|
|
(100 user/assistant pairs, 390x844, Android UA). `HISTORY_DELAY_MS` holds every
|
|
`chat_messages` reply to stand in for a slow server read. These are controlled
|
|
samples, not device timings.
|
|
|
|
| 1.5 s read delay | Reopen, before | Reopen, after |
|
|
| --- | ---: | ---: |
|
|
| 1x CPU: transcript in semantics (visits 2/3) | 1,661 / 1,641 ms | 550 / 414 ms |
|
|
| 4x CPU: frame 250 ms after tap | loading spinner | cached transcript |
|
|
|
|
Two costs add up:
|
|
|
|
1. The history read gates the first paint. Every visit waits for it, and the
|
|
real server projects the native transcript with tool state. The cache
|
|
removes this cost from reopen.
|
|
2. Rendering is itself slow for a long transcript. Opening at the end of the
|
|
top-anchored lazy `ListView` builds and lays out every row above the end
|
|
(101 first builds of the markdown view for 100 turns, with only 2 kept).
|
|
The probe recorded one main-thread long task per open of about 250-390 ms at
|
|
1x CPU and about 1-3 s at 4x CPU. Markdown parsing is not the cost: parsing
|
|
100 answers takes about 5 ms. **The cache does not remove this cost.** With
|
|
the cache it happens when the page is pushed instead of when the reply
|
|
arrives.
|
|
|
|
The real fix for (2) is a bottom-anchored transcript, for example a
|
|
`CustomScrollView` with a center sliver at the newest row. Opening would then
|
|
lay out only the visible tail. That rewrites follow, restore, older-page and
|
|
jump-control geometry, which about 66 chat geometry assertions cover, and it
|
|
changes how a short chat is aligned. It is left as a separate item rather than
|
|
mixed into this one.
|
|
|
|
## Evidence
|
|
|
|
- Probe: `HISTORY_DELAY_MS=1500 PROBE_PLATFORM=android CPU_RATE=4 node tool/chat_reopen_probe.mjs <web build> <out>`.
|
|
`runs_probe_server.mjs` gained an optional `chatHistoryDelayMs`/`onChatHistory`.
|
|
Its default replies immediately, as before.
|
|
- Captures: `build/reopen-before-d1500/visit-3-early.png` (spinner) and
|
|
`build/reopen-after-d1500/visit-3-early.png` (cached transcript), plus
|
|
`result.json` in both directories and in `build/reopen-{before,after}-cpu1/`.
|
|
- Tests: `test/features/chat/conversation/chat_transcript_cache_test.dart`
|
|
(bounds, merge, seeded controller, live frame during the read, refused read,
|
|
gap), `test/features/chat/chat_transcript_reopen_test.dart` (phone page and
|
|
wide pane), a cached restore in `chat_thread_scroll_test.dart`, and `/close`
|
|
in `chat_command_session_test.dart`.
|
|
- Not verified on a device. No APK, no deployment.
|