test(app): benchmark populated chat Back and document UX review

This commit is contained in:
2026-10-24 11:24:47 +01:00
parent 877c2f1acd
commit ac75da12e4
6 changed files with 197 additions and 3 deletions
+3
View File
@@ -111,6 +111,9 @@ class _AdaptiveShellState extends State<AdaptiveShell> {
@override
Widget build(BuildContext context) {
// Full-screen detail routes cover the shell themselves. Keep the root tabs
// mounted/layout-ready underneath, so Back does not resize or reveal them
// only after the outgoing page has finished its native animation.
final index = AppDestination.values.indexOf(_selected);
final details = destinationDetails(_selected);
final contents = AppDestination.values.map((destination) {
+55
View File
@@ -0,0 +1,55 @@
# UI/UX review: chat Back latency and save feedback
User requested another navigation/animation/feedback review, specifically chat
Back taking several seconds. No production chats/runs or personal devices used.
| Finding | Change/source | Focused verification | Landing | Delivery |
| --- | --- | --- | --- | --- |
| Unchanged answer parts reparsed/rebuilt during streaming | da44001a caches each Markdown render subtree until text or inherited appearance changes; link callbacks remain current | Deterministic reuse regression failed before fix; text/theme/link tests pass; selection element retained | Main | Pending |
| Usage-only updates rebuild transcript | 6d623197 separates transcript invalidation from usage/Info notifications | Both-host regression failed before fix; Info still updates live | Main | Pending |
| Open phone conversation becomes unread | 399d520c separates active reading identity from underlying list chrome | Both-host regression failed before fix; other chats still become unread | Main | Pending |
| Tabs are unmounted/revealed only after a detail finishes closing | 877c2f1a keeps the native destination mounted under its pushed route | Real Navigator regression; detail covers tabs, Back immediately reveals laid-out destination | Main | Pending |
| Server save closes editor before persistence succeeds | 25437b31 keeps draft/error/retry in editor, shows saving state, prevents duplicate sends | Agent: 88 tests, four pre-fix failures; 12 browser checks/6 captures across both hosts | Merged main | Pending |
## Measured Back behavior
`tool/chat_back_latency_probe.mjs` serves the normal built app against the
loopback protocol fixture. Android user-agent selects Flutter's actual Android
theme/transition. Chrome CPU throttling is 4x, viewport 390x844. Metrics measure
from before pointer dispatch to the first frame reporting the returned chat-list
route. Route completion/ownership semantics are unchanged between builds.
| Controlled scenario | Before route settled | After route settled | Before/after maximum frame gap |
| --- | ---: | ---: | ---: |
| 100 normal user/assistant pairs, quiet | 563 ms | 531 ms | 100 / 33 ms |
| 100 assistant messages folded into one turn, live text at 10 Hz | 1,153 ms | 563 ms | 400 / 117 ms |
| 199 assistant messages plus one user (one maximum history page), folded/live | 2,108 ms | 857 ms | 800 / 217 ms |
These are controlled browser samples, not device timings or statistical claims.
The largest case reproduced the reported multi-second delay. Initial open/settle
for that sample was about 4.05s for both builds; the change avoids repeating that
rendering work while streaming/closing rather than deferring it to opening.
- Before: installed Docker web bundle, source from prior release.
- After: `build/ux_review_web` from current source.
- Captures and raw JSON: `build/ux-review-before-android-{heavy,large-turn,max-page}/`
and matching `build/ux-review-after-android-*` directories.
- No explicit multi-second wait/network await was found in Back. Flutter's
normal Android toolbar/system-button fallback is 450ms, which cannot alone
explain the reproduced delay. Native transitions/predictive gestures retained.
- `PageRoute.completed` ownership guard retained to prevent re-push during exit.
- Optional `MAX_BACK_MS`/`MAX_FRAME_GAP_MS` probe budgets support repeatable lab
regression checks without imposing flaky wall-clock assertions on unit tests.
## Checks and limits
- 439 chat/shell/navigation tests passed, including predictive cancel/commit,
disclosure identity, paging, focus, text, drafts, follow/read position and Info.
- Full gate: clean analysis and **1,848 tests pass**, including added system
text-scaling coverage for cached Markdown.
- Settings evidence: `settings-server-editor-save-ux.md` and isolated
`tool/server_editor_save_probe.{dart,mjs}`. Root remount during successful
activation remains the root's existing behavior; no network wait added to Back.
- Native Android performance remains unverified: no ADB, device install or capture.
No transition-duration shortcuts, backend edits, workflow changes or production
mutation tests. Changes remain confined to app/.
@@ -13,7 +13,7 @@ import 'conversation/chat_thread_fixtures.dart';
class _Gateway extends FakeChatGateway {
@override
Future<JsonMap> call(String type, [JsonMap fields = const {}]) {
if (type == 'chat_sessions')
if (type == 'chat_sessions') {
return Future.value({
'chat_sessions': [
{
@@ -28,6 +28,7 @@ class _Gateway extends FakeChatGateway {
},
],
});
}
return super.call(type, fields);
}
}
@@ -4,6 +4,36 @@ import 'package:flutter_test/flutter_test.dart';
import 'package:pipelines_app/features/chat/ui/transcript/chat_markdown_view.dart';
void main() {
testWidgets('cached markdown still follows system text scaling', (
tester,
) async {
final scale = ValueNotifier<double>(1);
addTearDown(scale.dispose);
await tester.pumpWidget(
ValueListenableBuilder<double>(
valueListenable: scale,
builder: (_, value, _) => MaterialApp(
builder: (context, child) => MediaQuery(
data: MediaQuery.of(
context,
).copyWith(textScaler: TextScaler.linear(value)),
child: child!,
),
home: const Scaffold(
body: ChatMarkdownView(text: 'Scale this answer'),
),
),
),
);
final before = tester.getSize(find.text('Scale this answer')).height;
scale.value = 2;
await tester.pumpAndSettle();
expect(
tester.getSize(find.text('Scale this answer')).height,
greaterThan(before * 1.9),
);
});
testWidgets(
'unchanged markdown retains its rendered subtree across parent updates',
(tester) async {
+105
View File
@@ -0,0 +1,105 @@
// Measures normal-app Back against a populated loopback chat, never operator data.
// PROBE_PLATFORM=android CPU_RATE=4 CHAT_PAIRS=100 node tool/chat_back_latency_probe.mjs <web-build> <out>
import assert from 'node:assert/strict';
import { mkdir, mkdtemp, writeFile } from 'node:fs/promises';
import { once } from 'node:events';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { performance } from 'node:perf_hooks';
import { startFixture } from './runs_probe_server.mjs';
import { launchChrome, waitForPort, waitForPage, connect, sleep } from './pipelines_probe_cdp_lib.mjs';
import { axNodes, waitFor, capture } from './pipelines_probe_cdp_input.mjs';
const pairs = Number(process.env.CHAT_PAIRS ?? 100);
const text = '# Recorded answer\n\n' +
Array.from({ length: 12 }, (_, i) => `Paragraph ${i}: This is recorded fixture content with **formatting**, an [inert link](https://example.test), and enough text to wrap naturally on a phone.\n`).join('\n') +
'\n```dart\n' + Array.from({ length: 25 }, (_, i) => `final value${i} = ${i};`).join('\n') + '\n```\n';
const turns = Array.from({ length: pairs }, (_, i) => [
{ id: `u${i}`, role: 'user', parts: [{ type: 'text', text: `Recorded question ${i}` }] },
{ id: `a${i}`, role: 'assistant', parts: [{ id: `p${i}`, type: 'text', text }] },
]).flat();
const folded = process.env.CHAT_FOLDED === '1';
const messages = folded ? [turns[0], ...turns.filter(m => m.role === 'assistant')] : turns;
const output = resolve(process.argv[3] ?? 'build/chat-back-latency');
await mkdir(output, { recursive: true });
const fixture = await startFixture({ root: resolve(process.argv[2]), chatMessages: messages });
const profile = await mkdtemp(join(tmpdir(), 'chat-back-latency-'));
const chrome = launchChrome('/Applications/Google Chrome.app/Contents/MacOS/Google Chrome', profile, 'about:blank');
const log = { pairs, folded, streaming: process.env.CHAT_STREAM === '1', cpuRate: Number(process.env.CPU_RATE ?? 1), host: process.env.PROBE_PLATFORM ?? 'cupertino', checks: [], captures: [] };
let c;
let streamTimer;
const requests = new Set();
try {
c = await connect((await waitForPage(await waitForPort(join(profile, 'DevToolsActivePort')))).webSocketDebuggerUrl);
await c.send('Page.enable');
if (log.host === 'android') await c.send('Emulation.setUserAgentOverride', {
userAgent: 'Mozilla/5.0 (Linux; Android 16; BrowserFixture) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/155.0.0.0 Mobile Safari/537.36', platform: 'Linux armv8l',
});
if (log.host === 'material') await c.send('Emulation.setUserAgentOverride', {
userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/155.0.0.0 Safari/537.36', platform: 'Win32',
});
await c.send('Emulation.setDeviceMetricsOverride', { width: 390, height: 844, deviceScaleFactor: 1, mobile: true });
await c.send('Emulation.setCPUThrottlingRate', { rate: log.cpuRate });
const opened = performance.now();
await c.send('Page.navigate', { url: fixture.url + '/chat/ses_probechat' });
assert.ok(await waitFor(() => c.evaluate("Boolean(document.querySelector('flt-semantics-placeholder'))"), 30000));
// Mobile semantics activation requires a real pointer at the placeholder's
// centre; element.click() at (0,0) does not enable Android web semantics.
const rect = await c.evaluate("(() => { const r = document.querySelector('flt-semantics-placeholder').getBoundingClientRect(); return {x:(r.left+r.right)/2,y:(r.top+r.bottom)/2}; })()");
await c.send('Input.dispatchMouseEvent', { type: 'mousePressed', x: rect.x, y: rect.y, button: 'left', clickCount: 1 });
await c.send('Input.dispatchMouseEvent', { type: 'mouseReleased', x: rect.x, y: rect.y, button: 'left', clickCount: 1 });
assert.ok(await waitFor(async () => (await axNodes(c)).some(n => n.name?.value === 'Back'), 45000), 'chat Back ready');
await sleep(1800);
await c.evaluate('new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))');
log.openAndSettleMs = performance.now() - opened;
await capture(c, output, log, 'chat-before-back');
if (log.streaming) {
streamTimer = setInterval(() => {
const request = fetch(fixture.url + '/fixture/chat-event', { method: 'POST', body: JSON.stringify({
kind: 'text', message_id: `a${pairs-1}`, part: { type: 'text', id: `p${pairs-1}` }, delta: ' live',
}) });
requests.add(request);
request.finally(() => requests.delete(request));
}, 100);
await sleep(1200);
}
const backNodes = (await axNodes(c)).filter(n => /^Back(?:\s|$)/.test(n.name?.value ?? ''));
log.backControlNodes = backNodes.map(n => ({name:n.name?.value, role:n.role?.value, backend:n.backendDOMNodeId}));
const back = backNodes.find(n => n.backendDOMNodeId);
assert.ok(back, 'native Back control');
const { model } = await c.send('DOM.getBoxModel', { backendNodeId: back.backendDOMNodeId });
const [x1, y1, , , x3, y3] = model.content;
await c.evaluate(`(() => {
window.__back = {started: performance.now(), frames: [], longTasks: []};
new PerformanceObserver(list => { for (const e of list.getEntries()) window.__back.longTasks.push({start:e.startTime,duration:e.duration}); }).observe({entryTypes:['longtask']});
const frame = t => { window.__back.frames.push(t); if (location.pathname === '/chat') window.__back.settled=t; else requestAnimationFrame(frame); };
requestAnimationFrame(frame);
})()`);
const started = performance.now();
await c.send('Input.dispatchMouseEvent', { type: 'mousePressed', x: (x1+x3)/2, y: (y1+y3)/2, button: 'left', clickCount: 1 });
await c.send('Input.dispatchMouseEvent', { type: 'mouseReleased', x: (x1+x3)/2, y: (y1+y3)/2, button: 'left', clickCount: 1 });
assert.ok(await waitFor(() => c.evaluate('Boolean(window.__back.settled)'), 30000), 'Back returns to list without network response');
const metrics = await c.evaluate('window.__back');
log.backWallMs = performance.now() - started;
log.routeSettledMs = metrics.settled - metrics.started;
log.maxFrameGapMs = Math.max(0, ...metrics.frames.slice(1).map((t, i) => t - metrics.frames[i]));
log.longTasks = metrics.longTasks;
if (process.env.MAX_BACK_MS) assert.ok(log.routeSettledMs <= Number(process.env.MAX_BACK_MS), `Back exceeded ${process.env.MAX_BACK_MS} ms: ${log.routeSettledMs}`);
if (process.env.MAX_FRAME_GAP_MS) assert.ok(log.maxFrameGapMs <= Number(process.env.MAX_FRAME_GAP_MS), `Frame gap exceeded ${process.env.MAX_FRAME_GAP_MS} ms: ${log.maxFrameGapMs}`);
log.checks.push('pop restored chat list');
await sleep(150);
await capture(c, output, log, 'chat-after-back');
console.log(JSON.stringify(log, null, 2));
} catch (error) {
if (c) {
await capture(c, output, log, 'failure');
await writeFile(join(output, 'failure-ax.json'), JSON.stringify(await axNodes(c), null, 2));
}
throw error;
} finally {
clearInterval(streamTimer);
await Promise.allSettled([...requests]);
await writeFile(join(output, 'result.json'), JSON.stringify(log, null, 2));
c?.close(); const exited = once(chrome, 'exit'); chrome.kill(); await exited;
await fixture.close();
}
+2 -2
View File
@@ -310,7 +310,7 @@ async function staticFile(base, pathname, req, res) {
* It serves the built probe bundle and upgrades /connect on the same origin, so
* the normal web composition infers this server from Uri.base.
*/
export async function startFixture({ root, runs = RUNS, runDetail, files = {} }) {
export async function startFixture({ root, runs = RUNS, runDetail, files = {}, chatMessages }) {
const base = await realpath(root);
const connections = new Set();
const sockets = new Set();
@@ -564,7 +564,7 @@ export async function startFixture({ root, runs = RUNS, runDetail, files = {} })
client.send({
id: request.id, type: 'chat_history',
chat_history: {
messages: [
messages: chatMessages ?? [
{ id: 'm1', session_id: CHAT_SESSION.id, role: 'user',
parts: [{ type: 'text', text: 'Read-only probe question.' }] },
{ id: 'm2', session_id: CHAT_SESSION.id, role: 'assistant',