diff --git a/docs/chat-verification.md b/docs/chat-verification.md new file mode 100644 index 00000000..aa77c3b5 --- /dev/null +++ b/docs/chat-verification.md @@ -0,0 +1,169 @@ +# Chat verification (app chat lane, 17 Oct 2026) + +The chat surface gets its own end-to-end verification brief. It is a +**verification-only** pass: nothing was fixed and no app behaviour was changed. + +This document is the brief's required record. It is written by the +**implementation pass**, which per the brief's device policy verifies with the +build and the unit tests only. The rows that need the phone read +**device — visual review**: the visual reviewer owns the operator's Pixel 9 Pro +XL and captures them; this pass did not run `adb`, any Gradle device task, or the +device lock. + +## What this pass establishes, and what it does not + +Established here: + +- the exact revision built, and that it is main's own build (no main feature is + missing from it); +- the build and the unit tests, with the exact commands and their observed + results; +- a source-level map of every flow the brief names to the code path that + implements it and the unit test that pins it; +- the APK published to this run's artifacts directory and the LAN URL. + +Not established here (deferred to the visual reviewer, per the device policy): + +- anything that requires the phone: real-data captures, real round-trips, + streaming, the approval card, scroll feel, resilience, and every on-screen + rendering row marked below. + +No failure is reported as observed, because no device flow was executed in this +pass; the failure-triage details the brief asks for (capture, steps, exact error +text) can only come from the device pass. The two capability gaps that are +visible from the tree are recorded under "Known gaps" and are not device +failures. + +## Revision and build + +- **Run directory HEAD: `3ffc126`.** `git rev-parse --short HEAD` = `3ffc126` + and `git rev-parse --short main` = `3ffc126`, so this lane is **at main** (not + behind it). The operator checkout's main is also `3ffc126` + (`git -C /Users/alan/repos/pipelines rev-parse --short main`). **No feature of + main is absent from the build reviewed here** — this is main's build. +- **Server reachable.** `curl -s http://10.0.0.75:8080/healthz` → + `{"healthy":true}`. The phone's own connection to it is the visual reviewer's + to confirm. +- **APK published** as `pipelines-debug-wip-3ffc126.apk` + (18,552,316 bytes, sha256 + `5289b4f3d5dc6f9fd587e1e5decf075246b0e82b6fd7255335f2e27a2895aab7`), served at + **http://10.0.0.75:8080/artifacts/298/1/pipelines-debug-wip-3ffc126.apk** + (`HTTP/1.1 200 OK`, `Content-Type: application/vnd.android.package-archive`, + `Content-Length: 18552316`). + +### Commands and observed output + +Run in `android/` with `JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home` +(JDK 17.0.20.1), `ANDROID_HOME=/Users/alan/Library/Android/sdk`, and the cached +Gradle `~/.gradle/wrapper/dists/gradle-8.11.1-bin/bpt9gzteqjrbo1mjrsomdt32c/gradle-8.11.1/bin/gradle`. +Every command used `--no-daemon` as the brief requires. + +``` +gradle --offline --no-daemon --console=plain :app:testDebugUnitTest + → BUILD SUCCESSFUL in 5s (41 tasks up-to-date, from the warm build) +gradle --offline --no-daemon --console=plain :app:testDebugUnitTest --rerun-tasks + → BUILD SUCCESSFUL in 22s; testDebugUnitTest executed (22 tasks executed) +gradle --offline --no-daemon --console=plain :app:assembleDebug --rerun-tasks + → BUILD SUCCESSFUL in 25s; app-debug.apk 18,552,316 bytes, + sha256 5289b4f3…aab7 (byte-for-byte identical to the non-forced package) +``` + +The unit-test JUnit XMLs under +`android/app/build/test-results/testDebugUnitTest/` aggregate to: + +``` +60 suites / 545 tests / 0 failures / 0 errors / 0 skipped +``` + +with the chat suites green: `ChatProtocolTest`, `ChatStreamTest`, +`ChatApprovalTest`, `ChatScrollTest`, `ChatMultiChatStateTest`, +`ChatSessionManagementTest`, `ChatListFilterTest`, `ChatActivityTest`, +`ChatTranscriptTest`, `ChatFormatTest`, `ChatDirectoriesTest`, +`ChatAttachmentTest`, `ChatAttachmentStateTest`, `MarkdownTest`, +`ToolPanelPositionProviderTest`. + +## Per-flow table + +`Result` keys: **Pass (unit)** — the flow's logic is pinned by an executed, +green unit test; **Device — visual review** — the on-phone behaviour and capture +are the visual reviewer's; **Gap (server)** — the capability is absent because +the brokered server has no request for it. `Evidence path` is a repository path; +device captures will be added by the visual reviewer. + +| Flow | Result | Evidence path | Notes | +| --- | --- | --- | --- | +| Create — directory picker, home first + a recent | Pass (unit); device — visual review | `android/app/src/main/java/net/awan/pipelines/ui/chat/ChatDirectories.kt`, `ChatCreatePane.kt`; `android/app/src/test/java/net/awan/pipelines/ui/chat/ChatDirectoriesTest.kt` | `directoryChoices` puts the home first and keeps the server's recents order; `ChatDirectoriesTest.homeIsFirstAndIsTheDefaultChoice`, `recentsKeepTheServersOrderAndCreateInTheirFullPath`, `anEmptyOrBlankReplyStillYieldsTheHomeChoice`. On-screen picker not captured here. | +| Create — the model choice | Gap (server) | `ChatCreatePane.kt` (the note at the create footer); `ChatFormatTest.chatHeaderMeta*` | No `chat_models`/`chat_agents` request exists and `chat_prompt` drops `model`/`agent`, so the flow says so plainly and shows only the model the server reports. Needs the server catalogue to exist before it can be verified. | +| Create — session appears tagged `Operator` | Pass (unit); device — visual review | `ChatProtocol.kt` (`ChatSession.tag`, `TAG_OPERATOR`), `ChatFormat.kt` `sessionTagLabel`; `ChatSessionManagementTest.theKindChipComesFromTheServersOwnTag`, `ChatProtocolTest.chatSessionsCarryTheServersOwnKindTag` | The chip label derives from the server's own `tag`; an older server omitting it shows no chip. | +| List — recency order | Pass (unit); device — visual review | `ChatListPane.kt`; `ChatListFilter.kt`; `ChatListFilterTest.filteringKeepsTheRecencyOrderOfTheSurvivors` | The list is the server's order (MRU first, `internal/server/chat.go` `handleChatSessions`); the app never re-sorts it, only narrows. | +| List — activity marks | Pass (unit); device — visual review | `ChatListPane.kt` `SessionActivityBadge`; `ChatSeen.kt`; `ChatFormat.kt` `sessionActivity`; `ChatActivityTest` (5 cases) | Streaming / New / Open derivations pinned; distinct from each other and from idle. | +| List — search and filter | Pass (unit); device — visual review | `ChatListPane.kt` `ChatListSearchControl`; `ChatListFilter.kt`; `ChatListFilterTest` (9) | Case-insensitive over title and directory; blank matches all; survivors keep order; the create action is never hidden. | +| List — rename and remove | Pass (unit); device — visual review | `ChatListPane.kt`; `ChatStore.kt` `rename`/`remove`/`undoRemove`; `ChatSessionManagementTest.rename*`, `remove*`, `overrides*` | Both are local and honest (no server rename/delete); removal is undoable and the dialog says so. | +| List — no pipeline run session ever appears | Pass (unit + server source); device — visual review | `internal/server/chat.go` `handleChatSessions` (lists `registry.OperatorSessions` only); `ChatStore.kt` `visibleSessions`; `ChatSessionManagementTest.theListShowsTheServersSessionsVerbatimWhenNothingIsOverridden` | The server filters to operator sessions; the app performs no guessing. The brief's on-phone confirmation with real runs present is the visual reviewer's. | +| Conversation — sending a prompt | Pass (unit); device — visual review | `AppViewModel.kt` `sendChatPrompt`; `ChatProtocol.kt` `promptRequest`; `ChatProtocolTest.chatPromptRequestCarriesTheAttachedFileUris`, `chatRequestsRoundTripTheirBodies`; `ChatAttachmentStateTest.sendingClearsTheAttachmentsAndStillShowsThemOnTheOptimisticTurn` | Optimistic user turn; the wire shape is pinned. | +| Conversation — streaming reply arriving | Pass (unit); device — visual review | `ChatStore.kt` `applyDelta`/`upsertPart`; `ChatStreamTest.deltaSequenceGrowsRenderedTextAndTogglesRunning`; `ChatProtocolTest.chatEventDeltaCarriesTheTargetPartAndText` | Deltas grow the part and mark the thread running. | +| Conversation — tool chip expanding | Partial (position logic only); device — visual review | `ChatToolCalls.kt`; `ToolPanelPositionProviderTest` (3); `ChatFormatTest.prettyToolInput*` | The chip/panel tap, the popup overlay and the INPUT/OUTPUT/ERROR blocks are rendering that needs the device; the placement maths and the input formatting are pinned. | +| Conversation — diff card | Partial (diff logic); device — visual review | `ChatDiffCard.kt`; `ChatFormatTest.toolEditDiff*` (3), `toolEditDiffSummarisesAnOversizedEditInsteadOfRenderingEveryLine` | `+N -M` counts, line kinds and the oversized summary are pinned; the rendered card is the visual reviewer's. | +| Conversation — approval once / always / rejected | Pass (unit); device — visual review | `ChatApprovalCard.kt`; `ChatProtocol.kt` `replyRequest`; `ChatApprovalTest` (10) | The three decisions' wire values, the in-place settle, the server-resolved card and the failed-reply reopen are pinned. | +| Conversation — stopping a turn mid-stream | Pass (unit, wire); device — visual review | `AppViewModel.kt` `stopChat`; `ChatProtocol.kt` `interruptRequest`; `ChatProtocolTest.chatInterruptRequestCarriesTheRunningSession` | The interrupt frame is pinned; the Stop control's live behaviour needs the device. | +| Switching — two sessions, transcript + scroll each | Pass (unit); device — visual review | `ChatStore.kt` `open`/`activate`/`showList`; `MultiChatStateTest.switchingPreservesEachConversation`, `scrollPositionIsPerChat`, `draftsArePerChat` | Each open chat keeps messages, draft and scroll; a running chat re-syncs on return. | +| Scroll — opens at the newest message | Pass (unit); device — visual review | `ChatScroll.kt`; `ChatTranscript.kt` placement; `ChatScrollTest.aFreshConversationOpensAtItsNewestMessage`, `anItemAppendedPastTheFoldHasNoMeasurableGapSoItMustBeBroughtIntoView` | A chat with no saved position lands on its newest item, instantly (no animation). | +| Scroll — following while at the bottom | Pass (unit); device — visual review | `ChatTranscript.kt` follow effect; `ChatScrollTest.theFollowResumesAtTheBottomAndWithinTheThreshold`, `theFollowMeasuresAgainstTheContentNotTheLayoutThatStillLagsIt`; `Dimens.kt` `CHAT_FOLLOW_THRESHOLD_DP` (32dp) | Every arrival (message, delta, tool chip, grown composer, IME) follows while following. | +| Scroll — not moving while scrolled up | Pass (unit); device — visual review | `ChatTranscript.kt` `DragInteraction` handling; `ChatScrollTest.theFollowDoesNotResumeWhenTheEndIsNotInViewOrTheConversationIsEmpty`, `aSavedPositionAtTheVeryTopIsRestoredRatherThanTreatedAsNew` | Only the operator's own drag stops the follow; programmatic scrolls never un-stick it. | +| Scroll — jump-to-latest | Pass (unit); device — visual review | `ChatTranscript.kt` `SmallFloatingActionButton`; `ChatScrollTest.theJumpToLatestIsOfferedOnlyWhileTheOperatorIsAwayFromTheBottom` | Offered only while not following and something is genuinely below the fold. | +| Resilience — break the connection, reconnect + re-sync | Device — visual review | `AppViewModel.kt` `observeConnection` (`streamResumed().forEach { loadChatHistory(it) }`); `ChatStreamTest.streamResumedSettlesRunningAndReturnsEveryOpenChat` | The re-sync logic on the connected edge is pinned; **how the connection is broken is for the visual reviewer to choose and name**, together with whether the transcript re-syncs. | +| States — empty, loading, error | Partial (store states); device — visual review | `ChatListPane.kt` / `ChatConversationPane.kt` / `ChatCreatePane.kt` (`LoadingState`/`ErrorState`/`EmptyState`); `ChatStore.kt` `sessionsLoading`/`sessionsFailed`/`historyLoading`/`failed` | The state transitions exist and are asserted across stores/tests; the rendered states need the device. | +| Long conversation — hundreds of messages | Partial; device — visual review | `AppViewModel.kt` `CHAT_HISTORY_LIMIT` = 200; `ChatStore.kt` `history`/`upsertPart` | History is fetched up to 200 messages; how a several-hundred-message transcript scrolls and renders needs the device. | + +## Known gaps (from the tree, not device failures) + +- **No model choice.** The brokered server exposes no request to choose a chat's + model and no catalogue to offer one from, and `handleChatPrompt` drops the + `model`/`agent` the protocol carries. The create flow states this instead of + shipping a dead picker. Classification if seen on the phone: **server gap**, by + design and documented. +- **Attachments are not echoed back.** The server stores a user turn as text and + does not forward the prompt's `files` + (`internal/server/chat.go` → `broker.Prompt(ctx, sessionID, prompt.Text)`; + `internal/opencode/chat.go` posts only `{"text": …}`). The phone shows the file + from its own optimistic turn; a history round-trip will not carry it. + Classification: **server gap**. + +## What could not be verified, and why + +Everything that needs the phone. The brief's device policy is explicit: +"Implementation and ordinary code review verify with the build and the unit +tests only: do not run `adb`, do not run Gradle device tasks, and do not take the +device lock." This is the implementation pass, so the unresolved items are, +precisely: + +- every on-screen rendering and capture (create picker, list, conversation, tool + panel, diff card, approval card, empty/loading/error states, long transcript); +- every real-data round-trip against the deployed server (create a chat, send a + prompt, receive a streaming reply, answer an approval, stop a turn); +- the streaming and scroll *feel* (opening at the newest, following, staying put + when scrolled up, jump-to-latest); +- resilience: breaking the connection a named way and confirming the reconnect + and transcript re-sync; +- the wide-window checks (`wm size 2000x2244`) and the near-square unfolded + shape (`wm size 2076x2152`), then `wm size reset`; +- the per-flow evidence captures that fill the "device — visual review" rows + above and the failure-triage records the brief asks for. + +Why: the device belongs to the visual reviewer, and this pass changed nothing, so +it has no device evidence to offer rather than a fabricated one. + +## Device state after this pass + +This pass ran **no** `adb` command (including no `screencap`, no `wm size`, no +install) and did not take the device lock. It therefore neither read nor changed +the phone's installed build, its server URL or its app state, and it cannot +report which revision the phone is on. The brief's own closing steps are +consequently the visual reviewer's, not this pass's: + +- install and verify main's build with + `/private/var/folders/jp/q0qvjyg54fl_c5vhhk1ylz2m0000gn/T/opencode/deploy-app.sh main` + and report the revision the phone ends on; +- leave the app pointed at the real server + (`ws://10.0.0.75:8080/connect`, workspace `/Users/alan/repos/pipelines`), + force-stopped and relaunched, and say so in the evidence.