Record the chat end-to-end verification record
Add docs/chat-verification.md, the required record for the chat verification brief: per-flow table (flow, result, evidence path, notes), the known tree-level gaps, and the section on what could not be verified and why. Verification only; no app behaviour changed. Checked: packaged templates (internal/author/skill/references/) do not mention this record - no template or skill reference needed updating. Shared README and topic docs are intentionally not touched (single-writer fold).
This commit is contained in:
@@ -0,0 +1,169 @@
|
||||
# Chat verification (app chat lane, 17 Oct 2026)
|
||||
|
||||
The chat surface gets its own end-to-end verification brief. It is a
|
||||
**verification-only** pass: nothing was fixed and no app behaviour was changed.
|
||||
|
||||
This document is the brief's required record. It is written by the
|
||||
**implementation pass**, which per the brief's device policy verifies with the
|
||||
build and the unit tests only. The rows that need the phone read
|
||||
**device — visual review**: the visual reviewer owns the operator's Pixel 9 Pro
|
||||
XL and captures them; this pass did not run `adb`, any Gradle device task, or the
|
||||
device lock.
|
||||
|
||||
## What this pass establishes, and what it does not
|
||||
|
||||
Established here:
|
||||
|
||||
- the exact revision built, and that it is main's own build (no main feature is
|
||||
missing from it);
|
||||
- the build and the unit tests, with the exact commands and their observed
|
||||
results;
|
||||
- a source-level map of every flow the brief names to the code path that
|
||||
implements it and the unit test that pins it;
|
||||
- the APK published to this run's artifacts directory and the LAN URL.
|
||||
|
||||
Not established here (deferred to the visual reviewer, per the device policy):
|
||||
|
||||
- anything that requires the phone: real-data captures, real round-trips,
|
||||
streaming, the approval card, scroll feel, resilience, and every on-screen
|
||||
rendering row marked below.
|
||||
|
||||
No failure is reported as observed, because no device flow was executed in this
|
||||
pass; the failure-triage details the brief asks for (capture, steps, exact error
|
||||
text) can only come from the device pass. The two capability gaps that are
|
||||
visible from the tree are recorded under "Known gaps" and are not device
|
||||
failures.
|
||||
|
||||
## Revision and build
|
||||
|
||||
- **Run directory HEAD: `3ffc126`.** `git rev-parse --short HEAD` = `3ffc126`
|
||||
and `git rev-parse --short main` = `3ffc126`, so this lane is **at main** (not
|
||||
behind it). The operator checkout's main is also `3ffc126`
|
||||
(`git -C /Users/alan/repos/pipelines rev-parse --short main`). **No feature of
|
||||
main is absent from the build reviewed here** — this is main's build.
|
||||
- **Server reachable.** `curl -s http://10.0.0.75:8080/healthz` →
|
||||
`{"healthy":true}`. The phone's own connection to it is the visual reviewer's
|
||||
to confirm.
|
||||
- **APK published** as `pipelines-debug-wip-3ffc126.apk`
|
||||
(18,552,316 bytes, sha256
|
||||
`5289b4f3d5dc6f9fd587e1e5decf075246b0e82b6fd7255335f2e27a2895aab7`), served at
|
||||
**http://10.0.0.75:8080/artifacts/298/1/pipelines-debug-wip-3ffc126.apk**
|
||||
(`HTTP/1.1 200 OK`, `Content-Type: application/vnd.android.package-archive`,
|
||||
`Content-Length: 18552316`).
|
||||
|
||||
### Commands and observed output
|
||||
|
||||
Run in `android/` with `JAVA_HOME=/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home`
|
||||
(JDK 17.0.20.1), `ANDROID_HOME=/Users/alan/Library/Android/sdk`, and the cached
|
||||
Gradle `~/.gradle/wrapper/dists/gradle-8.11.1-bin/bpt9gzteqjrbo1mjrsomdt32c/gradle-8.11.1/bin/gradle`.
|
||||
Every command used `--no-daemon` as the brief requires.
|
||||
|
||||
```
|
||||
gradle --offline --no-daemon --console=plain :app:testDebugUnitTest
|
||||
→ BUILD SUCCESSFUL in 5s (41 tasks up-to-date, from the warm build)
|
||||
gradle --offline --no-daemon --console=plain :app:testDebugUnitTest --rerun-tasks
|
||||
→ BUILD SUCCESSFUL in 22s; testDebugUnitTest executed (22 tasks executed)
|
||||
gradle --offline --no-daemon --console=plain :app:assembleDebug --rerun-tasks
|
||||
→ BUILD SUCCESSFUL in 25s; app-debug.apk 18,552,316 bytes,
|
||||
sha256 5289b4f3…aab7 (byte-for-byte identical to the non-forced package)
|
||||
```
|
||||
|
||||
The unit-test JUnit XMLs under
|
||||
`android/app/build/test-results/testDebugUnitTest/` aggregate to:
|
||||
|
||||
```
|
||||
60 suites / 545 tests / 0 failures / 0 errors / 0 skipped
|
||||
```
|
||||
|
||||
with the chat suites green: `ChatProtocolTest`, `ChatStreamTest`,
|
||||
`ChatApprovalTest`, `ChatScrollTest`, `ChatMultiChatStateTest`,
|
||||
`ChatSessionManagementTest`, `ChatListFilterTest`, `ChatActivityTest`,
|
||||
`ChatTranscriptTest`, `ChatFormatTest`, `ChatDirectoriesTest`,
|
||||
`ChatAttachmentTest`, `ChatAttachmentStateTest`, `MarkdownTest`,
|
||||
`ToolPanelPositionProviderTest`.
|
||||
|
||||
## Per-flow table
|
||||
|
||||
`Result` keys: **Pass (unit)** — the flow's logic is pinned by an executed,
|
||||
green unit test; **Device — visual review** — the on-phone behaviour and capture
|
||||
are the visual reviewer's; **Gap (server)** — the capability is absent because
|
||||
the brokered server has no request for it. `Evidence path` is a repository path;
|
||||
device captures will be added by the visual reviewer.
|
||||
|
||||
| Flow | Result | Evidence path | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| Create — directory picker, home first + a recent | Pass (unit); device — visual review | `android/app/src/main/java/net/awan/pipelines/ui/chat/ChatDirectories.kt`, `ChatCreatePane.kt`; `android/app/src/test/java/net/awan/pipelines/ui/chat/ChatDirectoriesTest.kt` | `directoryChoices` puts the home first and keeps the server's recents order; `ChatDirectoriesTest.homeIsFirstAndIsTheDefaultChoice`, `recentsKeepTheServersOrderAndCreateInTheirFullPath`, `anEmptyOrBlankReplyStillYieldsTheHomeChoice`. On-screen picker not captured here. |
|
||||
| Create — the model choice | Gap (server) | `ChatCreatePane.kt` (the note at the create footer); `ChatFormatTest.chatHeaderMeta*` | No `chat_models`/`chat_agents` request exists and `chat_prompt` drops `model`/`agent`, so the flow says so plainly and shows only the model the server reports. Needs the server catalogue to exist before it can be verified. |
|
||||
| Create — session appears tagged `Operator` | Pass (unit); device — visual review | `ChatProtocol.kt` (`ChatSession.tag`, `TAG_OPERATOR`), `ChatFormat.kt` `sessionTagLabel`; `ChatSessionManagementTest.theKindChipComesFromTheServersOwnTag`, `ChatProtocolTest.chatSessionsCarryTheServersOwnKindTag` | The chip label derives from the server's own `tag`; an older server omitting it shows no chip. |
|
||||
| List — recency order | Pass (unit); device — visual review | `ChatListPane.kt`; `ChatListFilter.kt`; `ChatListFilterTest.filteringKeepsTheRecencyOrderOfTheSurvivors` | The list is the server's order (MRU first, `internal/server/chat.go` `handleChatSessions`); the app never re-sorts it, only narrows. |
|
||||
| List — activity marks | Pass (unit); device — visual review | `ChatListPane.kt` `SessionActivityBadge`; `ChatSeen.kt`; `ChatFormat.kt` `sessionActivity`; `ChatActivityTest` (5 cases) | Streaming / New / Open derivations pinned; distinct from each other and from idle. |
|
||||
| List — search and filter | Pass (unit); device — visual review | `ChatListPane.kt` `ChatListSearchControl`; `ChatListFilter.kt`; `ChatListFilterTest` (9) | Case-insensitive over title and directory; blank matches all; survivors keep order; the create action is never hidden. |
|
||||
| List — rename and remove | Pass (unit); device — visual review | `ChatListPane.kt`; `ChatStore.kt` `rename`/`remove`/`undoRemove`; `ChatSessionManagementTest.rename*`, `remove*`, `overrides*` | Both are local and honest (no server rename/delete); removal is undoable and the dialog says so. |
|
||||
| List — no pipeline run session ever appears | Pass (unit + server source); device — visual review | `internal/server/chat.go` `handleChatSessions` (lists `registry.OperatorSessions` only); `ChatStore.kt` `visibleSessions`; `ChatSessionManagementTest.theListShowsTheServersSessionsVerbatimWhenNothingIsOverridden` | The server filters to operator sessions; the app performs no guessing. The brief's on-phone confirmation with real runs present is the visual reviewer's. |
|
||||
| Conversation — sending a prompt | Pass (unit); device — visual review | `AppViewModel.kt` `sendChatPrompt`; `ChatProtocol.kt` `promptRequest`; `ChatProtocolTest.chatPromptRequestCarriesTheAttachedFileUris`, `chatRequestsRoundTripTheirBodies`; `ChatAttachmentStateTest.sendingClearsTheAttachmentsAndStillShowsThemOnTheOptimisticTurn` | Optimistic user turn; the wire shape is pinned. |
|
||||
| Conversation — streaming reply arriving | Pass (unit); device — visual review | `ChatStore.kt` `applyDelta`/`upsertPart`; `ChatStreamTest.deltaSequenceGrowsRenderedTextAndTogglesRunning`; `ChatProtocolTest.chatEventDeltaCarriesTheTargetPartAndText` | Deltas grow the part and mark the thread running. |
|
||||
| Conversation — tool chip expanding | Partial (position logic only); device — visual review | `ChatToolCalls.kt`; `ToolPanelPositionProviderTest` (3); `ChatFormatTest.prettyToolInput*` | The chip/panel tap, the popup overlay and the INPUT/OUTPUT/ERROR blocks are rendering that needs the device; the placement maths and the input formatting are pinned. |
|
||||
| Conversation — diff card | Partial (diff logic); device — visual review | `ChatDiffCard.kt`; `ChatFormatTest.toolEditDiff*` (3), `toolEditDiffSummarisesAnOversizedEditInsteadOfRenderingEveryLine` | `+N -M` counts, line kinds and the oversized summary are pinned; the rendered card is the visual reviewer's. |
|
||||
| Conversation — approval once / always / rejected | Pass (unit); device — visual review | `ChatApprovalCard.kt`; `ChatProtocol.kt` `replyRequest`; `ChatApprovalTest` (10) | The three decisions' wire values, the in-place settle, the server-resolved card and the failed-reply reopen are pinned. |
|
||||
| Conversation — stopping a turn mid-stream | Pass (unit, wire); device — visual review | `AppViewModel.kt` `stopChat`; `ChatProtocol.kt` `interruptRequest`; `ChatProtocolTest.chatInterruptRequestCarriesTheRunningSession` | The interrupt frame is pinned; the Stop control's live behaviour needs the device. |
|
||||
| Switching — two sessions, transcript + scroll each | Pass (unit); device — visual review | `ChatStore.kt` `open`/`activate`/`showList`; `MultiChatStateTest.switchingPreservesEachConversation`, `scrollPositionIsPerChat`, `draftsArePerChat` | Each open chat keeps messages, draft and scroll; a running chat re-syncs on return. |
|
||||
| Scroll — opens at the newest message | Pass (unit); device — visual review | `ChatScroll.kt`; `ChatTranscript.kt` placement; `ChatScrollTest.aFreshConversationOpensAtItsNewestMessage`, `anItemAppendedPastTheFoldHasNoMeasurableGapSoItMustBeBroughtIntoView` | A chat with no saved position lands on its newest item, instantly (no animation). |
|
||||
| Scroll — following while at the bottom | Pass (unit); device — visual review | `ChatTranscript.kt` follow effect; `ChatScrollTest.theFollowResumesAtTheBottomAndWithinTheThreshold`, `theFollowMeasuresAgainstTheContentNotTheLayoutThatStillLagsIt`; `Dimens.kt` `CHAT_FOLLOW_THRESHOLD_DP` (32dp) | Every arrival (message, delta, tool chip, grown composer, IME) follows while following. |
|
||||
| Scroll — not moving while scrolled up | Pass (unit); device — visual review | `ChatTranscript.kt` `DragInteraction` handling; `ChatScrollTest.theFollowDoesNotResumeWhenTheEndIsNotInViewOrTheConversationIsEmpty`, `aSavedPositionAtTheVeryTopIsRestoredRatherThanTreatedAsNew` | Only the operator's own drag stops the follow; programmatic scrolls never un-stick it. |
|
||||
| Scroll — jump-to-latest | Pass (unit); device — visual review | `ChatTranscript.kt` `SmallFloatingActionButton`; `ChatScrollTest.theJumpToLatestIsOfferedOnlyWhileTheOperatorIsAwayFromTheBottom` | Offered only while not following and something is genuinely below the fold. |
|
||||
| Resilience — break the connection, reconnect + re-sync | Device — visual review | `AppViewModel.kt` `observeConnection` (`streamResumed().forEach { loadChatHistory(it) }`); `ChatStreamTest.streamResumedSettlesRunningAndReturnsEveryOpenChat` | The re-sync logic on the connected edge is pinned; **how the connection is broken is for the visual reviewer to choose and name**, together with whether the transcript re-syncs. |
|
||||
| States — empty, loading, error | Partial (store states); device — visual review | `ChatListPane.kt` / `ChatConversationPane.kt` / `ChatCreatePane.kt` (`LoadingState`/`ErrorState`/`EmptyState`); `ChatStore.kt` `sessionsLoading`/`sessionsFailed`/`historyLoading`/`failed` | The state transitions exist and are asserted across stores/tests; the rendered states need the device. |
|
||||
| Long conversation — hundreds of messages | Partial; device — visual review | `AppViewModel.kt` `CHAT_HISTORY_LIMIT` = 200; `ChatStore.kt` `history`/`upsertPart` | History is fetched up to 200 messages; how a several-hundred-message transcript scrolls and renders needs the device. |
|
||||
|
||||
## Known gaps (from the tree, not device failures)
|
||||
|
||||
- **No model choice.** The brokered server exposes no request to choose a chat's
|
||||
model and no catalogue to offer one from, and `handleChatPrompt` drops the
|
||||
`model`/`agent` the protocol carries. The create flow states this instead of
|
||||
shipping a dead picker. Classification if seen on the phone: **server gap**, by
|
||||
design and documented.
|
||||
- **Attachments are not echoed back.** The server stores a user turn as text and
|
||||
does not forward the prompt's `files`
|
||||
(`internal/server/chat.go` → `broker.Prompt(ctx, sessionID, prompt.Text)`;
|
||||
`internal/opencode/chat.go` posts only `{"text": …}`). The phone shows the file
|
||||
from its own optimistic turn; a history round-trip will not carry it.
|
||||
Classification: **server gap**.
|
||||
|
||||
## What could not be verified, and why
|
||||
|
||||
Everything that needs the phone. The brief's device policy is explicit:
|
||||
"Implementation and ordinary code review verify with the build and the unit
|
||||
tests only: do not run `adb`, do not run Gradle device tasks, and do not take the
|
||||
device lock." This is the implementation pass, so the unresolved items are,
|
||||
precisely:
|
||||
|
||||
- every on-screen rendering and capture (create picker, list, conversation, tool
|
||||
panel, diff card, approval card, empty/loading/error states, long transcript);
|
||||
- every real-data round-trip against the deployed server (create a chat, send a
|
||||
prompt, receive a streaming reply, answer an approval, stop a turn);
|
||||
- the streaming and scroll *feel* (opening at the newest, following, staying put
|
||||
when scrolled up, jump-to-latest);
|
||||
- resilience: breaking the connection a named way and confirming the reconnect
|
||||
and transcript re-sync;
|
||||
- the wide-window checks (`wm size 2000x2244`) and the near-square unfolded
|
||||
shape (`wm size 2076x2152`), then `wm size reset`;
|
||||
- the per-flow evidence captures that fill the "device — visual review" rows
|
||||
above and the failure-triage records the brief asks for.
|
||||
|
||||
Why: the device belongs to the visual reviewer, and this pass changed nothing, so
|
||||
it has no device evidence to offer rather than a fabricated one.
|
||||
|
||||
## Device state after this pass
|
||||
|
||||
This pass ran **no** `adb` command (including no `screencap`, no `wm size`, no
|
||||
install) and did not take the device lock. It therefore neither read nor changed
|
||||
the phone's installed build, its server URL or its app state, and it cannot
|
||||
report which revision the phone is on. The brief's own closing steps are
|
||||
consequently the visual reviewer's, not this pass's:
|
||||
|
||||
- install and verify main's build with
|
||||
`/private/var/folders/jp/q0qvjyg54fl_c5vhhk1ylz2m0000gn/T/opencode/deploy-app.sh main`
|
||||
and report the revision the phone ends on;
|
||||
- leave the app pointed at the real server
|
||||
(`ws://10.0.0.75:8080/connect`, workspace `/Users/alan/repos/pipelines`),
|
||||
force-stopped and relaunched, and say so in the evidence.
|
||||
Reference in New Issue
Block a user