chat: attach files to a message from the composer
Let the operator attach one or more files to the next chat message and send them with it. The platform multi-document picker returns content URIs; the app resolves only each file's name, size and MIME (confirming it can be opened) and holds the attachments per chat, exactly like the draft. Nothing is read or transmitted until the message is sent. - Composer: an attach action, a strip of removable chips (image thumbnail or file glyph, name, size, remove), and a send that is blocked while any attachment is flagged. - Wire: one shared builder, ChatProtocol.promptRequest(sessionId, text, files), carries the attached URIs on the chat_prompt frame, in attachment order and omitted for a text-only prompt so that shape is unchanged. PipelinesRepository.chatPrompt takes the same list. - Transcript: a sent turn carries each attachment as a ChatProtocol.PART_FILE part, drawn through the existing UserParts -> FilePart display path rather than a second rendering. - Honest failures: an unreadable or over-24-MiB file is attached with its reason and blocks the send; a server rejection takes the optimistic bubble back out and returns the files flagged with the server's message. Attaching never transmits on its own, and attachments clear once the message is sent. Review follow-ups in the same slice: the composer now uses ui/Format.kt's humanBytes instead of a duplicate formatter; optimistic user-turn ids come from a monotonic counter so removing a rejected turn cannot re-mint a live key; restoreAttachments keeps a file attached between send and rejection; the thumbnail decode catches Exception rather than Throwable. Checks made for this commit: - The change is android/ only; the server, its Go source, the installer and the Docker files are untouched, no run is started from the app, and no dependency was added beyond the already-declared material-icons-extended. - Packaged authoring skill references (internal/author/skill/references/): no update needed. operations.md documents the brokered chat server, which this change does not alter; the server still stores user turns as text and ignores the prompt's files, so that reference stays accurate. - Packaged templates (internal/tui, internal/server) and the root README/topic docs: no update needed; none describes the Android chat composer. The Android-facing documentation lives in android/README.md, which this change updates with the attachment slice, its known server-side gap and its tests. Tests: ChatAttachmentTest (5 cases), ChatAttachmentStateTest (5 cases) and a new ChatProtocolTest case cover building the request with attachments (and omitting them for a text-only prompt), removing one, the unreadable and oversized paths, which files travel with the prompt, the optimistic turn's file parts, clearing after send, and the rejected path restoring the files flagged. Verified by the review stage with gradle --offline --no-daemon :app:testDebugUnitTest :app:assembleDebug -> BUILD SUCCESSFUL, 487 tests / 0 failures / 0 errors / 0 skipped.
This commit is contained in:
@@ -5538,3 +5538,106 @@ The composable's frame timing (placing the list before its first draw, the
|
||||
instant correction for a last message taller than the viewport, and the fallback
|
||||
that brings an item appended below the fold into view) is argued from the pinned
|
||||
Compose BOM (2024.10.00) and the pure decisions it delegates to, not observed.
|
||||
|
||||
## Attaching files to a message (17 Oct 2026)
|
||||
|
||||
A chat message could only be text even though the prompt already belongs to a
|
||||
turn the phone builds. This slice lets the operator attach one or more files to
|
||||
the next message, shows them with a name and an image thumbnail, lets one be
|
||||
removed, carries them on the prompt, and clears them once the message is sent.
|
||||
`android/` only; nothing in the Pipelines server, its Go source, the installer or
|
||||
the Docker files was touched, no dependency was added beyond the
|
||||
already-declared material-icons-extended, and no run is started from the app.
|
||||
|
||||
- **Attaching never transmits.** The picker (`ActivityResultContracts
|
||||
.OpenMultipleDocuments`, the platform picker) returns content URIs; the app
|
||||
resolves only each file's name, size and MIME and confirms it can be opened
|
||||
(`resolveChatAttachment`, `ui/chat/ChatAttachments.kt`). The bytes are not read
|
||||
here, so nothing leaves the device until the message carrying them is sent.
|
||||
- **What is attached.** `ChatThread.attachments` holds `ChatAttachment` values
|
||||
(uri/name/mime/size and an optional error), kept per chat exactly like the
|
||||
draft, so switching chats never loses them. Each gets a store-assigned id
|
||||
(`ChatStore.attach`) so it can be keyed and removed individually; the composer
|
||||
strip shows an image thumbnail (a downsampled local decode, off the main
|
||||
thread) or a file glyph, the name, the size, and a remove action.
|
||||
- **The prompt carries them.** The send builds the request through one builder,
|
||||
`ChatProtocol.promptRequest(sessionId, text, files)`: `files` is the attached
|
||||
URIs from `attachmentFiles`, in attachment order, and is omitted entirely for a
|
||||
text-only prompt (so the wire shape a text-only prompt always sent is
|
||||
unchanged). `PipelinesRepository.chatPrompt` takes the same `files` list.
|
||||
- **The transcript shows them.** A sent user turn carries each attachment as a
|
||||
`ChatProtocol.PART_FILE` part (`attachmentPart`), which `MessageItem`'s
|
||||
existing parts rendering draws (`UserParts` → `FilePart`): an image as a
|
||||
preview, anything else as an icon and its name. There is no second attachment
|
||||
list under the bubble.
|
||||
- **Honest failures.** A file that cannot be queried or opened, or is over
|
||||
`MAX_ATTACHMENT_BYTES` (25 MiB), is attached with the reason set against it
|
||||
(`attachmentError`), never dropped. `ChatThread.canSend` is false while any
|
||||
attachment is flagged, so the message is not sent without it; a server
|
||||
rejection (`sendChatPrompt`) takes the optimistic bubble back out, returns the
|
||||
files flagged with the server's message, and keeps the draft, so nothing is
|
||||
silently lost.
|
||||
- **Cleared after send.** `sendChatPrompt` clears the draft and the attachments
|
||||
once the prompt is built, while the optimistic turn still shows them.
|
||||
|
||||
**Known server-side gap (verified by reading the tree, not live).** The brokered
|
||||
server stores a user turn as text and does not forward the prompt's `files`
|
||||
(`internal/server/chat.go` calls `broker.Prompt(ctx, sessionID, prompt.Text)`,
|
||||
and `internal/opencode/chat.go` posts only `{"text": …}`), and the deployed
|
||||
server predates chat. So the attachment the phone sends is not yet echoed back in
|
||||
history; the transcript shows it from the app's own optimistic turn, and an
|
||||
end-to-end file round-trip needs the server-side lane. This slice is the client
|
||||
capability the request field and the display path already allow.
|
||||
|
||||
Tests (`ChatAttachmentTest` 5 cases, `ChatAttachmentStateTest` 5 cases, and a new
|
||||
`ChatProtocolTest` case): the request built with attachments (and omitted for a
|
||||
text-only prompt), removing one, the unreadable and oversized paths (reported by
|
||||
name, blocking the send), which files travel with the prompt, the optimistic
|
||||
turn's file parts, clearing after send, and the rejected path putting the files
|
||||
back flagged.
|
||||
|
||||
Evidence (this run's implementer pass, macOS, JDK 17.0.20.1,
|
||||
`ANDROID_HOME=/Users/alan/Library/Android/sdk`, Gradle 8.11.1 `--offline
|
||||
--no-daemon` — a single-use daemon is forked for the JVM settings):
|
||||
`gradle --offline --no-daemon :app:assembleDebug :app:testDebugUnitTest` →
|
||||
`BUILD SUCCESSFUL`; the JUnit XMLs total **485 tests / 0 failures / 0 errors / 0
|
||||
skipped**. The APK was published to this run's artifacts directory as
|
||||
`pipelines-debug-wip-d1a859e.apk` (18,470,240 bytes, sha256
|
||||
`378e7e8f03ca7e75108db1217aba3b81efca396fecfcbf9dec2a6bf964319dfa`; `d1a859e` is
|
||||
`git rev-parse --short HEAD`, the tree being uncommitted WIP on top of it) and is
|
||||
served at
|
||||
`http://10.0.0.75:8080/artifacts/249/1/pipelines-debug-wip-d1a859e.apk`
|
||||
(`HTTP/1.1 200 OK`, `Content-Type: application/vnd.android.package-archive`,
|
||||
`Content-Length: 18470240`).
|
||||
|
||||
Not verified in this pass: the picker, thumbnails, composer strip and sent
|
||||
message on a device — the brief keeps implementation verification to the build
|
||||
and the unit tests, and visual review owns the device; and the live file
|
||||
round-trip, which the server-side gap above makes unavailable.
|
||||
|
||||
### Review follow-ups (same slice, 17 Oct 2026)
|
||||
|
||||
A code-review pass found four concrete items, all fixed without changing the
|
||||
request shape or the display path:
|
||||
|
||||
- The composer's own byte formatter duplicated `ui/Format.kt`'s documented
|
||||
`humanBytes`. `attachmentError` and the chip's size line now call `humanBytes`,
|
||||
so a size reads the same in the composer as on the artifact screens (a sub-MiB
|
||||
size now keeps its one decimal, e.g. `900.0 KiB`).
|
||||
- Optimistic user-turn ids come from a monotonic counter instead of the count of
|
||||
live local turns. Removing a rejected turn no longer lets the next append mint
|
||||
the id it held, so the transcript's per-id keys can never duplicate.
|
||||
- `restoreAttachments` keeps a file attached between sending and the rejection
|
||||
(it previously replaced the whole list and silently dropped it).
|
||||
- The thumbnail decode catches `Exception`, not `Throwable`, so an `Error`-level
|
||||
failure is no longer swallowed into a null glyph.
|
||||
|
||||
Re-verified with `gradle --offline --no-daemon :app:testDebugUnitTest
|
||||
:app:assembleDebug` → `BUILD SUCCESSFUL`; the JUnit XMLs total **487 tests / 0
|
||||
failures / 0 errors / 0 skipped** (two new `ChatAttachmentStateTest` cases cover
|
||||
the counter and the kept file). The APK was republished for this stage as
|
||||
`pipelines-debug-wip-d1a859e.apk` (18,470,240 bytes, sha256
|
||||
`fbd640af7749cb483e5c01cd1ca06c074fc9db620049ab07d63c5a03c5298f63`) and is served
|
||||
at `http://10.0.0.75:8080/artifacts/249/3/pipelines-debug-wip-d1a859e.apk`
|
||||
(`HTTP/1.1 200 OK`, `Content-Type: application/vnd.android.package-archive`,
|
||||
`Content-Length: 18470240`).
|
||||
|
||||
@@ -253,17 +253,16 @@ class PipelinesRepository(
|
||||
).chatMessages ?: emptyList()
|
||||
}
|
||||
|
||||
/** Sends one prompt; the server echoes an ack and then pushes chat_event frames. */
|
||||
suspend fun chatPrompt(sessionId: String, text: String) {
|
||||
/**
|
||||
* Sends one prompt with its attached file URIs; the server echoes an ack and
|
||||
* then pushes chat_event frames. The attachments travel only here, with the
|
||||
* prompt the operator actually sent — never on their own.
|
||||
*/
|
||||
suspend fun chatPrompt(sessionId: String, text: String, files: List<String> = emptyList()) {
|
||||
if (sessionId.isBlank()) throw ProtocolException("Select a chat session first.")
|
||||
val trimmed = text.trim()
|
||||
if (trimmed.isEmpty()) throw ProtocolException("Enter a message before sending.")
|
||||
client.call(
|
||||
Message(
|
||||
type = Protocol.CHAT_PROMPT,
|
||||
chatPrompt = ChatPromptRequest(sessionId = sessionId, text = trimmed),
|
||||
),
|
||||
)
|
||||
client.call(ChatProtocol.promptRequest(sessionId, trimmed, files))
|
||||
}
|
||||
|
||||
suspend fun chatInterrupt(sessionId: String) {
|
||||
|
||||
@@ -69,7 +69,15 @@ data class ChatMessage(
|
||||
val parts: List<ChatPart> = emptyList(),
|
||||
)
|
||||
|
||||
/** One element of a [ChatMessage]; [type] selects which fields are meaningful. */
|
||||
/**
|
||||
* One element of a [ChatMessage]; [type] selects which fields are meaningful.
|
||||
*
|
||||
* A [ChatProtocol.PART_FILE] part is the operator's attachment: [uri] is the
|
||||
* local `content://` reference the picker returned and [mime] decides whether it
|
||||
* is drawn as an image or a plain file. That part exists only in the optimistic
|
||||
* user turn the app appends when it sends; the brokered server stores user turns
|
||||
* as text, so a history round-trip must not be expected to carry it back.
|
||||
*/
|
||||
@Serializable
|
||||
data class ChatPart(
|
||||
val type: String = "",
|
||||
@@ -79,6 +87,8 @@ data class ChatPart(
|
||||
val executed: Boolean = false,
|
||||
val state: ChatToolState? = null,
|
||||
val time: ChatTime? = null,
|
||||
val uri: String = "",
|
||||
val mime: String = "",
|
||||
)
|
||||
|
||||
/** The lifecycle of a tool call; [input] is a partial-JSON string while streaming, an object afterwards. */
|
||||
@@ -187,6 +197,13 @@ object ChatProtocol {
|
||||
const val PART_REASONING = "reasoning"
|
||||
const val PART_TOOL = "tool"
|
||||
|
||||
/**
|
||||
* A file the operator attached to a prompt. The value is the app's own local
|
||||
* part kind for an optimistic user turn; the [ChatMessage.parts] the server
|
||||
* returns never use it (the brokered server stores user turns as text).
|
||||
*/
|
||||
const val PART_FILE = "file"
|
||||
|
||||
const val EVENT_STEP = "step"
|
||||
const val EVENT_USAGE = "usage"
|
||||
|
||||
@@ -218,6 +235,18 @@ object ChatProtocol {
|
||||
else -> false
|
||||
}
|
||||
|
||||
/**
|
||||
* The `chat_prompt` frame the composer sends. [files] are the attached files'
|
||||
* URIs, in the order they were attached, and are omitted when empty (the
|
||||
* shape a text-only prompt has always sent). Built here (not inline in the
|
||||
* repository) so production and its unit test share the one wire shape.
|
||||
*/
|
||||
fun promptRequest(sessionId: String, text: String, files: List<String> = emptyList()): Message =
|
||||
Message(
|
||||
type = Protocol.CHAT_PROMPT,
|
||||
chatPrompt = ChatPromptRequest(sessionId = sessionId, text = text, files = files),
|
||||
)
|
||||
|
||||
/**
|
||||
* The `chat_interrupt` frame the composer's stop control sends: the server
|
||||
* aborts the in-flight turn for [sessionId]. Built here (not inline in the
|
||||
|
||||
@@ -36,9 +36,11 @@ import net.awan.pipelines.data.protocol.SessionEvidence
|
||||
import net.awan.pipelines.data.protocol.Template
|
||||
import net.awan.pipelines.data.transport.ConnectionState
|
||||
import net.awan.pipelines.data.transport.ServerUrl
|
||||
import net.awan.pipelines.ui.chat.ChatAttachment
|
||||
import net.awan.pipelines.ui.chat.ChatSessionOverrides
|
||||
import net.awan.pipelines.ui.chat.ChatState
|
||||
import net.awan.pipelines.ui.chat.ChatStore
|
||||
import net.awan.pipelines.ui.chat.attachmentFiles
|
||||
|
||||
/** Generic loading/empty/error/data state used by every screen. */
|
||||
data class LoadState<T>(
|
||||
@@ -823,6 +825,18 @@ class AppViewModel(
|
||||
|
||||
fun saveChatScroll(sessionId: String, index: Int, offset: Int) = chatStore.scroll(sessionId, index, offset)
|
||||
|
||||
/**
|
||||
* Attaches files the picker already resolved to the next message. Reading the
|
||||
* files' metadata is the caller's job (it needs the Android content resolver);
|
||||
* the store only records them, so attaching itself transmits nothing.
|
||||
*/
|
||||
fun attachChatFiles(sessionId: String, attachments: List<ChatAttachment>) {
|
||||
attachments.forEach { chatStore.attach(sessionId, it) }
|
||||
}
|
||||
|
||||
fun removeChatAttachment(sessionId: String, attachmentId: String) =
|
||||
chatStore.removeAttachment(sessionId, attachmentId)
|
||||
|
||||
// ---- Directory choices and starting a new chat ---------------------------
|
||||
|
||||
/**
|
||||
@@ -892,21 +906,32 @@ class AppViewModel(
|
||||
}
|
||||
}
|
||||
|
||||
/** Sends the active chat's draft. Read-only chats are never written to. */
|
||||
/**
|
||||
* Sends the active chat's draft with its attachments. Read-only chats are
|
||||
* never written to, and a chat with a flagged attachment is not sent at all:
|
||||
* the file is never dropped and the message never goes without it. The
|
||||
* attachments travel only with this prompt, and are cleared once it is sent;
|
||||
* a server rejection takes the unsent bubble back out and returns the files,
|
||||
* each flagged with the server's reason.
|
||||
*/
|
||||
fun sendChatPrompt(sessionId: String) {
|
||||
val thread = chatStore.value.thread(sessionId) ?: return
|
||||
if (thread.readOnly) return
|
||||
if (!thread.canSend) return
|
||||
val text = thread.draft.trim()
|
||||
if (text.isEmpty()) return
|
||||
chatStore.appendLocalUserMessage(sessionId, text)
|
||||
val attachments = thread.attachments
|
||||
val localId = chatStore.appendLocalUserMessage(sessionId, text, attachments) ?: return
|
||||
chatStore.clearDraft(sessionId)
|
||||
chatStore.clearAttachments(sessionId)
|
||||
chatStore.running(sessionId, true)
|
||||
viewModelScope.launch {
|
||||
try {
|
||||
repository.chatPrompt(sessionId, text)
|
||||
repository.chatPrompt(sessionId, text, attachmentFiles(attachments))
|
||||
} catch (t: Throwable) {
|
||||
if (t is CancellationException) throw t
|
||||
chatStore.failed(sessionId, ErrorMessages.describe(t))
|
||||
val message = ErrorMessages.describe(t)
|
||||
chatStore.removeLocalMessage(sessionId, localId)
|
||||
chatStore.failed(sessionId, message)
|
||||
chatStore.restoreAttachments(sessionId, attachments, message)
|
||||
chatStore.running(sessionId, false)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,272 @@
|
||||
package net.awan.pipelines.ui.chat
|
||||
|
||||
import android.content.Context
|
||||
import android.graphics.BitmapFactory
|
||||
import android.net.Uri
|
||||
import android.provider.OpenableColumns
|
||||
import androidx.compose.foundation.Image
|
||||
import androidx.compose.foundation.background
|
||||
import androidx.compose.foundation.layout.Arrangement
|
||||
import androidx.compose.foundation.layout.Box
|
||||
import androidx.compose.foundation.layout.Column
|
||||
import androidx.compose.foundation.layout.Row
|
||||
import androidx.compose.foundation.layout.fillMaxWidth
|
||||
import androidx.compose.foundation.layout.padding
|
||||
import androidx.compose.foundation.layout.size
|
||||
import androidx.compose.foundation.layout.widthIn
|
||||
import androidx.compose.material.icons.Icons
|
||||
import androidx.compose.material.icons.automirrored.filled.InsertDriveFile
|
||||
import androidx.compose.material.icons.filled.Close
|
||||
import androidx.compose.material.icons.filled.Image
|
||||
import androidx.compose.material3.Icon
|
||||
import androidx.compose.material3.IconButton
|
||||
import androidx.compose.material3.MaterialTheme
|
||||
import androidx.compose.material3.Surface
|
||||
import androidx.compose.material3.Text
|
||||
import androidx.compose.runtime.Composable
|
||||
import androidx.compose.runtime.LaunchedEffect
|
||||
import androidx.compose.runtime.getValue
|
||||
import androidx.compose.runtime.mutableStateOf
|
||||
import androidx.compose.runtime.remember
|
||||
import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.draw.clip
|
||||
import androidx.compose.ui.graphics.ImageBitmap
|
||||
import androidx.compose.ui.graphics.asImageBitmap
|
||||
import androidx.compose.ui.layout.ContentScale
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.platform.LocalDensity
|
||||
import androidx.compose.ui.text.style.TextOverflow
|
||||
import androidx.compose.ui.unit.Dp
|
||||
import androidx.compose.foundation.shape.RoundedCornerShape
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.withContext
|
||||
import net.awan.pipelines.ui.humanBytes
|
||||
import net.awan.pipelines.ui.theme.CHAT_ATTACHMENT_NAME_MAX_WIDTH_DP
|
||||
import net.awan.pipelines.ui.theme.CHAT_ATTACHMENT_PREVIEW_DP
|
||||
import net.awan.pipelines.ui.theme.CHAT_ATTACHMENT_THUMB_DP
|
||||
import net.awan.pipelines.ui.theme.ICON_BUTTON_SMALL_DP
|
||||
import net.awan.pipelines.ui.theme.ICON_INLINE_DP
|
||||
import net.awan.pipelines.ui.theme.ICON_LIST_DP
|
||||
import net.awan.pipelines.ui.theme.SPACING_XXS
|
||||
import net.awan.pipelines.ui.theme.SPACING_XS
|
||||
|
||||
/**
|
||||
* The attachment surface: resolving a picked file into a [ChatAttachment], the
|
||||
* local thumbnail a preview draws, and the two places an attachment is shown —
|
||||
* the composer's removable chips and the file part inside a sent message's
|
||||
* bubble. Geometry comes from the named tokens, so no call site writes a raw dp.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Resolves one picked [uri] into the attachment the composer holds. It reads only
|
||||
* the file's metadata and confirms it can be opened; the bytes are not read (and
|
||||
* nothing is transmitted) until the message carrying it is sent. A file that
|
||||
* cannot be queried or opened is returned with its error set rather than dropped.
|
||||
*/
|
||||
internal fun resolveChatAttachment(context: Context, uri: Uri): ChatAttachment {
|
||||
val resolver = context.contentResolver
|
||||
var name = uri.lastPathSegment.orEmpty()
|
||||
var size = 0L
|
||||
var readable = true
|
||||
try {
|
||||
resolver.query(uri, arrayOf(OpenableColumns.DISPLAY_NAME, OpenableColumns.SIZE), null, null, null)?.use { c ->
|
||||
if (c.moveToFirst()) {
|
||||
val nameIndex = c.getColumnIndex(OpenableColumns.DISPLAY_NAME)
|
||||
if (nameIndex >= 0 && !c.isNull(nameIndex)) name = c.getString(nameIndex)
|
||||
val sizeIndex = c.getColumnIndex(OpenableColumns.SIZE)
|
||||
if (sizeIndex >= 0 && !c.isNull(sizeIndex)) size = c.getLong(sizeIndex)
|
||||
}
|
||||
}
|
||||
} catch (_: Exception) {
|
||||
readable = false
|
||||
}
|
||||
if (readable) {
|
||||
// Opening and closing proves the content is reachable without reading it.
|
||||
try {
|
||||
resolver.openInputStream(uri)?.close() ?: run { readable = false }
|
||||
} catch (_: Exception) {
|
||||
readable = false
|
||||
}
|
||||
}
|
||||
val mime = runCatching { resolver.getType(uri) }.getOrNull().orEmpty()
|
||||
return ChatAttachment(
|
||||
uri = uri.toString(),
|
||||
name = name,
|
||||
mime = mime,
|
||||
sizeBytes = size,
|
||||
error = attachmentError(name, size, readable),
|
||||
)
|
||||
}
|
||||
|
||||
/** The composer's attachment strip: one removable chip per attached file. */
|
||||
@Composable
|
||||
internal fun ChatAttachmentStrip(
|
||||
attachments: List<ChatAttachment>,
|
||||
onRemove: (String) -> Unit,
|
||||
modifier: Modifier = Modifier,
|
||||
) {
|
||||
Column(
|
||||
modifier = modifier.fillMaxWidth().padding(bottom = CHAT_ITEM_GAP),
|
||||
verticalArrangement = Arrangement.spacedBy(CHAT_ITEM_GAP),
|
||||
) {
|
||||
attachments.forEach { attachment ->
|
||||
ChatAttachmentChip(attachment = attachment, onRemove = { onRemove(attachment.id) })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** One attached file in the composer: its preview, its name and its remove action. */
|
||||
@Composable
|
||||
private fun ChatAttachmentChip(attachment: ChatAttachment, onRemove: () -> Unit) {
|
||||
val failed = attachment.error != null
|
||||
Surface(
|
||||
shape = RoundedCornerShape(CHAT_ITEM_GAP),
|
||||
color = if (failed) {
|
||||
MaterialTheme.colorScheme.errorContainer
|
||||
} else {
|
||||
MaterialTheme.colorScheme.surfaceContainerHigh
|
||||
},
|
||||
) {
|
||||
Row(
|
||||
verticalAlignment = Alignment.CenterVertically,
|
||||
horizontalArrangement = Arrangement.spacedBy(CHAT_ITEM_GAP),
|
||||
modifier = Modifier.padding(start = CHAT_ITEM_GAP, top = SPACING_XS, end = SPACING_XXS, bottom = SPACING_XS),
|
||||
) {
|
||||
AttachmentThumb(attachment, CHAT_ATTACHMENT_THUMB_DP)
|
||||
Column(Modifier.widthIn(max = CHAT_ATTACHMENT_NAME_MAX_WIDTH_DP)) {
|
||||
Text(
|
||||
text = attachment.displayName,
|
||||
style = MaterialTheme.typography.labelMedium,
|
||||
maxLines = 1,
|
||||
overflow = TextOverflow.Ellipsis,
|
||||
color = if (failed) MaterialTheme.colorScheme.onErrorContainer else MaterialTheme.colorScheme.onSurface,
|
||||
)
|
||||
AttachmentMeta(attachment)
|
||||
}
|
||||
IconButton(onClick = onRemove, modifier = Modifier.size(ICON_BUTTON_SMALL_DP)) {
|
||||
Icon(
|
||||
Icons.Default.Close,
|
||||
contentDescription = "Remove ${attachment.displayName}",
|
||||
modifier = Modifier.size(ICON_INLINE_DP),
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** The chip's second line: why it cannot go, or how big it is. */
|
||||
@Composable
|
||||
private fun AttachmentMeta(attachment: ChatAttachment) {
|
||||
val error = attachment.error
|
||||
if (error != null) {
|
||||
Text(
|
||||
text = error,
|
||||
style = MaterialTheme.typography.bodySmall,
|
||||
color = MaterialTheme.colorScheme.onErrorContainer,
|
||||
maxLines = 2,
|
||||
overflow = TextOverflow.Ellipsis,
|
||||
)
|
||||
} else if (attachment.sizeBytes > 0) {
|
||||
Text(
|
||||
text = humanBytes(attachment.sizeBytes),
|
||||
style = MaterialTheme.typography.bodySmall,
|
||||
color = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
maxLines = 1,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
/** An attached file inside the sent message's bubble: the same part the transcript renders. */
|
||||
@Composable
|
||||
internal fun FilePart(part: net.awan.pipelines.data.protocol.ChatPart) {
|
||||
val attachment = ChatAttachment(uri = part.uri, name = part.name, mime = part.mime)
|
||||
if (attachment.isImage) {
|
||||
AttachmentThumb(attachment, CHAT_ATTACHMENT_PREVIEW_DP)
|
||||
} else {
|
||||
Row(
|
||||
verticalAlignment = Alignment.CenterVertically,
|
||||
horizontalArrangement = Arrangement.spacedBy(CHAT_ITEM_GAP),
|
||||
) {
|
||||
Box(Modifier.size(ICON_LIST_DP), contentAlignment = Alignment.Center) {
|
||||
Icon(Icons.AutoMirrored.Filled.InsertDriveFile, contentDescription = null)
|
||||
}
|
||||
Text(
|
||||
text = attachment.displayName,
|
||||
style = MaterialTheme.typography.bodyMedium,
|
||||
maxLines = 1,
|
||||
overflow = TextOverflow.Ellipsis,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** A square preview: the decoded thumbnail for an image, a glyph for anything else. */
|
||||
@Composable
|
||||
private fun AttachmentThumb(attachment: ChatAttachment, size: Dp) {
|
||||
val shape = RoundedCornerShape(SPACING_XXS)
|
||||
if (attachment.isImage) {
|
||||
val bitmap = rememberLocalThumbnail(attachment.uri, size)
|
||||
if (bitmap != null) {
|
||||
Image(
|
||||
bitmap = bitmap,
|
||||
contentDescription = attachment.displayName,
|
||||
contentScale = ContentScale.Crop,
|
||||
modifier = Modifier.size(size).clip(shape),
|
||||
)
|
||||
} else {
|
||||
Box(
|
||||
modifier = Modifier.size(size).clip(shape).background(MaterialTheme.colorScheme.surfaceContainerHighest),
|
||||
contentAlignment = Alignment.Center,
|
||||
) {
|
||||
Icon(Icons.Default.Image, contentDescription = "Image preview", modifier = Modifier.size(ICON_INLINE_DP))
|
||||
}
|
||||
}
|
||||
} else {
|
||||
Box(Modifier.size(size), contentAlignment = Alignment.Center) {
|
||||
Icon(
|
||||
Icons.AutoMirrored.Filled.InsertDriveFile,
|
||||
contentDescription = null,
|
||||
modifier = Modifier.size(ICON_INLINE_DP),
|
||||
tint = MaterialTheme.colorScheme.onSurfaceVariant,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Decodes a local content URI to roughly [size] on the longest edge, off the main
|
||||
* thread. Null while loading or when the file is not a decodable image, so the
|
||||
* caller falls back to a glyph instead of showing a broken frame.
|
||||
*/
|
||||
@Composable
|
||||
private fun rememberLocalThumbnail(uri: String, size: Dp): ImageBitmap? {
|
||||
val context = LocalContext.current
|
||||
val maxDim = with(LocalDensity.current) { size.roundToPx() }
|
||||
var bitmap by remember(uri, maxDim) { mutableStateOf<ImageBitmap?>(null) }
|
||||
LaunchedEffect(uri, maxDim) {
|
||||
bitmap = withContext(Dispatchers.IO) { decodeLocalThumbnail(context, uri, maxDim) }
|
||||
}
|
||||
return bitmap
|
||||
}
|
||||
|
||||
/** Decodes [uri] at or below [maxDim] on its longest edge; null on any failure. */
|
||||
private fun decodeLocalThumbnail(context: Context, uri: String, maxDim: Int): ImageBitmap? = try {
|
||||
val resolver = context.contentResolver
|
||||
val parsed = Uri.parse(uri)
|
||||
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
|
||||
resolver.openInputStream(parsed)?.use { BitmapFactory.decodeStream(it, null, bounds) }
|
||||
if (bounds.outWidth <= 0 || bounds.outHeight <= 0) {
|
||||
null
|
||||
} else {
|
||||
var sample = 1
|
||||
while (bounds.outWidth / sample > maxDim || bounds.outHeight / sample > maxDim) {
|
||||
sample *= 2
|
||||
}
|
||||
val options = BitmapFactory.Options().apply { inSampleSize = sample }
|
||||
resolver.openInputStream(parsed)?.use { BitmapFactory.decodeStream(it, null, options)?.asImageBitmap() }
|
||||
}
|
||||
} catch (_: Exception) {
|
||||
null
|
||||
}
|
||||
@@ -10,6 +10,7 @@ import net.awan.pipelines.data.protocol.ChatPermissionFrame
|
||||
import net.awan.pipelines.data.protocol.ChatProtocol
|
||||
import net.awan.pipelines.data.protocol.ChatSession
|
||||
import net.awan.pipelines.data.protocol.ChatToolState
|
||||
import net.awan.pipelines.ui.humanBytes
|
||||
|
||||
/**
|
||||
* Pure chat formatting helpers shared by the session list and the conversation,
|
||||
@@ -325,3 +326,49 @@ internal fun decisionLabel(decision: String): String = when (decision) {
|
||||
ChatProtocol.DECISION_REJECT -> "Rejected"
|
||||
else -> "Resolved"
|
||||
}
|
||||
|
||||
/**
|
||||
* The largest file the composer will attach. A phone-side guard, so a runaway
|
||||
* pick is refused with a reason instead of being handed to the server; the
|
||||
* server applies its own limits on top and a prompt it rejects comes back
|
||||
* flagged against the attachment.
|
||||
*/
|
||||
internal const val MAX_ATTACHMENT_BYTES: Long = 25L * 1024 * 1024
|
||||
|
||||
/**
|
||||
* Why an attachment cannot be sent, or null when it can. [readable] is the
|
||||
* picker's own answer to whether the file could be opened; a file that cannot be
|
||||
* read or that is over [MAX_ATTACHMENT_BYTES] is reported by name, so the
|
||||
* operator sees which one and why rather than losing it.
|
||||
*/
|
||||
internal fun attachmentError(name: String, sizeBytes: Long, readable: Boolean): String? {
|
||||
val display = name.ifBlank { "file" }
|
||||
return when {
|
||||
!readable -> "\"$display\" could not be read."
|
||||
sizeBytes > MAX_ATTACHMENT_BYTES ->
|
||||
"\"$display\" is too large (${humanBytes(MAX_ATTACHMENT_BYTES)} limit)."
|
||||
else -> null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The `files` carried by a prompt: the URIs of the attachments that can go. One
|
||||
* place derives the wire list from the attachments, so the request the operator
|
||||
* builds and the transcript's own file parts can never disagree about which
|
||||
* files are attached.
|
||||
*/
|
||||
internal fun attachmentFiles(attachments: List<ChatAttachment>): List<String> =
|
||||
attachments.filter { it.error == null }.map { it.uri }
|
||||
|
||||
/**
|
||||
* One attachment as a [ChatProtocol.PART_FILE] part of the optimistic user turn,
|
||||
* so the transcript shows the file through the same parts it already renders
|
||||
* rather than a second attachment list under the bubble.
|
||||
*/
|
||||
internal fun attachmentPart(attachment: ChatAttachment): ChatPart = ChatPart(
|
||||
type = ChatProtocol.PART_FILE,
|
||||
id = attachment.id,
|
||||
name = attachment.name,
|
||||
uri = attachment.uri,
|
||||
mime = attachment.mime,
|
||||
)
|
||||
|
||||
@@ -34,6 +34,7 @@ import androidx.compose.material.icons.Icons
|
||||
import androidx.compose.material.icons.automirrored.filled.ArrowBack
|
||||
import androidx.compose.material.icons.automirrored.filled.Send
|
||||
import androidx.compose.material.icons.filled.Add
|
||||
import androidx.compose.material.icons.filled.AttachFile
|
||||
import androidx.compose.material.icons.filled.Build
|
||||
import androidx.compose.material.icons.filled.Close
|
||||
import androidx.compose.material.icons.filled.Edit
|
||||
@@ -85,6 +86,7 @@ import androidx.compose.runtime.setValue
|
||||
import androidx.compose.ui.Alignment
|
||||
import androidx.compose.ui.Modifier
|
||||
import androidx.compose.ui.draw.clip
|
||||
import androidx.compose.ui.platform.LocalContext
|
||||
import androidx.compose.ui.platform.LocalDensity
|
||||
import androidx.compose.ui.semantics.Role
|
||||
import androidx.compose.ui.text.font.FontFamily
|
||||
@@ -99,10 +101,14 @@ import androidx.compose.ui.window.PopupPositionProvider
|
||||
import androidx.compose.ui.window.PopupProperties
|
||||
import androidx.compose.foundation.selection.selectable
|
||||
import androidx.activity.compose.PredictiveBackHandler
|
||||
import androidx.activity.compose.rememberLauncherForActivityResult
|
||||
import androidx.activity.result.contract.ActivityResultContracts
|
||||
import androidx.lifecycle.compose.collectAsStateWithLifecycle
|
||||
import kotlinx.coroutines.Dispatchers
|
||||
import kotlinx.coroutines.delay
|
||||
import kotlinx.coroutines.flow.collect
|
||||
import kotlinx.coroutines.launch
|
||||
import kotlinx.coroutines.withContext
|
||||
import net.awan.pipelines.data.protocol.ChatMessage
|
||||
import net.awan.pipelines.data.protocol.ChatPart
|
||||
import net.awan.pipelines.data.protocol.ChatProtocol
|
||||
@@ -820,6 +826,8 @@ private fun ChatConversationPane(
|
||||
ChatComposer(
|
||||
thread = thread,
|
||||
onDraft = { vm.updateChatDraft(thread.sessionId, it) },
|
||||
onAttach = { vm.attachChatFiles(thread.sessionId, it) },
|
||||
onRemoveAttachment = { vm.removeChatAttachment(thread.sessionId, it) },
|
||||
onSend = { vm.sendChatPrompt(thread.sessionId) },
|
||||
onStop = { vm.stopChat(thread.sessionId) },
|
||||
)
|
||||
@@ -1089,10 +1097,7 @@ private fun MessageItem(message: ChatMessage) {
|
||||
SelectionContainer {
|
||||
Column(Modifier.padding(CHAT_GUTTER)) {
|
||||
if (isUser) {
|
||||
Text(
|
||||
text = message.parts.joinToString("\n") { it.text }.trim(),
|
||||
style = MaterialTheme.typography.bodyMedium,
|
||||
)
|
||||
UserParts(message.parts)
|
||||
} else {
|
||||
AssistantParts(message.parts)
|
||||
}
|
||||
@@ -1102,6 +1107,24 @@ private fun MessageItem(message: ChatMessage) {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A user turn: its text, then the files that rode with it. The attachments are
|
||||
* the same [ChatProtocol.PART_FILE] parts [MessageItem] already renders, so the
|
||||
* sent message shows its file in the transcript through the one display path
|
||||
* rather than a second attachment list under the bubble.
|
||||
*/
|
||||
@Composable
|
||||
private fun UserParts(parts: List<ChatPart>) {
|
||||
val text = parts.filter { it.type == ChatProtocol.PART_TEXT || it.type.isBlank() }
|
||||
.joinToString("\n") { it.text }
|
||||
.trim()
|
||||
val files = parts.filter { it.type == ChatProtocol.PART_FILE }
|
||||
Column(verticalArrangement = Arrangement.spacedBy(CHAT_ITEM_GAP)) {
|
||||
if (text.isNotBlank()) Text(text, style = MaterialTheme.typography.bodyMedium)
|
||||
files.forEach { FilePart(it) }
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* An assistant turn's parts in order, with each run of consecutive tool calls
|
||||
* rendered as one wrapping chip group (mirroring the reference transcript's
|
||||
@@ -1424,10 +1447,24 @@ private fun DiffView(diff: ToolDiff) {
|
||||
private fun ChatComposer(
|
||||
thread: ChatThread,
|
||||
onDraft: (String) -> Unit,
|
||||
onAttach: (List<ChatAttachment>) -> Unit,
|
||||
onRemoveAttachment: (String) -> Unit,
|
||||
onSend: () -> Unit,
|
||||
onStop: () -> Unit,
|
||||
) {
|
||||
val action = composerAction(thread)
|
||||
val context = LocalContext.current
|
||||
val scope = rememberCoroutineScope()
|
||||
// The platform picker: the operator chooses files and they are only resolved
|
||||
// into attachments here — nothing is sent until the message is.
|
||||
val picker = rememberLauncherForActivityResult(ActivityResultContracts.OpenMultipleDocuments()) { uris ->
|
||||
if (uris.isNotEmpty()) {
|
||||
scope.launch {
|
||||
val resolved = withContext(Dispatchers.IO) { uris.map { resolveChatAttachment(context, it) } }
|
||||
onAttach(resolved)
|
||||
}
|
||||
}
|
||||
}
|
||||
Surface(color = MaterialTheme.colorScheme.surfaceContainer) {
|
||||
Column(Modifier.fillMaxWidth().padding(CHAT_GUTTER)) {
|
||||
if (thread.readOnly) {
|
||||
@@ -1438,7 +1475,16 @@ private fun ChatComposer(
|
||||
modifier = Modifier.padding(bottom = CHAT_ITEM_GAP),
|
||||
)
|
||||
}
|
||||
if (thread.attachments.isNotEmpty()) {
|
||||
ChatAttachmentStrip(attachments = thread.attachments, onRemove = onRemoveAttachment)
|
||||
}
|
||||
Row(verticalAlignment = Alignment.Bottom, horizontalArrangement = Arrangement.spacedBy(CHAT_COMPOSER_GAP)) {
|
||||
IconButton(
|
||||
onClick = { picker.launch(arrayOf("*/*")) },
|
||||
enabled = !thread.readOnly,
|
||||
) {
|
||||
Icon(Icons.Default.AttachFile, contentDescription = "Attach file")
|
||||
}
|
||||
OutlinedTextField(
|
||||
value = thread.draft,
|
||||
onValueChange = onDraft,
|
||||
|
||||
@@ -17,6 +17,30 @@ import net.awan.pipelines.data.protocol.ChatSession
|
||||
*/
|
||||
data class ChatScrollPosition(val index: Int = 0, val offset: Int = 0)
|
||||
|
||||
/**
|
||||
* One file the operator attached to a not-yet-sent message. [uri] is the
|
||||
* platform picker's `content://` reference; the file's bytes are never read
|
||||
* until the message is sent, so attaching transmits nothing. [error] is set when
|
||||
* the file cannot be read or is too large (or when the server rejected the
|
||||
* prompt), so the operator sees what happened to that attachment instead of it
|
||||
* being silently dropped. [id] is assigned by [ChatStore] and is unique per open
|
||||
* conversation, so the composer can key and remove one attachment.
|
||||
*/
|
||||
data class ChatAttachment(
|
||||
val id: String = "",
|
||||
val uri: String,
|
||||
val name: String = "",
|
||||
val mime: String = "",
|
||||
val sizeBytes: Long = 0,
|
||||
val error: String? = null,
|
||||
) {
|
||||
/** Draw an inline thumbnail only for an image the picker typed as one. */
|
||||
val isImage: Boolean get() = mime.startsWith("image/")
|
||||
|
||||
/** The name shown for the attachment, never blank. */
|
||||
val displayName: String get() = name.ifBlank { uri.substringAfterLast('/').ifBlank { "file" } }
|
||||
}
|
||||
|
||||
/**
|
||||
* One open conversation. The conversation itself, whether it is running, its
|
||||
* unsent composer draft and its scroll position all live here, so switching to
|
||||
@@ -46,6 +70,12 @@ data class ChatThread(
|
||||
val runningSince: Long = 0,
|
||||
/** The unsent composer text for this chat. */
|
||||
val draft: String = "",
|
||||
/**
|
||||
* The files attached to the next message, in the order they were picked.
|
||||
* Kept per chat like the draft, cleared only once the prompt is actually
|
||||
* sent, and restored (flagged with the server's error) if the send fails.
|
||||
*/
|
||||
val attachments: List<ChatAttachment> = emptyList(),
|
||||
/**
|
||||
* Where the operator left this conversation's transcript, or null while they
|
||||
* have not been there yet. Null is distinct from any position — including the
|
||||
@@ -71,8 +101,15 @@ data class ChatThread(
|
||||
*/
|
||||
val readOnly: Boolean get() = busy && !running
|
||||
|
||||
/** Whether the composer's send action is allowed. */
|
||||
val canSend: Boolean get() = !readOnly && draft.isNotBlank()
|
||||
/** At least one attachment is flagged; the prompt must not be sent with it. */
|
||||
val hasAttachmentErrors: Boolean get() = attachments.any { it.error != null }
|
||||
|
||||
/**
|
||||
* Whether the composer's send action is allowed. A message with a flagged
|
||||
* attachment is never sent: the operator must remove or re-attach it first,
|
||||
* so the file is never silently dropped.
|
||||
*/
|
||||
val canSend: Boolean get() = !readOnly && draft.isNotBlank() && !hasAttachmentErrors
|
||||
|
||||
/**
|
||||
* Non-negative elapsed time of an in-flight turn at [nowMillis], or 0 once it
|
||||
@@ -184,6 +221,17 @@ class ChatStore(
|
||||
|
||||
val value: ChatState get() = _state.value
|
||||
|
||||
/** Monotonic source of attachment ids, so two attachments are never keyed alike. */
|
||||
private var attachmentSeq = 0
|
||||
|
||||
/**
|
||||
* Monotonic source of optimistic user-turn ids. Counting the local messages a
|
||||
* thread currently holds would re-mint the id of a turn that was removed (a
|
||||
* rejected send), so a later append could collide with a surviving turn and
|
||||
* duplicate a key the transcript's list is keyed by; a counter never repeats.
|
||||
*/
|
||||
private var localMessageSeq = 0
|
||||
|
||||
// ---- Session list --------------------------------------------------------
|
||||
|
||||
fun sessionsLoading() {
|
||||
@@ -395,6 +443,48 @@ class ChatStore(
|
||||
|
||||
fun clearDraft(sessionId: String) = update(sessionId) { it.copy(draft = "") }
|
||||
|
||||
// ---- Attachments ---------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Attaches one resolved file to the chat's next message and returns it with
|
||||
* the id the store assigned, or null when the chat is not open. The file's
|
||||
* bytes are not read here: attaching never transmits anything.
|
||||
*/
|
||||
fun attach(sessionId: String, attachment: ChatAttachment): ChatAttachment? {
|
||||
var stored: ChatAttachment? = null
|
||||
update(sessionId) { thread ->
|
||||
stored = attachment.copy(id = attachment.id.ifBlank { "att-" + (++attachmentSeq) })
|
||||
thread.copy(attachments = thread.attachments + stored!!)
|
||||
}
|
||||
return stored
|
||||
}
|
||||
|
||||
/** Removes one attachment from the next message. */
|
||||
fun removeAttachment(sessionId: String, attachmentId: String) = update(sessionId) {
|
||||
it.copy(attachments = it.attachments.filterNot { a -> a.id == attachmentId })
|
||||
}
|
||||
|
||||
/** Drops every attachment, used once the message carrying them is sent. */
|
||||
fun clearAttachments(sessionId: String) = update(sessionId) {
|
||||
if (it.attachments.isEmpty()) it else it.copy(attachments = emptyList())
|
||||
}
|
||||
|
||||
/**
|
||||
* Puts a rejected attachment set back on the chat with [error] flagged against
|
||||
* it. Used when the server rejected the prompt: nothing is silently dropped,
|
||||
* the files return to the composer and each one says why it did not go. Any
|
||||
* file attached in the meantime is kept, so a pick made between sending and
|
||||
* the rejection is not lost when the earlier set comes back.
|
||||
*/
|
||||
fun restoreAttachments(sessionId: String, attachments: List<ChatAttachment>, error: String) =
|
||||
update(sessionId) { thread ->
|
||||
val restored = attachments.map { it.copy(error = error) }
|
||||
val restoredIds = restored.mapTo(mutableSetOf()) { it.id }
|
||||
thread.copy(
|
||||
attachments = restored + thread.attachments.filterNot { it.id in restoredIds },
|
||||
)
|
||||
}
|
||||
|
||||
fun scroll(sessionId: String, index: Int, offset: Int) =
|
||||
update(sessionId) { it.copy(scroll = ChatScrollPosition(index, offset)) }
|
||||
|
||||
@@ -483,16 +573,35 @@ class ChatStore(
|
||||
|
||||
/**
|
||||
* Appends the operator's just-sent prompt optimistically, so the bubble is
|
||||
* visible before the ack and history round-trip. History later reconciles it.
|
||||
* visible before the ack and history round-trip. [attachments] ride as file
|
||||
* parts of that same turn, which is the one display path the transcript
|
||||
* already renders. Returns the local message id, so a failed send can take
|
||||
* the unsent bubble back out. Null when the chat is not open.
|
||||
*/
|
||||
fun appendLocalUserMessage(sessionId: String, text: String) = update(sessionId) {
|
||||
val message = ChatMessage(
|
||||
id = LOCAL_MESSAGE_PREFIX + (it.messages.count { m -> m.id.startsWith(LOCAL_MESSAGE_PREFIX) } + 1),
|
||||
sessionId = sessionId,
|
||||
role = ChatProtocol.ROLE_USER,
|
||||
parts = listOf(ChatPart(type = ChatProtocol.PART_TEXT, text = text)),
|
||||
)
|
||||
it.copy(messages = it.messages + message)
|
||||
fun appendLocalUserMessage(
|
||||
sessionId: String,
|
||||
text: String,
|
||||
attachments: List<ChatAttachment> = emptyList(),
|
||||
): String? {
|
||||
var id: String? = null
|
||||
update(sessionId) { thread ->
|
||||
id = LOCAL_MESSAGE_PREFIX + (++localMessageSeq)
|
||||
val message = ChatMessage(
|
||||
id = id!!,
|
||||
sessionId = sessionId,
|
||||
role = ChatProtocol.ROLE_USER,
|
||||
parts = listOf(ChatPart(type = ChatProtocol.PART_TEXT, text = text)) +
|
||||
attachments.map { attachmentPart(it) },
|
||||
)
|
||||
thread.copy(messages = thread.messages + message)
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
/** Removes one optimistic user turn (a send the server rejected). */
|
||||
fun removeLocalMessage(sessionId: String, messageId: String) = update(sessionId) {
|
||||
if (!messageId.startsWith(LOCAL_MESSAGE_PREFIX)) return@update it
|
||||
it.copy(messages = it.messages.filterNot { m -> m.id == messageId })
|
||||
}
|
||||
|
||||
// ---- Streaming events ----------------------------------------------------
|
||||
|
||||
@@ -228,6 +228,15 @@ internal val CHAT_TITLE_MAX_WIDTH_DP = 160.dp
|
||||
/** A chat message bubble's maximum width, so long prose stays readable. */
|
||||
internal val CHAT_BUBBLE_MAX_WIDTH_DP = 560.dp
|
||||
|
||||
/** An attachment's own thumbnail in the composer strip. */
|
||||
internal val CHAT_ATTACHMENT_THUMB_DP = 40.dp
|
||||
|
||||
/** An image attachment's preview inside the sent message's bubble. */
|
||||
internal val CHAT_ATTACHMENT_PREVIEW_DP = 160.dp
|
||||
|
||||
/** An attachment's name column stops here so its remove action keeps its place. */
|
||||
internal val CHAT_ATTACHMENT_NAME_MAX_WIDTH_DP = 180.dp
|
||||
|
||||
/**
|
||||
* How close to the end of the transcript the viewport must be for new content to
|
||||
* keep following it: about one comfortable line of text, so the operator can be a
|
||||
|
||||
@@ -218,6 +218,32 @@ class ChatProtocolTest {
|
||||
assertEquals(ChatProtocol.DECISION_ONCE, decodedReply.chatReply!!.decision)
|
||||
}
|
||||
|
||||
/**
|
||||
* The prompt the composer builds carries its attached files, in the order they
|
||||
* were attached, and a text-only prompt omits the field entirely so the shape
|
||||
* a text-only prompt has always sent is unchanged.
|
||||
*/
|
||||
@Test
|
||||
fun chatPromptRequestCarriesTheAttachedFileUris() {
|
||||
val request = ChatProtocol.promptRequest(
|
||||
sessionId = "ses_1",
|
||||
text = "look at these",
|
||||
files = listOf("content://media/1", "content://media/2"),
|
||||
)
|
||||
assertEquals(Protocol.CHAT_PROMPT, request.type)
|
||||
assertEquals(listOf("content://media/1", "content://media/2"), request.chatPrompt!!.files)
|
||||
|
||||
val encoded = PipelinesJson.encodeToString(Message.serializer(), request.copy(id = "android-1"))
|
||||
val decoded = PipelinesJson.decodeFromString(Message.serializer(), encoded)
|
||||
assertEquals(listOf("content://media/1", "content://media/2"), decoded.chatPrompt!!.files)
|
||||
|
||||
val textOnly = PipelinesJson.encodeToString(
|
||||
Message.serializer(),
|
||||
ChatProtocol.promptRequest("ses_1", "hi").copy(id = "android-2"),
|
||||
)
|
||||
assertTrue("a text-only prompt must not grow a files field: $textOnly", !textOnly.contains("files"))
|
||||
}
|
||||
|
||||
/**
|
||||
* The stop control's interrupt request: pressing it while a turn is in flight
|
||||
* must send a `chat_interrupt` carrying the running session's id and nothing
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
package net.awan.pipelines.ui.chat
|
||||
|
||||
import net.awan.pipelines.data.protocol.ChatMessage
|
||||
import net.awan.pipelines.data.protocol.ChatPart
|
||||
import net.awan.pipelines.data.protocol.ChatProtocol
|
||||
import net.awan.pipelines.data.protocol.ChatSession
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertFalse
|
||||
import org.junit.Assert.assertNotNull
|
||||
import org.junit.Assert.assertNull
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Test
|
||||
|
||||
/**
|
||||
* The attachment lifecycle on [ChatStore]: files attach without transmitting,
|
||||
* one can be removed, a flagged one blocks the send, sending clears them while
|
||||
* the optimistic turn still shows them, and a server rejection puts them back
|
||||
* with the reason rather than dropping them.
|
||||
*/
|
||||
class ChatAttachmentStateTest {
|
||||
|
||||
private val session = ChatSession(id = "a", title = "Chat A")
|
||||
|
||||
private fun store(): ChatStore = ChatStore().apply { open(session) }
|
||||
|
||||
private fun attachment(uri: String, name: String, error: String? = null) =
|
||||
ChatAttachment(uri = uri, name = name, mime = "image/png", error = error)
|
||||
|
||||
@Test
|
||||
fun attachingRecordsEachFileWithADistinctIdAndOneCanBeRemoved() {
|
||||
val store = store()
|
||||
val first = store.attach("a", attachment("content://1", "one.png"))!!
|
||||
val second = store.attach("a", attachment("content://2", "two.png"))!!
|
||||
assertTrue(first.id.isNotBlank())
|
||||
assertTrue(first.id != second.id)
|
||||
assertEquals(listOf("one.png", "two.png"), store.value.thread("a")!!.attachments.map { it.name })
|
||||
|
||||
store.removeAttachment("a", first.id)
|
||||
assertEquals(listOf("two.png"), store.value.thread("a")!!.attachments.map { it.name })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun aFlaggedAttachmentBlocksSendingUntilItIsRemoved() {
|
||||
val store = store()
|
||||
val ok = store.attach("a", attachment("content://1", "one.png"))!!
|
||||
val bad = store.attach("a", attachment("content://2", "broken.png", error = "could not be read"))!!
|
||||
store.draft("a", "hello")
|
||||
|
||||
assertTrue(store.value.thread("a")!!.hasAttachmentErrors)
|
||||
assertFalse(store.value.thread("a")!!.canSend)
|
||||
|
||||
// Removing the flagged one makes the message sendable again.
|
||||
store.removeAttachment("a", bad.id)
|
||||
assertFalse(store.value.thread("a")!!.hasAttachmentErrors)
|
||||
assertTrue(store.value.thread("a")!!.canSend)
|
||||
assertNotNull(ok)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun sendingClearsTheAttachmentsAndStillShowsThemOnTheOptimisticTurn() {
|
||||
val store = store()
|
||||
val one = store.attach("a", attachment("content://1", "one.png"))!!
|
||||
store.draft("a", "look")
|
||||
|
||||
val messageId = store.appendLocalUserMessage("a", "look", store.value.thread("a")!!.attachments)!!
|
||||
store.clearDraft("a")
|
||||
store.clearAttachments("a")
|
||||
|
||||
// Cleared for the next message...
|
||||
assertTrue(store.value.thread("a")!!.attachments.isEmpty())
|
||||
assertEquals("", store.value.thread("a")!!.draft)
|
||||
// ...but the just-sent turn carries the attachment as a file part.
|
||||
val turn = store.value.thread("a")!!.messages.single()
|
||||
assertEquals(messageId, turn.id)
|
||||
assertEquals(ChatProtocol.ROLE_USER, turn.role)
|
||||
assertEquals("look", turn.parts.first { it.type == ChatProtocol.PART_TEXT }.text)
|
||||
val file = turn.parts.single { it.type == ChatProtocol.PART_FILE }
|
||||
assertEquals(one.uri, file.uri)
|
||||
assertEquals("one.png", file.name)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun aRejectedSendDropsTheBubbleAndReturnsTheFileFlagged() {
|
||||
val store = store()
|
||||
val files = listOf(store.attach("a", attachment("content://1", "one.png"))!!)
|
||||
store.draft("a", "look")
|
||||
val messageId = store.appendLocalUserMessage("a", "look", files)!!
|
||||
store.clearDraft("a")
|
||||
store.clearAttachments("a")
|
||||
|
||||
// The server rejected the prompt: take the unsent bubble back out and put
|
||||
// the file back, flagged with the server's reason. Nothing is dropped.
|
||||
store.removeLocalMessage("a", messageId)
|
||||
store.restoreAttachments("a", files, "the server rejected the prompt")
|
||||
|
||||
assertTrue(store.value.thread("a")!!.messages.isEmpty())
|
||||
val restored = store.value.thread("a")!!.attachments.single()
|
||||
assertEquals("one.png", restored.name)
|
||||
assertEquals("the server rejected the prompt", restored.error)
|
||||
assertFalse(store.value.thread("a")!!.canSend)
|
||||
}
|
||||
|
||||
/**
|
||||
* A file picked after sending but before the rejection rides along: restoring
|
||||
* the rejected set must not replace the one attached in the meantime.
|
||||
*/
|
||||
@Test
|
||||
fun restoringARejectedSetKeepsAFileAttachedInTheMeantime() {
|
||||
val store = store()
|
||||
val sent = store.attach("a", attachment("content://1", "one.png"))!!
|
||||
store.draft("a", "look")
|
||||
store.appendLocalUserMessage("a", "look", listOf(sent))
|
||||
store.clearAttachments("a")
|
||||
val later = store.attach("a", attachment("content://2", "two.png"))!!
|
||||
|
||||
store.restoreAttachments("a", listOf(sent), "the server rejected the prompt")
|
||||
|
||||
val attachments = store.value.thread("a")!!.attachments
|
||||
assertEquals(listOf("one.png", "two.png"), attachments.map { it.name })
|
||||
assertEquals("the server rejected the prompt", attachments.single { it.name == "one.png" }.error)
|
||||
assertNull(attachments.single { it.name == "two.png" }.error)
|
||||
assertNotNull(later)
|
||||
}
|
||||
|
||||
/**
|
||||
* A rejected send takes its turn out; the id it held is never minted again, so
|
||||
* the next local turn cannot collide with a surviving one (duplicate keys the
|
||||
* transcript's list would choke on).
|
||||
*/
|
||||
@Test
|
||||
fun aRemovedTurnIdIsNeverReused() {
|
||||
val store = store()
|
||||
val first = store.appendLocalUserMessage("a", "one")!!
|
||||
store.removeLocalMessage("a", first)
|
||||
val second = store.appendLocalUserMessage("a", "two")!!
|
||||
|
||||
assertTrue(first != second)
|
||||
assertEquals(1, store.value.thread("a")!!.messages.size)
|
||||
assertEquals(second, store.value.thread("a")!!.messages.single().id)
|
||||
}
|
||||
|
||||
/** Removing a message is a no-op unless it is one of this device's own turns. */
|
||||
@Test
|
||||
fun onlyLocalMessagesCanBeRemoved() {
|
||||
val store = store()
|
||||
store.history(
|
||||
"a",
|
||||
listOf(
|
||||
ChatMessage(
|
||||
id = "srv-1",
|
||||
sessionId = "a",
|
||||
role = ChatProtocol.ROLE_USER,
|
||||
parts = listOf(ChatPart(type = ChatProtocol.PART_TEXT, text = "from the server")),
|
||||
),
|
||||
),
|
||||
)
|
||||
store.removeLocalMessage("a", "srv-1")
|
||||
assertEquals(1, store.value.thread("a")!!.messages.size)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
package net.awan.pipelines.ui.chat
|
||||
|
||||
import net.awan.pipelines.data.protocol.ChatProtocol
|
||||
import net.awan.pipelines.ui.humanBytes
|
||||
import org.junit.Assert.assertEquals
|
||||
import org.junit.Assert.assertNull
|
||||
import org.junit.Assert.assertTrue
|
||||
import org.junit.Test
|
||||
|
||||
/**
|
||||
* The pure attachment helpers the composer and its model share: what makes one
|
||||
* un-sendable, which files travel with a prompt, and how an attachment becomes
|
||||
* the file part the transcript renders. No device, server or Compose runtime.
|
||||
*/
|
||||
class ChatAttachmentTest {
|
||||
|
||||
@Test
|
||||
fun unreadableFileIsReportedByNameAndNeverSendable() {
|
||||
val error = attachmentError(name = "shot.png", sizeBytes = 10, readable = false)!!
|
||||
assertTrue(error.contains("shot.png"))
|
||||
assertTrue(error.contains("could not be read"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun oversizedFileIsRejectedWithTheLimit() {
|
||||
val error = attachmentError(name = "movie.mp4", sizeBytes = MAX_ATTACHMENT_BYTES + 1, readable = true)!!
|
||||
assertTrue(error.contains("movie.mp4"))
|
||||
assertTrue(error.contains("too large"))
|
||||
assertTrue(error.contains(humanBytes(MAX_ATTACHMENT_BYTES)))
|
||||
|
||||
// Exactly at the limit is still allowed.
|
||||
assertNull(attachmentError(name = "ok.png", sizeBytes = MAX_ATTACHMENT_BYTES, readable = true))
|
||||
assertNull(attachmentError(name = "ok.png", sizeBytes = 0, readable = true))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun anUnnamedAttachmentStillNamesItself() {
|
||||
assertTrue(attachmentError(name = "", sizeBytes = 1, readable = false)!!.contains("file"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun onlyAttachmentsWithoutAnErrorTravelWithThePrompt() {
|
||||
val ok = ChatAttachment(uri = "content://ok", name = "ok.png", mime = "image/png")
|
||||
val bad = ChatAttachment(uri = "content://bad", name = "bad.png", error = "rejected")
|
||||
assertEquals(listOf("content://ok"), attachmentFiles(listOf(ok, bad)))
|
||||
assertTrue(attachmentFiles(listOf(bad)).isEmpty())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun anAttachmentBecomesAFilePartOnTheOptimisticTurn() {
|
||||
val attachment = ChatAttachment(
|
||||
id = "att-1",
|
||||
uri = "content://media/1",
|
||||
name = "shot.png",
|
||||
mime = "image/png",
|
||||
)
|
||||
val part = attachmentPart(attachment)
|
||||
assertEquals(ChatProtocol.PART_FILE, part.type)
|
||||
assertEquals("att-1", part.id)
|
||||
assertEquals("shot.png", part.name)
|
||||
assertEquals("content://media/1", part.uri)
|
||||
assertEquals("image/png", part.mime)
|
||||
assertTrue(attachment.isImage)
|
||||
assertEquals("shot.png", attachment.displayName)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user