Read chat replies aloud with word highlighting
An assistant reply gets Read aloud beside Copy; while it reads it becomes Pause/Resume and Stop, one reply at a time app-wide. The spoken word is highlighted in the rendered reply (across multi-part replies) and the transcript scrolls only when that word leaves the viewport. Reading stops when its chat closes or is swapped, another reply starts, or the app goes to the background. Markdown is spoken as sentences; code blocks are announced as "Code block.". Speech goes through a SpeechEngine seam with a flutter_tts implementation (Android, macOS, web; hidden on Linux) and a fake for tests. Settings has a Read aloud section: a voice picker grouped by language (device language first, a sample per voice), a rate slider and reset, saved per device. flutter_tts has no Swift package, so macOS/iOS gain the CocoaPods integration (Podfiles with the Runner deployment targets); Android declares the TTS_SERVICE query. The capture fixture gains ?speech=fake.
This commit is contained in:
@@ -0,0 +1,55 @@
|
||||
# Read aloud
|
||||
|
||||
An assistant reply has Read aloud beside Copy (same `IconButton` style). While
|
||||
that reply reads it becomes Pause/Resume and Stop; one reply reads at a time
|
||||
app-wide. The spoken word is highlighted (theme `primaryContainer`) in the
|
||||
rendered reply and the transcript scrolls only when that word leaves the
|
||||
viewport. Settings > Read aloud picks the voice (grouped by language, device
|
||||
language first, a sample per voice), the rate (0.5x to 2.0x) and resets both;
|
||||
they are saved on the device.
|
||||
|
||||
## Structure
|
||||
|
||||
- `lib/features/chat/speech/speech_engine.dart`: the `SpeechEngine` seam
|
||||
(speak, pause, stop, setVoice, setRate, voices, start/pause/resume/
|
||||
progress/completion/cancel/error callbacks).
|
||||
- `flutter_tts_speech_engine.dart`: the `flutter_tts` implementation. The rate
|
||||
is a multiplier (1.0 = normal); Apple platforms get half of it on their 0..1
|
||||
scale. Linux reports unsupported, so the button and section are hidden.
|
||||
- `speech_controller.dart`: the app's single reader (`speechController`, a fake
|
||||
in tests via `debugSpeechController`). State changes notify the controller;
|
||||
the spoken word is the separate `word` notifier. Stops on `hidden`, `paused`
|
||||
or `detached` (not `inactive`, which is only a lost focus).
|
||||
- `speech_document.dart`: Markdown to sentences, with each spoken character's
|
||||
part and source offset. Code blocks (``` and ~~~) are "Code block.", tables
|
||||
"Table." then cells; rules are skipped.
|
||||
- `chat_markdown_speech.dart`: highlight spans and the reveal scroll for
|
||||
`ChatMarkdownView`.
|
||||
- Reader ids are `<sessionId>/<row id>` so a chat that closes or is swapped
|
||||
for another stops only its own reply (`stopScope`).
|
||||
|
||||
## Platform notes
|
||||
|
||||
- Resume speaks the paused text again; Android, Apple and web plugins all
|
||||
treat that as a resume. Android then reports progress against the remaining
|
||||
suffix, which the controller maps back to the whole text.
|
||||
- After a stop the controller waits (at most 400 ms) for the platform cancel,
|
||||
so a late cancel cannot end the next reply and web speech is idle before
|
||||
the next utterance.
|
||||
- Apple and web `flutter_tts` cannot clear a chosen voice. After a voice was
|
||||
set in a run, "System default" uses the device language's first voice.
|
||||
- Platforms without word boundaries read without a highlight.
|
||||
- `flutter_tts` has no Swift package, so macOS (and iOS) builds use CocoaPods
|
||||
for it: `Podfile` and the `Pods-Runner` xcconfig includes are the generated
|
||||
plugin integration.
|
||||
|
||||
## Verification
|
||||
|
||||
Tests (fake engine): `test/features/chat/speech/`,
|
||||
`test/features/chat/ui/transcript/chat_read_aloud_button_test.dart`,
|
||||
`chat_markdown_read_aloud_test.dart`,
|
||||
`test/features/chat/conversation/chat_thread_read_aloud_test.dart`,
|
||||
`test/features/settings/settings_read_aloud_section_test.dart`.
|
||||
|
||||
Not verified on a device: real speech output, word boundaries and voices on
|
||||
Android, macOS and browsers.
|
||||
Reference in New Issue
Block a user