Files
pipelines/app/notes/chat-interactive-terminal.md
T
alan 2332143e5b Fit chat terminals to their output, keep pastes and keys in the terminal
Finished terminals are sized synchronously from their rows at the view's
width (the post-frame fit made the transcript re-anchor forever); blank rows
xterm.dart keeps when rewrapping are dropped; desktop typing uses key events;
the composer leaves Cmd/Ctrl-V to a focused terminal; Ctrl-A reaches the
process. The probe fixture gains coloured terminal runs and an interactive
login. Notes record behaviour, protocol, security and verification.
2026-11-09 20:51:59 +00:00

171 lines
10 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Interactive chat shell commands (terminal)
Operator shell commands (`!cmd`, Run command…; see `chat-shell-commands.md`)
now run in a real pseudo-terminal and the card draws them with a terminal
emulator. Auth flows that print a link, wait for a pasted token and continue
(`gh auth login`, `claude login`, `npm login`, ssh prompts) work in the chat.
## Behaviour
- **Server (macOS, Linux):** every `chat_shell_run` starts `$SHELL -lc <cmd>`
inside a PTY (`github.com/creack/pty` v1.1.24) as the leader of a new
session with the PTY as controlling terminal (`Setsid`, `Setctty`), so the
session id is the process group Stop signals (SIGINT, then SIGKILL after the
grace period). `TERM=xterm-256color`; cwd and the rest of the environment are
unchanged. Size comes from the request (`cols`, `rows`), clamped to
20–500 x 4–200, default 100x30. The PTY controller is closed when the command
ends. Other platforms keep the previous pipe mode; input and resize answer
"interactive terminals are not supported on this server".
- **Card:** the output area is an `xterm` 4.0.0 (xterm.dart) `TerminalView`
fed the raw stream in order. One emulator per run lives in the controller
(`ChatShellTerminals`), so scrolling a card away or opening Full screen keeps
the same state.
- Running: 12 rows in the reading column; click/tap to focus and type
(hardware keys, Ctrl-C, arrows, Tab, Esc go to the process). Paste:
Cmd-V on Apple platforms, Ctrl-V or Ctrl-Shift-V elsewhere, plus the
toolbar's Paste button. Ctrl-A and other Ctrl keys reach the process. A key
toolbar under the terminal: Paste, Enter, Ctrl-C, Tab, Esc, arrows. On
Android/iOS the toolbar also has a text field ("Type input, then Enter")
for the soft keyboard, with a Hide typing toggle; it is emptied when sent.
- The terminal is resized to its layout (debounced 150 ms); the process gets
SIGWINCH. New runs start at the last laid-out size.
- Printed `http(s)` links open in the browser on tap (wrapped links are
joined). Other schemes are ignored.
- Finished runs are drawn read-only, shrunk to the rows their output takes
(at most 12), cursor hidden. Full screen opens the same terminal in a page.
- Colours: background/foreground/cursor/selection from the host theme; the
16 ANSI colours are xterm.dart's palette, darkened on light surfaces.
- Copy output copies the server's plain-text rendering; Send to agent shares
a bounded plain-text tail (no escape codes).
- Pipe-mode and older runs (no `terminal`) are drawn in the same emulator with
`\n` drawn as a new line, read-only.
## Protocol
- `chat_shell_run` request adds optional `cols`, `rows`; records add
`terminal`, `cols`, `rows` (current size) and, only in
`chat_shell_run_output`, `text` (plain-text rendering).
- `chat_shell_input {session_id, run_id, data}`: `data` is UTF-8 text written
byte-for-byte (its UTF-8 encoding). Control characters are JSON escapes:
Ctrl-C `\u0003`, Enter `\r`, arrows `\u001b[A`… Paste is text (bracketed when
the program asked). At most 16 KiB per request (clients split longer pastes
at 4096 UTF-16 units); invalid UTF-8 is refused. Requests from one
connection are applied in order through a 64-entry queue; a full queue is
refused ("the command is not reading its input"). Reply has no body.
- `chat_shell_resize {session_id, run_id, cols, rows}`: reply echoes the run.
The last resize from any client wins.
- Output frames are unchanged: raw terminal bytes (UTF-8, partial characters
held back) with absolute byte offsets `[offset, end)`, which are the stream
sequence numbers. `chat_shell_run_output` takes `after`: the reply's
`output_start` is `max(after, retained start)`, so a reconnecting client
fetches only what it missed and sees a gap if it fell out of the 1 MiB tail.
The app does this on reconnect for runs it holds.
- Persistence: `chat_shell_runs` gains `terminal`, `cols`, `rows`,
`output_text` (additive columns). `output` is the raw stream tail (1 MiB),
`output_text` its rendering (bounded to 1 MiB). Rows recorded before this
change get a rendering computed on read.
## Security
- Any authenticated client watching the chat may type into or resize a
running command, not only the one that started it (the same authority that
can already start and stop commands in that chat). Input is checked against
the chat's operator session like every other shell request.
- Input is never persisted or logged: the server does not log
`chat_shell_input`/`chat_shell_resize` request lines at all (keystrokes
would otherwise be one log line each), and only what the terminal echoes
becomes output. Password prompts disable echo, so hidden input never
appears in output, frames, `output` or `output_text`.
- The client keeps no copy of typed input: keys go from the emulator straight
into a send queue that is cleared as each request is sent (and discarded on
a refusal); the mobile field is cleared on send and can hide its text.
- When the terminal has focus, the chat composer leaves Cmd/Ctrl-V alone
(`ChatComposerPasteFocus.registerInput`), so a pasted token goes only to the
process.
## Limits and known gaps
- Plain-text rendering is not an emulator: it strips CSI/OSC/DCS sequences,
applies `\r`, backspace and line erase; full-screen programs render as their
last lines. The card's emulator draws them properly.
- Every watching client's emulator answers terminal queries (e.g. cursor
position) for a live run, so two open clients send two answers.
xterm.dart 4.0.0 answers DSR 6 zero-based; the app overrides it to the
correct one-based answer.
- xterm.dart 4.0.0 keeps blank rows when it rewraps; the app drops blank rows
below the cursor after a resize so the top of the output stays in view.
- Desktop typing uses key events (no IME composition for CJK input). Touch
platforms use the field.
- A descendant that keeps the PTY open after the command exits is read for
1 s; on macOS the PTY has no read deadline, so its reader goroutine ends
only when that descendant closes it (the run itself is finished and the
group killed).
- Mobile behaviour (soft keyboard field) is verified by widget tests only; no
device test ran. Android/Linux builds were not run (web and macOS were).
## Files
- Server: `internal/protocol/chat_shell_terminal.go`, `chat_shell_run.go`;
`internal/server/chat_shell_terminal{,_unix,_other}.go`,
`chat_shell_plain.go`, `chat_shell_runner.go`, `chat_shell_supervise.go`,
`chat_shell_run_handlers.go`, `chat_shell_share.go`, `chat.go`,
`websocket.go`; `internal/pipeline/chat_shell_runs.go`; `go.mod`, `go.sum`.
- App: `lib/features/chat/shell/chat_shell_terminal_{session,view,keys,theme,links,page}.dart`,
`chat_shell_terminals.dart`, `chat_shell_output_view.dart`,
`chat_shell_run{,_card}.dart`, `chat_shell_card_actions.dart`,
`chat_shell_runs_{controller,repository}.dart`, `chat_shell_metrics.dart`;
`lib/features/chat/conversation/chat_composer_paste_focus.dart`;
`pubspec.yaml`/`pubspec.lock` (`xterm: ^4.0.0`, pure Dart; pub.dev lists
Android, iOS, Linux, macOS, web, Windows); probe fixture
`tool/perf/chat_shell_fixture.dart` (`?shell=1`: coloured runs and an
interactive `gh auth login` that hides the token and finishes on Enter).
## Verification
Flutter 3.44.6 / Dart 3.12.2; macOS host.
- `go test ./internal/protocol ./internal/pipeline ./internal/server -count=1`:
`ok …/protocol 0.205s`, `ok …/pipeline 15.838s`, `ok …/server 140.086s`.
Real-PTY tests (`chat_shell_terminal_test.go`, `_ws_test.go`): a typed line
reaches `read`; `stty -echo` input is in no frame, output or text; `\x03`
interrupts; `stty size` sees the start size and a resize; resume by offset
over the websocket from a second client; raw + plain text persisted;
input bounds (16 KiB, empty, invalid UTF-8, full queue); size clamping.
`go test -run ChatShell -count=5`: ok. `GOOS=freebsd go build
./internal/server` (pipe fallback) and `GOOS=linux GOARCH=amd64 go test -c`
compile. (`-race` shows an existing data race in
`internal/opencode` `nativeRequest`, unrelated to this change.)
- `tool/memwatch/full.sh test/features/chat`: `PASS: 1124 passed, 0 failed`.
New: `test/features/chat/shell/chat_shell_terminal_test.dart` (desktop keys,
Cmd-V paste not taken by the composer, Paste button, touch toolbar and
hidden field cleared on send, resize sent, finished run read-only, colours,
reconnect resume), `chat_shell_terminal_session_test.dart` (links incl.
wrapped and non-http, query answers not resent on replay, one-based DSR,
gap redraw, rewrap keeps first line, row fitting).
- `tool/analyze.sh`: clean. `flutter build web`: built.
`FLUTTER_XCODE_ARCHS=arm64 flutter build macos`: built (34.9MB).
- Real Chrome (`tool/capture`, probe `?chats=2&shell=1`, 880x1000, Material and
Cupertino, light and dark): `/tmp/chat-interactive-terminal/<platform>-<theme>/`
`01-login-prompt.png` (coloured output, underlined link, prompt and cursor),
`02-login-focused.png` (focused cursor), `03-login-done.png` (5 typed keys
and Enter reached the fixture: "received a 5-character token", nothing
echoed; the card shrank to its output, read-only), `04-full-screen.png`.
- Scratch server end to end (`go build ./cmd/pipelines-server`, `env -i`
with HOME/XDG_DATA_HOME/DB under `/tmp/pty-e2e`, OS-chosen loopback port,
operator session inserted with sqlite3, Node websocket driver
`/tmp/pty-e2e/drive.mjs`; server stopped afterwards, no processes left):
```
started run_45be206dedb0454409c537b6 terminal=true size=90x20
resize reply: chat_shell_resize 120x40
input reply: chat_shell_input
final: status=exited exit=0 bytes=145
"tty: xterm-256color\r\n20 90\r\n\u001b[1;32mOpen https://example.com/login\u001b[0m\r\nPaste token: \r\n40 120\r\ngot 13 characters\r\nName: operator\r\nhello operator\r\n"
secret in stream: false
resume after 119: output_start=119 output="operator\r\nhello operator\r\n"
secret in text: false
late input: error this command is not running
run_…|exited|0|1|120|40|145|125|0 (sqlite: terminal, cols, rows, raw, text, SECRET offset)
```
The server log has no `chat_shell_input` or `chat_shell_resize` lines.