Files
pipelines/app/notes/chat-interactive-terminal.md
alan 2332143e5b Fit chat terminals to their output, keep pastes and keys in the terminal
Finished terminals are sized synchronously from their rows at the view's
width (the post-frame fit made the transcript re-anchor forever); blank rows
xterm.dart keeps when rewrapping are dropped; desktop typing uses key events;
the composer leaves Cmd/Ctrl-V to a focused terminal; Ctrl-A reaches the
process. The probe fixture gains coloured terminal runs and an interactive
login. Notes record behaviour, protocol, security and verification.
2026-11-09 20:51:59 +00:00

10 KiB
Raw Permalink Blame History

Interactive chat shell commands (terminal)

Operator shell commands (!cmd, Run command…; see chat-shell-commands.md) now run in a real pseudo-terminal and the card draws them with a terminal emulator. Auth flows that print a link, wait for a pasted token and continue (gh auth login, claude login, npm login, ssh prompts) work in the chat.

Behaviour

  • Server (macOS, Linux): every chat_shell_run starts $SHELL -lc <cmd> inside a PTY (github.com/creack/pty v1.1.24) as the leader of a new session with the PTY as controlling terminal (Setsid, Setctty), so the session id is the process group Stop signals (SIGINT, then SIGKILL after the grace period). TERM=xterm-256color; cwd and the rest of the environment are unchanged. Size comes from the request (cols, rows), clamped to 20–500 x 4–200, default 100x30. The PTY controller is closed when the command ends. Other platforms keep the previous pipe mode; input and resize answer "interactive terminals are not supported on this server".
  • Card: the output area is an xterm 4.0.0 (xterm.dart) TerminalView fed the raw stream in order. One emulator per run lives in the controller (ChatShellTerminals), so scrolling a card away or opening Full screen keeps the same state.
    • Running: 12 rows in the reading column; click/tap to focus and type (hardware keys, Ctrl-C, arrows, Tab, Esc go to the process). Paste: Cmd-V on Apple platforms, Ctrl-V or Ctrl-Shift-V elsewhere, plus the toolbar's Paste button. Ctrl-A and other Ctrl keys reach the process. A key toolbar under the terminal: Paste, Enter, Ctrl-C, Tab, Esc, arrows. On Android/iOS the toolbar also has a text field ("Type input, then Enter") for the soft keyboard, with a Hide typing toggle; it is emptied when sent.
    • The terminal is resized to its layout (debounced 150 ms); the process gets SIGWINCH. New runs start at the last laid-out size.
    • Printed http(s) links open in the browser on tap (wrapped links are joined). Other schemes are ignored.
    • Finished runs are drawn read-only, shrunk to the rows their output takes (at most 12), cursor hidden. Full screen opens the same terminal in a page.
    • Colours: background/foreground/cursor/selection from the host theme; the 16 ANSI colours are xterm.dart's palette, darkened on light surfaces.
    • Copy output copies the server's plain-text rendering; Send to agent shares a bounded plain-text tail (no escape codes).
  • Pipe-mode and older runs (no terminal) are drawn in the same emulator with \n drawn as a new line, read-only.

Protocol

  • chat_shell_run request adds optional cols, rows; records add terminal, cols, rows (current size) and, only in chat_shell_run_output, text (plain-text rendering).
  • chat_shell_input {session_id, run_id, data}: data is UTF-8 text written byte-for-byte (its UTF-8 encoding). Control characters are JSON escapes: Ctrl-C \u0003, Enter \r, arrows \u001b[A… Paste is text (bracketed when the program asked). At most 16 KiB per request (clients split longer pastes at 4096 UTF-16 units); invalid UTF-8 is refused. Requests from one connection are applied in order through a 64-entry queue; a full queue is refused ("the command is not reading its input"). Reply has no body.
  • chat_shell_resize {session_id, run_id, cols, rows}: reply echoes the run. The last resize from any client wins.
  • Output frames are unchanged: raw terminal bytes (UTF-8, partial characters held back) with absolute byte offsets [offset, end), which are the stream sequence numbers. chat_shell_run_output takes after: the reply's output_start is max(after, retained start), so a reconnecting client fetches only what it missed and sees a gap if it fell out of the 1 MiB tail. The app does this on reconnect for runs it holds.
  • Persistence: chat_shell_runs gains terminal, cols, rows, output_text (additive columns). output is the raw stream tail (1 MiB), output_text its rendering (bounded to 1 MiB). Rows recorded before this change get a rendering computed on read.

Security

  • Any authenticated client watching the chat may type into or resize a running command, not only the one that started it (the same authority that can already start and stop commands in that chat). Input is checked against the chat's operator session like every other shell request.
  • Input is never persisted or logged: the server does not log chat_shell_input/chat_shell_resize request lines at all (keystrokes would otherwise be one log line each), and only what the terminal echoes becomes output. Password prompts disable echo, so hidden input never appears in output, frames, output or output_text.
  • The client keeps no copy of typed input: keys go from the emulator straight into a send queue that is cleared as each request is sent (and discarded on a refusal); the mobile field is cleared on send and can hide its text.
  • When the terminal has focus, the chat composer leaves Cmd/Ctrl-V alone (ChatComposerPasteFocus.registerInput), so a pasted token goes only to the process.

Limits and known gaps

  • Plain-text rendering is not an emulator: it strips CSI/OSC/DCS sequences, applies \r, backspace and line erase; full-screen programs render as their last lines. The card's emulator draws them properly.
  • Every watching client's emulator answers terminal queries (e.g. cursor position) for a live run, so two open clients send two answers. xterm.dart 4.0.0 answers DSR 6 zero-based; the app overrides it to the correct one-based answer.
  • xterm.dart 4.0.0 keeps blank rows when it rewraps; the app drops blank rows below the cursor after a resize so the top of the output stays in view.
  • Desktop typing uses key events (no IME composition for CJK input). Touch platforms use the field.
  • A descendant that keeps the PTY open after the command exits is read for 1 s; on macOS the PTY has no read deadline, so its reader goroutine ends only when that descendant closes it (the run itself is finished and the group killed).
  • Mobile behaviour (soft keyboard field) is verified by widget tests only; no device test ran. Android/Linux builds were not run (web and macOS were).

Files

  • Server: internal/protocol/chat_shell_terminal.go, chat_shell_run.go; internal/server/chat_shell_terminal{,_unix,_other}.go, chat_shell_plain.go, chat_shell_runner.go, chat_shell_supervise.go, chat_shell_run_handlers.go, chat_shell_share.go, chat.go, websocket.go; internal/pipeline/chat_shell_runs.go; go.mod, go.sum.
  • App: lib/features/chat/shell/chat_shell_terminal_{session,view,keys,theme,links,page}.dart, chat_shell_terminals.dart, chat_shell_output_view.dart, chat_shell_run{,_card}.dart, chat_shell_card_actions.dart, chat_shell_runs_{controller,repository}.dart, chat_shell_metrics.dart; lib/features/chat/conversation/chat_composer_paste_focus.dart; pubspec.yaml/pubspec.lock (xterm: ^4.0.0, pure Dart; pub.dev lists Android, iOS, Linux, macOS, web, Windows); probe fixture tool/perf/chat_shell_fixture.dart (?shell=1: coloured runs and an interactive gh auth login that hides the token and finishes on Enter).

Verification

Flutter 3.44.6 / Dart 3.12.2; macOS host.

  • go test ./internal/protocol ./internal/pipeline ./internal/server -count=1: ok …/protocol 0.205s, ok …/pipeline 15.838s, ok …/server 140.086s. Real-PTY tests (chat_shell_terminal_test.go, _ws_test.go): a typed line reaches read; stty -echo input is in no frame, output or text; \x03 interrupts; stty size sees the start size and a resize; resume by offset over the websocket from a second client; raw + plain text persisted; input bounds (16 KiB, empty, invalid UTF-8, full queue); size clamping. go test -run ChatShell -count=5: ok. GOOS=freebsd go build ./internal/server (pipe fallback) and GOOS=linux GOARCH=amd64 go test -c compile. (-race shows an existing data race in internal/opencode nativeRequest, unrelated to this change.)

  • tool/memwatch/full.sh test/features/chat: PASS: 1124 passed, 0 failed. New: test/features/chat/shell/chat_shell_terminal_test.dart (desktop keys, Cmd-V paste not taken by the composer, Paste button, touch toolbar and hidden field cleared on send, resize sent, finished run read-only, colours, reconnect resume), chat_shell_terminal_session_test.dart (links incl. wrapped and non-http, query answers not resent on replay, one-based DSR, gap redraw, rewrap keeps first line, row fitting).

  • tool/analyze.sh: clean. flutter build web: built. FLUTTER_XCODE_ARCHS=arm64 flutter build macos: built (34.9MB).

  • Real Chrome (tool/capture, probe ?chats=2&shell=1, 880x1000, Material and Cupertino, light and dark): /tmp/chat-interactive-terminal/<platform>-<theme>/ 01-login-prompt.png (coloured output, underlined link, prompt and cursor), 02-login-focused.png (focused cursor), 03-login-done.png (5 typed keys and Enter reached the fixture: "received a 5-character token", nothing echoed; the card shrank to its output, read-only), 04-full-screen.png.

  • Scratch server end to end (go build ./cmd/pipelines-server, env -i with HOME/XDG_DATA_HOME/DB under /tmp/pty-e2e, OS-chosen loopback port, operator session inserted with sqlite3, Node websocket driver /tmp/pty-e2e/drive.mjs; server stopped afterwards, no processes left):

    started run_45be206dedb0454409c537b6 terminal=true size=90x20
    resize reply: chat_shell_resize 120x40
    input reply: chat_shell_input
    final: status=exited exit=0 bytes=145
    "tty: xterm-256color\r\n20 90\r\n\u001b[1;32mOpen https://example.com/login\u001b[0m\r\nPaste token: \r\n40 120\r\ngot 13 characters\r\nName: operator\r\nhello operator\r\n"
    secret in stream: false
    resume after 119: output_start=119 output="operator\r\nhello operator\r\n"
    secret in text: false
    late input: error this command is not running
    run_…|exited|0|1|120|40|145|125|0   (sqlite: terminal, cols, rows, raw, text, SECRET offset)
    

    The server log has no chat_shell_input or chat_shell_resize lines.