From f80edfabab787dc5b10a693fc27f232a96e1731a Mon Sep 17 00:00:00 2001 From: Alan Silva Date: Sat, 7 Nov 2026 01:39:49 +0000 Subject: [PATCH] Record Flutter terminal popup integration research --- docs/notes/terminal-popup-research.md | 111 ++++++++++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 docs/notes/terminal-popup-research.md diff --git a/docs/notes/terminal-popup-research.md b/docs/notes/terminal-popup-research.md new file mode 100644 index 00000000..eeaa95b3 --- /dev/null +++ b/docs/notes/terminal-popup-research.md @@ -0,0 +1,111 @@ +# Terminal popup in the Flutter client — research (2026-11-07) + +Status: research only. No code changed. Decision pending with the operator. + +## Question + +Can the Flutter client embed a full interactive terminal as a popup, and would the +integration be good enough to ship? "Good enough" means: real shell, full-screen TUIs +(vim, htop, claude, codex) render correctly, keyboard works the way a terminal user +expects, copy/paste, resize, scrollback, light/dark theme, and it survives the client +reconnecting. + +## What a terminal would have to be here + +The client ships on web and Android only (desktop targets are not built). Neither can +spawn a local shell: browsers have no process API, and an Android PTY would be a shell on +the phone, not where the chats run. So the only useful design is a **remote PTY on the +Pipelines server host** (the Mac), streamed over the existing `/connect` websocket. The +terminal the operator sees is a shell on the server machine, the same machine the Claude +and Codex sessions run on. That is also the useful one: tail logs, inspect worktrees, +run git, open the TUI. + +## Rendering options evaluated + +### A. `xterm2` (Dart terminal emulator, maintained fork of `xterm.dart`) + +- pub.dev: xterm2 5.4.0 published 2026-10-21, publisher sofluffy.io, MIT, Flutter >= 3.27 + (we are on 3.44.6). Platform support passes for all six platforms including Web; WASM + compatible. 130/160 pub points; the missing points are dartdoc coverage and one stale + dependency constraint (`equatable`). Low adoption so far (about 4.6k downloads, 5 likes). +- Upstream `TerminalStudio/xterm.dart` is abandoned (last meaningful activity 2024, + 78 open issues, web issues like "text weirdly spaced on Web" #198 never fixed there). +- The fork's 5.x changelog is serious and frequent (5.0 Jul 11, 5.1 Jul 19, 5.2 Jul 25, + 5.3 Jul 28, 5.4 Oct 21): OSC 8 hyperlinks, DEC synchronized output, kitty keyboard + protocol, CSI u, mouse reporting incl. wheel and back/forward, IME composition fixes, + bounded clipboard, scrollback search, semantic prompts (OSC 133/633), Unicode 17 widths, + procedural box-drawing glyphs, copy/paste callbacks, clipboard shortcuts, performance + work on dense TUI output. This is the feature set modern TUIs (claude, codex, lazygit) + need. +- Wiring is trivial: `Terminal()` + `TerminalView(terminal)`; bytes in via + `terminal.write`, keystrokes out via `terminal.onOutput`, size via `onResize`. +- Rendering is a Flutter CustomPainter, so it inherits our theme, text scaling and the + app's font handling, and sits naturally inside a dialog/sheet/side pane with the rest of + the widget tree. Focus, shortcuts and overlays are plain Flutter. +- Risks: + - Single maintainer, 4 months old as a fork. Pin the version; expect to patch. + - Web is "passes platform check", not "battle tested". Upstream had web text-spacing + bugs; the fork rewrote glyph layout in 5.1/5.3 but nobody has confirmed it on our + CanvasKit build. Must be verified first. + - Browser-reserved shortcuts (Ctrl+W, Ctrl+T, Ctrl+N, Cmd+L) cannot be captured on web + by any approach. Ctrl+C/V/A/Tab must be arbitrated between the terminal, the app's + chat shortcuts (ctrl+tab chat switching) and the browser. Known class of problem + (Canonical workshops #127); solvable with an outer `Focus`/`Shortcuts` scope that + decides who wins when the popup is open. + - Android: the soft keyboard has no Ctrl/Esc/Tab/arrows. A usable mobile terminal needs + an extra key bar (Esc, Tab, Ctrl, arrows, `/`, `-`, `|`). Upstream issues #163/#176 + (autocorrect and suggestions corrupting input) were reported there; the fork's IME work + claims to address composition but this is unverified by us. + +### B. `xterm.js` inside a web view + +- Web: `HtmlElementView` with an iframe hosting xterm.js. xterm.js is the gold standard + renderer, but the iframe is a foreign body: Flutter receives no pointer events inside it, + focus hand-off is one-way, theme/font must be re-implemented in CSS, and the popup + cannot overlap it with Flutter widgets. It also only exists on web; Android would need + `webview_flutter` (already a dependency) with a second, different integration, and the + keyboard/IME story in an Android WebView terminal is poor. +- Verdict: two integrations, neither of which feels native. Rejected for a "very good + integration" bar. + +### C. Native/text fallback (a scrolling log of command output) + +- Not a terminal. Does not run TUIs. Rejected for this goal, though fine as a separate + "run a command" feature. + +## Server side (Go) + +- New capability in `internal/server`: `terminal_open` (spawn `$SHELL -l` on a PTY via + `github.com/creack/pty`, cwd = chat worktree or home, cols/rows), `terminal_input` + (base64 bytes), `terminal_resize`, `terminal_close`, and an unsolicited `terminal_output` + frame (base64 chunks, coalesced). Fits the existing one-JSON-envelope-per-message + transport and the 2 MiB read limit. Binary websocket frames are not needed. +- The `claudehost` pattern (a detached host process that spools output and survives + server restarts) is the right model if a terminal should survive a server restart or a + client reconnect; a first version can simply die with the connection. +- Security is the real cost. The web app is reachable on the tailnet (8443) and the + `/connect` websocket has no per-user auth today; a terminal is a full shell as the server + user on the Mac. Shipping it requires at least: an explicit server flag to enable + terminals (same posture as `AllowAuthoring`), loopback or tailnet-only binding + confirmed, and ideally a per-session confirmation. This is not optional. + +## Effort estimate + +- Spike (verify renderer on web): 0.5 day. Add `xterm2` in a scratch branch, feed it a + recorded `vim`/`htop`/`claude` byte stream, check glyph spacing, colours, cursor, IME, + theme switch, resize, on CanvasKit in Chrome and on the foldable emulator. +- Server PTY commands + protocol: 1 day incl. tests. +- Client popup (dialog on wide layouts, full-screen sheet on mobile), focus arbitration + with existing chat shortcuts, extra key bar on Android, theme bridging: 1.5 to 2 days. +- Hardening (enable flag, reconnect behaviour, session cleanup, docs): 0.5 to 1 day. + +## Recommendation + +Feasible, and `xterm2` is the only path that can meet the "very good integration" bar on +both web and Android with one implementation. Do not commit to the feature until the +half-day renderer spike passes on our actual web build; that spike decides it. If it does, +build it as a server-host shell with an explicit enable flag, not as an always-on feature. + +Sources: pub.dev/packages/xterm2 (score, changelog), github.com/TerminalStudio/xterm.dart +issues, docs.flutter.dev platform-integration/web/web-content-in-flutter, +github.com/canonical/workshops/issues/127, github.com/creack/pty.