Files
pipelines/docs/notes/terminal-popup-research.md
T

6.7 KiB

Terminal popup in the Flutter client — research (2026-11-07)

Status: research only. No code changed. Decision pending with the operator.

Question

Can the Flutter client embed a full interactive terminal as a popup, and would the integration be good enough to ship? "Good enough" means: real shell, full-screen TUIs (vim, htop, claude, codex) render correctly, keyboard works the way a terminal user expects, copy/paste, resize, scrollback, light/dark theme, and it survives the client reconnecting.

What a terminal would have to be here

The client ships on web and Android only (desktop targets are not built). Neither can spawn a local shell: browsers have no process API, and an Android PTY would be a shell on the phone, not where the chats run. So the only useful design is a remote PTY on the Pipelines server host (the Mac), streamed over the existing /connect websocket. The terminal the operator sees is a shell on the server machine, the same machine the Claude and Codex sessions run on. That is also the useful one: tail logs, inspect worktrees, run git, open the TUI.

Rendering options evaluated

A. xterm2 (Dart terminal emulator, maintained fork of xterm.dart)

  • pub.dev: xterm2 5.4.0 published 2026-10-21, publisher sofluffy.io, MIT, Flutter >= 3.27 (we are on 3.44.6). Platform support passes for all six platforms including Web; WASM compatible. 130/160 pub points; the missing points are dartdoc coverage and one stale dependency constraint (equatable). Low adoption so far (about 4.6k downloads, 5 likes).
  • Upstream TerminalStudio/xterm.dart is abandoned (last meaningful activity 2024, 78 open issues, web issues like "text weirdly spaced on Web" #198 never fixed there).
  • The fork's 5.x changelog is serious and frequent (5.0 Jul 11, 5.1 Jul 19, 5.2 Jul 25, 5.3 Jul 28, 5.4 Oct 21): OSC 8 hyperlinks, DEC synchronized output, kitty keyboard protocol, CSI u, mouse reporting incl. wheel and back/forward, IME composition fixes, bounded clipboard, scrollback search, semantic prompts (OSC 133/633), Unicode 17 widths, procedural box-drawing glyphs, copy/paste callbacks, clipboard shortcuts, performance work on dense TUI output. This is the feature set modern TUIs (claude, codex, lazygit) need.
  • Wiring is trivial: Terminal() + TerminalView(terminal); bytes in via terminal.write, keystrokes out via terminal.onOutput, size via onResize.
  • Rendering is a Flutter CustomPainter, so it inherits our theme, text scaling and the app's font handling, and sits naturally inside a dialog/sheet/side pane with the rest of the widget tree. Focus, shortcuts and overlays are plain Flutter.
  • Risks:
    • Single maintainer, 4 months old as a fork. Pin the version; expect to patch.
    • Web is "passes platform check", not "battle tested". Upstream had web text-spacing bugs; the fork rewrote glyph layout in 5.1/5.3 but nobody has confirmed it on our CanvasKit build. Must be verified first.
    • Browser-reserved shortcuts (Ctrl+W, Ctrl+T, Ctrl+N, Cmd+L) cannot be captured on web by any approach. Ctrl+C/V/A/Tab must be arbitrated between the terminal, the app's chat shortcuts (ctrl+tab chat switching) and the browser. Known class of problem (Canonical workshops #127); solvable with an outer Focus/Shortcuts scope that decides who wins when the popup is open.
    • Android: the soft keyboard has no Ctrl/Esc/Tab/arrows. A usable mobile terminal needs an extra key bar (Esc, Tab, Ctrl, arrows, /, -, |). Upstream issues #163/#176 (autocorrect and suggestions corrupting input) were reported there; the fork's IME work claims to address composition but this is unverified by us.

B. xterm.js inside a web view

  • Web: HtmlElementView with an iframe hosting xterm.js. xterm.js is the gold standard renderer, but the iframe is a foreign body: Flutter receives no pointer events inside it, focus hand-off is one-way, theme/font must be re-implemented in CSS, and the popup cannot overlap it with Flutter widgets. It also only exists on web; Android would need webview_flutter (already a dependency) with a second, different integration, and the keyboard/IME story in an Android WebView terminal is poor.
  • Verdict: two integrations, neither of which feels native. Rejected for a "very good integration" bar.

C. Native/text fallback (a scrolling log of command output)

  • Not a terminal. Does not run TUIs. Rejected for this goal, though fine as a separate "run a command" feature.

Server side (Go)

  • New capability in internal/server: terminal_open (spawn $SHELL -l on a PTY via github.com/creack/pty, cwd = chat worktree or home, cols/rows), terminal_input (base64 bytes), terminal_resize, terminal_close, and an unsolicited terminal_output frame (base64 chunks, coalesced). Fits the existing one-JSON-envelope-per-message transport and the 2 MiB read limit. Binary websocket frames are not needed.
  • The claudehost pattern (a detached host process that spools output and survives server restarts) is the right model if a terminal should survive a server restart or a client reconnect; a first version can simply die with the connection.
  • Security is the real cost. The web app is reachable on the tailnet (8443) and the /connect websocket has no per-user auth today; a terminal is a full shell as the server user on the Mac. Shipping it requires at least: an explicit server flag to enable terminals (same posture as AllowAuthoring), loopback or tailnet-only binding confirmed, and ideally a per-session confirmation. This is not optional.

Effort estimate

  • Spike (verify renderer on web): 0.5 day. Add xterm2 in a scratch branch, feed it a recorded vim/htop/claude byte stream, check glyph spacing, colours, cursor, IME, theme switch, resize, on CanvasKit in Chrome and on the foldable emulator.
  • Server PTY commands + protocol: 1 day incl. tests.
  • Client popup (dialog on wide layouts, full-screen sheet on mobile), focus arbitration with existing chat shortcuts, extra key bar on Android, theme bridging: 1.5 to 2 days.
  • Hardening (enable flag, reconnect behaviour, session cleanup, docs): 0.5 to 1 day.

Recommendation

Feasible, and xterm2 is the only path that can meet the "very good integration" bar on both web and Android with one implementation. Do not commit to the feature until the half-day renderer spike passes on our actual web build; that spike decides it. If it does, build it as a server-host shell with an explicit enable flag, not as an always-on feature.

Sources: pub.dev/packages/xterm2 (score, changelog), github.com/TerminalStudio/xterm.dart issues, docs.flutter.dev platform-integration/web/web-content-in-flutter, github.com/canonical/workshops/issues/127, github.com/creack/pty.