Record Flutter terminal popup integration research
This commit is contained in:
@@ -0,0 +1,111 @@
|
||||
# Terminal popup in the Flutter client — research (2026-11-07)
|
||||
|
||||
Status: research only. No code changed. Decision pending with the operator.
|
||||
|
||||
## Question
|
||||
|
||||
Can the Flutter client embed a full interactive terminal as a popup, and would the
|
||||
integration be good enough to ship? "Good enough" means: real shell, full-screen TUIs
|
||||
(vim, htop, claude, codex) render correctly, keyboard works the way a terminal user
|
||||
expects, copy/paste, resize, scrollback, light/dark theme, and it survives the client
|
||||
reconnecting.
|
||||
|
||||
## What a terminal would have to be here
|
||||
|
||||
The client ships on web and Android only (desktop targets are not built). Neither can
|
||||
spawn a local shell: browsers have no process API, and an Android PTY would be a shell on
|
||||
the phone, not where the chats run. So the only useful design is a **remote PTY on the
|
||||
Pipelines server host** (the Mac), streamed over the existing `/connect` websocket. The
|
||||
terminal the operator sees is a shell on the server machine, the same machine the Claude
|
||||
and Codex sessions run on. That is also the useful one: tail logs, inspect worktrees,
|
||||
run git, open the TUI.
|
||||
|
||||
## Rendering options evaluated
|
||||
|
||||
### A. `xterm2` (Dart terminal emulator, maintained fork of `xterm.dart`)
|
||||
|
||||
- pub.dev: xterm2 5.4.0 published 2026-10-21, publisher sofluffy.io, MIT, Flutter >= 3.27
|
||||
(we are on 3.44.6). Platform support passes for all six platforms including Web; WASM
|
||||
compatible. 130/160 pub points; the missing points are dartdoc coverage and one stale
|
||||
dependency constraint (`equatable`). Low adoption so far (about 4.6k downloads, 5 likes).
|
||||
- Upstream `TerminalStudio/xterm.dart` is abandoned (last meaningful activity 2024,
|
||||
78 open issues, web issues like "text weirdly spaced on Web" #198 never fixed there).
|
||||
- The fork's 5.x changelog is serious and frequent (5.0 Jul 11, 5.1 Jul 19, 5.2 Jul 25,
|
||||
5.3 Jul 28, 5.4 Oct 21): OSC 8 hyperlinks, DEC synchronized output, kitty keyboard
|
||||
protocol, CSI u, mouse reporting incl. wheel and back/forward, IME composition fixes,
|
||||
bounded clipboard, scrollback search, semantic prompts (OSC 133/633), Unicode 17 widths,
|
||||
procedural box-drawing glyphs, copy/paste callbacks, clipboard shortcuts, performance
|
||||
work on dense TUI output. This is the feature set modern TUIs (claude, codex, lazygit)
|
||||
need.
|
||||
- Wiring is trivial: `Terminal()` + `TerminalView(terminal)`; bytes in via
|
||||
`terminal.write`, keystrokes out via `terminal.onOutput`, size via `onResize`.
|
||||
- Rendering is a Flutter CustomPainter, so it inherits our theme, text scaling and the
|
||||
app's font handling, and sits naturally inside a dialog/sheet/side pane with the rest of
|
||||
the widget tree. Focus, shortcuts and overlays are plain Flutter.
|
||||
- Risks:
|
||||
- Single maintainer, 4 months old as a fork. Pin the version; expect to patch.
|
||||
- Web is "passes platform check", not "battle tested". Upstream had web text-spacing
|
||||
bugs; the fork rewrote glyph layout in 5.1/5.3 but nobody has confirmed it on our
|
||||
CanvasKit build. Must be verified first.
|
||||
- Browser-reserved shortcuts (Ctrl+W, Ctrl+T, Ctrl+N, Cmd+L) cannot be captured on web
|
||||
by any approach. Ctrl+C/V/A/Tab must be arbitrated between the terminal, the app's
|
||||
chat shortcuts (ctrl+tab chat switching) and the browser. Known class of problem
|
||||
(Canonical workshops #127); solvable with an outer `Focus`/`Shortcuts` scope that
|
||||
decides who wins when the popup is open.
|
||||
- Android: the soft keyboard has no Ctrl/Esc/Tab/arrows. A usable mobile terminal needs
|
||||
an extra key bar (Esc, Tab, Ctrl, arrows, `/`, `-`, `|`). Upstream issues #163/#176
|
||||
(autocorrect and suggestions corrupting input) were reported there; the fork's IME work
|
||||
claims to address composition but this is unverified by us.
|
||||
|
||||
### B. `xterm.js` inside a web view
|
||||
|
||||
- Web: `HtmlElementView` with an iframe hosting xterm.js. xterm.js is the gold standard
|
||||
renderer, but the iframe is a foreign body: Flutter receives no pointer events inside it,
|
||||
focus hand-off is one-way, theme/font must be re-implemented in CSS, and the popup
|
||||
cannot overlap it with Flutter widgets. It also only exists on web; Android would need
|
||||
`webview_flutter` (already a dependency) with a second, different integration, and the
|
||||
keyboard/IME story in an Android WebView terminal is poor.
|
||||
- Verdict: two integrations, neither of which feels native. Rejected for a "very good
|
||||
integration" bar.
|
||||
|
||||
### C. Native/text fallback (a scrolling log of command output)
|
||||
|
||||
- Not a terminal. Does not run TUIs. Rejected for this goal, though fine as a separate
|
||||
"run a command" feature.
|
||||
|
||||
## Server side (Go)
|
||||
|
||||
- New capability in `internal/server`: `terminal_open` (spawn `$SHELL -l` on a PTY via
|
||||
`github.com/creack/pty`, cwd = chat worktree or home, cols/rows), `terminal_input`
|
||||
(base64 bytes), `terminal_resize`, `terminal_close`, and an unsolicited `terminal_output`
|
||||
frame (base64 chunks, coalesced). Fits the existing one-JSON-envelope-per-message
|
||||
transport and the 2 MiB read limit. Binary websocket frames are not needed.
|
||||
- The `claudehost` pattern (a detached host process that spools output and survives
|
||||
server restarts) is the right model if a terminal should survive a server restart or a
|
||||
client reconnect; a first version can simply die with the connection.
|
||||
- Security is the real cost. The web app is reachable on the tailnet (8443) and the
|
||||
`/connect` websocket has no per-user auth today; a terminal is a full shell as the server
|
||||
user on the Mac. Shipping it requires at least: an explicit server flag to enable
|
||||
terminals (same posture as `AllowAuthoring`), loopback or tailnet-only binding
|
||||
confirmed, and ideally a per-session confirmation. This is not optional.
|
||||
|
||||
## Effort estimate
|
||||
|
||||
- Spike (verify renderer on web): 0.5 day. Add `xterm2` in a scratch branch, feed it a
|
||||
recorded `vim`/`htop`/`claude` byte stream, check glyph spacing, colours, cursor, IME,
|
||||
theme switch, resize, on CanvasKit in Chrome and on the foldable emulator.
|
||||
- Server PTY commands + protocol: 1 day incl. tests.
|
||||
- Client popup (dialog on wide layouts, full-screen sheet on mobile), focus arbitration
|
||||
with existing chat shortcuts, extra key bar on Android, theme bridging: 1.5 to 2 days.
|
||||
- Hardening (enable flag, reconnect behaviour, session cleanup, docs): 0.5 to 1 day.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Feasible, and `xterm2` is the only path that can meet the "very good integration" bar on
|
||||
both web and Android with one implementation. Do not commit to the feature until the
|
||||
half-day renderer spike passes on our actual web build; that spike decides it. If it does,
|
||||
build it as a server-host shell with an explicit enable flag, not as an always-on feature.
|
||||
|
||||
Sources: pub.dev/packages/xterm2 (score, changelog), github.com/TerminalStudio/xterm.dart
|
||||
issues, docs.flutter.dev platform-integration/web/web-content-in-flutter,
|
||||
github.com/canonical/workshops/issues/127, github.com/creack/pty.
|
||||
Reference in New Issue
Block a user