Claude Code chats survive server restarts in a detachable host

A pipelines-claude-host process owns each claude -p CLI and its pipes,
spools its stream-json output with offsets and serves a private Unix
socket. The server detaches on shutdown and reattaches on start,
replaying the spool to rebuild turns, approvals, questions, background
tasks and the live model. Explicit close, idle release and CLI exit stop
the host; a crashed host falls back to --resume with a notice.
This commit is contained in:
2026-11-04 16:09:56 +00:00
parent fa6c0864db
commit c810437aa5
19 changed files with 2396 additions and 37 deletions
+67 -3
View File
@@ -43,9 +43,73 @@ credential or alter global settings. `PIPELINES_CLAUDE` may name the executable;
otherwise `claude` is resolved from `PATH` and common per-user locations.
A new session pins a generated UUID with `--session-id`. When the process is
released (idle or server restart), the next prompt resumes the same conversation
with `--resume <id>`, so context and the selected model survive. Chat creation is
local: the CLI starts on the first prompt or model-catalog read.
released (idle, an explicit close, or a host that crashed), the next prompt
resumes the same conversation with `--resume <id>`, so context and the selected
model survive. Chat creation is local: the CLI starts on the first prompt or
model-catalog read.
### Surviving a server restart
A server restart does not stop a Claude Code chat. The installed server runs each
CLI inside `pipelines-claude-host` (`cmd/pipelines-claude-host`,
`internal/claudehost`), a small helper installed next to `pipelines-server`. The
host starts in its own session and process group, so neither launchd nor the
supervisor stops it with the server (the systemd unit uses `KillMode=process`
for the same reason). The host owns the CLI's stdin/stdout. It appends every
output line to a per-session spool file with byte offsets and serves a Unix
socket. The socket is mode 0600 in a 0700 directory:
`<user cache dir>/pipelines/claude-hosts/<hash of the database path>/`. Two
servers of one user never adopt each other's chats. Each host also writes a
registry file (`<key>.json`) there.
The socket speaks one JSON object per line: `attach` (deliver output from an
offset; a newer attach replaces the older connection), `in` (one stdin frame),
`ack` (output processed up to an offset), `status` and `close`. Interrupts,
model and mode switches travel as the CLI's own control requests inside `in`. The
host snoops these frames to keep the CLI's initialize response (models, account),
the applied model and permission mode, a turn's running state, and the control
requests nobody has answered yet (`can_use_tool`, `AskUserQuestion`).
On shutdown the server only disconnects (`Detach`). On start it reads the
registry and reattaches to every live host before serving. It replays the spool
from its start without broadcasting, which rebuilds the session's stream state,
background commands and agents, and pending approvals and questions. Requests
already answered are skipped. Output after the last acknowledged offset, written
while no server was attached, is then processed live. The turn is shown running
if it was running at that offset, watchers get a resync, and the start-up turn
probe keeps observing it for notifications. An approval or question asked before
the restart can be answered after it. Stop works as usual, and the CLI is never
restarted or initialized twice.
The host stops the CLI (EOF, then SIGINT, then SIGKILL to its process group) and
removes its files only when one of these happens:
- The operator closes the chat (`chat_close`).
- The server's idle release or live-process cap fires.
- The CLI exits; the server then gets its stderr.
- With no server attached, the CLI has been idle for the same 20 minutes, or
no server has attached for 24 hours.
If the host itself dies, the server first tries to reconnect once. If that fails,
it stops the orphaned CLI's process group (only when that pid's command line
still names the session) and removes the host files. A running turn ends with
the notice "the Claude Code host stopped unexpectedly; the next prompt resumes
this session with --resume". The next prompt then resumes the session.
Limits:
- A container restart (Docker) still ends every host. So does a reboot or
logout. The transcript survives and `--resume` applies.
- The spool keeps at most 64 MiB of acknowledged output. Older output is
dropped, so background-task state from before that point is not rebuilt after
a restart. Unacknowledged output is dropped only beyond 256 MiB; the server
then logs the gap and clients reload the transcript.
- Turn bookkeeping that lived only in server memory restarts fresh: the turn id,
and which mid-turn messages are still outstanding. Output that begins a
follow-up turn after a reconnect is adopted as a running turn.
- Without the host binary (`PIPELINES_CLAUDE_HOST=off`, or not found next to the
server or on `PATH`), the CLI is the server's child as before and a restart
resumes it.
Text and reasoning stream from the CLI's partial-message events; completed
messages are de-duplicated against the streamed text. Tool calls appear as normal