Claude Code chats survive server restarts in a detachable host
A pipelines-claude-host process owns each claude -p CLI and its pipes, spools its stream-json output with offsets and serves a private Unix socket. The server detaches on shutdown and reattaches on start, replaying the spool to rebuild turns, approvals, questions, background tasks and the live model. Explicit close, idle release and CLI exit stop the host; a crashed host falls back to --resume with a notice.
This commit is contained in:
+67
-3
@@ -43,9 +43,73 @@ credential or alter global settings. `PIPELINES_CLAUDE` may name the executable;
|
||||
otherwise `claude` is resolved from `PATH` and common per-user locations.
|
||||
|
||||
A new session pins a generated UUID with `--session-id`. When the process is
|
||||
released (idle or server restart), the next prompt resumes the same conversation
|
||||
with `--resume <id>`, so context and the selected model survive. Chat creation is
|
||||
local: the CLI starts on the first prompt or model-catalog read.
|
||||
released (idle, an explicit close, or a host that crashed), the next prompt
|
||||
resumes the same conversation with `--resume <id>`, so context and the selected
|
||||
model survive. Chat creation is local: the CLI starts on the first prompt or
|
||||
model-catalog read.
|
||||
|
||||
### Surviving a server restart
|
||||
|
||||
A server restart does not stop a Claude Code chat. The installed server runs each
|
||||
CLI inside `pipelines-claude-host` (`cmd/pipelines-claude-host`,
|
||||
`internal/claudehost`), a small helper installed next to `pipelines-server`. The
|
||||
host starts in its own session and process group, so neither launchd nor the
|
||||
supervisor stops it with the server (the systemd unit uses `KillMode=process`
|
||||
for the same reason). The host owns the CLI's stdin/stdout. It appends every
|
||||
output line to a per-session spool file with byte offsets and serves a Unix
|
||||
socket. The socket is mode 0600 in a 0700 directory:
|
||||
`<user cache dir>/pipelines/claude-hosts/<hash of the database path>/`. Two
|
||||
servers of one user never adopt each other's chats. Each host also writes a
|
||||
registry file (`<key>.json`) there.
|
||||
|
||||
The socket speaks one JSON object per line: `attach` (deliver output from an
|
||||
offset; a newer attach replaces the older connection), `in` (one stdin frame),
|
||||
`ack` (output processed up to an offset), `status` and `close`. Interrupts,
|
||||
model and mode switches travel as the CLI's own control requests inside `in`. The
|
||||
host snoops these frames to keep the CLI's initialize response (models, account),
|
||||
the applied model and permission mode, a turn's running state, and the control
|
||||
requests nobody has answered yet (`can_use_tool`, `AskUserQuestion`).
|
||||
|
||||
On shutdown the server only disconnects (`Detach`). On start it reads the
|
||||
registry and reattaches to every live host before serving. It replays the spool
|
||||
from its start without broadcasting, which rebuilds the session's stream state,
|
||||
background commands and agents, and pending approvals and questions. Requests
|
||||
already answered are skipped. Output after the last acknowledged offset, written
|
||||
while no server was attached, is then processed live. The turn is shown running
|
||||
if it was running at that offset, watchers get a resync, and the start-up turn
|
||||
probe keeps observing it for notifications. An approval or question asked before
|
||||
the restart can be answered after it. Stop works as usual, and the CLI is never
|
||||
restarted or initialized twice.
|
||||
|
||||
The host stops the CLI (EOF, then SIGINT, then SIGKILL to its process group) and
|
||||
removes its files only when one of these happens:
|
||||
|
||||
- The operator closes the chat (`chat_close`).
|
||||
- The server's idle release or live-process cap fires.
|
||||
- The CLI exits; the server then gets its stderr.
|
||||
- With no server attached, the CLI has been idle for the same 20 minutes, or
|
||||
no server has attached for 24 hours.
|
||||
|
||||
If the host itself dies, the server first tries to reconnect once. If that fails,
|
||||
it stops the orphaned CLI's process group (only when that pid's command line
|
||||
still names the session) and removes the host files. A running turn ends with
|
||||
the notice "the Claude Code host stopped unexpectedly; the next prompt resumes
|
||||
this session with --resume". The next prompt then resumes the session.
|
||||
|
||||
Limits:
|
||||
|
||||
- A container restart (Docker) still ends every host. So does a reboot or
|
||||
logout. The transcript survives and `--resume` applies.
|
||||
- The spool keeps at most 64 MiB of acknowledged output. Older output is
|
||||
dropped, so background-task state from before that point is not rebuilt after
|
||||
a restart. Unacknowledged output is dropped only beyond 256 MiB; the server
|
||||
then logs the gap and clients reload the transcript.
|
||||
- Turn bookkeeping that lived only in server memory restarts fresh: the turn id,
|
||||
and which mid-turn messages are still outstanding. Output that begins a
|
||||
follow-up turn after a reconnect is adopted as a running turn.
|
||||
- Without the host binary (`PIPELINES_CLAUDE_HOST=off`, or not found next to the
|
||||
server or on `PATH`), the CLI is the server's child as before and a restart
|
||||
resumes it.
|
||||
|
||||
Text and reasoning stream from the CLI's partial-message events; completed
|
||||
messages are de-duplicated against the streamed text. Tool calls appear as normal
|
||||
|
||||
Reference in New Issue
Block a user