300 lines
18 KiB
Markdown
300 lines
18 KiB
Markdown
# Chat backends
|
|
|
|
Chats can be created on more than one harness. OpenCode remains the default;
|
|
Codex is available when its App Server control socket exists, and Claude Code
|
|
when the `claude` CLI is installed. The selected backend is stored on the
|
|
operator-session row and returned as `backend` on every `chat_sessions` row, so a
|
|
restart does not have to guess from the session id.
|
|
|
|
When a `chat_prompt` is accepted during a running turn, the reply sets
|
|
`chat_steered: true` when the backend hands it to that turn; otherwise it sets
|
|
`chat_queued: true` while the prompt waits for the turn to finish.
|
|
|
|
## Restarts keep sessions running
|
|
|
|
A server restart or stop does not stop chat sessions. Only an operator action
|
|
stops one: closing the chat, or Stop on its turn. The server durably journals
|
|
chat work as it changes (a turn in flight, a pending question or approval, or
|
|
running background tasks), and refreshes `chat_live_sessions` on shutdown.
|
|
It then detaches each backend instead of closing it, and never sends an
|
|
interrupt.
|
|
|
|
On the next start each recorded chat is reattached. The server asks the harness
|
|
whether the turn is still running, re-subscribes to its event stream with no
|
|
client attached, re-reads pending approvals/questions and background shells,
|
|
and holds queued prompts behind the turn. If a turn ended while the server was
|
|
down, or its end event was lost with the old stream, the end is folded in from
|
|
the harness's history. Clients get a `resync`, the reply notifies once (with
|
|
the text the harness recorded), and queued prompts are delivered in order. The
|
|
startup discovery checks all open registered chats for work not yet recorded,
|
|
including after a crash. Unavailable sessions remain journaled and retry;
|
|
browser requests wait until journaled turns and queued prompts are restored.
|
|
|
|
Per backend (the contract is `chatShutdownDetacher.Detach`, which
|
|
`chatManager.Close` calls instead of `Close`, plus `chatSessionReleaser` for
|
|
chat_close; see `internal/server/chat_manager.go` and `chat_detach.go`):
|
|
|
|
- **OpenCode**: sessions live in the OpenCode service. An attached endpoint
|
|
(the operator's CLI-managed service) already outlives the server. Without
|
|
one, a V2 server now uses a detached Pipelines-owned chat service
|
|
(`opencode serve` in its own session, registered in
|
|
`<cache>/pipelines/private/opencode-chat-service.json`). The next server
|
|
reuses it at the same address and password, and it is not stopped on
|
|
shutdown. A V1 serve child still ends with the server.
|
|
- **Codex**: threads and turns live in the App Server. The installed server
|
|
connects through `pipelines-codex-host`, which retains the original App Server
|
|
connection and unanswered approvals/questions while Pipelines detaches. A new
|
|
server receives those same callbacks and reconciles ordinary turn state through
|
|
readback. Direct mode (hosting disabled or helper absent) still loses pending
|
|
connection-bound interactions on restart.
|
|
- **Claude Code**: its detachable hosts keep the CLI running (see the Claude
|
|
Code section).
|
|
|
|
`PIPELINES_SHUTDOWN_SESSIONS=stop` restores the old shutdown, which closes the
|
|
chat clients and records nothing. See [Installation](install.md#restarts-keep-sessions-running).
|
|
|
|
## Codex
|
|
|
|
The adapter in `internal/codex` connects to the Codex App Server's Unix
|
|
WebSocket control socket (`$CODEX_HOME/app-server-control/app-server-control.sock`,
|
|
falling back to `~/.codex`). It uses the operator's existing Codex login. A new
|
|
chat issues `thread/start`; a prompt issues `turn/start`; history comes from
|
|
`thread/read`; questions map from `item/tool/requestUserInput` to the normal
|
|
question card and `chat_answer` returns the JSON-RPC response. Threads created
|
|
this way are ordinary Codex threads and remain visible to the Codex app.
|
|
|
|
New threads retain the `workspace-write` sandbox and inherit Codex's configured
|
|
approval policy (Pipelines no longer forces `never`). Command, file-change,
|
|
network and explicit permission requests appear in the web approval cards. Each
|
|
JSON-RPC callback has its own identity, including Code Mode subcommands sharing
|
|
one item. Allow once maps to `accept`, session grants to `acceptForSession`, and
|
|
Reject to `decline`; explicit permission profiles grant only requested fields.
|
|
The web labels session scope explicitly, never as a permanent policy change.
|
|
|
|
Pending questions and approvals survive browser reconnects while the App Server
|
|
connection lives. Standard MCP form/URL elicitation supports typed answers,
|
|
Accept/Decline/Cancel and safe external links. Extended OpenAI elicitation is not
|
|
advertised; unknown request formats return an explicit unsupported RPC error,
|
|
not a fake empty success or silent auto-approval. App Server socket loss does
|
|
not provide durable callback recovery; a disconnected transport is reported as
|
|
an error rather than claiming old requests are still actionable.
|
|
|
|
### Pending interactions across restarts
|
|
|
|
The installed `pipelines-codex-host` helper owns one WebSocket connection to the
|
|
existing Codex App Server Unix socket. It never starts a second Codex daemon or
|
|
changes Codex settings. Helpers are scoped by database path and upstream socket,
|
|
with mode-0600 Unix sockets and singleton locks under the mode-0700 directory
|
|
`<user cache dir>/pipelines/codex-hosts/<database hash>/`.
|
|
|
|
The helper completes upstream initialize -> initialized exactly once before
|
|
accepting clients. On reconnect it returns the cached initialization result and replays
|
|
only unanswered server requests, preserving exact callback IDs and parameters.
|
|
Responses and `serverRequest/resolved` remove pending entries. Old/new client RPC
|
|
IDs are remapped so a delayed response for an old Pipelines process cannot answer
|
|
a new process's call. A new attach closes the obsolete downstream connection.
|
|
Ordinary notifications are not spooled; the existing thread readback repairs gaps.
|
|
|
|
State is bounded to 128 callbacks / 8 MiB and 256 in-flight client calls, with a
|
|
16 MiB frame limit. Exceeding pending bounds closes the bridge without granting,
|
|
declining or otherwise auto-answering an interaction. A detached bridge keeps its original upstream connection indefinitely by default.
|
|
Only explicitly supplied positive `-idle`/`-max-detached` limits enable expiry.
|
|
An upstream App Server restart, a bridge crash, or expiry loses its connection-
|
|
bound callbacks: no disk JSON can recreate them on a new connection. The initial
|
|
deployment cannot recover requests already owned by the old direct connection.
|
|
|
|
`PIPELINES_CODEX_HOST` overrides the helper executable; `off` disables hosting.
|
|
The server otherwise finds it beside `pipelines-server`, then on PATH. Without
|
|
hosting, direct-mode fixtures and intentional direct connections retain their
|
|
previous behaviour and limitation. Installing a new helper binary does not kill
|
|
an existing helper or its pending requests.
|
|
|
|
## Claude Code
|
|
|
|
The adapter in `internal/claude` owns one Claude Code CLI process per active
|
|
session. It starts `claude -p` with stream-json input and output, partial
|
|
messages, and `--permission-prompt-tool=stdio`, then performs the SDK-style
|
|
`initialize` handshake before any turn. The operator's existing Claude Code login
|
|
and configuration are used unchanged; Pipelines does not store a second
|
|
credential or alter global settings. `PIPELINES_CLAUDE` may name the executable;
|
|
otherwise `claude` is resolved from `PATH` and common per-user locations.
|
|
|
|
A new session pins a generated UUID with `--session-id`. When the process is
|
|
released (idle, an explicit close, or a host that crashed), the next prompt
|
|
resumes the same conversation with `--resume <id>`, so context and the selected
|
|
model survive. Chat creation is local: the CLI starts on the first prompt or
|
|
model-catalog read.
|
|
|
|
### Surviving a server restart
|
|
|
|
A server restart does not stop a Claude Code chat. The installed server runs each
|
|
CLI inside `pipelines-claude-host` (`cmd/pipelines-claude-host`,
|
|
`internal/claudehost`), a small helper installed next to `pipelines-server`. The
|
|
host starts in its own session and process group, so neither launchd nor the
|
|
supervisor stops it with the server (the systemd unit uses `KillMode=process`
|
|
for the same reason). The host owns the CLI's stdin/stdout. It appends every
|
|
output line to a per-session spool file with byte offsets and serves a Unix
|
|
socket. The socket is mode 0600 in a 0700 directory:
|
|
`<user cache dir>/pipelines/claude-hosts/<hash of the database path>/`. Two
|
|
servers of one user never adopt each other's chats. Each host also writes a
|
|
registry file (`<key>.json`) there.
|
|
|
|
The socket speaks one JSON object per line: `attach` (deliver output from an
|
|
offset; a newer attach replaces the older connection), `in` (one stdin frame),
|
|
`ack` (output processed up to an offset), `status` and `close`. Interrupts,
|
|
model and mode switches travel as the CLI's own control requests inside `in`. The
|
|
host snoops these frames to keep the CLI's initialize response (models, account),
|
|
the applied model and permission mode, a turn's running state, and the control
|
|
requests nobody has answered yet (`can_use_tool`, `AskUserQuestion`).
|
|
|
|
On shutdown the server only disconnects (`Detach`). On start it reads the
|
|
registry and reattaches to every live host before serving. It replays the spool
|
|
from its start without broadcasting, which rebuilds the session's stream state,
|
|
background commands and agents, and pending approvals and questions. Requests
|
|
already answered are skipped. Output after the last acknowledged offset, written
|
|
while no server was attached, is then processed live. The turn is shown running
|
|
if it was running at that offset, watchers get a resync, and the start-up turn
|
|
probe keeps observing it for notifications. An approval or question asked before
|
|
the restart can be answered after it. Stop works as usual, and the CLI is never
|
|
restarted or initialized twice.
|
|
|
|
The host stops the CLI (EOF, then SIGINT, then SIGKILL to its process group) and
|
|
removes its files only when one of these happens:
|
|
|
|
- The operator closes the chat (`chat_close`).
|
|
- A genuinely idle unhosted CLI is released. For hosted sessions, server cache
|
|
eviction only detaches; the host owns authoritative background-task liveness.
|
|
- The CLI exits; the server then gets its stderr.
|
|
- With no server attached, the CLI has been genuinely idle for 20 minutes,
|
|
with no foreground turn, pending interaction, or native background task.
|
|
There is no default hard detached lifetime limit.
|
|
|
|
If the control connection is lost, the server first tries to reconnect. A verified
|
|
live host is preserved when its socket is temporarily unavailable, and another
|
|
CLI is not started alongside it. Dead host recovery verifies process identity
|
|
before stopping an orphan. On attachment, the host's authoritative task snapshot
|
|
restores running task identities even after older spool records were trimmed.
|
|
Legacy hosts without that snapshot remain attached during cache eviction, so
|
|
their older detached idle timers cannot mistake background work for an idle CLI.
|
|
|
|
Limits:
|
|
|
|
- A container restart (Docker) still ends every host. So does a reboot or
|
|
logout. The transcript survives and `--resume` applies.
|
|
- The spool keeps at most 64 MiB of acknowledged output. Older output is
|
|
dropped. Current hosts retain running task identities separately from output.
|
|
Legacy hosts with an incomplete replay report task visibility as unknown,
|
|
rather than claiming there are no tasks. Unacknowledged output is dropped only
|
|
beyond 256 MiB; the server
|
|
then logs the gap and clients reload the transcript.
|
|
- Turn bookkeeping that lived only in server memory restarts fresh: the turn id,
|
|
and which mid-turn messages are still outstanding. Output that begins a
|
|
follow-up turn after a reconnect is adopted as a running turn.
|
|
- Without the host binary (`PIPELINES_CLAUDE_HOST=off`, or not found next to the
|
|
server or on `PATH`), the CLI is the server's child as before and a restart
|
|
resumes it.
|
|
|
|
Text and reasoning stream from the CLI's partial-message events; completed
|
|
messages are de-duplicated against the streamed text. Tool calls appear as normal
|
|
tool cards, Bash commands as the existing background-command rows, and results
|
|
attach to the call. `can_use_tool` becomes the normal approval card: Allow once
|
|
returns the original input, Always allow forwards the CLI's own permission
|
|
suggestions, and Reject returns a deny. `AskUserQuestion` becomes the normal
|
|
question card and `chat_answer` returns the chosen answers. Interrupt sends the
|
|
CLI's own `interrupt` control request.
|
|
|
|
The model catalog comes from the CLI's initialize response and is switched live
|
|
with `set_model`; Pipelines persists the choice for the next turn so a resumed
|
|
process starts with it. Usage and context totals come from the CLI's `get_usage`
|
|
and `get_context_usage` queries: session cost, processed tokens, context tokens
|
|
and the model window are reported values, not estimates. Compaction boundaries
|
|
surface as native compaction activity.
|
|
|
|
History is read from Claude Code's own session transcript
|
|
(`$CLAUDE_CONFIG_DIR/projects/*/<id>.jsonl` or `~/.claude`). Entries are merged by
|
|
message id, tool results attach to their calls, subagent sidechains and local
|
|
command noise are excluded, and paging is bounded. A session file larger than
|
|
64 MiB is reported as exceeding the history bound rather than silently truncated.
|
|
The live user turn keeps the client's optimistic id when it can be matched to the
|
|
recorded prompt.
|
|
|
|
## Fresh context (`/new`)
|
|
|
|
The composer's `/new` sends `chat_new_session`. It keeps the chat and clears the
|
|
agent's context, like `/new` or `/clear` in the harness's own TUI. The server
|
|
creates a fresh native session on the chat's own harness, in the same directory
|
|
and with the same title. OpenCode makes a new `ses_` session, Codex a new thread
|
|
and Claude Code a new session UUID. The server then re-keys the
|
|
`operator_sessions` row in one transaction. The new row keeps the title,
|
|
directory, tag, backend, colour, star, creation time, model override and mode.
|
|
The old row is not deleted: it records `replaced_by`, is no longer listed, and
|
|
its native session can still be resumed in a terminal.
|
|
|
|
The row is re-keyed rather than mapped from a stable id because every other
|
|
layer is keyed by the native id: rooms, queues, transcripts, shells, exports and
|
|
resume commands. The reply, and the `chat_session_changed` push to other
|
|
connections, carry the new row with `previous_id`. Each client swaps that row in
|
|
place, and an open conversation follows it to the fresh, empty transcript. A
|
|
running turn is refused, so stop it first. A second `/new` from an id that has
|
|
already been replaced is refused too.
|
|
|
|
Claude Code also accepts `/clear` in `-p --input-format stream-json` mode, but
|
|
the CLI then picks the new session id itself. Re-keying keeps one path for all
|
|
three harnesses and lets the server choose the id.
|
|
|
|
## Chat list activity
|
|
|
|
Every `chat_sessions` row carries `activity` (turn running, running background
|
|
tasks), and `chat_activity` pushes each change to every connection (see
|
|
`docs/cli.md`). Background tasks are the same shells all three harnesses already
|
|
report: OpenCode shells, Codex `commandExecution` items, and Claude Code
|
|
background Bash and Agent/Task subagents. A chat whose last viewer left stays
|
|
observed while its tasks run, so their exits are seen. That is bounded by the
|
|
unattended limits (16 chats, 6 hours) and by the harness: when a Claude Code
|
|
process exits, its background tasks are reported exited because they ran inside
|
|
it. After an OpenCode or Codex stream reconnects, the shell set is re-read from
|
|
the harness.
|
|
|
|
## Web interaction contract
|
|
|
|
All advertised chat adapters (OpenCode V2, Codex and Claude Code) implement the
|
|
same server contract for uploads, models, pending approvals, questions, form
|
|
replies, interrupts, command readback and bounded output. Compile-time assertions
|
|
and browser regressions protect this contract; support is not satisfied by text
|
|
streaming alone. Pi's pipeline executor is not advertised as a Pi web-chat
|
|
adapter, and a harness whose CLI is absent is not presented as a working choice.
|
|
|
|
Codex uploads use native `localImage` for verified supported images. Other files
|
|
are retained intact with an explicit server-local reference; the model's own
|
|
tools inspect them rather than pretending their contents were already loaded.
|
|
The saved next-turn model is also respected with uploads. Model sandbox and
|
|
approval policy remain in force. Claude Code receives images as native image
|
|
blocks, small text files inline, and larger files by absolute server path; the
|
|
saved next-turn model is applied before the turn.
|
|
|
|
Codex's paged history retains tool and reasoning items, including structured MCP
|
|
errors and Code Mode outputs, instead of discarding everything except text.
|
|
Unknown item kinds remain inspectable. Tool detail is explicitly truncated at
|
|
64 KiB, not silently represented as complete. Running commands and their output
|
|
use real command items plus read-only thread history; no unsupported terminal
|
|
endpoint, process control, resumed turn or experimental flag is invented.
|
|
Output reads are bounded and verify thread ownership. The installed Codex
|
|
version exposes no API for controlling an agent-owned terminal through the
|
|
separate, client-owned `command/exec` process IDs; those IDs are never conflated.
|
|
|
|
A fresh Codex connection can accept a turn without subscribing to its native
|
|
thread notifications. The adapter therefore reconciles the latest turn through
|
|
the read-only paged API and emits an authoritative-history resync when it changes.
|
|
It does not call `thread/resume` on open/model selection, so merely watching a
|
|
chat cannot awaken a queued goal. Native turn IDs protect the queue/UI from stale
|
|
completion events. A real, isolated tool-free turn passed after replacing the
|
|
owning connection; no command, permission grant or global setting was involved.
|
|
|
|
An explicit Send can load a persisted thread if `turn/start` is refused with
|
|
the exact application error `thread not found: <owned-thread-id>`. Only then
|
|
does the adapter call `thread/resume` (metadata only, no policy/configuration
|
|
override) and submit the original message/model/files. Transport failures and
|
|
unrelated errors are never retried. This loading is not performed by browsing,
|
|
history reads or model selection. As in Codex's native client, explicitly
|
|
continuing a thread also resumes that thread's configured execution context.
|