Retain Codex interaction callbacks across server restarts
This commit is contained in:
+35
-4
@@ -36,10 +36,12 @@ chat_close; see `internal/server/chat_manager.go` and `chat_detach.go`):
|
||||
`<cache>/pipelines/private/opencode-chat-service.json`). The next server
|
||||
reuses it at the same address and password, and it is not stopped on
|
||||
shutdown. A V1 serve child still ends with the server.
|
||||
- **Codex**: threads and turns live in the App Server. Detaching closes the
|
||||
control connection, and the next server reconciles through `thread/read`.
|
||||
An approval or question the App Server sent to the old connection is bound to
|
||||
it, so one still pending at shutdown cannot be answered after the restart.
|
||||
- **Codex**: threads and turns live in the App Server. The installed server
|
||||
connects through `pipelines-codex-host`, which retains the original App Server
|
||||
connection and unanswered approvals/questions while Pipelines detaches. A new
|
||||
server receives those same callbacks and reconciles ordinary turn state through
|
||||
readback. Direct mode (hosting disabled or helper absent) still loses pending
|
||||
connection-bound interactions on restart.
|
||||
- **Claude Code**: its detachable hosts keep the CLI running (see the Claude
|
||||
Code section).
|
||||
|
||||
@@ -72,6 +74,35 @@ not a fake empty success or silent auto-approval. App Server socket loss does
|
||||
not provide durable callback recovery; a disconnected transport is reported as
|
||||
an error rather than claiming old requests are still actionable.
|
||||
|
||||
### Pending interactions across restarts
|
||||
|
||||
The installed `pipelines-codex-host` helper owns one WebSocket connection to the
|
||||
existing Codex App Server Unix socket. It never starts a second Codex daemon or
|
||||
changes Codex settings. Helpers are scoped by database path and upstream socket,
|
||||
with mode-0600 Unix sockets and singleton locks under the mode-0700 directory
|
||||
`<user cache dir>/pipelines/codex-hosts/<database hash>/`.
|
||||
|
||||
On reconnect the helper returns its cached initialization handshake and replays
|
||||
only unanswered server requests, preserving exact callback IDs and parameters.
|
||||
Responses and `serverRequest/resolved` remove pending entries. Old/new client RPC
|
||||
IDs are remapped so a delayed response for an old Pipelines process cannot answer
|
||||
a new process's call. A new attach closes the obsolete downstream connection.
|
||||
Ordinary notifications are not spooled; the existing thread readback repairs gaps.
|
||||
|
||||
State is bounded to 128 callbacks / 8 MiB and 256 in-flight client calls, with a
|
||||
16 MiB frame limit. Exceeding pending bounds closes the bridge without granting,
|
||||
declining or otherwise auto-answering an interaction. A detached bridge expires
|
||||
after 20 minutes with no pending interactions, or after 24 hours even if pending.
|
||||
An upstream App Server restart, a bridge crash, or expiry loses its connection-
|
||||
bound callbacks: no disk JSON can recreate them on a new connection. The initial
|
||||
deployment cannot recover requests already owned by the old direct connection.
|
||||
|
||||
`PIPELINES_CODEX_HOST` overrides the helper executable; `off` disables hosting.
|
||||
The server otherwise finds it beside `pipelines-server`, then on PATH. Without
|
||||
hosting, direct-mode fixtures and intentional direct connections retain their
|
||||
previous behaviour and limitation. Installing a new helper binary does not kill
|
||||
an existing helper or its pending requests.
|
||||
|
||||
## Claude Code
|
||||
|
||||
The adapter in `internal/claude` owns one Claude Code CLI process per active
|
||||
|
||||
+4
-3
@@ -42,8 +42,9 @@ Linux needs `systemctl --user`. Other platforms are refused.
|
||||
2. Builds the Flutter web app in Docker (pinned to `app/.flutter-version`).
|
||||
Builds are cached in `~/.cache/pipelines/app-builds` by a digest of the app
|
||||
sources, so an unchanged app is reused.
|
||||
3. Builds `pipelines-server`, `pipelines` and `pipelines-claude-host` (keeps
|
||||
Claude Code chats alive across server restarts) with `go build -trimpath`.
|
||||
3. Builds `pipelines-server`, `pipelines`, `pipelines-claude-host` and
|
||||
`pipelines-codex-host` (retain Claude processes and Codex interaction callbacks
|
||||
across server restarts) with `go build -trimpath`.
|
||||
4. Installs into the prefix: binaries in `bin/`, web app in `lib/app-web`, web
|
||||
proxy in `lib/web`, `lib/manage.py`, and an empty `lib/app-updates` for APKs.
|
||||
5. Runs `pipelines skill install` (the authoring skill for OpenCode and Claude
|
||||
@@ -239,7 +240,7 @@ autostart, not boot-without-login.
|
||||
| Path | Contents |
|
||||
| --- | --- |
|
||||
| `<prefix>/pipelines-install.json` | manifest (ports, paths, install args, environment) |
|
||||
| `<prefix>/bin/` | `pipelines`, `pipelines-server`, `pipelines-claude-host`, `ntfy` |
|
||||
| `<prefix>/bin/` | `pipelines`, `pipelines-server`, `pipelines-claude-host`, `pipelines-codex-host`, `ntfy` |
|
||||
| `<prefix>/data/pipelines.db` | SQLite database (definitions, runs, chats, settings) |
|
||||
| `<prefix>/data/artifacts/` | run artifacts and session exports |
|
||||
| `<prefix>/data/ntfy/` | ntfy config, cache and accounts |
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# Codex pending interactions across Pipelines restarts
|
||||
|
||||
The App Server's pending approval/question callback IDs belong to the connection
|
||||
that received them. Its current official schema exposes no pending-request
|
||||
listing/replay method. Saving JSON in the Pipelines database cannot make an old
|
||||
callback answerable on a new upstream connection.
|
||||
|
||||
Added `pipelines-codex-host`: a detached, private local WebSocket bridge that owns
|
||||
the existing App Server connection. It does not launch Codex. The installed
|
||||
server scopes hosts by database and upstream socket, finds/reuses the helper,
|
||||
and initializes only its downstream connection after a restart. The host caches
|
||||
the one upstream initialization and replays exactly the still-unanswered server
|
||||
requests. Responses or matching serverRequest/resolved notifications remove
|
||||
pending entries. Client call IDs are translated to unique host IDs and routed
|
||||
only to the originating client; replacing a client discards old routes. Client
|
||||
and callback RPC IDs are capped at 256 bytes.
|
||||
|
||||
Unix sockets are mode 0600 under an owner-verified mode-0700 directory. Per-scope
|
||||
file locks prevent simultaneous helpers. The existing daemon connection, not a
|
||||
PID or a model statement, owns pending callbacks. Pending memory is capped at
|
||||
128 requests / 8 MiB; calls at 256; incoming frames at 16 MiB. Overflow closes
|
||||
rather than answering an interaction. With no Pipelines client, helpers expire
|
||||
after 20 minutes with no pending callbacks, or after 24 hours even when pending.
|
||||
Ordinary notifications are not spooled; current thread readback repairs gaps.
|
||||
|
||||
Native installation, tester packages and Docker include the helper; Go-bin
|
||||
uninstallation verifies and removes its executable. Missing helper or explicit
|
||||
PIPELINES_CODEX_HOST=off retains direct mode and logs its restart limitation.
|
||||
Direct-mode fixtures still pass. Hosts run independently of a launching request's
|
||||
context and are not killed by installing a replacement binary.
|
||||
|
||||
Verification:
|
||||
- `go test -race ./internal/codexhost ./internal/codex ./internal/installation
|
||||
./cmd/pipelines-server` passes. Real Unix-socket fixtures cover pending approval
|
||||
and question responses across client replacement, duplicate response rejection,
|
||||
resolved callbacks, matching thread ownership, old/new RPC-ID collision
|
||||
isolation, detached arrival/replay, bounded pending state, singleton ownership,
|
||||
private permissions, shutdown cleanup, and actual helper process spawning/reuse.
|
||||
- The actual Codex adapter's Open -> Detach -> Open integration retains both
|
||||
permission and question maps, answers their original IDs, initializes upstream
|
||||
once, and does not replay resolved requests on a third client.
|
||||
- Existing direct Codex tests pass; `go build ./...`, `go vet ./...`, and all 53
|
||||
native-installer Python tests pass. Full Go integration is the manager's check.
|
||||
|
||||
No live service, conversation, daemon, credential or deployment was changed.
|
||||
The initial deployment cannot rescue callbacks already owned by the old direct
|
||||
connection; pending interactions must settle before that first replacement.
|
||||
A helper crash, upstream App Server restart, or expiry breaks callback identity
|
||||
and remains an explicit limitation. No disk reconstruction is claimed.
|
||||
Reference in New Issue
Block a user