Preserve background shell visibility across readback races

Reconcile authoritative shell snapshots with bounded lifecycle journals so concurrent exits and creates cannot be overwritten, coalesce observation setup, and retain verified ownership across reconnects. Add deterministic reconciliation and cancellation regressions plus stable limited-visibility handling.\n\nChecked packaged templates and authoring skill references: none need updating. The task-owned docs note records the stale shared chat documentation for docs-fold; shared documentation was not edited.
This commit is contained in:
2026-10-18 12:54:36 +01:00
parent 101e5e8dfe
commit e9d902213e
12 changed files with 1036 additions and 25 deletions
+37
View File
@@ -0,0 +1,37 @@
# Background shell visibility
The chat server now opens the broker stream for read-only subscribers and reconciles verified running shell operations from OpenCode's authoritative `/api/shell` readback. The packaged OpenCode skill references and chat documentation should describe that shell visibility survives reconnects only when OpenCode reports explicit `metadata.sessionID` ownership; shared working directories are not ownership evidence.
## Current source behavior
Observers share one initial readback per room/stream attachment. The adapter and hub
fold bounded lifecycle journals over each snapshot, so a stale read cannot resurrect
an exited shell or erase a concurrent create. Shell replay and live delivery enqueue
in fold order. Cancellation rejects old-room callbacks and releases local identity
state before reconnect. Journals retain at most 256 identities and 1 MiB of id/command
text; capacity/read failures surface stable limited-visibility text, never broker
response bodies. No shell polling was added. OpenCode's terminal `timeout` and
`killed` statuses are excluded from retained running operations as well as `exited`.
## Verification evidence — 2026-10-18
`opencode api get /openapi.json` authenticated through supported service discovery.
The live schema advertises shell list/detail/output and generic `metadata`; it does
not type `metadata.sessionID`. A throwaway session
`ses_eb12ab7b3ffe0ZAle7PHpftHMG` and its own `sleep 1` shell
`sh_14ed57134001e6fuYqvn62pKit` verified actual create/detail readback preserving
explicit `metadata.sessionID`, status `running`, subsequent `exited`/exit 0, and
absence from the running list. This verifies explicit API metadata, not automatic
agent-tool ownership injection. The CLI event attempt yielded no captured events;
live-provider event ordering is **not tested**. Forced ordering evidence comes from
channel/barrier regressions, including a real WebSocket handler on a loopback scratch
server with fresh SQLite, not from the deployed service.
## Documentation follow-up / deployment
`docs/chat-verification.md` and the chat subscribe/replay section of `docs/cli.md`
are stale: docs-fold should describe authoritative ownership/readback, coalesced
observation, bounded reconciliation and limited-visibility errors, distinguishing
fixture ordering evidence from the live readback above. Shared docs and packaged
skill references were not edited here. No deployment, installation, service restart
or physical-phone/UI verification was performed; this changes no rendered layout.
+198 -11
View File
@@ -28,7 +28,9 @@ var ErrPermissionUnsupported = errors.New("OpenCode permission approval is not i
// semantics, the session-id shape and the 64 KiB / 1 MiB bounds of Connection so
// no second transport or credential path is introduced.
type Chat struct {
conn *Connection
conn *Connection
shellMu sync.Mutex
shells map[string]*ChatStreamState
// mu serializes the adapter's HTTP calls. Connection records its last
// operation and status on the shared connection, so two calls must not
// overlap; the permission poller runs alongside the read loop's
@@ -294,6 +296,114 @@ func decodeChatPermissions(sessionID string, data []byte) ([]protocol.ChatPermis
return frames, nil
}
// Shells reads the endpoint's authoritative shell list and returns only running
// operations whose metadata explicitly names sessionID. The shell API is global;
// filtering locally avoids guessing an unsupported query filter and never assigns
// ownership from a shared working directory. A successful read also seeds the
// stream state used to attribute later shell.exited events, which carry no session.
func (c *Chat) Shells(ctx context.Context, sessionID string) ([]protocol.ChatShell, error) {
if !ValidSessionID(sessionID) {
return nil, fmt.Errorf("invalid OpenCode session id")
}
state, err := c.streamStateContext(ctx, sessionID)
if err != nil {
return nil, err
}
// One read per state; waiting and I/O never hold the event-state mutex.
state.mu.Lock()
if state.readGate == nil {
state.readGate = make(chan struct{}, 1)
}
gate := state.readGate
state.mu.Unlock()
select {
case gate <- struct{}{}:
case <-ctx.Done():
return nil, ctx.Err()
}
defer func() { <-gate }()
state.mu.Lock()
state.readback = &ShellSnapshot{}
state.mu.Unlock()
defer func() { state.mu.Lock(); state.readback = nil; state.mu.Unlock() }()
data, err := c.call(ctx, "get", "/api/shell", nil)
if err != nil {
return nil, err
}
var response struct {
Data []chatShellInfo `json:"data"`
}
if err := json.Unmarshal(data, &response); err != nil || response.Data == nil {
if err := json.Unmarshal(data, &response.Data); err != nil {
return nil, fmt.Errorf("invalid OpenCode shell response")
}
}
out := make([]protocol.ChatShell, 0, len(response.Data))
for _, info := range response.Data {
if info.ID == "" || info.shellSessionID() != sessionID || (info.Status != "" && info.Status != protocol.ChatShellRunning) {
continue
}
status := info.Status
if status == "" {
status = protocol.ChatShellRunning
}
out = append(out, protocol.ChatShell{ID: info.ID, Status: status, Command: info.Command})
}
state.mu.Lock()
defer state.mu.Unlock()
out, err = state.readback.Merge(out)
if err != nil {
return nil, err
}
if err := ctx.Err(); err != nil {
return nil, err
}
ids := make([]string, 0, len(out))
for _, shell := range out {
ids = append(ids, shell.ID)
}
state.replaceShellsLocked(ids)
return out, nil
}
// streamState returns the session's retained, verified shell identities. It is
// deliberately kept across stream replacement so an exit after reconnect remains
// attributable; Shells reconciles its contents from authoritative readback on
// each attach rather than replacing the pointer a live scanner is using.
func (c *Chat) streamState(sessionID string) *ChatStreamState {
state, _ := c.streamStateContext(context.Background(), sessionID)
return state
}
func (c *Chat) streamStateContext(ctx context.Context, sessionID string) (*ChatStreamState, error) {
c.shellMu.Lock()
defer c.shellMu.Unlock()
if err := ctx.Err(); err != nil {
return nil, err
}
if c.shells == nil {
c.shells = map[string]*ChatStreamState{}
}
state := c.shells[sessionID]
if state == nil {
state = &ChatStreamState{}
c.shells[sessionID] = state
}
return state, nil
}
// ReleaseShells discards verified shell identities after the last room observer
// disconnects. A future attach calls Shells for a fresh authoritative snapshot,
// so this bounds state without attributing a foreign shell during the gap.
func (c *Chat) ReleaseShells(sessionID string) {
if sessionID == "" {
return
}
c.shellMu.Lock()
delete(c.shells, sessionID)
c.shellMu.Unlock()
}
// Stream subscribes to the OpenCode event firehose and invokes emit for every
// frame belonging to sessionID. It reads one bounded SSE frame at a time and
// never polls the per-session message log; the global firehose is the only
@@ -306,6 +416,12 @@ func (c *Chat) Stream(ctx context.Context, id string, ready func(), emit func(pr
if !sessionID.MatchString(id) {
return fmt.Errorf("invalid OpenCode session id")
}
// Pin state before opening/announcing the stream so a canceled scanner
// cannot acquire a replacement room's identity map.
state, err := c.streamStateContext(ctx, id)
if err != nil {
return err
}
body, err := c.conn.EventStream(ctx)
if err != nil {
return err
@@ -314,7 +430,7 @@ func (c *Chat) Stream(ctx context.Context, id string, ready func(), emit func(pr
if ready != nil {
ready()
}
return scanChatEvents(ctx, body, id, emit)
return c.scanChatEventsWithState(ctx, body, id, state, emit)
}
// scanChatEvents turns an SSE body into protocol frames, one bounded frame at a
@@ -322,10 +438,13 @@ func (c *Chat) Stream(ctx context.Context, id string, ready func(), emit func(pr
// malformed JSON fails the stream instead of producing a malformed frame. The
// last assistant message id is retained so a later tool/step event that omits it
// still attributes every frame to a stable message.
func scanChatEvents(ctx context.Context, body io.Reader, sessionID string, emit func(protocol.ChatEventFrame) error) error {
func (c *Chat) scanChatEvents(ctx context.Context, body io.Reader, sessionID string, emit func(protocol.ChatEventFrame) error) error {
return c.scanChatEventsWithState(ctx, body, sessionID, c.streamState(sessionID), emit)
}
func (c *Chat) scanChatEventsWithState(ctx context.Context, body io.Reader, sessionID string, state *ChatStreamState, emit func(protocol.ChatEventFrame) error) error {
scanner := bufio.NewScanner(body)
scanner.Buffer(make([]byte, 0, 64<<10), MaxResponse)
state := &ChatStreamState{}
for scanner.Scan() {
if err := ctx.Err(); err != nil {
return err
@@ -346,7 +465,7 @@ func scanChatEvents(ctx context.Context, body io.Reader, sessionID string, emit
continue
}
if frame.MessageID != "" {
state.LastMessageID = frame.MessageID
state.setLastMessageID(frame.MessageID)
}
if err := emit(frame); err != nil {
return err
@@ -449,8 +568,29 @@ func (i *chatShellInfo) shellSessionID() string {
// what attributes the exit to its session: another session's shell is an
// unknown id and is ignored.
type ChatStreamState struct {
mu sync.Mutex
LastMessageID string
Shells map[string]struct{}
readGate chan struct{}
readback *ShellSnapshot
}
func (s *ChatStreamState) lastMessageID() string {
if s == nil {
return ""
}
s.mu.Lock()
defer s.mu.Unlock()
return s.LastMessageID
}
func (s *ChatStreamState) setLastMessageID(id string) {
if s == nil || id == "" {
return
}
s.mu.Lock()
s.LastMessageID = id
s.mu.Unlock()
}
// knowsShell reports whether id is a shell this stream has already surfaced for
@@ -459,6 +599,8 @@ func (s *ChatStreamState) knowsShell(id string) bool {
if s == nil || id == "" {
return false
}
s.mu.Lock()
defer s.mu.Unlock()
_, ok := s.Shells[id]
return ok
}
@@ -468,6 +610,8 @@ func (s *ChatStreamState) markShell(id string) {
if s == nil || id == "" {
return
}
s.mu.Lock()
defer s.mu.Unlock()
if s.Shells == nil {
s.Shells = map[string]struct{}{}
}
@@ -476,12 +620,37 @@ func (s *ChatStreamState) markShell(id string) {
// forgetShell drops id once its exit has been surfaced.
func (s *ChatStreamState) forgetShell(id string) {
if s == nil || s.Shells == nil {
if s == nil {
return
}
s.mu.Lock()
defer s.mu.Unlock()
if s.Shells == nil {
return
}
delete(s.Shells, id)
}
// replaceShells applies one authoritative running-shell snapshot without
// replacing the state object an already attached event scanner owns.
func (s *ChatStreamState) replaceShells(ids []string) {
if s == nil {
return
}
s.mu.Lock()
defer s.mu.Unlock()
s.replaceShellsLocked(ids)
}
func (s *ChatStreamState) replaceShellsLocked(ids []string) {
s.Shells = make(map[string]struct{}, len(ids))
for _, id := range ids {
if id != "" {
s.Shells[id] = struct{}{}
}
}
}
// NormalizeChatEvent maps one OpenCode firehose `data:` payload to a
// protocol.ChatEventFrame, folding the small per-session context ([stream]) it
// needs across frames. ok is false for an event type this adapter does not
@@ -528,7 +697,7 @@ func NormalizeChatEvent(payload []byte, sessionID string, stream *ChatStreamStat
messageID = data.MessageIDAlt
}
if messageID == "" && stream != nil {
messageID = stream.LastMessageID
messageID = stream.lastMessageID()
}
// Every pushed frame must carry a stable message id so the client can
// attribute it; without one this is not a chat frame.
@@ -590,20 +759,38 @@ func normalizeShellEvent(eventType string, data chatEvent, sessionID string, str
if status == "" {
status = protocol.ChatShellRunning
}
stream.markShell(data.Info.ID)
shell := &protocol.ChatShell{ID: data.Info.ID, Status: status, Command: data.Info.Command}
if stream != nil {
stream.mu.Lock()
stream.readback.Record(shell)
if stream.Shells == nil {
stream.Shells = map[string]struct{}{}
}
if status == protocol.ChatShellRunning && len(stream.Shells) < chatShellIdentityLimit {
stream.Shells[shell.ID] = struct{}{}
}
stream.mu.Unlock()
}
return protocol.ChatEventFrame{
SessionID: sessionID,
Kind: protocol.ChatEventShell,
Shell: &protocol.ChatShell{ID: data.Info.ID, Status: status, Command: data.Info.Command},
Shell: shell,
}, true, nil
}
if data.ID == "" {
return protocol.ChatEventFrame{}, false, fmt.Errorf("malformed OpenCode shell.exited event")
}
if !stream.knowsShell(data.ID) {
if stream == nil {
return protocol.ChatEventFrame{}, false, nil
}
stream.mu.Lock()
stream.readback.Record(&protocol.ChatShell{ID: data.ID, Status: protocol.ChatShellExited})
_, known := stream.Shells[data.ID]
delete(stream.Shells, data.ID)
stream.mu.Unlock()
if !known {
return protocol.ChatEventFrame{}, false, nil
}
stream.forgetShell(data.ID)
status := data.Status
if status == "" {
status = protocol.ChatShellExited
@@ -0,0 +1,116 @@
package opencode
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"testing"
"time"
"git.a-wan.net/alan/pipelines/internal/protocol"
)
func TestChatShellReadbackBarrier(t *testing.T) {
for _, known := range []bool{false, true} {
t.Run(fmt.Sprint(known), func(t *testing.T) {
read, apply := make(chan struct{}), make(chan struct{})
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
close(read)
select {
case <-apply:
fmt.Fprint(w, `{"data":[{"id":"sh_old","status":"running","metadata":{"sessionID":"ses_1"}},{"id":"sh_foreign","status":"running","metadata":{"sessionID":"ses_other"}}]}`)
case <-r.Context().Done():
}
}))
defer srv.Close()
c := NewChat(&Connection{Backend: "v2", URL: srv.URL})
state := c.streamState("ses_1")
if known {
state.markShell("sh_old")
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
result := make(chan []protocol.ChatShell, 1)
errs := make(chan error, 1)
go func() { shells, err := c.Shells(ctx, "ses_1"); result <- shells; errs <- err }()
select {
case <-read:
case <-ctx.Done():
t.Fatal("read did not reach barrier")
}
// Both a previously verified exit and an exit first learned while
// reading must suppress the stale running snapshot.
frame, ok, err := NormalizeChatEvent([]byte(`{"type":"shell.exited","data":{"id":"sh_old","status":"exited","exit":0}}`), "ses_1", state)
if err != nil || ok != known || (ok && frame.Shell.Status != protocol.ChatShellExited) {
t.Fatalf("exit = %+v %v %v", frame, ok, err)
}
frame, ok, err = NormalizeChatEvent([]byte(`{"type":"shell.created","data":{"info":{"id":"sh_new","status":"running","command":"sleep 10","metadata":{"sessionID":"ses_1"}}}}`), "ses_1", state)
if err != nil || !ok || frame.Shell.ID != "sh_new" {
t.Fatalf("create = %+v %v %v", frame, ok, err)
}
close(apply)
got := <-result
if err := <-errs; err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].ID != "sh_new" || got[0].Command != "sleep 10" {
t.Fatalf("readback lost events/leaked ownership: %+v", got)
}
if state.knowsShell("sh_old") || state.knowsShell("sh_foreign") || !state.knowsShell("sh_new") {
t.Fatal("adapter identity state regressed")
}
// User-visible subsequent exit remains attributable after reconciliation.
frame, ok, err = NormalizeChatEvent([]byte(`{"type":"shell.exited","data":{"id":"sh_new","status":"exited","exit":0}}`), "ses_1", state)
if err != nil || !ok || frame.Shell.ID != "sh_new" {
t.Fatalf("new shell exit lost: %+v %v %v", frame, ok, err)
}
})
}
}
func TestChatShellReadbackCancelWaiter(t *testing.T) {
read := make(chan struct{})
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { close(read); <-r.Context().Done() }))
defer srv.Close()
c := NewChat(&Connection{Backend: "v2", URL: srv.URL})
ctx, cancel := context.WithCancel(context.Background())
result := make(chan error, 2)
go func() { _, err := c.Shells(ctx, "ses_1"); result <- err }()
select {
case <-read:
case <-time.After(time.Second):
t.Fatal("read stuck")
}
go func() { _, err := c.Shells(ctx, "ses_1"); result <- err }()
cancel()
for range 2 {
select {
case err := <-result:
if err == nil {
t.Fatal("canceled read succeeded")
}
case <-time.After(time.Second):
t.Fatal("read or waiter stuck")
}
}
old := c.streamState("ses_1")
c.ReleaseShells("ses_1")
if c.streamState("ses_1") == old {
t.Fatal("reconnect reused released state")
}
}
func TestShellSnapshotBounded(t *testing.T) {
var snapshot ShellSnapshot
for i := 0; i <= chatShellIdentityLimit; i++ {
snapshot.Record(&protocol.ChatShell{ID: fmt.Sprintf("sh_%d", i), Status: protocol.ChatShellExited})
}
if len(snapshot.changes) != chatShellIdentityLimit {
t.Fatalf("unbounded journal: %d", len(snapshot.changes))
}
if _, err := snapshot.Merge([]protocol.ChatShell{{ID: "sh_0", Status: protocol.ChatShellRunning}}); err == nil {
t.Fatal("overflow accepted stale snapshot")
}
}
+72
View File
@@ -0,0 +1,72 @@
package opencode
import (
"fmt"
"sort"
"git.a-wan.net/alan/pipelines/internal/protocol"
)
// ShellSnapshot records changes while an authoritative read is in flight. Its
// owner serializes access. Unknown exits may be recorded, but can only remove an
// id: they never establish ownership. Overflow rejects the snapshot, not events.
type ShellSnapshot struct {
changes map[string]protocol.ChatShell
bytes int
overflow bool
}
const chatShellIdentityLimit = 256
func (s *ShellSnapshot) Record(shell *protocol.ChatShell) {
if s == nil || shell == nil || shell.ID == "" || s.overflow {
return
}
if s.changes == nil {
s.changes = make(map[string]protocol.ChatShell)
}
old := s.changes[shell.ID]
n := s.bytes - len(old.ID) - len(old.Command) + len(shell.ID) + len(shell.Command)
if n > MaxResponse || (old.ID == "" && len(s.changes) >= chatShellIdentityLimit) {
s.overflow = true
return
}
// A terminal event wins even if a duplicate create is delivered later.
if old.ID != "" && old.Status != protocol.ChatShellRunning {
return
}
s.changes[shell.ID] = *shell
s.bytes = n
}
func (s *ShellSnapshot) Merge(shells []protocol.ChatShell) ([]protocol.ChatShell, error) {
if s.overflow {
return nil, fmt.Errorf("shell reconciliation capacity exceeded")
}
byID := make(map[string]protocol.ChatShell)
for _, shell := range shells {
if shell.ID != "" && shell.Status == protocol.ChatShellRunning {
byID[shell.ID] = shell
}
}
for id, shell := range s.changes {
if shell.Status == protocol.ChatShellRunning {
byID[id] = shell
} else {
delete(byID, id)
}
}
if len(byID) > chatShellIdentityLimit {
return nil, fmt.Errorf("shell reconciliation capacity exceeded")
}
ids := make([]string, 0, len(byID))
for id := range byID {
ids = append(ids, id)
}
sort.Strings(ids)
out := make([]protocol.ChatShell, 0, len(ids))
for _, id := range ids {
out = append(out, byID[id])
}
return out, nil
}
+42
View File
@@ -395,3 +395,45 @@ func TestOpenChatRejectsUnrunnableCLI(t *testing.T) {
t.Fatal("a missing CLI must fail detection, not open an endpoint")
}
}
// TestChatShellReadbackSeedsReplacementStream proves an authoritative shell
// readback survives a later stream replacement: shell.exited has no session id,
// so it must still clear the verified shell after reconnect.
func TestChatShellReadbackSeedsReplacementStream(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/shell":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"data":[{"id":"sh_1","status":"running","command":"sleep 30","metadata":{"sessionID":"ses_1"}},{"id":"sh_other","status":"running","command":"sleep 30","metadata":{"sessionID":"ses_other"}}]}`))
case "/api/event":
w.Header().Set("Content-Type", "text/event-stream")
_, _ = w.Write([]byte("data: {\"type\":\"shell.exited\",\"data\":{\"id\":\"sh_1\",\"status\":\"exited\",\"exit\":0}}\n\n"))
}
}))
defer srv.Close()
chat := NewChat(&Connection{Backend: "v2", URL: srv.URL})
shells, err := chat.Shells(context.Background(), "ses_1")
if err != nil || len(shells) != 1 || shells[0].ID != "sh_1" {
t.Fatalf("Shells = %+v, %v", shells, err)
}
var got []protocol.ChatEventFrame
if err := chat.Stream(context.Background(), "ses_1", nil, func(frame protocol.ChatEventFrame) error { got = append(got, frame); return nil }); err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].Shell == nil || got[0].Shell.ID != "sh_1" || got[0].Shell.Status != protocol.ChatShellExited {
t.Fatalf("exit after readback = %+v", got)
}
}
func TestChatReleaseShellsBoundsDisconnectedState(t *testing.T) {
chat := NewChat(&Connection{})
state := chat.streamState("ses_1")
state.markShell("sh_1")
chat.ReleaseShells("ses_1")
chat.shellMu.Lock()
_, retained := chat.shells["ses_1"]
chat.shellMu.Unlock()
if retained {
t.Fatal("released session shell state was retained")
}
}
+12
View File
@@ -32,9 +32,21 @@ type chatBroker interface {
// Permissions lists the session's pending OpenCode approval requests so the
// handler can push chat_permission frames.
Permissions(ctx context.Context, sessionID string) ([]protocol.ChatPermissionFrame, error)
// Shells returns only shell operations whose ownership is authoritatively
// verified for sessionID. It is the reconnect/readback source.
Shells(ctx context.Context, sessionID string) ([]protocol.ChatShell, error)
Reply(ctx context.Context, request protocol.ChatReplyRequest) error
}
// chatShellReleaser is implemented by brokers that retain verified shell ids for
// attributing exit events. The hub calls it only after the last observer leaves
// a room; a later attach obtains a fresh authoritative snapshot.
// This is a bounded local-cache operation: it must not perform I/O or call the
// hub, which serializes it with room replacement under its mutex.
type chatShellReleaser interface {
ReleaseShells(sessionID string)
}
// operatorRegistry is the server-side operator-session seam. *pipeline.Store
// satisfies it. It is a table of its own, separate from the pipeline stage
// ownership (SessionOwner/run_session_owners) that settles a run: chat_sessions
+11 -1
View File
@@ -133,6 +133,13 @@ func (h *chatHub) apply(frame protocol.ChatEventFrame) {
}
h.mu.Lock()
defer h.mu.Unlock()
h.applyLocked(frame)
}
func (h *chatHub) applyLocked(frame protocol.ChatEventFrame) {
if frame.SessionID == "" {
return
}
turn := h.turnLocked(frame.SessionID)
switch frame.Kind {
case protocol.ChatEventStep:
@@ -144,6 +151,9 @@ func (h *chatHub) apply(frame protocol.ChatEventFrame) {
case protocol.ChatEventShell:
// A shell frame carries no message id (it outlives the tool call that
// started it), so it is folded before the message-id gate below.
if room := h.rooms[frame.SessionID]; room != nil {
room.shellReadback.Record(frame.Shell)
}
turn.applyShell(frame.Shell)
return
}
@@ -240,7 +250,7 @@ func (t *chatTurn) applyShell(shell *protocol.ChatShell) {
if shell == nil || shell.ID == "" {
return
}
if shell.Status == protocol.ChatShellExited {
if shell.Status != protocol.ChatShellRunning {
old, ok := t.shells[shell.ID]
if !ok {
return
+2
View File
@@ -82,6 +82,8 @@ func (c *channelChat) Session(context.Context, string) (protocol.ChatSession, bo
return protocol.ChatSession{}, false, nil
}
func (c *channelChat) Shells(context.Context, string) ([]protocol.ChatShell, error) { return nil, nil }
func (c *channelChat) Permissions(context.Context, string) ([]protocol.ChatPermissionFrame, error) {
c.mu.Lock()
defer c.mu.Unlock()
+181 -5
View File
@@ -106,6 +106,11 @@ type chatQueuedPrompt struct {
acked bool
}
type chatObservation struct {
done chan struct{}
err error // published by closing done; belongs to this attempt, not the next
}
// chatRoom is one session's shared live channel. Every field is guarded by the
// owning chatHub's mutex.
type chatRoom struct {
@@ -117,6 +122,11 @@ type chatRoom struct {
queue []*chatQueuedPrompt
running bool
streaming bool
// observing is non-nil while one caller is attaching the shared broker
// stream. It prevents concurrent subscribers from opening duplicate global
// event streams before streaming can be published.
observing *chatObservation
shellReadback *opencode.ShellSnapshot
// buffering holds the turn's frames until the prompting client's ack has been
// written, so the ack always precedes the frames it acknowledges. It is set
// only while the very first turn subscribes to the firehose.
@@ -160,6 +170,81 @@ func (h *chatHub) roomLocked(sessionID string) *chatRoom {
return room
}
// ensureObserving starts the firehose even for a read-only subscriber, then
// reconciles retained shells from the endpoint's authoritative list. A bounded
// journal folds concurrent lifecycle events over that snapshot atomically. All
// observers share this reconciliation, with no broker I/O under the hub mutex.
func (h *chatHub) ensureObserving(room *chatRoom, broker chatBroker) error {
if broker == nil {
return errors.New("chat is not configured on this server")
}
h.mu.Lock()
if err := room.ctx.Err(); err != nil {
h.mu.Unlock()
return err
}
if waiting := room.observing; waiting != nil {
h.mu.Unlock()
select {
case <-waiting.done:
return waiting.err
case <-room.ctx.Done():
return room.ctx.Err()
}
}
if room.streaming {
h.mu.Unlock()
return nil
}
room.observing = &chatObservation{done: make(chan struct{})}
waiting := room.observing
room.shellReadback = &opencode.ShellSnapshot{}
h.mu.Unlock()
var observeErr error
defer func() {
h.mu.Lock()
room.shellReadback = nil
waiting.err = observeErr
room.observing = nil
close(waiting.done)
h.mu.Unlock()
}()
if err := h.startStream(room, broker); err != nil {
observeErr = err
return err
}
shells, err := broker.Shells(room.ctx, room.sessionID)
h.mu.Lock()
if room.ctx.Err() != nil || h.rooms[room.sessionID] != room {
h.mu.Unlock()
observeErr = context.Canceled
return observeErr
}
if err == nil {
shells, err = room.shellReadback.Merge(shells)
}
if err != nil {
// The firehose remains useful when its authoritative shell readback is
// temporarily unavailable. Do not turn that read failure into a chat
// failure (or claim the last retained snapshot is current): clear the
// replay cache and tell attached clients that shell visibility is limited.
// Preserve events already folded during this read, but clear stale replay.
live, mergeErr := room.shellReadback.Merge(nil)
if mergeErr == nil {
h.replaceShellsLocked(room.sessionID, live)
}
h.mu.Unlock()
// Never log broker bodies/URLs: they may contain credentials or private output.
log.Printf("chat shell readback session=%s failed (%T)", room.sessionID, err)
h.fanout(room.sessionID, protocol.Message{Type: protocol.Error, Error: "background shell visibility is limited"}, "")
return nil
}
h.replaceShellsLocked(room.sessionID, shells)
room.shellReadback = nil
h.mu.Unlock()
return nil
}
// attach registers write as one client's subscription to sessionID's stream and
// returns it, or the existing subscription when the same connection has already
// attached. It is idempotent so a client may prompt and subscribe without
@@ -209,6 +294,11 @@ func (h *chatHub) detach(sessionID string, sub *chatSub) {
if len(room.subs) == 0 {
delete(h.rooms, sessionID)
room.cancel()
// ReleaseShells is a local identity-cache operation (no broker I/O).
// Complete it before a reconnect can acquire this session's new room.
if releaser, ok := room.broker.(chatShellReleaser); ok {
releaser.ReleaseShells(sessionID)
}
}
h.mu.Unlock()
}
@@ -265,9 +355,17 @@ func (h *chatHub) broadcastUser(frame protocol.ChatEventFrame, exceptClientID st
// once it is written. A step-ended frame finishes the turn, letting the queue
// advance.
func (h *chatHub) emit(frame protocol.ChatEventFrame) {
h.apply(frame)
h.emitRoom(nil, frame)
}
func (h *chatHub) emitRoom(source *chatRoom, frame protocol.ChatEventFrame) {
h.mu.Lock()
room := h.rooms[frame.SessionID]
if source != nil && (source != room || source.ctx.Err() != nil) {
h.mu.Unlock()
return
}
h.applyLocked(frame)
buffered := false
var targets []*chatSub
if room != nil {
@@ -283,6 +381,22 @@ func (h *chatHub) emit(frame protocol.ChatEventFrame) {
if room != nil && frame.Kind == protocol.ChatEventStep && frame.Step != nil && frame.Step.Status == protocol.ChatStepEnded {
h.finishTurnLocked(room)
}
// Shell replay and live lifecycle delivery must enqueue in the same order
// as folding. Otherwise an exit can overtake an older running replay.
if frame.Kind == protocol.ChatEventShell && !buffered {
msg := protocol.Message{Type: protocol.ChatEvent, ChatEvent: &frame}
var slow []*chatSub
for _, sub := range targets {
if !enqueueChatLocked(sub, msg) {
slow = append(slow, sub)
}
}
h.mu.Unlock()
for _, sub := range slow {
sub.close()
}
return
}
h.mu.Unlock()
if buffered {
return
@@ -293,6 +407,62 @@ func (h *chatHub) emit(frame protocol.ChatEventFrame) {
}
}
// enqueueChatLocked never blocks or calls detach under the hub mutex.
func enqueueChatLocked(sub *chatSub, msg protocol.Message) bool {
select {
case <-sub.done:
return true
default:
}
select {
case sub.ch <- msg:
return true
default:
return false
}
}
func (h *chatHub) replayShells(sub *chatSub) {
h.mu.Lock()
slow := false
if h.rooms[sub.room.sessionID] == sub.room && sub.room.subs[sub.clientID] == sub {
turn := h.turnLocked(sub.room.sessionID)
for _, id := range turn.shellOrder {
shell := turn.shells[id]
frame := protocol.ChatEventFrame{SessionID: sub.room.sessionID, Kind: protocol.ChatEventShell, Shell: &shell}
if !enqueueChatLocked(sub, protocol.Message{Type: protocol.ChatEvent, ChatEvent: &frame}) {
slow = true
break
}
}
}
h.mu.Unlock()
if slow {
sub.close()
}
}
// replaceShells reconciles a successful authoritative readback. It drops stale
// retained entries (including exits missed while disconnected) and upserts the
// verified running set without fabricating a lifecycle event for foreign shells.
func (h *chatHub) replaceShells(sessionID string, shells []protocol.ChatShell) {
h.mu.Lock()
defer h.mu.Unlock()
h.replaceShellsLocked(sessionID, shells)
}
func (h *chatHub) replaceShellsLocked(sessionID string, shells []protocol.ChatShell) {
turn := h.turnLocked(sessionID)
for _, shell := range turn.shells {
turn.shrink(len(shell.Command))
}
turn.shells = nil
turn.shellOrder = nil
for i := range shells {
turn.applyShell(&shells[i])
}
}
// finishTurnLocked marks the room's turn finished and releases the turn runner's
// wait. Callers must hold h.mu.
func (h *chatHub) finishTurnLocked(room *chatRoom) {
@@ -331,14 +501,20 @@ func (h *chatHub) flushBuffered(sessionID string) {
targets = append(targets, sub)
}
}
h.mu.Unlock()
var slow []*chatSub
for i := range pending {
frame := pending[i]
msg := protocol.Message{Type: protocol.ChatEvent, ChatEvent: &frame}
for _, sub := range targets {
sub.send(msg)
if !enqueueChatLocked(sub, msg) {
slow = append(slow, sub)
}
}
}
h.mu.Unlock()
for _, sub := range slow {
sub.close()
}
}
// submitPrompt accepts one prompt: it starts the turn runner immediately when no
@@ -476,7 +652,7 @@ func (h *chatHub) runTurn(room *chatRoom, p *chatQueuedPrompt, done chan struct{
return
}
if first {
if err := h.startStream(room, broker); err != nil {
if err := h.ensureObserving(room, broker); err != nil {
h.failPrompt(room, p, err)
h.endRoomTurn(room)
return
@@ -550,7 +726,7 @@ func (h *chatHub) startStream(room *chatRoom, broker chatBroker) error {
result <- broker.Stream(ctx, room.sessionID, func() {
once.Do(func() { close(attached) })
}, func(frame protocol.ChatEventFrame) error {
h.emit(frame)
h.emitRoom(room, frame)
return nil
})
}()
@@ -0,0 +1,261 @@
package server
import (
"context"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
"git.a-wan.net/alan/pipelines/internal/protocol"
"github.com/coder/websocket"
"github.com/coder/websocket/wsjson"
)
type shellBarrierBroker struct {
*fakeChat
read chan struct{}
apply chan struct{}
events chan shellBarrierEvent
stopped chan struct{}
reads atomic.Int32
snapshot []protocol.ChatShell
}
// Exercise the real websocket handler on a loopback scratch server and a fresh
// SQLite database. A marker on the shared stream bounds the replay assertion
// without sleeps, polling or a negative-read timeout.
func TestChatShellReconciliationScratchServer(t *testing.T) {
b := newShellBarrierBroker()
b.snapshot = []protocol.ChatShell{{ID: "sh_old", Status: protocol.ChatShellRunning}}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
srv := httptest.NewServer(HandlerWithOptions(ctx, Options{Chat: b}, newOperatorStore(t)))
defer srv.Close()
conn, _, err := websocket.Dial(ctx, "ws"+strings.TrimPrefix(srv.URL, "http")+"/connect", nil)
if err != nil {
t.Fatal(err)
}
defer conn.CloseNow()
subscribe := func(id string) {
t.Helper()
if err := wsjson.Write(ctx, conn, protocol.Message{Type: protocol.ChatSubscribe, ID: id, ChatSubscribe: &protocol.ChatQuery{SessionID: "ses_1"}}); err != nil {
t.Fatal(err)
}
}
read := func() protocol.Message {
t.Helper()
var m protocol.Message
if err := wsjson.Read(ctx, conn, &m); err != nil {
t.Fatal(err)
}
return m
}
if m := read(); m.Type != protocol.PipelineList {
t.Fatalf("server greeting = %s", m.Type)
}
subscribe("first")
if m := read(); m.ID != "first" || m.Type != protocol.ChatSubscribe {
t.Fatalf("ack = %+v", m)
}
shellBarrierWait(t, b.read)
done := make(chan struct{})
b.events <- shellBarrierEvent{protocol.ChatShell{ID: "sh_old", Status: protocol.ChatShellExited}, done}
shellBarrierWait(t, done)
if m := read(); m.ChatEvent == nil || m.ChatEvent.Shell.Status != protocol.ChatShellExited {
t.Fatalf("exit not visible: %+v", m)
}
close(b.apply)
subscribe("second")
for {
m := read()
if m.ChatEvent != nil && m.ChatEvent.Shell.Status == protocol.ChatShellRunning {
t.Fatalf("stale replay: %+v", m)
}
if m.ID == "second" {
break
}
}
done = make(chan struct{})
b.events <- shellBarrierEvent{protocol.ChatShell{ID: "sh_marker", Status: protocol.ChatShellExited}, done}
shellBarrierWait(t, done)
for {
m := read()
if m.ChatEvent == nil {
continue
}
if m.ChatEvent.Shell.Status == protocol.ChatShellRunning {
t.Fatalf("ghost visible after apply: %+v", m)
}
if m.ChatEvent.Shell.ID == "sh_marker" {
break
}
}
if b.reads.Load() != 1 {
t.Fatalf("shared observer read count = %d", b.reads.Load())
}
conn.CloseNow()
shellBarrierWait(t, b.stopped)
}
type shellBarrierEvent struct {
shell protocol.ChatShell
done chan struct{}
}
func newShellBarrierBroker() *shellBarrierBroker {
return &shellBarrierBroker{fakeChat: scriptedChat(), read: make(chan struct{}, 8), apply: make(chan struct{}), events: make(chan shellBarrierEvent), stopped: make(chan struct{})}
}
func (b *shellBarrierBroker) Shells(ctx context.Context, _ string) ([]protocol.ChatShell, error) {
b.reads.Add(1)
snapshot := append([]protocol.ChatShell(nil), b.snapshot...)
b.read <- struct{}{}
select {
case <-b.apply:
return snapshot, nil
case <-ctx.Done():
return nil, ctx.Err()
}
}
func (b *shellBarrierBroker) Stream(ctx context.Context, id string, ready func(), emit func(protocol.ChatEventFrame) error) error {
defer close(b.stopped)
ready()
for {
select {
case <-ctx.Done():
return ctx.Err()
case event := <-b.events:
if err := emit(protocol.ChatEventFrame{SessionID: id, Kind: protocol.ChatEventShell, Shell: &event.shell}); err != nil {
return err
}
close(event.done)
}
}
}
func shellBarrierWait(t *testing.T, ch <-chan struct{}) {
t.Helper()
select {
case <-ch:
case <-time.After(3 * time.Second):
t.Fatal("barrier timed out")
}
}
func TestChatShellReconciliationBarrier(t *testing.T) {
for _, status := range []string{protocol.ChatShellExited, protocol.ChatShellRunning, "timeout", "killed"} {
t.Run(status, func(t *testing.T) {
b := newShellBarrierBroker()
b.snapshot = []protocol.ChatShell{{ID: "sh_old", Status: protocol.ChatShellRunning}}
h := newChatHub()
messages := make(chan protocol.Message, 32)
sub := h.attach("ses_1", "one", "", b, func(m protocol.Message) error { messages <- m; return nil })
t.Cleanup(sub.close)
result := make(chan error, 2)
go func() { result <- h.ensureObserving(sub.room, b) }()
shellBarrierWait(t, b.read)
// A second observer must share the in-flight read, not apply an
// independently completing older/newer snapshot out of order.
go func() { result <- h.ensureObserving(sub.room, b) }()
shell := protocol.ChatShell{ID: "sh_old", Status: status}
if status == protocol.ChatShellRunning {
shell.ID = "sh_new"
shell.Command = "sleep 10"
}
done := make(chan struct{})
select {
case b.events <- shellBarrierEvent{shell, done}:
case <-time.After(time.Second):
t.Fatal("event blocked behind readback")
}
shellBarrierWait(t, done)
select {
case m := <-messages:
if m.ChatEvent == nil || m.ChatEvent.Shell == nil || m.ChatEvent.Shell.ID != shell.ID || m.ChatEvent.Shell.Status != status {
t.Fatalf("live shell = %+v", m)
}
case <-time.After(time.Second):
t.Fatal("live event not visible during read")
}
close(b.apply)
for range 2 {
select {
case err := <-result:
if err != nil {
t.Fatal(err)
}
case <-time.After(time.Second):
t.Fatal("observer stuck")
}
}
if b.reads.Load() != 1 {
t.Fatalf("redundant/out-of-order reads: %d", b.reads.Load())
}
snapshot := h.shellSnapshot("ses_1")
if status == protocol.ChatShellRunning {
if len(snapshot) != 2 {
t.Fatalf("new create lost: %+v", snapshot)
}
} else if len(snapshot) != 0 {
t.Fatalf("terminal shell resurrected: %+v", snapshot)
}
// Assert the same retained state is replayed to the client.
h.replayShells(sub)
for range len(snapshot) {
select {
case m := <-messages:
if m.ChatEvent == nil || m.ChatEvent.Shell.Status != protocol.ChatShellRunning {
t.Fatalf("replay = %+v", m)
}
case <-time.After(time.Second):
t.Fatal("replay missing")
}
}
sub.close()
shellBarrierWait(t, b.stopped)
})
}
}
func TestChatShellReconciliationCancelReconnectBarrier(t *testing.T) {
b := newShellBarrierBroker()
h := newChatHub()
sub := h.attach("ses_1", "one", "", b, func(protocol.Message) error { return nil })
results := make(chan error, 2)
go func() { results <- h.ensureObserving(sub.room, b) }()
shellBarrierWait(t, b.read)
go func() { results <- h.ensureObserving(sub.room, b) }()
sub.close()
for range 2 {
select {
case err := <-results:
if err == nil {
t.Fatal("canceled observer succeeded")
}
case <-time.After(time.Second):
t.Fatal("canceled waiter stuck")
}
}
shellBarrierWait(t, b.stopped)
next := newShellBarrierBroker()
next.snapshot = []protocol.ChatShell{{ID: "sh_new", Status: protocol.ChatShellRunning}}
close(next.apply)
newSub := h.attach("ses_1", "two", "", next, func(protocol.Message) error { return nil })
t.Cleanup(newSub.close)
if err := h.ensureObserving(newSub.room, next); err != nil {
t.Fatal(err)
}
// A late callback from the canceled stream cannot poison the new room,
// nor can a foreign-session callback from the current stream.
h.emitRoom(sub.room, protocol.ChatEventFrame{SessionID: "ses_1", Kind: protocol.ChatEventShell, Shell: &protocol.ChatShell{ID: "sh_ghost", Status: protocol.ChatShellRunning}})
h.emitRoom(newSub.room, protocol.ChatEventFrame{SessionID: "ses_foreign", Kind: protocol.ChatEventShell, Shell: &protocol.ChatShell{ID: "sh_foreign", Status: protocol.ChatShellRunning}})
got := h.shellSnapshot("ses_1")
if len(got) != 1 || got[0].ID != "sh_new" || len(h.shellSnapshot("ses_foreign")) != 0 {
t.Fatalf("reconnect state = %+v", got)
}
newSub.close()
shellBarrierWait(t, next.stopped)
}
+8 -5
View File
@@ -111,7 +111,14 @@ func handleChatSubscribe(request protocol.Message, broker chatBroker, hub *chatH
_ = write(protocol.Message{Type: protocol.Error, ID: request.ID, Error: "could not attach to session " + query.SessionID})
return
}
// Acknowledge before the stream is allowed to fan out: otherwise an immediately
// available retained shell can overtake the subscribe reply on this connection.
_ = write(protocol.Message{Type: protocol.ChatSubscribe, ID: request.ID})
if err := hub.ensureObserving(sub.room, broker); err != nil {
sub.close()
_ = write(protocol.Message{Type: protocol.Error, ID: request.ID, Error: "could not observe session " + query.SessionID})
return
}
// Replay every still-unresolved approval request the room holds, not only the
// most recent one, so a client that attaches after two requests were raised
// still sees both cards.
@@ -124,9 +131,5 @@ func handleChatSubscribe(request protocol.Message, broker chatBroker, hub *chatH
// shows them rather than nothing until they exit. The message snapshot cannot
// carry a shell — a shell outlives the turn and is not a turn — so this is the
// shell half of the resume.
for _, shell := range hub.shellSnapshot(query.SessionID) {
running := shell
frame := protocol.ChatEventFrame{SessionID: query.SessionID, Kind: protocol.ChatEventShell, Shell: &running}
sub.send(protocol.Message{Type: protocol.ChatEvent, ChatEvent: &frame})
}
hub.replayShells(sub)
}
+96 -3
View File
@@ -3,6 +3,7 @@ package server
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/http/httptest"
@@ -52,6 +53,8 @@ type fakeChat struct {
permissionSessionID string
permissions []protocol.ChatPermissionFrame
permissionErr error
shells []protocol.ChatShell
shellErr error
sessionCalls int
sessions []protocol.ChatSession
sessionErr error
@@ -62,6 +65,8 @@ type fakeChat struct {
started chan struct{}
done chan struct{}
startOnce sync.Once
streamCalls int
readyGate <-chan struct{}
block bool
subscribed bool
promptBeforeSubscribe bool
@@ -91,13 +96,18 @@ func (f *fakeChat) Prompt(context.Context, string, string) error {
func (f *fakeChat) Stream(ctx context.Context, _ string, ready func(), emit func(protocol.ChatEventFrame) error) error {
f.mu.Lock()
f.subscribed = true
f.streamCalls++
readyGate := f.readyGate
f.mu.Unlock()
if ready != nil {
ready()
}
if f.started != nil {
f.startOnce.Do(func() { close(f.started) })
}
if readyGate != nil {
<-readyGate
}
if ready != nil {
ready()
}
if f.block {
<-ctx.Done()
if f.done != nil {
@@ -116,6 +126,42 @@ func (f *fakeChat) Stream(ctx context.Context, _ string, ready func(), emit func
return nil
}
// TestChatHubConcurrentObserversShareStream pins the start-up handoff: two
// clients that subscribe before the broker says ready still share one firehose.
func TestChatHubConcurrentObserversShareStream(t *testing.T) {
broker := scriptedChat()
broker.block = true
broker.started = make(chan struct{})
gate := make(chan struct{})
broker.readyGate = gate
hub := newChatHub()
hub.mu.Lock()
room := hub.roomLocked("ses_1")
hub.mu.Unlock()
errs := make(chan error, 2)
go func() { errs <- hub.ensureObserving(room, broker) }()
select {
case <-broker.started:
case <-time.After(time.Second):
t.Fatal("first observer did not start the broker stream")
}
go func() { errs <- hub.ensureObserving(room, broker) }()
close(gate)
for range 2 {
if err := <-errs; err != nil {
t.Fatalf("ensureObserving: %v", err)
}
}
broker.mu.Lock()
calls := broker.streamCalls
broker.mu.Unlock()
if calls != 1 {
t.Fatalf("broker Stream calls = %d, want 1", calls)
}
room.cancel()
}
func (f *fakeChat) Interrupt(context.Context, string) error {
f.mu.Lock()
defer f.mu.Unlock()
@@ -151,6 +197,12 @@ func (f *fakeChat) Reply(_ context.Context, request protocol.ChatReplyRequest) e
// Permissions returns the seeded pending requests, or blocks until the caller's
// context is canceled when permBlock is set (used to prove the poller stops on
// disconnect). Every call is counted so a test can assert polling happened.
func (f *fakeChat) Shells(context.Context, string) ([]protocol.ChatShell, error) {
f.mu.Lock()
defer f.mu.Unlock()
return append([]protocol.ChatShell(nil), f.shells...), f.shellErr
}
func (f *fakeChat) Permissions(ctx context.Context, sessionID string) ([]protocol.ChatPermissionFrame, error) {
f.mu.Lock()
f.permissionCalls++
@@ -682,6 +734,9 @@ func TestChatInterruptEndsRunningPrompt(t *testing.T) {
case r.Method == http.MethodGet && r.URL.Path == "/api/permission/request":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"location":{"directory":"/work"},"data":[]}`))
case r.Method == http.MethodGet && r.URL.Path == "/api/shell":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"data":[]}`))
case r.Method == http.MethodGet && r.URL.Path == "/api/event":
flusher, ok := w.(http.Flusher)
if !ok {
@@ -995,6 +1050,7 @@ func TestChatPromptStreamsShellFrame(t *testing.T) {
// shows the background process instead of nothing until it exits.
func TestChatSubscribeReplaysRunningShell(t *testing.T) {
broker := scriptedChat()
broker.shells = []protocol.ChatShell{{ID: "sh_1", Status: protocol.ChatShellRunning, Command: "sleep 120"}}
broker.script = append([]protocol.ChatEventFrame{
{SessionID: "ses_1", Kind: protocol.ChatEventShell, Shell: &protocol.ChatShell{ID: "sh_1", Status: protocol.ChatShellRunning, Command: "sleep 120"}},
}, broker.script...)
@@ -1039,6 +1095,40 @@ func TestChatSubscribeReplaysRunningShell(t *testing.T) {
}
}
// TestChatSubscribeKeepsStreamWhenShellReadbackFails ensures a temporary
// authoritative readback failure does not make chat unavailable or replay a
// stale process as running. The subscriber receives an explicit limitation
// while its broker stream remains attached for future lifecycle events.
func TestChatSubscribeKeepsStreamWhenShellReadbackFails(t *testing.T) {
broker := scriptedChat()
broker.block = true
broker.shellErr = errors.New("shell endpoint unavailable: https://user:secret@example.invalid private response body")
ctx, _, conn := dialChat(t, broker)
if err := wsjson.Write(ctx, conn, protocol.Message{Type: protocol.ChatSubscribe, ID: "sub1", ChatSubscribe: &protocol.ChatQuery{SessionID: "ses_1"}}); err != nil {
t.Fatal(err)
}
var ack protocol.Message
if err := wsjson.Read(ctx, conn, &ack); err != nil {
t.Fatal(err)
}
if ack.Type != protocol.ChatSubscribe || ack.ID != "sub1" {
t.Fatalf("subscribe ack = %+v", ack)
}
var limited protocol.Message
if err := wsjson.Read(ctx, conn, &limited); err != nil {
t.Fatal(err)
}
if limited.Type != protocol.Error || limited.Error != "background shell visibility is limited" {
t.Fatalf("limited visibility notice = %+v", limited)
}
broker.mu.Lock()
calls := broker.streamCalls
broker.mu.Unlock()
if calls != 1 {
t.Fatalf("broker Stream calls = %d, want 1", calls)
}
}
// TestChatPromptSubscribesBeforePrompt pins the subscribe-before-prompt order:
// the firehose has no replay cursor, so the subscription must be attached before
// the prompt is posted or deltas emitted in between are lost.
@@ -1079,6 +1169,9 @@ func realAdapterChat(t *testing.T, frames []string) []protocol.ChatEventFrame {
case r.Method == http.MethodGet && r.URL.Path == "/api/permission/request":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"location":{"directory":"/work"},"data":[]}`))
case r.Method == http.MethodGet && r.URL.Path == "/api/shell":
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"data":[]}`))
case r.Method == http.MethodGet && r.URL.Path == "/api/event":
streams.Add(1)
w.Header().Set("Content-Type", "text/event-stream")