aa8eddd686
A stopping server no longer interrupts harness sessions. Runs are canceled with a shutdown cause, so stage clients leave admitted sessions running and owned; start-up recovery re-adopts live OpenCode V2 stage sessions (wait, read back the result, continue the graph), restarts unadmitted visits, continues runs between stages, and falls back to interrupted with a reason when the harness is gone. Chats with live work are recorded on shutdown and reattached on start; turns that ended during the downtime are folded in from history. The server-spawned V2 chat serve is now a detached owned service. PIPELINES_SHUTDOWN_SESSIONS=stop restores the old behaviour.
62 lines
2.2 KiB
Go
62 lines
2.2 KiB
Go
package stage
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"sync"
|
|
)
|
|
|
|
// A Pipelines server restart must not stop harness work. The runner cancels a
|
|
// run's context with ErrServerShutdown as its cause when the server process is
|
|
// stopping (as opposed to an operator cancel, whose cause is context.Canceled).
|
|
// A stage client that sees this cause leaves its admitted session running: it
|
|
// neither interrupts it nor clears its recorded ownership, and returns an error
|
|
// wrapping ErrDetached. The next server process re-adopts the session through
|
|
// WithResumeSession and waits for it instead of prompting again.
|
|
|
|
// ErrServerShutdown is the cancellation cause of a run context whose server is
|
|
// stopping or restarting.
|
|
var ErrServerShutdown = errors.New("pipelines server is shutting down; harness sessions are left running")
|
|
|
|
// ErrDetached reports that a stage left its admitted harness session running
|
|
// for the next server process to re-adopt.
|
|
var ErrDetached = errors.New("stage session detached for server restart")
|
|
|
|
// Detached reports whether ctx was canceled because the server is stopping.
|
|
func Detached(ctx context.Context) bool {
|
|
return ctx.Err() != nil && errors.Is(context.Cause(ctx), ErrServerShutdown)
|
|
}
|
|
|
|
type resumeKey struct{}
|
|
|
|
type resumeHolder struct {
|
|
mu sync.Mutex
|
|
session string
|
|
}
|
|
|
|
// WithResumeSession marks the next model generation under ctx as a re-adoption
|
|
// of an already admitted session: the client attaches to sessionID, waits for it
|
|
// to settle and reads its final result, without creating a session or sending a
|
|
// prompt. The mark is consumed by the first generation, so a later retry or
|
|
// repair attempt starts fresh as usual.
|
|
func WithResumeSession(ctx context.Context, sessionID string) context.Context {
|
|
if sessionID == "" {
|
|
return ctx
|
|
}
|
|
return context.WithValue(ctx, resumeKey{}, &resumeHolder{session: sessionID})
|
|
}
|
|
|
|
// TakeResumeSession returns and consumes the session marked by
|
|
// WithResumeSession, or "" when the generation is a fresh one.
|
|
func TakeResumeSession(ctx context.Context) string {
|
|
holder, _ := ctx.Value(resumeKey{}).(*resumeHolder)
|
|
if holder == nil {
|
|
return ""
|
|
}
|
|
holder.mu.Lock()
|
|
defer holder.mu.Unlock()
|
|
id := holder.session
|
|
holder.session = ""
|
|
return id
|
|
}
|