Run OpenCode stages on a Pipelines-owned stage service with their own limits

Stages now run on a detached Pipelines-owned opencode serve started with
OPENCODE_CONFIG_CONTENT carrying stage tool_output (16384 bytes / 400 lines)
and compaction.auto defaults, configurable through PIPELINES_OPENCODE_STAGE_*.
Chats keep their existing service and configuration; a server-spawned chat
serve can take PIPELINES_OPENCODE_CHAT_CONFIG. The service shares the user's
OpenCode config, credentials and session database, keeps a stable private
registration so retained references (recovery, interrupt, export) stay valid,
revives on the same port after a crash, replaces itself when idle after a
settings change, and is stopped on shutdown only when idle.
PIPELINES_OPENCODE_STAGE_SERVICE=shared restores the previous behaviour.
This commit is contained in:
2026-11-04 14:04:35 +00:00
parent fa6c0864db
commit b655278674
12 changed files with 977 additions and 15 deletions
+3
View File
@@ -137,6 +137,9 @@ func main() {
defer chats.Close()
}
srv := &http.Server{Addr: *addr, Handler: server.HandlerWithOptions(ctx, options, store), ReadHeaderTimeout: 5 * time.Second}
// Deferred before WaitRuns so it runs after the runs settle: the owned stage
// service is stopped only when no stage session is still active on it.
defer opencode.StopIdleStageService()
defer server.WaitRuns(srv.Handler)
go func() {
<-ctx.Done()
+68
View File
@@ -36,6 +36,74 @@ provider spending limits separate. Existing definitions, defaults and published
versions are never rewritten. Editing the setting requires publishing a new
pipeline version as usual.
## OpenCode limits for stages and chats
OpenCode reads its tool-output and compaction limits only from configuration;
there is no per-session or per-agent setting. Pipeline stages therefore run on
their own OpenCode process, the **stage service**, so stages and chats can use
different limits:
- **Stages** run on a Pipelines-owned `opencode serve` on `127.0.0.1`, started
lazily by the first stage with `OPENCODE_CONFIG_CONTENT` holding the stage
limits merged over any inherited `OPENCODE_CONFIG_CONTENT`. It is the same
binary, user and XDG directories as your normal OpenCode, so it reads your
configuration, directory `opencode.json` files and provider credentials, and
writes sessions to the same OpenCode database (SQLite WAL; OpenCode sets a
busy timeout). Stage sessions stay visible to `opencode --session <id>`.
- **Chats** keep the OpenCode service they already use and its configuration.
Pipelines never writes `~/.config/opencode`. `PIPELINES_OPENCODE_CHAT_CONFIG`
(a JSON object) applies only when the server spawns its own chat
`opencode serve`. A shared or `-opencode-endpoint` service keeps the limits in
your OpenCode configuration (`tool_output`, `compaction`).
Server environment settings (add them to `opencode_env` in the install
manifest, or the server's environment):
| Setting | Default | Meaning |
| --- | --- | --- |
| `PIPELINES_OPENCODE_STAGE_TOOL_OUTPUT_MAX_BYTES` | `16384` | stage `tool_output.max_bytes`; `0` keeps OpenCode's default (51200) |
| `PIPELINES_OPENCODE_STAGE_TOOL_OUTPUT_MAX_LINES` | `400` | stage `tool_output.max_lines`; `0` keeps OpenCode's default (2000) |
| `PIPELINES_OPENCODE_STAGE_COMPACTION` | `on` | `on`/`off` set `compaction.auto`; `opencode` leaves it unset |
| `PIPELINES_OPENCODE_STAGE_CONFIG` | none | JSON object merged last, e.g. `{"compaction":{"keep":{"tokens":20000}}}` |
| `PIPELINES_OPENCODE_STAGE_SERVICE` | `owned` | `shared` runs stages on the CLI-managed service again, without stage limits |
| `PIPELINES_OPENCODE_CHAT_CONFIG` | none | JSON object for a server-spawned chat `opencode serve` only |
Over-limit tool output is truncated by OpenCode itself. Bash keeps the tail
with a `[showing lines X-Y of N; full output saved to <file>]` note, so the
model can still read the full file.
How the stage service behaves:
- **Registration.** Its URL, password and pid are in the owner-only file
`opencode-stage-service.json` (log: `opencode-stage-service.log`) under the
user cache directory's `pipelines/private`. A stage's retained reference
points at this file, so restart recovery, interruption and `session.html`
export reach the same service. References made before this change still point
at the CLI-managed service and keep working.
- **Health and restarts.** Every stage admission checks it with the
authenticated `/api/info`. A crashed service is started again on the same
port and password, also when a retained reference is used after it exited.
Start-up is bounded (20 s per attempt, 3 attempts). If it cannot start, the
stage fails with `OpenCode stage service exited before becoming ready (see
<log>)` or a timeout error. It never falls back to the shared service, because
that would silently drop the stage limits. Use
`PIPELINES_OPENCODE_STAGE_SERVICE=shared` to opt out deliberately.
- **Changed settings or an upgraded OpenCode** restart the service when no
session is active on it. A busy service keeps its old limits until it is idle.
- **Lifetime.** It runs detached in its own process session, so a stopping
Pipelines service does not take active stage work with it. On server shutdown
it is stopped only when no session is active on it. Otherwise restart
recovery interrupts the retained sessions through it, as before.
- `pipelines debug` is unchanged: it opens a fresh interactive harness
conversation, not the stage session.
Claude Code and Codex are chat-only harnesses (stages accept only
`harness=opencode` or `harness=pi`), so they need no stage limit. Their chats
keep their own defaults. Claude Code has `BASH_MAX_OUTPUT_LENGTH` and
`MAX_MCP_OUTPUT_TOKENS` (environment or `--settings`). Codex has
`tool_output_token_limit` (config override `-c`). Pi stages keep Pi's built-in
tool-output truncation; Pipelines sets no Pi limit.
## Harness defaults
Configure two independent model defaults:
+7 -5
View File
@@ -131,11 +131,13 @@ new`, `edit` and `debug` sessions.
`/usr/local/bin`, `/usr/bin`, `~/.local/bin`. `./install` requires `opencode`
on `PATH` and records its absolute path.
**Launched as:** the server starts its own loopback `opencode serve` for chat
and for stage sessions, using your normal OpenCode configuration and login.
A server started with `-opencode-endpoint URL` (or
`PIPELINES_OPENCODE_ENDPOINT`) attaches chat to that service instead; the
native installer does not set this.
**Launched as:** chat uses the OpenCode service the server attached to (an
endpoint from `-opencode-endpoint URL` / `PIPELINES_OPENCODE_ENDPOINT`, or a
loopback `opencode serve` the server spawns). Stage sessions run on a separate
Pipelines-owned loopback `opencode serve`, the *stage service*, which uses your
normal OpenCode configuration, login and session database but adds the stage
tool-output and compaction limits. See "OpenCode limits for stages and chats"
in [harnesses-and-control.md](harnesses-and-control.md).
**Install and sign in:** `pipelines harness setup opencode` (runs
`opencode auth login` when no provider is connected). Pipelines reads the
+1
View File
@@ -49,6 +49,7 @@ func fakeNativeService(t *testing.T) {
t.Setenv("HOME", t.TempDir())
state := t.TempDir()
t.Setenv("XDG_STATE_HOME", state)
t.Setenv("PIPELINES_OPENCODE_STAGE_SERVICE", "shared") // fixture models the shared CLI-managed service
t.Setenv("XDG_CACHE_HOME", t.TempDir())
if err := os.Mkdir(filepath.Join(state, "opencode"), 0700); err != nil {
t.Fatal(err)
+30 -6
View File
@@ -48,6 +48,9 @@ type Connection struct {
// tell an authentication refusal (worth re-pairing) from any other refusal.
// It is set only by the request path and cleared by the attach probe.
unauthorized bool
// configContent is the OPENCODE_CONFIG_CONTENT an owned serve process is
// started with; empty keeps the inherited environment unchanged.
configContent string
}
// Reference retains the execution service, never provider auth. V1 retains its
@@ -110,6 +113,13 @@ func FromReference(path string) (*Connection, error) {
if !filepath.IsAbs(reference.CredentialFile) {
return nil, fmt.Errorf("invalid V2 service credential reference")
}
// A retained reference to the owned stage service revives it on its
// recorded URL and password if it exited; it never selects another one.
if isOwnedStageRegistration(reference.CredentialFile) {
if err := reviveOwnedStageService(reference.Directory); err != nil {
return nil, err
}
}
c, err := serviceConnection(reference.CredentialFile, reference.Directory)
if err != nil {
return nil, err
@@ -244,7 +254,25 @@ func OpenChat(ctx context.Context, executable, directory string) (*Connection, e
if err != nil {
return nil, err
}
return OpenStream(ctx, executable, directory, backend)
c, err := newConnection(ctx, executable, directory, backend)
if err != nil {
return nil, err
}
// Chat-only limits apply to this spawned V2 process only; the operator's
// shared service and any attached endpoint keep their own configuration.
overrides, err := chatConfigOverrides()
if err != nil {
return nil, err
}
if overrides != nil && backend == "v2" {
if c.configContent, err = configContent(os.Getenv("OPENCODE_CONFIG_CONTENT"), overrides); err != nil {
return nil, err
}
}
if err := c.startServe(ctx); err != nil {
return nil, err
}
return c, nil
}
// newConnection resolves the executable/directory defaults, detects the backend
@@ -304,11 +332,7 @@ func (c *Connection) startServe(ctx context.Context) error {
c.password = hex.EncodeToString(secret)
c.process = exec.Command(c.Executable, "serve", "--hostname", "127.0.0.1", "--port", "0")
c.process.Dir = c.Directory
for _, entry := range os.Environ() {
if !strings.HasPrefix(entry, "OPENCODE_SERVER_PASSWORD=") && !strings.HasPrefix(entry, "OPENCODE_SERVER_USERNAME=") {
c.process.Env = append(c.process.Env, entry)
}
}
c.process.Env = serviceEnvironment(c.configContent)
c.process.Env = append(c.process.Env, "OPENCODE_SERVER_PASSWORD="+c.password, "OPENCODE_SERVER_USERNAME="+c.serverUser())
stdout, err := c.process.StdoutPipe()
if err != nil {
+162
View File
@@ -0,0 +1,162 @@
package opencode
import (
"encoding/json"
"fmt"
"os"
"strconv"
"strings"
)
// Stage and chat OpenCode limits. OpenCode reads tool_output and compaction
// only from configuration (there is no per-session or per-agent knob), so
// stages get their own limits by running on a Pipelines-owned `opencode serve`
// started with OPENCODE_CONFIG_CONTENT. Chats keep the operator's OpenCode
// configuration unless PIPELINES_OPENCODE_CHAT_CONFIG is set, which applies only
// to a chat serve process the server spawns itself.
const (
// StageServiceEnvKey selects the stage service: "owned" (default) runs
// stages on the Pipelines-owned service with the stage limits; "shared"
// keeps the previous behaviour (the CLI-managed service, no stage limits).
StageServiceEnvKey = "PIPELINES_OPENCODE_STAGE_SERVICE"
// StageToolOutputMaxBytesEnvKey / StageToolOutputMaxLinesEnvKey set
// tool_output.max_bytes / max_lines. 0 leaves OpenCode's own default.
StageToolOutputMaxBytesEnvKey = "PIPELINES_OPENCODE_STAGE_TOOL_OUTPUT_MAX_BYTES"
StageToolOutputMaxLinesEnvKey = "PIPELINES_OPENCODE_STAGE_TOOL_OUTPUT_MAX_LINES"
// StageCompactionEnvKey is "on" (default), "off", or "opencode" to leave
// OpenCode's own compaction setting.
StageCompactionEnvKey = "PIPELINES_OPENCODE_STAGE_COMPACTION"
// StageConfigEnvKey is an optional JSON object merged last over the stage
// overrides (for example {"compaction":{"keep":{"tokens":20000}}}).
StageConfigEnvKey = "PIPELINES_OPENCODE_STAGE_CONFIG"
// ChatConfigEnvKey is an optional JSON object applied to the chat serve
// process the server spawns itself. It cannot change a shared or
// operator-supplied endpoint, which keeps the operator's configuration.
ChatConfigEnvKey = "PIPELINES_OPENCODE_CHAT_CONFIG"
DefaultStageToolOutputMaxBytes = 16384
DefaultStageToolOutputMaxLines = 400
)
// StageServiceShared reports whether the operator selected the shared
// CLI-managed service for stages.
func StageServiceShared() (bool, error) {
switch value := strings.ToLower(strings.TrimSpace(os.Getenv(StageServiceEnvKey))); value {
case "", "owned":
return false, nil
case "shared":
return true, nil
default:
return false, fmt.Errorf("%s must be owned or shared, not %q", StageServiceEnvKey, value)
}
}
// StageConfigOverrides returns the OpenCode configuration the stage service is
// started with. It is deterministic so its encoding identifies the service.
func StageConfigOverrides() (map[string]any, error) {
overrides := map[string]any{}
toolOutput := map[string]any{}
for _, limit := range []struct {
key, field string
fallback int
}{
{StageToolOutputMaxBytesEnvKey, "max_bytes", DefaultStageToolOutputMaxBytes},
{StageToolOutputMaxLinesEnvKey, "max_lines", DefaultStageToolOutputMaxLines},
} {
value := limit.fallback
if raw := strings.TrimSpace(os.Getenv(limit.key)); raw != "" {
parsed, err := strconv.Atoi(raw)
if err != nil || parsed < 0 {
return nil, fmt.Errorf("%s must be a non-negative integer, not %q", limit.key, raw)
}
value = parsed
}
if value > 0 {
toolOutput[limit.field] = value
}
}
if len(toolOutput) > 0 {
overrides["tool_output"] = toolOutput
}
switch value := strings.ToLower(strings.TrimSpace(os.Getenv(StageCompactionEnvKey))); value {
case "", "on":
overrides["compaction"] = map[string]any{"auto": true}
case "off":
overrides["compaction"] = map[string]any{"auto": false}
case "opencode":
default:
return nil, fmt.Errorf("%s must be on, off or opencode, not %q", StageCompactionEnvKey, value)
}
if raw := strings.TrimSpace(os.Getenv(StageConfigEnvKey)); raw != "" {
var extra map[string]any
if err := json.Unmarshal([]byte(raw), &extra); err != nil || extra == nil {
return nil, fmt.Errorf("%s must be a JSON object", StageConfigEnvKey)
}
mergeConfig(overrides, extra)
}
return overrides, nil
}
// chatConfigOverrides returns the optional chat-only configuration, or nil.
func chatConfigOverrides() (map[string]any, error) {
raw := strings.TrimSpace(os.Getenv(ChatConfigEnvKey))
if raw == "" {
return nil, nil
}
var overrides map[string]any
if err := json.Unmarshal([]byte(raw), &overrides); err != nil || overrides == nil {
return nil, fmt.Errorf("%s must be a JSON object", ChatConfigEnvKey)
}
return overrides, nil
}
// configContent merges overrides over an inherited OPENCODE_CONFIG_CONTENT (the
// operator's own, if any) so neither silently replaces the other. Overrides win.
func configContent(inherited string, overrides map[string]any) (string, error) {
merged := map[string]any{}
if strings.TrimSpace(inherited) != "" {
if err := json.Unmarshal([]byte(inherited), &merged); err != nil || merged == nil {
return "", fmt.Errorf("the inherited OPENCODE_CONFIG_CONTENT is not a JSON object")
}
}
mergeConfig(merged, overrides)
data, err := json.Marshal(merged)
if err != nil {
return "", err
}
return string(data), nil
}
// mergeConfig deep-merges src into dst: nested objects merge, other values replace.
func mergeConfig(dst, src map[string]any) {
for key, value := range src {
if nested, ok := value.(map[string]any); ok {
if existing, ok := dst[key].(map[string]any); ok {
mergeConfig(existing, nested)
continue
}
copied := map[string]any{}
mergeConfig(copied, nested)
dst[key] = copied
continue
}
dst[key] = value
}
}
// serviceEnvironment is the inherited environment without the variables the
// owned serve process must receive from Pipelines only.
func serviceEnvironment(content string) []string {
var env []string
for _, entry := range os.Environ() {
if strings.HasPrefix(entry, "OPENCODE_SERVER_PASSWORD=") || strings.HasPrefix(entry, "OPENCODE_SERVER_USERNAME=") ||
(content != "" && strings.HasPrefix(entry, "OPENCODE_CONFIG_CONTENT=")) {
continue
}
env = append(env, entry)
}
if content != "" {
env = append(env, "OPENCODE_CONFIG_CONTENT="+content)
}
return env
}
+411
View File
@@ -0,0 +1,411 @@
package opencode
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"log"
"net"
"net/url"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"sync"
"syscall"
"time"
)
// The owned stage service is one Pipelines-owned V2 `opencode serve` process,
// separate from the CLI-managed service chats use, started with the stage
// limits in OPENCODE_CONFIG_CONTENT. It runs as the same user with the same
// XDG directories, so it reads the operator's configuration and provider
// credentials (stored in OpenCode's SQLite database) and writes sessions to
// the same database; OpenCode opens it in WAL mode with a busy timeout, so two
// processes share it safely.
//
// It is detached (its own session) and its loopback URL and password are
// recorded in a private registration file that is kept across restarts, so a
// retained stage reference stays valid: restart recovery, interruption and
// session export reach the same URL. A dead service is revived on the same port
// and password before use. The server stops it on shutdown only when no
// session is active on it.
// ownedStageStartTimeout bounds one start attempt; attempts retry a few times
// because a concurrent OpenCode migration can briefly lock the database.
const ownedStageStartTimeout = 20 * time.Second
const ownedStageStartAttempts = 3
// ownedStageProbeTimeout bounds the health check of an existing service.
const ownedStageProbeTimeout = 3 * time.Second
var ownedStageMu sync.Mutex
type ownedStageState struct {
URL string `json:"url"`
Password string `json:"password"`
PID int `json:"pid"`
// Identity hashes the executable and the configuration content, so a
// changed limit or upgraded OpenCode replaces an idle service.
Identity string `json:"identity"`
}
// StageServiceRegistrationPath is the private registration of the owned stage
// service, beside the private connection references.
func StageServiceRegistrationPath() (string, error) {
cache, err := os.UserCacheDir()
if err != nil {
return "", err
}
return filepath.Join(cache, "pipelines", "private", "opencode-stage-service.json"), nil
}
func isOwnedStageRegistration(path string) bool {
owned, err := StageServiceRegistrationPath()
return err == nil && filepath.Clean(path) == filepath.Clean(owned)
}
// ensureOwnedStageService returns a connection to the healthy owned stage
// service, starting or reviving it when needed. It fails with a clear error and
// never falls back to another service: the stage limits are a guarantee.
func ensureOwnedStageService(ctx context.Context, executable, directory string) (*Connection, error) {
overrides, err := StageConfigOverrides()
if err != nil {
return nil, err
}
content, err := configContent(os.Getenv("OPENCODE_CONFIG_CONTENT"), overrides)
if err != nil {
return nil, err
}
path, err := StageServiceRegistrationPath()
if err != nil {
return nil, err
}
if err := os.MkdirAll(filepath.Dir(path), 0700); err != nil {
return nil, err
}
ownedStageMu.Lock()
defer ownedStageMu.Unlock()
unlock, err := lockFile(path + ".lock")
if err != nil {
return nil, fmt.Errorf("OpenCode stage service lock: %w", err)
}
defer unlock()
identity := ownedStageIdentity(executable, content)
state, readErr := readOwnedStageState(path)
port := 0
if readErr == nil {
current := &Connection{Backend: "v2", URL: state.URL, password: state.Password}
pid, healthy := current.ownedInfo(ctx)
switch {
case healthy && state.Identity == identity:
return ownedStageConnection(path, executable, directory)
case healthy && current.ownedIdle(ctx):
log.Printf("OpenCode stage service configuration changed; restarting the idle service")
stopOwnedPID(pid)
case healthy:
// Never interrupt active stage work to apply a new limit.
log.Printf("OpenCode stage service configuration changed; keeping the busy service until it is idle")
return ownedStageConnection(path, executable, directory)
}
if u, err := url.Parse(state.URL); err == nil {
port, _ = strconv.Atoi(u.Port())
}
} else {
state = ownedStageState{}
}
if state.Password == "" {
secret := make([]byte, 32)
if _, err := rand.Read(secret); err != nil {
return nil, err
}
state.Password = hex.EncodeToString(secret)
}
var lastErr error
for attempt := 0; attempt < ownedStageStartAttempts; attempt++ {
if attempt > 0 {
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(time.Duration(attempt) * time.Second):
}
}
started, err := startOwnedStage(ctx, executable, port, state.Password, content)
if err == nil {
started.Identity = identity
if err := writeOwnedStageState(path, started); err != nil {
stopOwnedPID(started.PID)
return nil, err
}
return ownedStageConnection(path, executable, directory)
}
lastErr = err
if ctx.Err() != nil {
return nil, ctx.Err()
}
// A recorded port that another process now holds cannot be reused.
if port != 0 && !portFree(port) {
port = 0
}
}
return nil, lastErr
}
func ownedStageConnection(path, executable, directory string) (*Connection, error) {
c, err := serviceConnection(path, directory)
if err != nil {
return nil, err
}
c.Executable = executable
return c, nil
}
// reviveOwnedStageService makes a retained reference to the owned service
// usable again after the service exited, on the recorded URL and password.
// A healthy service is used as is, so recovery never depends on the current
// limit settings being valid.
func reviveOwnedStageService(directory string) error {
ctx, cancel := context.WithTimeout(context.Background(), ownedStageStartTimeout*ownedStageStartAttempts)
defer cancel()
if path, err := StageServiceRegistrationPath(); err == nil {
if state, err := readOwnedStageState(path); err == nil {
if _, healthy := (&Connection{Backend: "v2", URL: state.URL, password: state.Password}).ownedInfo(ctx); healthy {
return nil
}
}
}
_, err := ensureOwnedStageService(ctx, Executable(), directory)
return err
}
func ownedStageIdentity(executable, content string) string {
resolved := executable
if path, err := exec.LookPath(executable); err == nil {
resolved = path
}
if path, err := filepath.EvalSymlinks(resolved); err == nil {
resolved = path
}
sum := sha256.Sum256([]byte(resolved + "\x00" + content))
return hex.EncodeToString(sum[:])
}
// startOwnedStage starts one detached serve process and waits until it answers
// the authenticated V2 info contract.
func startOwnedStage(ctx context.Context, executable string, port int, password, content string) (ownedStageState, error) {
if port == 0 {
free, err := freePort()
if err != nil {
return ownedStageState{}, err
}
port = free
}
base := "http://127.0.0.1:" + strconv.Itoa(port)
cmd := exec.Command(executable, "serve", "--hostname", "127.0.0.1", "--port", strconv.Itoa(port))
if home, err := os.UserHomeDir(); err == nil {
cmd.Dir = home
}
cmd.Env = append(serviceEnvironment(content), "OPENCODE_SERVER_PASSWORD="+password, "OPENCODE_SERVER_USERNAME=opencode")
// Its own session: a stopping Pipelines service (and its process group)
// does not take active stage work with it.
cmd.SysProcAttr = &syscall.SysProcAttr{Setsid: true}
logPath, _ := StageServiceRegistrationPath()
logPath = strings.TrimSuffix(logPath, ".json") + ".log"
logFile, err := os.OpenFile(logPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0600)
if err != nil {
return ownedStageState{}, err
}
cmd.Stdout, cmd.Stderr = logFile, logFile
cmd.Stdin = nil
if err := cmd.Start(); err != nil {
logFile.Close()
return ownedStageState{}, fmt.Errorf("OpenCode stage service could not start: %w", err)
}
logFile.Close()
exited := make(chan struct{})
go func() { _ = cmd.Wait(); close(exited) }()
conn := &Connection{Backend: "v2", URL: base, password: password}
startup, cancel := context.WithTimeout(ctx, ownedStageStartTimeout)
defer cancel()
for {
if pid, healthy := conn.ownedInfo(startup); healthy {
if pid == 0 {
pid = cmd.Process.Pid
}
return ownedStageState{URL: base, Password: password, PID: pid}, nil
}
select {
case <-exited:
return ownedStageState{}, fmt.Errorf("OpenCode stage service exited before becoming ready (see %s)", logPath)
case <-startup.Done():
_ = cmd.Process.Kill()
if ctx.Err() != nil {
return ownedStageState{}, ctx.Err()
}
return ownedStageState{}, fmt.Errorf("OpenCode stage service did not become ready within %s (see %s)", ownedStageStartTimeout, logPath)
case <-time.After(150 * time.Millisecond):
}
}
}
// ownedInfo authenticates against the service and returns its reported pid.
// The password proves the process is the one Pipelines started.
func (c *Connection) ownedInfo(ctx context.Context) (int, bool) {
probe, cancel := context.WithTimeout(ctx, ownedStageProbeTimeout)
defer cancel()
data, err := c.nativeRequest(probe, "get", "/api/info", nil, "")
if err != nil {
return 0, false
}
var info struct {
Version string `json:"version"`
PID int `json:"pid"`
}
if json.Unmarshal(data, &info) != nil || info.Version == "" {
return 0, false
}
return info.PID, true
}
// ownedIdle reports whether no session is active on this process; any error
// counts as busy so active work is never stopped on a guess.
func (c *Connection) ownedIdle(ctx context.Context) bool {
probe, cancel := context.WithTimeout(ctx, ownedStageProbeTimeout)
defer cancel()
data, err := c.nativeRequest(probe, "get", "/api/session/active", nil, "")
if err != nil {
return false
}
var active struct {
Data map[string]json.RawMessage `json:"data"`
}
return json.Unmarshal(data, &active) == nil && active.Data != nil && len(active.Data) == 0
}
// StopIdleStageService stops the owned stage service on server shutdown when no
// session is active on it. The registration (URL and password) is kept, so a
// later start reuses the same endpoint and retained references stay valid. A
// busy service keeps running so restart recovery can interrupt it properly.
func StopIdleStageService() {
path, err := StageServiceRegistrationPath()
if err != nil {
return
}
ownedStageMu.Lock()
defer ownedStageMu.Unlock()
state, err := readOwnedStageState(path)
if err != nil {
return
}
ctx, cancel := context.WithTimeout(context.Background(), 2*ownedStageProbeTimeout)
defer cancel()
conn := &Connection{Backend: "v2", URL: state.URL, password: state.Password}
pid, healthy := conn.ownedInfo(ctx)
if !healthy {
return
}
if !conn.ownedIdle(ctx) {
log.Printf("OpenCode stage service has active sessions; leaving it running")
return
}
stopOwnedPID(pid)
}
// stopOwnedPID terminates a process identified through the authenticated info
// contract, escalating after a bounded grace.
func stopOwnedPID(pid int) {
if pid <= 0 {
return
}
process, err := os.FindProcess(pid)
if err != nil {
return
}
if process.Signal(syscall.SIGTERM) != nil {
return
}
deadline := time.Now().Add(5 * time.Second)
for time.Now().Before(deadline) {
if process.Signal(syscall.Signal(0)) != nil {
return
}
time.Sleep(100 * time.Millisecond)
}
_ = process.Kill()
}
func readOwnedStageState(path string) (ownedStageState, error) {
if err := checkPrivateFile(path, serviceCredentialLimit); err != nil {
return ownedStageState{}, err
}
data, err := os.ReadFile(path)
if err != nil {
return ownedStageState{}, err
}
var state ownedStageState
if err := json.Unmarshal(data, &state); err != nil || state.Password == "" {
return ownedStageState{}, fmt.Errorf("invalid OpenCode stage service registration")
}
if _, err := NormalizeEndpointURL(state.URL); err != nil {
return ownedStageState{}, fmt.Errorf("invalid OpenCode stage service registration")
}
return state, nil
}
func writeOwnedStageState(path string, state ownedStageState) error {
data, err := json.Marshal(state)
if err != nil {
return err
}
f, err := os.CreateTemp(filepath.Dir(path), ".opencode-stage-service-*.json")
if err != nil {
return err
}
_, writeErr := f.Write(data)
closeErr := f.Close()
if writeErr == nil {
writeErr = closeErr
}
if writeErr == nil {
writeErr = os.Rename(f.Name(), path)
}
if writeErr != nil {
_ = os.Remove(f.Name())
}
return writeErr
}
func lockFile(path string) (func(), error) {
f, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0600)
if err != nil {
return nil, err
}
if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX); err != nil {
f.Close()
return nil, err
}
return func() { _ = syscall.Flock(int(f.Fd()), syscall.LOCK_UN); f.Close() }, nil
}
func freePort() (int, error) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
return 0, err
}
defer listener.Close()
return listener.Addr().(*net.TCPAddr).Port, nil
}
func portFree(port int) bool {
listener, err := net.Listen("tcp", "127.0.0.1:"+strconv.Itoa(port))
if err != nil {
return false
}
listener.Close()
return true
}
+277
View File
@@ -0,0 +1,277 @@
package opencode
import (
"context"
"encoding/json"
"net/http"
"os"
"path/filepath"
"strings"
"syscall"
"testing"
"time"
)
// fakeOpenCodeEnv makes the test binary act as `opencode serve`: TestMain sees
// it and serves the minimal V2 contract the owned stage service uses.
const fakeOpenCodeEnv = "PIPELINES_TEST_FAKE_OPENCODE"
func TestMain(m *testing.M) {
if os.Getenv(fakeOpenCodeEnv) == "1" {
runFakeOpenCodeServe()
return
}
os.Exit(m.Run())
}
// runFakeOpenCodeServe serves /api/info (with pid), /api/session/active (busy
// while $FAKE_BUSY_FILE exists) and /api/config (the received
// OPENCODE_CONFIG_CONTENT) on the --port argument, behind basic auth.
func runFakeOpenCodeServe() {
port := ""
for i, arg := range os.Args {
if arg == "--port" && i+1 < len(os.Args) {
port = os.Args[i+1]
}
}
if len(os.Args) < 2 || os.Args[1] != "serve" || port == "" {
os.Exit(3)
}
if os.Getenv("FAKE_FAIL") == "1" {
os.Exit(4)
}
password := os.Getenv("OPENCODE_SERVER_PASSWORD")
mux := http.NewServeMux()
auth := func(next func(w http.ResponseWriter)) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
user, got, ok := r.BasicAuth()
if !ok || user != "opencode" || got != password {
w.WriteHeader(http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
next(w)
}
}
mux.HandleFunc("/api/info", auth(func(w http.ResponseWriter) {
_ = json.NewEncoder(w).Encode(map[string]any{"version": "fake", "pid": os.Getpid()})
}))
mux.HandleFunc("/api/session/active", auth(func(w http.ResponseWriter) {
if _, err := os.Stat(os.Getenv("FAKE_BUSY_FILE")); err == nil {
_, _ = w.Write([]byte(`{"data":{"ses_busy":{"type":"running"}}}`))
return
}
_, _ = w.Write([]byte(`{"data":{}}`))
}))
mux.HandleFunc("/api/config", auth(func(w http.ResponseWriter) {
_ = json.NewEncoder(w).Encode(map[string]string{"content": os.Getenv("OPENCODE_CONFIG_CONTENT")})
}))
_ = http.ListenAndServe("127.0.0.1:"+port, mux)
}
func fakeOwnedStageEnv(t *testing.T) string {
t.Helper()
t.Setenv("HOME", t.TempDir()) // os.UserCacheDir uses HOME on macOS
t.Setenv("XDG_CACHE_HOME", t.TempDir())
t.Setenv(OpenCodeServiceStateEnvKey, t.TempDir())
t.Setenv(fakeOpenCodeEnv, "1")
t.Setenv("FAKE_BUSY_FILE", filepath.Join(t.TempDir(), "busy"))
for _, key := range []string{StageServiceEnvKey, StageToolOutputMaxBytesEnvKey, StageToolOutputMaxLinesEnvKey, StageCompactionEnvKey, StageConfigEnvKey, "OPENCODE_CONFIG_CONTENT", "FAKE_FAIL"} {
t.Setenv(key, "")
}
executable, err := os.Executable()
if err != nil {
t.Fatal(err)
}
t.Cleanup(StopIdleStageService)
t.Cleanup(func() {
path, _ := StageServiceRegistrationPath()
if state, err := readOwnedStageState(path); err == nil {
_ = syscall.Kill(state.PID, syscall.SIGKILL)
}
})
return executable
}
func fakeConfigContent(t *testing.T, c *Connection) map[string]any {
t.Helper()
data, err := c.nativeRequest(context.Background(), "get", "/api/config", nil, "")
if err != nil {
t.Fatal(err)
}
var response struct{ Content string }
if err := json.Unmarshal(data, &response); err != nil {
t.Fatal(err)
}
var content map[string]any
if err := json.Unmarshal([]byte(response.Content), &content); err != nil {
t.Fatalf("stage service got no JSON config content: %q", response.Content)
}
return content
}
func ownedPID(t *testing.T, c *Connection) int {
t.Helper()
pid, ok := c.ownedInfo(context.Background())
if !ok {
t.Fatal("owned stage service is not healthy")
}
return pid
}
func TestStageConfigOverridesDefaultsAndSettings(t *testing.T) {
for _, key := range []string{StageToolOutputMaxBytesEnvKey, StageToolOutputMaxLinesEnvKey, StageCompactionEnvKey, StageConfigEnvKey} {
t.Setenv(key, "")
}
got, err := StageConfigOverrides()
if err != nil {
t.Fatal(err)
}
data, _ := json.Marshal(got)
if string(data) != `{"compaction":{"auto":true},"tool_output":{"max_bytes":16384,"max_lines":400}}` {
t.Fatalf("defaults = %s", data)
}
t.Setenv(StageToolOutputMaxBytesEnvKey, "0")
t.Setenv(StageToolOutputMaxLinesEnvKey, "100")
t.Setenv(StageCompactionEnvKey, "opencode")
t.Setenv(StageConfigEnvKey, `{"compaction":{"keep":{"tokens":20000}},"tool_output":{"max_lines":50}}`)
got, err = StageConfigOverrides()
if err != nil {
t.Fatal(err)
}
data, _ = json.Marshal(got)
if string(data) != `{"compaction":{"keep":{"tokens":20000}},"tool_output":{"max_lines":50}}` {
t.Fatalf("configured = %s", data)
}
for key, value := range map[string]string{StageToolOutputMaxBytesEnvKey: "lots", StageCompactionEnvKey: "maybe", StageConfigEnvKey: "[1]"} {
t.Run(key, func(t *testing.T) {
t.Setenv(key, value)
if _, err := StageConfigOverrides(); err == nil || !strings.Contains(err.Error(), key) {
t.Fatalf("invalid %s accepted: %v", key, err)
}
})
}
t.Setenv(StageServiceEnvKey, "elsewhere")
if _, err := StageServiceShared(); err == nil {
t.Fatal("invalid stage service selection accepted")
}
}
func TestConfigContentKeepsInheritedConfiguration(t *testing.T) {
content, err := configContent(`{"model":"a/b","tool_output":{"max_lines":9,"max_bytes":1}}`, map[string]any{"tool_output": map[string]any{"max_bytes": 2}})
if err != nil {
t.Fatal(err)
}
if content != `{"model":"a/b","tool_output":{"max_bytes":2,"max_lines":9}}` {
t.Fatalf("merged = %s", content)
}
if _, err := configContent(`not json`, nil); err == nil {
t.Fatal("invalid inherited content accepted")
}
}
func TestOwnedStageServiceStartsReusesRevivesAndReplacesWhenIdle(t *testing.T) {
executable := fakeOwnedStageEnv(t)
directory := t.TempDir()
first, err := OpenStage(context.Background(), executable, directory, "v2")
if err != nil {
t.Fatal(err)
}
path, _ := StageServiceRegistrationPath()
if first.credentialFile != path || first.Directory != directory || !strings.HasPrefix(first.URL, "http://127.0.0.1:") {
t.Fatalf("stage connection = %+v", first)
}
if info, err := os.Stat(path); err != nil || info.Mode().Perm() != 0600 {
t.Fatalf("registration not private: %v", err)
}
if content := fakeConfigContent(t, first); content["tool_output"].(map[string]any)["max_bytes"] != float64(16384) || content["compaction"].(map[string]any)["auto"] != true {
t.Fatalf("stage limits not applied: %v", content)
}
pid := ownedPID(t, first)
// Reused, not restarted.
second, err := OpenStage(context.Background(), executable, t.TempDir(), "v2")
if err != nil || second.URL != first.URL || ownedPID(t, second) != pid {
t.Fatalf("healthy service was not reused: %v", err)
}
// A retained reference survives a crash: it revives on the same URL.
reference, err := first.Reference()
if err != nil {
t.Fatal(err)
}
_ = syscall.Kill(pid, syscall.SIGKILL)
waitGone(t, pid)
revived, err := FromReference(reference)
if err != nil {
t.Fatal(err)
}
if revived.URL != first.URL || revived.Directory != directory {
t.Fatalf("revived on another endpoint: %s", revived.URL)
}
pid = ownedPID(t, revived)
// A changed limit replaces an idle service on the same URL and password.
t.Setenv(StageToolOutputMaxBytesEnvKey, "1000")
replaced, err := OpenStage(context.Background(), executable, directory, "v2")
if err != nil {
t.Fatal(err)
}
if replaced.URL != first.URL || ownedPID(t, replaced) == pid {
t.Fatal("idle service was not replaced on its endpoint")
}
if fakeConfigContent(t, replaced)["tool_output"].(map[string]any)["max_bytes"] != float64(1000) {
t.Fatal("replacement did not receive the new limit")
}
if _, err := FromReference(reference); err != nil {
t.Fatalf("retained reference broke after replacement: %v", err)
}
// A busy service is never stopped to apply a change, nor at shutdown.
pid = ownedPID(t, replaced)
if err := os.WriteFile(os.Getenv("FAKE_BUSY_FILE"), nil, 0600); err != nil {
t.Fatal(err)
}
t.Setenv(StageToolOutputMaxBytesEnvKey, "2000")
busy, err := OpenStage(context.Background(), executable, directory, "v2")
if err != nil || ownedPID(t, busy) != pid {
t.Fatalf("busy service was replaced: %v", err)
}
StopIdleStageService()
if ownedPID(t, busy) != pid {
t.Fatal("busy service was stopped at shutdown")
}
_ = os.Remove(os.Getenv("FAKE_BUSY_FILE"))
StopIdleStageService()
waitGone(t, pid)
}
func TestOwnedStageServiceStartFailureIsClearAndBounded(t *testing.T) {
executable := fakeOwnedStageEnv(t)
t.Setenv("FAKE_FAIL", "1")
started := time.Now()
_, err := OpenStage(context.Background(), executable, t.TempDir(), "v2")
if err == nil || !strings.Contains(err.Error(), "OpenCode stage service exited before becoming ready") {
t.Fatalf("start failure = %v", err)
}
if time.Since(started) > 15*time.Second {
t.Fatal("start failure was not bounded")
}
t.Setenv("FAKE_FAIL", "")
t.Setenv(StageCompactionEnvKey, "sometimes")
if _, err := OpenStage(context.Background(), executable, t.TempDir(), "v2"); err == nil || !strings.Contains(err.Error(), StageCompactionEnvKey) {
t.Fatalf("invalid setting = %v", err)
}
}
func waitGone(t *testing.T, pid int) {
t.Helper()
deadline := time.Now().Add(10 * time.Second)
for time.Now().Before(deadline) {
if syscall.Kill(pid, 0) != nil {
return
}
time.Sleep(50 * time.Millisecond)
}
t.Fatalf("process %d still running", pid)
}
+13 -4
View File
@@ -6,9 +6,11 @@ import (
"os"
)
// OpenStage is the admission-only connection path. Unlike retained-session
// reads and interruption, a new stage may discover/start the CLI-managed V2
// service before creating any session. Chat endpoint overrides are not involved.
// OpenStage is the admission-only connection path. V2 stages run on the
// Pipelines-owned stage service (stage limits, see stage_owned.go) unless the
// operator selected the shared CLI-managed service, which a new stage may then
// discover/start before creating any session. Chat endpoint overrides are not
// involved either way.
func OpenStage(ctx context.Context, executable, directory, backend string) (*Connection, error) {
c, err := newConnection(ctx, executable, directory, backend)
if err != nil {
@@ -17,7 +19,14 @@ func OpenStage(ctx context.Context, executable, directory, backend string) (*Con
if c.Backend != "v2" {
return Open(ctx, c.Executable, c.Directory, c.Backend)
}
return ensureStageService(ctx, c.Executable, c.Directory)
shared, err := StageServiceShared()
if err != nil {
return nil, err
}
if shared {
return ensureStageService(ctx, c.Executable, c.Directory)
}
return ensureOwnedStageService(ctx, c.Executable, c.Directory)
}
// ensureStageService delegates cold-start discovery to the selected CLI's
+1
View File
@@ -15,6 +15,7 @@ func coldServiceCLI(t *testing.T, script string) (string, string) {
t.Setenv("HOME", t.TempDir())
root := t.TempDir()
t.Setenv(OpenCodeServiceStateEnvKey, root)
t.Setenv(StageServiceEnvKey, "shared") // fixture models the shared CLI-managed service
t.Setenv("XDG_CACHE_HOME", t.TempDir())
marker := filepath.Join(root, "discovery-called")
t.Setenv("TEST_DISCOVERY_MARKER", marker)
+1
View File
@@ -67,6 +67,7 @@ func localExportService(t *testing.T, endpoint string) {
t.Setenv("XDG_CACHE_HOME", t.TempDir())
state := t.TempDir()
t.Setenv(opencode.OpenCodeServiceStateEnvKey, state)
t.Setenv(opencode.StageServiceEnvKey, "shared") // these fixtures model the shared CLI-managed service
if err := os.Mkdir(filepath.Join(state, "opencode"), 0700); err != nil {
t.Fatal(err)
}
+3
View File
@@ -18,6 +18,7 @@ func fakeExecutionService(t *testing.T) {
t.Setenv("HOME", t.TempDir()) // os.UserCacheDir uses HOME on macOS, not XDG_CACHE_HOME.
state := t.TempDir()
t.Setenv(opencode.OpenCodeServiceStateEnvKey, state)
t.Setenv(opencode.StageServiceEnvKey, "shared") // fixture models the shared CLI-managed service
t.Setenv("XDG_CACHE_HOME", t.TempDir())
if err := os.Mkdir(filepath.Join(state, "opencode"), 0700); err != nil {
t.Fatal(err)
@@ -33,6 +34,7 @@ func TestV2StagePersistsExecutionReferenceBeforePrompt(t *testing.T) {
t.Setenv("HOME", t.TempDir())
state := t.TempDir()
t.Setenv(opencode.OpenCodeServiceStateEnvKey, state)
t.Setenv(opencode.StageServiceEnvKey, "shared")
t.Setenv("XDG_CACHE_HOME", t.TempDir())
t.Setenv(opencode.EndpointEnvKey, "http://127.0.0.1:2")
t.Setenv("OPENCODE_PASSWORD", "foreign")
@@ -98,6 +100,7 @@ func TestV2ColdDiscoveryWithoutRegistrationNeverCreatesSession(t *testing.T) {
t.Setenv("HOME", t.TempDir())
root := t.TempDir()
t.Setenv(opencode.OpenCodeServiceStateEnvKey, root)
t.Setenv(opencode.StageServiceEnvKey, "shared")
t.Setenv("XDG_CACHE_HOME", t.TempDir())
t.Setenv(opencode.EndpointEnvKey, "http://127.0.0.1:1")
marker := filepath.Join(root, "cli-calls")