diff --git a/README.md b/README.md index 488e8750..057e4f55 100644 --- a/README.md +++ b/README.md @@ -123,6 +123,7 @@ pipelines harness list # status of all four **Also send to ntfy** for devices that still use ntfy. Pi runs pipeline stages (`harness=pi`) but is not offered as a chat harness. +Claude Code runs chats and pipeline stages (`harness=claude`). See [Harnesses](docs/harnesses.md). ## Architecture diff --git a/docs/harnesses-and-control.md b/docs/harnesses-and-control.md index 3d9aa22e..34281c71 100644 --- a/docs/harnesses-and-control.md +++ b/docs/harnesses-and-control.md @@ -100,8 +100,8 @@ How the stage service behaves: - `pipelines debug` is unchanged: it opens a fresh interactive harness conversation, not the stage session. -Claude Code and Codex are chat-only harnesses (stages accept only -`harness=opencode` or `harness=pi`), so they need no stage limit. Their chats +Codex is a chat-only harness and Claude Code stages (`harness=claude`) run the +CLI's own print mode, so neither has a stage limit here. Their chats keep their own defaults. Claude Code has `BASH_MAX_OUTPUT_LENGTH` and `MAX_MCP_OUTPUT_TOKENS` (environment or `--settings`). Codex has `tool_output_token_limit` (config override `-c`). Pi stages keep Pi's built-in @@ -176,7 +176,8 @@ instead of being silently redirected or downgraded to plaintext. `provider-check on a Docker-linked profile runs in-container and preserves the caller's output mode (`--json` only when requested) with exact stdout/exit propagation. -Put `harness=pi` in a model stage's configs to select Pi. nvim edits definitions +Put `harness=pi` in a model stage's configs to select Pi, or `harness=claude` +with `provider=anthropic` for Claude Code ([harnesses](harnesses.md#claude-code-stages)). nvim edits definitions or displays debug evidence; it does not execute model stages. For bounded review loops, add a fenced `limits` JSON object to the reviewer: diff --git a/docs/harnesses.md b/docs/harnesses.md index 7983f38e..db3690a5 100644 --- a/docs/harnesses.md +++ b/docs/harnesses.md @@ -8,10 +8,10 @@ harness's own login, which `pipelines harness setup ` opens for you. | --- | --- | --- | --- | --- | | OpenCode | yes (default) | yes (default) | queued until the turn ends | yes | | Pi | yes (`harness=pi`) | no | n/a | yes | -| Claude Code | no | yes | yes, delivered into the running turn | no | +| Claude Code | yes (`harness=claude`) | yes | yes, delivered into the running turn | no | | Codex | no | yes | yes, via `turn/steer` | no | -Stage configs accept only `harness=opencode` or `harness=pi`. +Stage configs accept `harness=opencode`, `harness=pi` or `harness=claude`. ## Set up harnesses @@ -77,6 +77,19 @@ reasoning_effort=high [harnesses and control](harnesses-and-control.md). Use the exact IDs your harness lists (`opencode models`, `pi --list-models`). +A stage on Claude Code uses the server user's Claude sign-in and a Claude model +ID, for example a cheaper model for a small change: + +````markdown +```configs +harness=claude +provider=anthropic +model=claude-haiku-5-5 +``` +```` + +See [Claude Code stages](#claude-code-stages) for what such a stage can do. + **Per run**, without changing the definition: ```sh @@ -114,6 +127,7 @@ its own catalog. pipelines doctor # health + OpenCode API discovery, no inference pipelines provider-check --model P/M # one tiny tool-free generation (may cost) pipelines provider-check --harness pi --model P/M +pipelines provider-check --harness claude --model anthropic/claude-haiku-5-5 python3 scripts/check-pi-rpc.py # Pi RPC and policy, no inference ``` @@ -173,7 +187,7 @@ return to model selection. ## Claude Code -**Used for:** chat only. +**Used for:** chat, and pipeline stages with `harness=claude` (see [Claude Code stages](#claude-code-stages)). **Found by:** `PIPELINES_CLAUDE`, then `PATH`, then `~/.local/bin/claude`, `/opt/homebrew/bin/claude`, `/usr/local/bin/claude`. The server checks again @@ -211,9 +225,37 @@ for the next turn. **Mid-turn messages:** yes. The message is written to the running CLI's stdin and read at its next step. Model changes apply from the next turn. +### Claude Code stages + +A model stage with `harness=claude`, `provider=anthropic` and +`model=` runs one `claude -p` process per visit in the run +directory, signed in as the server user (the same sign-in the chats use; a run +directory that chooses its own account keeps it). + +- **Tools.** Only the tools in the stage's `tools` block exist for the session + (`read`, `glob`, `grep`, `edit`, `shell`, `webfetch`, `websearch`). Nothing + else is approved: the CLI runs with `--permission-mode dontAsk`, and file + edits are approved only below the run directory and the stage's artifacts + directory. An undeclared tool or an edit elsewhere is refused by the CLI. +- **Not supported; the stage or run is refused, never widened:** a + `permissions` block, a run launched with `--write-path`, `delivery=true` + (landing stays on OpenCode), and `reasoning_effort` `none` or `minimal`. +- **Isolation.** User, project and local Claude Code settings, hooks, MCP + servers, plugins and slash commands are not loaded. A `CLAUDE.md` in the run + directory is still read by the CLI. +- **Usage.** Cost and tokens come from the CLI's final result and count toward + stage and run budgets; a result without usage fails a budgeted stage. +- **Failures.** Diagnostics are fixed text. A billing, quota or rate-limit + refusal stops the run without retries; a missing sign-in says so. The CLI's + redacted stream is kept as the session export and evidence. +- **Recovery.** A server restart during a Claude Code stage stops that process + and fails the visit; it is not reattached. +- **Overrides.** `--model` and `--stage-model` must give an + `anthropic/` for these stages; the launch is refused otherwise. + ## Codex -**Used for:** chat only. +**Used for:** chat, and pipeline stages with `harness=claude` (see [Claude Code stages](#claude-code-stages)). **Found by:** the Codex App Server control socket at `$CODEX_HOME/app-server-control/app-server-control.sock` (default diff --git a/internal/author/skill/references/format.md b/internal/author/skill/references/format.md index e479fa1e..4a133401 100644 --- a/internal/author/skill/references/format.md +++ b/internal/author/skill/references/format.md @@ -101,7 +101,9 @@ evidence route to `blocked` when declared, else `error`; never fabricate a resul - `provider`/`model`: exact IDs validated during server setup. Templates (`pipelines new NAME --template KEY`) fill them in; copy those IDs into a hand-written draft. Availability needs `pipelines provider-check`. -- `harness=opencode|pi`; omission means OpenCode (`references/harnesses.md`). +- `harness=opencode|pi|claude`; omission means OpenCode (`references/harnesses.md`). + `claude` needs `provider=anthropic` and a Claude model ID, and accepts no + `permissions` block, `delivery=true` or run write paths. - `retries` 0–100, default 2 (three attempts). Retries cover transport failures and timeouts; a stage with `edit`/`shell` (or no tools block) never replays admitted work. diff --git a/internal/author/skill/references/harnesses.md b/internal/author/skill/references/harnesses.md index f679e232..55713f11 100644 --- a/internal/author/skill/references/harnesses.md +++ b/internal/author/skill/references/harnesses.md @@ -4,7 +4,8 @@ Authoring needs only the `harness=` config (`references/format.md`). The rest is for an operator setting up or debugging the execution computer. OpenCode and Pi run interactive authoring, debugging and model stages; Claude Code -and Codex are chat backends. nvim is a file editor/read-only debug packet viewer, +is a chat backend that also runs model stages (`harness=claude`, +`provider=anthropic`, declared tools only); Codex is a chat backend. nvim is a file editor/read-only debug packet viewer, not a stage backend. Only configured harnesses appear in pickers, default first. CLI `new/edit/debug --harness opencode|pi|nvim` overrides the default, and `pipelines harness` launches the configured default in the invocation directory. diff --git a/internal/claude/auth_env.go b/internal/claude/auth_env.go index af845294..fa519387 100644 --- a/internal/claude/auth_env.go +++ b/internal/claude/auth_env.go @@ -165,3 +165,14 @@ func hasEnv(env []string, key string) bool { } return false } + +// stageAuth resolves the sign-in for stage processes once, like a Chat's. +var stageAuth authEnv + +// StageEnv is the environment for a non-interactive CLI the server starts for a +// pipeline stage: base (the run directory's environment) plus the same sign-in +// fallbacks chats use (the private service token, then the login file) when +// base itself chooses no account. +func StageEnv(ctx context.Context, binary string, base []string) []string { + return stageAuth.resolve(ctx, binary, withServiceToken(append([]string{}, base...))) +} diff --git a/internal/cli/app.go b/internal/cli/app.go index f85516bc..b7ac7f56 100644 --- a/internal/cli/app.go +++ b/internal/cli/app.go @@ -402,8 +402,14 @@ func (a App) Run(ctx context.Context, args []string) error { return err } generator = &stage.PiClient{Executable: profile.Executable, Directory: cwd, Tools: []string{}, OnOwnership: func(id, reference string, active bool) error { session = id; return nil }} + case stage.ClaudeBackend: + cwd, err := os.Getwd() + if err != nil { + return err + } + generator = &stage.ClaudeStageClient{Directory: cwd, Tools: []string{}, OnOwnership: func(id, reference string, active bool) error { session = id; return nil }} default: - return fmt.Errorf("provider-check harness must be opencode or pi") + return fmt.Errorf("provider-check harness must be opencode, pi or claude") } text, err := generator.Generate(check, stage.ModelConfig{Harness: o.harness, Provider: provider, Model: model}, "Provider connection validation only. Do not use tools. Reply with exactly PIPELINES_PROVIDER_OK.") if err == nil && strings.TrimSpace(text) != "PIPELINES_PROVIDER_OK" { diff --git a/internal/server/claude_stage_run_test.go b/internal/server/claude_stage_run_test.go new file mode 100644 index 00000000..5581059a --- /dev/null +++ b/internal/server/claude_stage_run_test.go @@ -0,0 +1,100 @@ +package server + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + + "git.a-wan.net/alan/pipelines/internal/pipeline" + "git.a-wan.net/alan/pipelines/internal/protocol" +) + +// A stand-in Claude Code CLI: it answers the stage with a valid final message. +const fakeClaudeForRun = `#!/bin/sh +if [ "$1" = auth ]; then echo '{"loggedIn":true}'; exit 0; fi +session=""; model=""; prev="" +for a in "$@"; do + case "$prev" in + --session-id|--resume) session="$a" ;; + --model) model="$a" ;; + esac + prev="$a" +done +cat > /dev/null +echo '{"type":"system","subtype":"init","session_id":"'"$session"'","model":"'"$model"'"}' +echo '{"type":"assistant","session_id":"'"$session"'","message":{"role":"assistant","content":[{"type":"text","text":"working"}]}}' +printf '%s\n' '{"type":"result","subtype":"success","is_error":false,"result":"# outcome\nsuccess\n\n# handoff\n\nChanged nothing.\n","session_id":"'"$session"'","total_cost_usd":0.5,"usage":{"input_tokens":4,"output_tokens":6}}' +` + +// TestRunnerRunsAClaudeHarnessStage drives a whole run whose only stage uses +// harness=claude through the real runner path: client selection, session +// ownership, usage and the session export gate. +func TestRunnerRunsAClaudeHarnessStage(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + root, err := filepath.EvalSymlinks(t.TempDir()) + if err != nil { + t.Fatal(err) + } + t.Setenv("HOME", root) + t.Setenv("XDG_CACHE_HOME", root) + t.Setenv("CLAUDE_CODE_OAUTH_TOKEN", "") + executable := filepath.Join(root, "claude") + if err := os.WriteFile(executable, []byte(fakeClaudeForRun), 0700); err != nil { + t.Fatal(err) + } + t.Setenv("PIPELINES_CLAUDE", executable) + store, err := pipeline.Open(filepath.Join(root, "runs.db")) + if err != nil { + t.Fatal(err) + } + defer store.Close() + source := strings.Replace(pipeline.Template([]string{"one"}, true), "provider=opencode-go\nmodel=deepseek-v4.1-flash\n", "harness=claude\nprovider=anthropic\nmodel=claude-haiku-5-5\n", 1) + if !strings.Contains(source, "harness=claude") { + t.Fatal("template changed; update this test") + } + if _, err := store.Submit(ctx, "test", source); err != nil { + t.Fatal(err) + } + items, _ := store.List(ctx) + r, err := newRunner(ctx, store) + if err != nil { + t.Fatal(err) + } + workspace := filepath.Join(root, "work") + if err := os.Mkdir(workspace, 0700); err != nil { + t.Fatal(err) + } + request := protocol.Message{PipelineID: items[0].ID, Version: 1, RequestKey: "claude", Workspace: workspace, Input: map[string]any{"prompt": "task"}} + // An override to another provider cannot run on this harness. + refused := request + refused.Model = "opencode-go/deepseek-v4.1-flash" + if _, err := r.start(refused); err == nil || !strings.Contains(err.Error(), "anthropic/") { + t.Fatal("a non-Claude model override was admitted:", err) + } + run, err := r.start(request) + if err != nil { + t.Fatal(err) + } + r.wg.Wait() + detail, err := store.GetRun(ctx, run.ID) + if err != nil { + t.Fatal(err) + } + if detail.Status != "succeeded" || len(detail.Steps) != 1 { + t.Fatalf("run: %s %+v", detail.Status, detail.Steps) + } + step := detail.Steps[0] + if step.Outcome != "success" || step.SessionID == "" || step.Provider != "anthropic" || step.Model != "claude-haiku-5-5" { + t.Fatalf("step: %+v", step) + } + if step.SessionExport == nil || step.SessionExport.Backend != "claude" || step.SessionExport.Diagnostic != "" { + t.Fatalf("session export: %+v", step.SessionExport) + } + if step.Usage == nil { + t.Fatal("no usage recorded") + } + t.Logf("usage %+v export %+v", *step.Usage, *step.SessionExport) +} diff --git a/internal/server/evidence.go b/internal/server/evidence.go index a8fcc56e..6ad699a2 100644 --- a/internal/server/evidence.go +++ b/internal/server/evidence.go @@ -26,6 +26,8 @@ func (r *runner) evidence(ctx context.Context, id int64) ([]protocol.SessionEvid item.Backend = owner.Backend if owner.Backend == "pi" { item.Data, err = stage.PiEvidence(owner.Reference) + } else if owner.Backend == stage.ClaudeBackend { + item.Data, err = stage.ClaudeStageEvidence(owner.Reference) } else { item.Data, err = (stage.OpenCodeClient{Directory: owner.Directory, Backend: owner.Backend}).SessionEvidence(ctx, owner.ID) } diff --git a/internal/server/run_reattach.go b/internal/server/run_reattach.go index bc7d90d7..f58832f0 100644 --- a/internal/server/run_reattach.go +++ b/internal/server/run_reattach.go @@ -67,6 +67,9 @@ var interruptRecoveredSession = func(ctx context.Context, owner pipeline.Session if owner.Backend == "pi" { return stage.InterruptPi(ctx, owner.Reference) } + if owner.Backend == stage.ClaudeBackend { + return stage.InterruptClaudeStage(ctx, owner.Reference) + } return (stage.OpenCodeClient{Backend: owner.Backend, Directory: owner.Directory, Reference: owner.Reference}).InterruptAndConfirm(ctx, owner.ID) } @@ -128,6 +131,9 @@ func planReattach(ctx context.Context, s *pipeline.Store) ([]reattachPlan, map[i if owner.Backend == "pi" { name = "Pi" } + if owner.Backend == stage.ClaudeBackend { + name = "Claude Code" + } reasons[row.ID] = fmt.Sprintf("%s stage sessions belong to the stopped server process, so stage %s could not be reattached", name, last.Stage) continue } diff --git a/internal/server/runner.go b/internal/server/runner.go index 8a179193..aeb4df4c 100644 --- a/internal/server/runner.go +++ b/internal/server/runner.go @@ -14,6 +14,7 @@ import ( "sync" "time" + "git.a-wan.net/alan/pipelines/internal/claude" "git.a-wan.net/alan/pipelines/internal/filediff" "git.a-wan.net/alan/pipelines/internal/harness" "git.a-wan.net/alan/pipelines/internal/opencode" @@ -223,13 +224,20 @@ func (r *runner) startWithSchedule(request protocol.Message, origin *pipeline.Sc // admission: an unknown stage key or a malformed provider/model must never // insert a run row. The resolution is O(#overrides + #stages) with no model or // network call, and the same helper is the authoritative precedence rule. - if _, err := pipeline.EffectiveModels(d.Graph, request.Model, request.StageModels); err != nil { + launchModels, err := pipeline.EffectiveModels(d.Graph, request.Model, request.StageModels) + if err != nil { return nil, err } for _, s := range d.Graph.Stages { if stage.EffectiveType(s.Type) != stage.ModelType { continue } + if s.Configs["harness"] == stage.ClaudeBackend && launchModels[s.Key].Provider != stage.ClaudeProvider { + return nil, fmt.Errorf("stage %q runs on the claude harness and needs an %s/ override", s.Key, stage.ClaudeProvider) + } + if s.Configs["harness"] == stage.ClaudeBackend && r.execute == nil && !claude.Available() { + return nil, fmt.Errorf("Claude Code is not installed on this execution server; run pipelines harness setup claude there") + } if s.Configs["harness"] == "pi" && r.execute == nil { configured, err := harness.Load() if err != nil { @@ -1941,6 +1949,16 @@ func (r *runner) runLoop(ctx context.Context, id int64, graph protocol.Graph, na result, err = (stage.ModelStage{Client: piClient}).Run(stageCtx, def, input) stageSession = piClient.SessionID client.Transcript = piClient.Transcript + } else if modelConfig.Harness == stage.ClaudeBackend { + claudeClient := &stage.ClaudeStageClient{Directory: workspace, Tools: tools, WritePaths: request.WritePaths, ArtifactsDir: artifactsDir} + claudeClient.OnOwnership = func(session, reference string, active bool) error { + c, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + return r.store.RunSession(c, id, sequence, session, active, stage.ClaudeBackend, reference) + } + result, err = (stage.ModelStage{Client: claudeClient}).Run(stageCtx, def, input) + stageSession = claudeClient.SessionID + client.Transcript = claudeClient.Transcript } else if useAgent { result, err = (stage.ModelStage{Client: client}).Run(stageCtx, def, input) stageSession = client.SessionID @@ -2242,7 +2260,7 @@ func (r *runner) runLoop(ctx context.Context, id int64, graph protocol.Graph, na func (r *runner) exportStageSession(ctx context.Context, owner pipeline.SessionOwner, artifactsDir string, runID int64, sequence int, key string, modelConfig stage.ModelConfig) *protocol.SessionExport { evidence := &protocol.SessionExport{SessionID: owner.ID, Backend: owner.Backend, Directory: owner.Directory, RecordedAt: time.Now().UTC().Format(time.RFC3339)} defer classifySessionExport(evidence) - if owner.Backend != "v1" && owner.Backend != "v2" && !(owner.Backend == "pi" && owner.Reference != "") { + if owner.Backend != "v1" && owner.Backend != "v2" && !((owner.Backend == "pi" || owner.Backend == stage.ClaudeBackend) && owner.Reference != "") { evidence.Source = "unavailable" evidence.Diagnostic = "session export unavailable: harness has no native export path" return evidence diff --git a/internal/server/session_export_repair.go b/internal/server/session_export_repair.go index 1b869245..0f01c289 100644 --- a/internal/server/session_export_repair.go +++ b/internal/server/session_export_repair.go @@ -87,7 +87,7 @@ func (r *runner) repairSessionExportOwned(ctx context.Context, id int64, sequenc } return r.sessionExportUpdatedDetail(recordCtx, id) } - if owner.Backend != "v1" && owner.Backend != "v2" && owner.Backend != "pi" { + if owner.Backend != "v1" && owner.Backend != "v2" && owner.Backend != "pi" && owner.Backend != stage.ClaudeBackend { return recordFailure("native export unavailable for original harness") } root, err := os.OpenRoot(r.artifactsRoot) diff --git a/internal/stage/claude_stage.go b/internal/stage/claude_stage.go new file mode 100644 index 00000000..6d919ddd --- /dev/null +++ b/internal/stage/claude_stage.go @@ -0,0 +1,518 @@ +package stage + +import ( + "bufio" + "bytes" + "context" + "crypto/rand" + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "syscall" + "time" + + "git.a-wan.net/alan/pipelines/internal/claude" + "git.a-wan.net/alan/pipelines/internal/opencode" + "git.a-wan.net/alan/pipelines/internal/workenv" +) + +// ClaudeBackend names a stage session run by the Claude Code CLI in ownership +// records and exports. +const ClaudeBackend = "claude" + +// ClaudeProvider is the provider a `harness=claude` stage declares. Claude Code +// signs in with the execution user's own Claude login; no provider key is read. +const ClaudeProvider = "anthropic" + +// claudeStageSessions is the retained evidence directory under the user cache. +const claudeStageSessions = "claude-stage-sessions" + +// Fixed diagnostics: no CLI output, provider message or credential is copied. +var ( + errClaudeMissing = errors.New("Claude Code is not installed on the execution server; run pipelines harness setup claude there") + errClaudeSignedOut = errors.New("Claude Code is not signed in on the execution server; run pipelines harness setup claude there") + errClaudeNoResult = errors.New("Claude Code ended without a final result") +) + +// ClaudeStageClient runs one stage visit with the Claude Code CLI in print +// mode: the prompt on stdin, the final assistant text out. Only the stage's +// declared tools exist for the session (`--tools`), and nothing outside them +// is approved (`--permission-mode dontAsk`): an undeclared tool is absent, and +// an edit outside the run directory or the stage's artifacts directory is +// refused by the CLI, not merely discouraged. User, project and local settings +// (hooks, plugins, MCP servers, slash commands) are not loaded. +type ClaudeStageClient struct { + Executable, Directory, SessionID, Reference, Transcript string + Tools []string + WritePaths []string + ArtifactsDir string + Policy ToolPolicy + OnOwnership func(string, string, bool) error + WorkStarted bool + + budget *Budget + // Session totals across the visit's invocations (the prompt and one + // outcome follow-up share a session and report per invocation). + usd float64 + input, output, reasoning, cacheRead, cacheW int64 +} + +func (c *ClaudeStageClient) BudgetCapable() bool { return true } + +func (c *ClaudeStageClient) workAdmitted() bool { return c.WorkStarted } + +// claudeTools maps the stage's canonical tool names to Claude Code's builtins. +func claudeTools(tools []string) ([]string, error) { + if tools == nil { + tools = legacyTools + } + names := []string{} + for _, tool := range tools { + switch tool { + case "read": + names = append(names, "Read") + case "glob": + names = append(names, "Glob") + case "grep": + names = append(names, "Grep") + case "edit": + names = append(names, "Edit", "Write") + case "shell": + names = append(names, "Bash") + case "webfetch": + names = append(names, "WebFetch") + case "websearch": + names = append(names, "WebSearch") + default: + return nil, fmt.Errorf("Claude Code has no audited builtin for %s; choose read/glob/grep/edit/shell/webfetch/websearch", tool) + } + } + return names, nil +} + +// claudeAllowRules are the pre-approved permission rules. File edits are +// approved only below the run directory and the stage's artifacts directory; +// everything else a declared tool does not need approval for stays denied by +// dontAsk. +func claudeAllowRules(tools []string, directory, artifactsDir string) []string { + rules := []string{} + edit := false + for _, tool := range tools { + switch tool { + case "Edit": + edit = true + case "Write": + default: + rules = append(rules, tool) + } + } + if edit { + // `//path` is an absolute path in Claude Code permission rules; an Edit + // rule covers every builtin that writes files. + rules = append(rules, "Edit(/"+filepath.ToSlash(directory)+"/**)") + if artifactsDir != "" { + rules = append(rules, "Edit(/"+filepath.ToSlash(artifactsDir)+"/**)") + } + } + return rules +} + +func (c *ClaudeStageClient) args(config ModelConfig, tools []string, effort string, resume bool) []string { + args := []string{"-p", "--output-format", "stream-json", "--verbose", + "--model", config.Model, + "--permission-mode", "dontAsk", + "--setting-sources", "", + "--strict-mcp-config", + "--disable-slash-commands", + "--tools", strings.Join(tools, ",")} + if rules := claudeAllowRules(tools, c.Directory, c.ArtifactsDir); len(rules) > 0 { + args = append(args, "--allowedTools") + args = append(args, rules...) + } + if c.ArtifactsDir != "" { + args = append(args, "--add-dir", c.ArtifactsDir) + } + if effort != "" { + args = append(args, "--effort", effort) + } + if resume { + args = append(args, "--resume", c.SessionID) + } else { + args = append(args, "--session-id", c.SessionID) + } + return args +} + +func (c *ClaudeStageClient) Generate(ctx context.Context, config ModelConfig, prompt string) (text string, err error) { + c.WorkStarted = false + c.usd, c.input, c.output, c.reasoning, c.cacheRead, c.cacheW = 0, 0, 0, 0, 0, 0 + c.budget = config.Budget + if c.budget == nil { + c.budget = NewBudget(0, 0) + } + if config.Provider != ClaudeProvider { + return "", fmt.Errorf("the claude harness needs provider=%s", ClaudeProvider) + } + effort, err := normalizeReasoningEffort(config.ReasoningEffort) + if err != nil { + return "", err + } + if effort == "none" || effort == "minimal" { + return "", fmt.Errorf("the claude harness supports reasoning_effort low, medium, high, xhigh or max") + } + if len(c.WritePaths) > 0 || c.Policy.Edit != nil || c.Policy.Shell != nil { + // The other harnesses enforce these narrower scopes themselves. Claude + // Code's rule syntax is not audited for them, so refuse, never widen. + return "", fmt.Errorf("the claude harness cannot enforce a permissions block or a narrowed write scope; use the opencode harness for this stage") + } + tools, err := claudeTools(c.Tools) + if err != nil { + return "", err + } + if c.Executable == "" { + c.Executable = claude.BinaryPath() + } + if c.Executable == "" { + return "", fmt.Errorf("%w: %w", ErrHarnessUnreachable, errClaudeMissing) + } + if _, statErr := os.Stat(c.Executable); statErr != nil { + return "", fmt.Errorf("%w: %w", ErrHarnessUnreachable, errClaudeMissing) + } + cache, err := os.UserCacheDir() + if err != nil { + return "", err + } + root := filepath.Join(cache, "pipelines", claudeStageSessions) + if err = os.MkdirAll(root, 0700); err != nil { + return "", err + } + if c.Reference, err = os.MkdirTemp(root, "stage-*"); err != nil { + return "", err + } + if c.SessionID, err = newClaudeSessionID(); err != nil { + return "", err + } + env := claudeStageEnvironment(c.environment(ctx), c.ArtifactsDir) + owned := false + defer func() { + // Quiescence is proven per invocation (see run); only then is the + // evidence marked settled and ownership released. + if errors.Is(err, ErrStageCleanup) { + text = "" + return + } + _ = os.WriteFile(filepath.Join(c.Reference, "quiescent"), []byte("quiescent"), 0600) + if owned { + if saveErr := c.OnOwnership(c.SessionID, c.Reference, false); saveErr != nil { + err = errors.Join(err, saveErr, ErrStageCleanup) + text = "" + } + } + }() + if c.OnOwnership != nil { + if err = c.OnOwnership(c.SessionID, c.Reference, true); err != nil { + return "", err + } + owned = true + } + if text, err = c.run(ctx, env, c.args(config, tools, effort, false), config.Model, prompt); err != nil { + return "", err + } + // One outcome repair follow-up in the same session. + repair := outputRepairFrom(ctx) + if message, ok := repair.followUp(text); ok { + reply, err := c.run(ctx, env, c.args(config, tools, effort, true), config.Model, message) + if err != nil { + return "", err + } + text = repair.merge(text, reply) + } + return text, nil +} + +// environment is the run directory's environment (the server's when the +// directory changes nothing) plus the server's Claude sign-in fallbacks. A +// directory that sets or unsets an account variable owns the account: no +// fallback is added, as in a terminal opened there. +func (c *ClaudeStageClient) environment(ctx context.Context) []string { + run, _ := workenv.FromContext(ctx) + base := run.Command(c.Directory) + if base == nil { + base = os.Environ() + } + owned := map[string]bool{} + for _, key := range workenv.AccountEnvKeys { + owned[key] = true + } + for _, key := range append(append([]string{}, run.Set...), run.Unset...) { + if owned[key] { + return base + } + } + return claude.StageEnv(ctx, c.Executable, base) +} + +// claudeStageEnvironment drops an inherited artifacts directory (the argument +// is authoritative) and exports this stage's. +func claudeStageEnvironment(inherited []string, artifactsDir string) []string { + env := make([]string, 0, len(inherited)+1) + for _, entry := range inherited { + if strings.HasPrefix(entry, ArtifactsEnv+"=") { + continue + } + env = append(env, entry) + } + if artifactsDir != "" { + env = append(env, ArtifactsEnv+"="+artifactsDir) + } + return env +} + +func newClaudeSessionID() (string, error) { + var b [16]byte + if _, err := rand.Read(b[:]); err != nil { + return "", err + } + b[6] = (b[6] & 0x0f) | 0x40 + b[8] = (b[8] & 0x3f) | 0x80 + return fmt.Sprintf("%x-%x-%x-%x-%x", b[0:4], b[4:6], b[6:8], b[8:10], b[10:16]), nil +} + +// claudeStreamRecord is the part of a stream-json line the stage reads. Text +// fields are used for the result only, never copied into a diagnostic. +type claudeStreamRecord struct { + Type string `json:"type"` + Subtype string `json:"subtype"` + SessionID string `json:"session_id"` + Model string `json:"model"` + IsError bool `json:"is_error"` + Result string `json:"result"` + Error string `json:"error"` + CostUSD *float64 `json:"total_cost_usd"` + Usage *struct { + Input *int64 `json:"input_tokens"` + Output *int64 `json:"output_tokens"` + CacheRead *int64 `json:"cache_read_input_tokens"` + CacheWrite *int64 `json:"cache_creation_input_tokens"` + Details *struct { + Thinking *int64 `json:"thinking_tokens"` + } `json:"output_tokens_details"` + } `json:"usage"` + RateLimit *struct { + Status string `json:"status"` + } `json:"rate_limit_info"` + Message *struct { + Error string `json:"error"` + } `json:"message"` +} + +// run executes one CLI invocation and returns its final result text. The +// process group is always gone when it returns; otherwise ErrStageCleanup. +func (c *ClaudeStageClient) run(ctx context.Context, env, args []string, model, prompt string) (text string, err error) { + cmd := exec.Command(c.Executable, args...) + cmd.Dir = c.Directory + cmd.Env = env + cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} + cmd.Stdin = strings.NewReader(prompt) + stderr, err := os.OpenFile(filepath.Join(c.Reference, "stderr.log"), os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0600) + if err != nil { + return "", err + } + defer stderr.Close() + cmd.Stderr = stderr + out, err := cmd.StdoutPipe() + if err != nil { + return "", err + } + stream, err := os.OpenFile(filepath.Join(c.Reference, "stream.jsonl"), os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0600) + if err != nil { + return "", err + } + defer stream.Close() + if err = cmd.Start(); err != nil { + return "", fmt.Errorf("%w: %w", ErrHarnessUnreachable, errClaudeMissing) + } + owner, _ := json.Marshal(map[string]any{"pid": cmd.Process.Pid, "session_id": c.SessionID}) + if err = os.WriteFile(filepath.Join(c.Reference, "owner.json"), owner, 0600); err != nil { + _ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) + _ = cmd.Wait() + return "", err + } + // The prompt is admitted once the process has it: tool work may follow. + c.WorkStarted = true + var final *claudeStreamRecord + var refusal, limited bool + var authFailed, sessionMismatch, modelMismatch bool + scanned := make(chan error, 1) + go func() { + scanner := bufio.NewScanner(out) + scanner.Buffer(make([]byte, 64<<10), 16<<20) + for scanner.Scan() { + line := scanner.Bytes() + var record claudeStreamRecord + if json.Unmarshal(line, &record) != nil { + continue + } + redacted := RedactEvidence(string(line)) + _, _ = stream.WriteString(redacted + "\n") + if len(c.Transcript) < 32<<10 { + c.Transcript += redacted + "\n" + } + switch record.Type { + case "system": + if record.Subtype == "init" { + sessionMismatch = record.SessionID != c.SessionID + modelMismatch = record.Model != model + } + case "rate_limit_event": + if record.RateLimit != nil && record.RateLimit.Status == "rejected" { + limited = true + } + case "assistant": + if record.Message != nil { + switch record.Message.Error { + case "billing_error", "rate_limit": + refusal = true + case "authentication_failed": + authFailed = true + } + } + switch record.Error { + case "billing_error", "rate_limit": + refusal = true + case "authentication_failed": + authFailed = true + } + case "result": + copied := record + final = &copied + } + } + scanned <- scanner.Err() + }() + waited := make(chan error, 1) + go func() { + scanErr := <-scanned + waitErr := cmd.Wait() + if scanErr != nil && waitErr == nil { + waitErr = scanErr + } + waited <- waitErr + }() + var waitErr error + select { + case waitErr = <-waited: + case <-ctx.Done(): + // Timeout or cancellation: stop the whole process group, then prove it. + _ = syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM) + select { + case <-waited: + case <-time.After(5 * time.Second): + _ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) + <-waited + } + waitErr = ctx.Err() + } + // The CLI's exit alone does not prove its shell descendants exited. + if !claudeGroupGone(cmd.Process.Pid) { + _ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) + deadline := time.Now().Add(5 * time.Second) + for !claudeGroupGone(cmd.Process.Pid) && time.Now().Before(deadline) { + time.Sleep(50 * time.Millisecond) + } + if !claudeGroupGone(cmd.Process.Pid) { + return "", errors.Join(waitErr, ErrStageCleanup) + } + } + if ctx.Err() != nil { + return "", ctx.Err() + } + if final != nil { + if usageErr := c.observe(final); usageErr != nil { + return "", usageErr + } + } + switch { + case authFailed: + return "", fmt.Errorf("%w: %w", ErrHarnessTransport, errClaudeSignedOut) + case refusal || (limited && (final == nil || final.IsError)): + return "", fmt.Errorf("%w: %w", ErrHarnessTransport, opencode.ErrProviderQuota) + case sessionMismatch || modelMismatch: + return "", fmt.Errorf("%w: Claude Code did not start the requested session and model", ErrHarnessTransport) + case final == nil: + if waitErr != nil { + return "", fmt.Errorf("%w: Claude Code exited with an error before a result (see the retained stderr log)", ErrHarnessTransport) + } + return "", fmt.Errorf("%w: %w", ErrHarnessTransport, errClaudeNoResult) + case final.IsError || final.Subtype != "success": + return "", fmt.Errorf("%w: Claude Code reported an error result (%s)", ErrHarnessTransport, sanitizedClaudeName(final.Subtype)) + case strings.TrimSpace(final.Result) == "": + return "", fmt.Errorf("%w: %w", ErrHarnessTransport, errClaudeNoResult) + } + return final.Result, nil +} + +func claudeGroupGone(pid int) bool { + return errors.Is(syscall.Kill(-pid, 0), syscall.ESRCH) +} + +// sanitizedClaudeName bounds a CLI-reported identifier to a safe token. +func sanitizedClaudeName(name string) string { + var b bytes.Buffer + for _, r := range name { + if b.Len() >= 48 { + break + } + if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' || r == '-' { + b.WriteRune(r) + } + } + if b.Len() == 0 { + return "unknown" + } + return b.String() +} + +// observe adds one invocation's reported usage to the visit's session totals. +// A result without cost and token counts fails closed when a budget is set. +func (c *ClaudeStageClient) observe(final *claudeStreamRecord) error { + u := final.Usage + if final.CostUSD == nil || u == nil || u.Input == nil || u.Output == nil { + if c.budget.Enabled() { + return fmt.Errorf("%w: Claude Code result reported no usage", ErrBudgetAccounting) + } + return nil + } + value := func(p *int64) int64 { + if p == nil { + return 0 + } + return *p + } + thinking := int64(0) + if u.Details != nil { + thinking = value(u.Details.Thinking) + } + for _, n := range []int64{*u.Input, *u.Output, value(u.CacheRead), value(u.CacheWrite), thinking} { + if n < 0 { + return fmt.Errorf("%w: invalid token count", ErrBudgetAccounting) + } + } + c.usd += *final.CostUSD + c.input += *u.Input + // Claude reports thinking tokens inside output_tokens; keep the split. + c.output += max(*u.Output-thinking, 0) + c.reasoning += thinking + c.cacheRead += value(u.CacheRead) + c.cacheW += value(u.CacheWrite) + usd, in, out, reasoning, read, write := c.usd, c.input, c.output, c.reasoning, c.cacheRead, c.cacheW + return c.budget.observe(c.SessionID, reportedUsage{USD: &usd, Tokens: &struct { + Input, Output, Reasoning *int64 + Cache *struct{ Read, Write *int64 } + Total *int64 + }{Input: &in, Output: &out, Reasoning: &reasoning, Cache: &struct{ Read, Write *int64 }{Read: &read, Write: &write}}}) +} diff --git a/internal/stage/claude_stage_export.go b/internal/stage/claude_stage_export.go new file mode 100644 index 00000000..530f1409 --- /dev/null +++ b/internal/stage/claude_stage_export.go @@ -0,0 +1,259 @@ +package stage + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "strings" + "syscall" + "time" + + "git.a-wan.net/alan/pipelines/internal/opencode" + "git.a-wan.net/alan/pipelines/internal/protocol" +) + +// claudeStageRoot opens a retained Claude stage evidence directory after +// checking that it is a server-owned, private `stage-*` directory directly +// under the cache's claude-stage-sessions directory, with no symlink on the +// way. It mirrors the Pi evidence checks. +func claudeStageRoot(reference string) (*os.Root, error) { + fail := fmt.Errorf("Claude stage session unavailable: invalid retained evidence") + cache, err := os.UserCacheDir() + if err != nil || !filepath.IsAbs(reference) || filepath.Clean(reference) != reference { + return nil, fail + } + base := filepath.Join(cache, "pipelines", claudeStageSessions) + if filepath.Dir(reference) != base || !strings.HasPrefix(filepath.Base(reference), "stage-") { + return nil, fail + } + cacheRoot, err := os.OpenRoot(cache) + if err != nil { + return nil, fail + } + defer cacheRoot.Close() + rel, err := filepath.Rel(cache, reference) + if err != nil { + return nil, fail + } + components := strings.Split(rel, string(filepath.Separator)) + for i := range components { + info, err := cacheRoot.Lstat(filepath.Join(components[:i+1]...)) + if err != nil || !info.IsDir() || !piOwned(info) || info.Mode().Perm()&0022 != 0 { + return nil, fail + } + } + info, err := cacheRoot.Lstat(rel) + if err != nil || info.Mode().Perm()&0077 != 0 { + return nil, fail + } + root, err := cacheRoot.OpenRoot(rel) + if err != nil { + return nil, fail + } + opened, err := root.Stat(".") + if err != nil || !os.SameFile(info, opened) { + root.Close() + return nil, fail + } + return root, nil +} + +func claudeStageRead(root *os.Root, name string, limit int64) ([]byte, bool, error) { + f, err := piOpenEvidence(root, name) + if err != nil { + return nil, false, err + } + defer f.Close() + data, err := io.ReadAll(io.LimitReader(f, limit+1)) + if int64(len(data)) > limit { + return data[:limit], true, err + } + return data, false, err +} + +type claudeStageOwner struct { + PID int `json:"pid"` + SessionID string `json:"session_id"` +} + +func claudeStageOwnerOf(root *os.Root) (claudeStageOwner, error) { + var owner claudeStageOwner + data, cut, err := claudeStageRead(root, "owner.json", 16<<10) + if err != nil || cut || json.Unmarshal(data, &owner) != nil || owner.PID <= 1 || owner.SessionID == "" { + return owner, fmt.Errorf("invalid retained Claude stage owner") + } + return owner, nil +} + +// exportClaudeStageSession reads only a settled, server-owned Claude stage +// session from the stream the stage retained. It starts no CLI and resumes, +// interrupts or deletes nothing. +func exportClaudeStageSession(ctx context.Context, request SessionExportRequest) (opencode.ExportedSession, error) { + session := opencode.ExportedSession{ID: request.SessionID, Backend: ClaudeBackend, Attempts: 1} + fail := func() (opencode.ExportedSession, error) { + return session, fmt.Errorf("Claude stage session unavailable: invalid, active or unsettled retained evidence") + } + if err := ctx.Err(); err != nil { + return session, err + } + root, err := claudeStageRoot(request.Reference) + if err != nil || request.SessionID == "" { + return fail() + } + defer root.Close() + owner, err := claudeStageOwnerOf(root) + if err != nil || owner.SessionID != request.SessionID { + return fail() + } + quiet, _, err := claudeStageRead(root, "quiescent", 32) + if err != nil || string(quiet) != "quiescent" || !errors.Is(syscall.Kill(owner.PID, 0), syscall.ESRCH) || !claudeGroupGone(owner.PID) { + return fail() + } + data, cut, err := claudeStageRead(root, "stream.jsonl", SessionExportSourceLimit) + if err != nil { + return fail() + } + session.Truncated = cut + if cut { + // A cut JSONL record is never fabricated or partially decoded. + if i := bytes.LastIndexByte(data, '\n'); i >= 0 { + data = data[:i+1] + } else { + data = nil + } + } + for _, line := range bytes.Split(bytes.TrimSpace(data), []byte{'\n'}) { + if err := ctx.Err(); err != nil { + return session, err + } + if len(bytes.TrimSpace(line)) == 0 { + continue + } + var entry struct { + Type string `json:"type"` + UUID string `json:"uuid"` + Timestamp string `json:"timestamp"` + Message struct { + ID string `json:"id"` + Role string `json:"role"` + Model string `json:"model"` + Content json.RawMessage `json:"content"` + } `json:"message"` + } + if json.Unmarshal(line, &entry) != nil { + return fail() + } + raw := opencode.ExportedPart{Type: "claude_stream_entry", Raw: append(json.RawMessage(nil), line...)} + if entry.Type != "assistant" && entry.Type != "user" { + // init, rate-limit and result records ride with the neighbouring + // message as native evidence, not as invented turns. + if n := len(session.Messages); n > 0 { + session.Messages[n-1].Parts = append(session.Messages[n-1].Parts, raw) + } else { + session.Messages = append(session.Messages, opencode.ExportedMessage{Role: "system", Parts: []opencode.ExportedPart{raw}}) + } + continue + } + m := opencode.ExportedMessage{ID: entry.Message.ID, Role: entry.Message.Role, Model: entry.Message.Model} + if m.ID == "" { + m.ID = entry.UUID + } + if m.Role == "" { + m.Role = entry.Type + } + if stamp, err := time.Parse(time.RFC3339Nano, entry.Timestamp); err == nil { + m.CreatedAt = stamp.UnixMilli() + } + var text string + if json.Unmarshal(entry.Message.Content, &text) == nil { + m.Parts = append(m.Parts, opencode.ExportedPart{Type: protocol.ChatPartText, Text: text}) + } else { + var parts []json.RawMessage + _ = json.Unmarshal(entry.Message.Content, &parts) + for _, part := range parts { + var p struct { + Type, Text, Thinking, ID, Name string + ToolUseID string `json:"tool_use_id"` + IsError bool `json:"is_error"` + Input json.RawMessage `json:"input"` + } + if json.Unmarshal(part, &p) != nil { + continue + } + switch p.Type { + case "text": + m.Parts = append(m.Parts, opencode.ExportedPart{Type: protocol.ChatPartText, Text: p.Text}) + case "thinking": + m.Parts = append(m.Parts, opencode.ExportedPart{Type: protocol.ChatPartReasoning, Text: p.Thinking}) + case "tool_use": + m.Parts = append(m.Parts, opencode.ExportedPart{Type: protocol.ChatPartTool, ID: p.ID, Name: p.Name, State: &protocol.ChatToolState{Input: p.Input}}) + case "tool_result": + status := "completed" + if p.IsError { + status = "error" + } + m.Parts = append(m.Parts, opencode.ExportedPart{Type: protocol.ChatPartTool, ID: p.ToolUseID, State: &protocol.ChatToolState{Status: status}}) + } + } + } + m.Parts = append(m.Parts, raw) + session.Messages = append(session.Messages, m) + } + if len(session.Messages) == 0 { + return session, opencode.ErrEmptySessionExport + } + return session, nil +} + +// ClaudeStageEvidence returns the retained stream of a Claude stage session as +// redacted JSONL text (bounded), for the run's debug evidence. +func ClaudeStageEvidence(reference string) (string, error) { + root, err := claudeStageRoot(reference) + if err != nil { + return "", err + } + defer root.Close() + data, _, err := claudeStageRead(root, "stream.jsonl", 8<<20) + return RedactEvidence(string(data)), err +} + +// InterruptClaudeStage stops a retained Claude stage process group (one the +// stopped server left running) and confirms it is gone. Without that proof the +// recovery fails closed. +func InterruptClaudeStage(ctx context.Context, reference string) error { + root, err := claudeStageRoot(reference) + if err != nil { + return errors.Join(err, ErrStageCleanup) + } + defer root.Close() + owner, err := claudeStageOwnerOf(root) + if err != nil { + return errors.Join(err, ErrStageCleanup) + } + if quiet, _, err := claudeStageRead(root, "quiescent", 32); err == nil && string(quiet) == "quiescent" && claudeGroupGone(owner.PID) { + return nil + } + _ = syscall.Kill(-owner.PID, syscall.SIGTERM) + ticker := time.NewTicker(100 * time.Millisecond) + defer ticker.Stop() + deadline := time.Now().Add(5 * time.Second) + for { + if errors.Is(syscall.Kill(owner.PID, 0), syscall.ESRCH) && claudeGroupGone(owner.PID) { + _ = os.WriteFile(filepath.Join(reference, "quiescent"), []byte("quiescent"), 0600) + return nil + } + if time.Now().After(deadline) { + _ = syscall.Kill(-owner.PID, syscall.SIGKILL) + } + select { + case <-ctx.Done(): + return errors.Join(ctx.Err(), ErrStageCleanup) + case <-ticker.C: + } + } +} diff --git a/internal/stage/claude_stage_live_test.go b/internal/stage/claude_stage_live_test.go new file mode 100644 index 00000000..5f0aa5aa --- /dev/null +++ b/internal/stage/claude_stage_live_test.go @@ -0,0 +1,72 @@ +package stage + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// TestClaudeStageLive runs one real, cheap stage visit through the installed +// Claude Code CLI and its sign-in. It is skipped unless PIPELINES_CLAUDE_LIVE +// names a model (for example claude-haiku-5-5); provider charges apply. +func TestClaudeStageLive(t *testing.T) { + model := os.Getenv("PIPELINES_CLAUDE_LIVE") + if model == "" { + t.Skip("set PIPELINES_CLAUDE_LIVE= to run") + } + root, err := filepath.EvalSymlinks(t.TempDir()) + if err != nil { + t.Fatal(err) + } + work, outside := filepath.Join(root, "work"), filepath.Join(root, "outside") + for _, dir := range []string{work, outside} { + if err := os.Mkdir(dir, 0700); err != nil { + t.Fatal(err) + } + } + if err := os.WriteFile(filepath.Join(work, "note.txt"), []byte("colour: blue\n"), 0600); err != nil { + t.Fatal(err) + } + client := &ClaudeStageClient{Directory: work, Tools: []string{"read", "edit"}} + budget := NewBudget(1, 0) + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute) + defer cancel() + text, err := client.Generate(ctx, ModelConfig{Harness: ClaudeBackend, Provider: ClaudeProvider, Model: model, Budget: budget}, + "Do these steps in order. 1. In note.txt in the current directory, change blue to green. "+ + "2. Try once to create the file "+filepath.Join(outside, "escape.txt")+" containing the word escaped; if that is refused, do not retry. "+ + "3. Try once to run the shell command `touch shell.txt`; if you have no shell tool, skip it. "+ + "Then reply with one line saying which steps worked.") + if err != nil { + t.Fatal(err) + } + t.Logf("reply: %s", text) + t.Logf("usage: %+v", budget.Snapshot()) + if note, _ := os.ReadFile(filepath.Join(work, "note.txt")); !strings.Contains(string(note), "green") { + t.Fatalf("declared edit inside the run directory did not happen: %q", note) + } + if _, err := os.Stat(filepath.Join(outside, "escape.txt")); err == nil { + t.Fatal("an edit outside the run directory was allowed") + } + if _, err := os.Stat(filepath.Join(work, "shell.txt")); err == nil { + t.Fatal("an undeclared shell tool ran") + } + if used := budget.Snapshot(); used.UsedUSD <= 0 || used.UsedTokens <= 0 { + t.Fatalf("no usage recorded: %+v", used) + } + session, err := exportClaudeStageSession(ctx, SessionExportRequest{Backend: ClaudeBackend, SessionID: client.SessionID, Reference: client.Reference}) + if err != nil || len(session.Messages) < 2 { + t.Fatalf("export: %d messages, %v", len(session.Messages), err) + } + stream, _ := ClaudeStageEvidence(client.Reference) + for _, line := range strings.Split(stream, "\n") { + if strings.Contains(line, `"subtype":"init"`) || strings.Contains(line, `"permission_denials"`) { + if len(line) > 1500 { + line = line[:1500] + } + t.Log(line) + } + } +} diff --git a/internal/stage/claude_stage_test.go b/internal/stage/claude_stage_test.go new file mode 100644 index 00000000..2f9db2f2 --- /dev/null +++ b/internal/stage/claude_stage_test.go @@ -0,0 +1,367 @@ +package stage + +import ( + "context" + "errors" + "os" + "path/filepath" + "slices" + "strconv" + "strings" + "syscall" + "testing" + "time" + + "git.a-wan.net/alan/pipelines/internal/opencode" +) + +// fakeClaude is a stand-in Claude Code CLI. It records its arguments and +// prompt, then prints a canned stream-json conversation for FAKE_CLAUDE_CASE. +const fakeClaude = `#!/bin/sh +if [ "$1" = auth ]; then echo '{"loggedIn":true}'; exit 0; fi +session=""; model=""; resumed=0; prev="" +for a in "$@"; do + printf '%s\n' "$a" >> "$FAKE_CLAUDE_OUT/args" + case "$prev" in + --session-id) session="$a" ;; + --resume) session="$a"; resumed=1 ;; + --model) model="$a" ;; + esac + prev="$a" +done +echo "--end--" >> "$FAKE_CLAUDE_OUT/args" +cat >> "$FAKE_CLAUDE_OUT/prompt" +init() { echo '{"type":"system","subtype":"init","session_id":"'"$1"'","model":"'"$2"'","tools":["Read"],"permissionMode":"dontAsk"}'; } +usage='"total_cost_usd":0.25,"usage":{"input_tokens":7,"output_tokens":5,"cache_read_input_tokens":3,"cache_creation_input_tokens":2,"output_tokens_details":{"thinking_tokens":1}}' +case "$FAKE_CLAUDE_CASE" in + success) + init "$session" "$model" + echo '{"type":"assistant","session_id":"'"$session"'","message":{"role":"assistant","content":[{"type":"tool_use","id":"t1","name":"Read","input":{"file_path":"a.txt"}}]}}' + echo '{"type":"user","session_id":"'"$session"'","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"t1","content":"file body"}]}}' + echo '{"type":"rate_limit_event","rate_limit_info":{"status":"allowed"}}' + echo '{"type":"assistant","session_id":"'"$session"'","message":{"role":"assistant","content":[{"type":"text","text":"done"}]}}' + echo '{"type":"result","subtype":"success","is_error":false,"result":"done","session_id":"'"$session"'",'"$usage"'}' + ;; + repair) + init "$session" "$model" + if [ "$resumed" = 1 ]; then text='# outcome\nsuccess\n'; else text='Report kept.'; fi + printf '%s\n' '{"type":"result","subtype":"success","is_error":false,"result":"'"$text"'","session_id":"'"$session"'",'"$usage"'}' + ;; + nousage) + init "$session" "$model" + echo '{"type":"result","subtype":"success","is_error":false,"result":"done","session_id":"'"$session"'"}' + ;; + hang) + init "$session" "$model" + sleep 60 & + echo $! > "$FAKE_CLAUDE_OUT/child" + wait + ;; + exit) + echo "PROVIDER-BODY-SECRET" >&2 + exit 3 + ;; + quota) + init "$session" "$model" + echo '{"type":"assistant","session_id":"'"$session"'","error":"billing_error","message":{"role":"assistant","content":[{"type":"text","text":"PROVIDER-BODY-SECRET"}]}}' + echo '{"type":"result","subtype":"success","is_error":true,"result":"PROVIDER-BODY-SECRET","session_id":"'"$session"'",'"$usage"'}' + exit 1 + ;; + limited) + init "$session" "$model" + echo '{"type":"rate_limit_event","rate_limit_info":{"status":"rejected"}}' + echo '{"type":"result","subtype":"success","is_error":true,"result":"PROVIDER-BODY-SECRET","session_id":"'"$session"'",'"$usage"'}' + exit 1 + ;; + auth) + init "$session" "$model" + echo '{"type":"assistant","session_id":"'"$session"'","error":"authentication_failed","message":{"role":"assistant","content":[{"type":"text","text":"PROVIDER-BODY-SECRET"}]}}' + echo '{"type":"result","subtype":"success","is_error":true,"result":"PROVIDER-BODY-SECRET","session_id":"'"$session"'",'"$usage"'}' + exit 1 + ;; + maxturns) + init "$session" "$model" + echo '{"type":"result","subtype":"error_max_turns","is_error":true,"result":"PROVIDER-BODY-SECRET","session_id":"'"$session"'",'"$usage"'}' + ;; + othermodel) + init "$session" "some-other-model" + echo '{"type":"result","subtype":"success","is_error":false,"result":"done","session_id":"'"$session"'",'"$usage"'}' + ;; +esac +` + +func fakeClaudeClient(t *testing.T, which string) (*ClaudeStageClient, string, *[]bool) { + t.Helper() + root, err := filepath.EvalSymlinks(t.TempDir()) + if err != nil { + t.Fatal(err) + } + t.Setenv("XDG_CACHE_HOME", root) + t.Setenv("HOME", root) + t.Setenv("FAKE_CLAUDE_OUT", root) + t.Setenv("FAKE_CLAUDE_CASE", which) + // The fake needs no account; keep the developer's own out of the test. + t.Setenv("CLAUDE_CODE_OAUTH_TOKEN", "") + executable := filepath.Join(root, "claude") + if err := os.WriteFile(executable, []byte(fakeClaude), 0700); err != nil { + t.Fatal(err) + } + work := filepath.Join(root, "work") + if err := os.Mkdir(work, 0700); err != nil { + t.Fatal(err) + } + active := &[]bool{} + client := &ClaudeStageClient{Executable: executable, Directory: work, Tools: []string{"read"}, OnOwnership: func(_, _ string, value bool) error { + *active = append(*active, value) + return nil + }} + return client, root, active +} + +func fakeClaudeArgs(t *testing.T, root string) [][]string { + t.Helper() + data, err := os.ReadFile(filepath.Join(root, "args")) + if err != nil { + t.Fatal(err) + } + var calls [][]string + for _, call := range strings.Split(string(data), "--end--\n") { + if call != "" { + calls = append(calls, strings.Split(strings.TrimSuffix(call, "\n"), "\n")) + } + } + return calls +} + +func claudeConfig(budget *Budget) ModelConfig { + return ModelConfig{Harness: ClaudeBackend, Provider: ClaudeProvider, Model: "claude-haiku-5-5", Budget: budget} +} + +func TestClaudeStageSuccessUsageAndExport(t *testing.T) { + client, root, active := fakeClaudeClient(t, "success") + budget := NewBudget(10, 1000) + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + text, err := client.Generate(ctx, claudeConfig(budget), "the task") + if err != nil || text != "done" { + t.Fatal(text, err) + } + used := budget.Snapshot() + // 7 input + 5 output (1 of it thinking) + 3 cache read + 2 cache write. + if used.UsedUSD != 0.25 || used.UsedTokens != 17 { + t.Fatalf("usage %+v", used) + } + if !slices.Equal(*active, []bool{true, false}) || !client.WorkStarted { + t.Fatal("ownership", *active, client.WorkStarted) + } + prompt, _ := os.ReadFile(filepath.Join(root, "prompt")) + if string(prompt) != "the task" { + t.Fatalf("prompt %q", prompt) + } + if !strings.Contains(client.Transcript, `"tool_use"`) { + t.Fatal("transcript", client.Transcript) + } + session, err := exportClaudeStageSession(ctx, SessionExportRequest{Backend: ClaudeBackend, SessionID: client.SessionID, Reference: client.Reference}) + if err != nil || len(session.Messages) != 4 { + t.Fatalf("export %+v %v", session, err) + } + if _, err := exportClaudeStageSession(ctx, SessionExportRequest{Backend: ClaudeBackend, SessionID: "another", Reference: client.Reference}); err == nil { + t.Fatal("export accepted a session the evidence does not own") + } + if data, err := ClaudeStageEvidence(client.Reference); err != nil || !strings.Contains(data, `"result"`) { + t.Fatal("evidence", err) + } +} + +func TestClaudeStageDeclaredToolsAreTheOnlyTools(t *testing.T) { + for _, test := range []struct { + name string + tools []string + artifacts bool + want func(work, artifacts string) []string + }{ + {"none", []string{}, false, func(_, _ string) []string { + return []string{"--tools", ""} + }}, + {"read only", []string{"read", "glob", "grep"}, false, func(_, _ string) []string { + return []string{"--tools", "Read,Glob,Grep", "--allowedTools", "Read", "Glob", "Grep"} + }}, + {"edit and shell", []string{"read", "edit", "shell"}, true, func(work, artifacts string) []string { + return []string{"--tools", "Read,Edit,Write,Bash", "--allowedTools", "Read", "Bash", "Edit(/" + work + "/**)", "Edit(/" + artifacts + "/**)", "--add-dir", artifacts} + }}, + } { + t.Run(test.name, func(t *testing.T) { + client, root, _ := fakeClaudeClient(t, "success") + client.Tools = test.tools + artifacts := "" + if test.artifacts { + artifacts = filepath.Join(root, "artifacts") + client.ArtifactsDir = artifacts + } + if _, err := client.Generate(context.Background(), claudeConfig(nil), "task"); err != nil { + t.Fatal(err) + } + calls := fakeClaudeArgs(t, root) + if len(calls) != 1 { + t.Fatal(calls) + } + want := append([]string{"-p", "--output-format", "stream-json", "--verbose", "--model", "claude-haiku-5-5", + "--permission-mode", "dontAsk", "--setting-sources", "", "--strict-mcp-config", "--disable-slash-commands"}, + test.want(client.Directory, artifacts)...) + want = append(want, "--session-id", client.SessionID) + if !slices.Equal(calls[0], want) { + t.Fatalf("args\n got %q\nwant %q", calls[0], want) + } + }) + } +} + +func TestClaudeStageRefusesWhatItCannotEnforce(t *testing.T) { + for name, change := range map[string]func(*ClaudeStageClient, *ModelConfig){ + "write paths": func(c *ClaudeStageClient, _ *ModelConfig) { c.WritePaths = []string{"app"} }, + "edit permissions": func(c *ClaudeStageClient, _ *ModelConfig) { c.Policy.Edit = []string{"app/**"} }, + "shell permissions": func(c *ClaudeStageClient, _ *ModelConfig) { c.Policy.Shell = []string{"go test"} }, + "unknown tool": func(c *ClaudeStageClient, _ *ModelConfig) { c.Tools = []string{"browser"} }, + "other provider": func(_ *ClaudeStageClient, m *ModelConfig) { m.Provider = "opencode-go" }, + "no reasoning": func(_ *ClaudeStageClient, m *ModelConfig) { m.ReasoningEffort = "none" }, + } { + t.Run(name, func(t *testing.T) { + client, root, active := fakeClaudeClient(t, "success") + config := claudeConfig(nil) + change(client, &config) + if _, err := client.Generate(context.Background(), config, "task"); err == nil { + t.Fatal("accepted") + } + if _, err := os.Stat(filepath.Join(root, "args")); err == nil || len(*active) != 0 || client.WorkStarted { + t.Fatal("the CLI was started for a refused stage") + } + }) + } +} + +func TestClaudeStageMissingExecutableIsRetryableReachFailure(t *testing.T) { + client, root, _ := fakeClaudeClient(t, "success") + client.Executable = filepath.Join(root, "absent") + _, err := client.Generate(context.Background(), claudeConfig(nil), "task") + if !errors.Is(err, ErrHarnessUnreachable) || client.WorkStarted { + t.Fatal(err, client.WorkStarted) + } +} + +func TestClaudeStageFailuresAreFixedText(t *testing.T) { + for which, check := range map[string]func(error) bool{ + "exit": func(err error) bool { return errors.Is(err, ErrHarnessTransport) }, + "quota": func(err error) bool { return errors.Is(err, opencode.ErrProviderQuota) }, + "limited": func(err error) bool { return errors.Is(err, opencode.ErrProviderQuota) }, + "auth": func(err error) bool { return errors.Is(err, errClaudeSignedOut) }, + "maxturns": func(err error) bool { return strings.Contains(err.Error(), "(error_max_turns)") }, + "othermodel": func(err error) bool { return errors.Is(err, ErrHarnessTransport) }, + } { + t.Run(which, func(t *testing.T) { + client, _, active := fakeClaudeClient(t, which) + text, err := client.Generate(context.Background(), claudeConfig(nil), "task") + if err == nil || text != "" || !check(err) { + t.Fatal(text, err) + } + if strings.Contains(err.Error(), "PROVIDER-BODY-SECRET") { + t.Fatal("diagnostic copied CLI output:", err) + } + if !slices.Equal(*active, []bool{true, false}) || errors.Is(err, ErrStageCleanup) { + t.Fatal("ownership/cleanup", *active, err) + } + }) + } +} + +func TestClaudeStageBudgetAccounting(t *testing.T) { + client, _, _ := fakeClaudeClient(t, "nousage") + if _, err := client.Generate(context.Background(), claudeConfig(NewBudget(0, 100)), "task"); !errors.Is(err, ErrBudgetAccounting) { + t.Fatal(err) + } + client, _, _ = fakeClaudeClient(t, "success") + if _, err := client.Generate(context.Background(), claudeConfig(NewBudget(0, 5)), "task"); !errors.Is(err, ErrBudgetExceeded) { + t.Fatal(err) + } +} + +func TestClaudeStageOutcomeRepairResumesTheSameSession(t *testing.T) { + client, root, _ := fakeClaudeClient(t, "repair") + budget := NewBudget(0, 1000) + ctx := withOutputRepair(context.Background(), []string{"error", "success"}) + text, err := client.Generate(ctx, claudeConfig(budget), "task") + if err != nil || !strings.HasPrefix(text, "# outcome\nsuccess\n") || !strings.Contains(text, "Report kept.") { + t.Fatalf("%q %v", text, err) + } + calls := fakeClaudeArgs(t, root) + if len(calls) != 2 || !slices.Contains(calls[0], "--session-id") || !slices.Equal(calls[1][len(calls[1])-2:], []string{"--resume", client.SessionID}) { + t.Fatal(calls) + } + // Both invocations count: the session total is cumulative. + if budget.Snapshot().UsedTokens != 34 { + t.Fatalf("usage %+v", budget.Snapshot()) + } +} + +func TestClaudeStageTimeoutStopsTheProcessGroup(t *testing.T) { + client, root, active := fakeClaudeClient(t, "hang") + ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond) + defer cancel() + _, err := client.Generate(ctx, claudeConfig(nil), "task") + if !errors.Is(err, context.DeadlineExceeded) || errors.Is(err, ErrStageCleanup) { + t.Fatal(err) + } + if !slices.Equal(*active, []bool{true, false}) { + t.Fatal("ownership", *active) + } + if err := InterruptClaudeStage(context.Background(), client.Reference); err != nil { + t.Fatal(err) + } + data, err := os.ReadFile(filepath.Join(root, "child")) + if err != nil { + t.Fatal("the fake never recorded its child:", err) + } + child, err := strconv.Atoi(strings.TrimSpace(string(data))) + if err != nil { + t.Fatal(err) + } + deadline := time.Now().Add(2 * time.Second) + for !errors.Is(syscall.Kill(child, 0), syscall.ESRCH) { + if time.Now().After(deadline) { + t.Fatal("the CLI's child process survived the timeout") + } + time.Sleep(20 * time.Millisecond) + } +} + +func TestClaudeHarnessValidation(t *testing.T) { + definition := func(configs map[string]string, blocks map[string]string) Definition { + all := map[string]string{"prompt": "do it", "configs": "x"} + for key, value := range blocks { + all[key] = value + } + return Definition{Key: "work", Type: ModelType, Blocks: all, Configs: configs, Edges: map[string]string{"success": "#success"}} + } + claude := func(extra map[string]string) map[string]string { + configs := map[string]string{"harness": "claude", "provider": "anthropic", "model": "claude-haiku-5-5"} + for key, value := range extra { + configs[key] = value + } + return configs + } + spec, err := (ModelStage{}).Spec(definition(claude(nil), map[string]string{"tools": "read\nedit\nshell"})) + if err != nil || spec.Config.Harness != ClaudeBackend { + t.Fatal(spec.Config, err) + } + for name, def := range map[string]Definition{ + "unknown harness": definition(map[string]string{"harness": "codex", "provider": "anthropic", "model": "claude-haiku-5-5"}, nil), + "capitalised": definition(map[string]string{"harness": "Claude", "provider": "anthropic", "model": "claude-haiku-5-5"}, nil), + "other provider": definition(claude(map[string]string{"provider": "opencode-go"}), nil), + "delivery": definition(claude(map[string]string{"delivery": "true"}), nil), + "permissions block": definition(claude(nil), map[string]string{"permissions": `{"edit":["app/**"]}`}), + "no reasoning": definition(claude(map[string]string{"reasoning_effort": "none"}), nil), + } { + if _, err := (ModelStage{}).Spec(def); err == nil { + t.Fatal(name, "accepted") + } + } +} diff --git a/internal/stage/model.go b/internal/stage/model.go index e1d03745..e4e17340 100644 --- a/internal/stage/model.go +++ b/internal/stage/model.go @@ -91,8 +91,8 @@ func (ModelStage) Spec(def Definition) (ModelSpec, error) { if backend == "" { backend = "opencode" } - if backend != "opencode" && backend != "pi" { - return ModelSpec{}, fmt.Errorf("harness must be opencode or pi") + if backend != "opencode" && backend != "pi" && backend != ClaudeBackend { + return ModelSpec{}, fmt.Errorf("harness must be opencode, pi or claude") } effort, err := normalizeReasoningEffort(def.Configs["reasoning_effort"]) if err != nil { @@ -118,6 +118,21 @@ func (ModelStage) Spec(def Definition) (ModelSpec, error) { return ModelSpec{}, err } } + if backend == ClaudeBackend { + if def.Configs["provider"] != ClaudeProvider { + return ModelSpec{}, fmt.Errorf("the claude harness needs provider=%s", ClaudeProvider) + } + tools, _, _ := ToolNames(def) + if _, err := claudeTools(tools); err != nil { + return ModelSpec{}, err + } + if _, present := def.Blocks["permissions"]; present { + return ModelSpec{}, fmt.Errorf("the claude harness cannot enforce a permissions block; use the opencode harness for this stage") + } + if effort == "none" || effort == "minimal" { + return ModelSpec{}, fmt.Errorf("the claude harness supports reasoning_effort low, medium, high, xhigh or max") + } + } retries := DefaultRetries if value := def.Configs["retries"]; value != "" { retries, err = strconv.Atoi(value) diff --git a/internal/stage/model_run.go b/internal/stage/model_run.go index 92ab0a07..6ff7d0b6 100644 --- a/internal/stage/model_run.go +++ b/internal/stage/model_run.go @@ -30,6 +30,9 @@ func (m ModelStage) Run(ctx context.Context, def Definition, input Values) (Resu if spec.Config.Harness == "pi" { return Result{}, fmt.Errorf("Pi stages require an audited Pi client") } + if spec.Config.Harness == ClaudeBackend { + return Result{}, fmt.Errorf("Claude stages require a Claude stage client") + } tools, _, _ := ToolNames(def) if len(tools) > 0 { return Result{}, fmt.Errorf("tool-enabled stages require an audited agent client") @@ -55,6 +58,9 @@ func (m ModelStage) Run(ctx context.Context, def Definition, input Values) (Resu if pi, ok := client.(*PiClient); ok { pi.Policy, _ = StagePermissions(def) } + if claudeClient, ok := client.(*ClaudeStageClient); ok { + claudeClient.Policy, _ = StagePermissions(def) + } if err := ctx.Err(); err != nil { return Result{}, err } diff --git a/internal/stage/session_export.go b/internal/stage/session_export.go index f7562b2d..a0166e31 100644 --- a/internal/stage/session_export.go +++ b/internal/stage/session_export.go @@ -92,7 +92,7 @@ const ( // a sanitized error and writes nothing when the read fails; it never mutates, // resumes, interrupts or deletes the session. func ExportSession(ctx context.Context, request SessionExportRequest) (SessionExportResult, error) { - if request.Backend != "pi" && !opencode.ValidSessionID(request.SessionID) { + if request.Backend != "pi" && request.Backend != ClaudeBackend && !opencode.ValidSessionID(request.SessionID) { return SessionExportResult{}, fmt.Errorf("session export: invalid session ID") } if request.ArtifactsDir == "" { @@ -109,6 +109,8 @@ func ExportSession(ctx context.Context, request SessionExportRequest) (SessionEx var err error if request.Backend == "pi" { session, err = exportPiSession(ctx, request) + } else if request.Backend == ClaudeBackend { + session, err = exportClaudeStageSession(ctx, request) } else { var connections []*opencode.Connection if request.Backend == "v2" && request.Reference != "" { @@ -221,6 +223,9 @@ func sessionMetadata(session opencode.ExportedSession, request SessionExportRequ if session.Backend == "pi" { meta = append(meta, sessionMeta{"format", "pi_native_jsonl"}, sessionMeta{"reference", request.Reference}) } + if session.Backend == ClaudeBackend { + meta = append(meta, sessionMeta{"format", "claude_stream_jsonl"}, sessionMeta{"reference", request.Reference}) + } if request.Model != "" { meta = append(meta, sessionMeta{"model", request.Model}) }