Commit Graph

24 Commits

Author SHA1 Message Date
alan 46f85556af Retry bounded budget reads after transient timeouts 2026-11-05 15:48:14 +00:00
alan cedfc22e45 Pipeline runs use the workspace's directory environment
Resolve the run workspace's environment with workenv (direnv, parent
.envrc, mise; same account rule as Claude chats) at run start and resume,
and apply it to Pi stage processes, a per-environment Pipelines-owned
OpenCode stage service (keyed by a hash of the change, reused across runs,
idle ones stopped after PIPELINES_OPENCODE_STAGE_IDLE_TIMEOUT, revived per
registration), delivery-gate checks and merges, plain-stage opencode CLI
calls and file-change capture git reads. Record the directory, .envrc
files, sources and variable names (never values) on the run detail; a
blocked .envrc becomes a run warning.
2026-11-04 18:08:42 +00:00
alan 88dfc051e5 Model stages: outcome-first output shape and one in-session outcome repair
The prompt now says output.md must start with # outcome and the outcome
word alone on the next line, then # handoff, then detail. Runs 241/243
followed the old "start with # handoff" wording and dropped the outcome.

When a completed turn lacks a valid outcome, the OpenCode/agent and Pi
clients send one short follow-up in the same session asking only for the
outcome line. An outcome-only reply goes at the top of the original report;
a still-invalid reply fails with the same validation error as before. The
follow-up's tokens accrue to the session usage. No outcome is inferred.

Missing tools/evidence now name blocked when declared, else error. The
parser tolerates a trailing period on the outcome word.
2026-11-04 17:20:58 +00:00
alan aa8eddd686 Keep OpenCode/Codex chat and stage sessions running across server restarts
A stopping server no longer interrupts harness sessions. Runs are canceled
with a shutdown cause, so stage clients leave admitted sessions running and
owned; start-up recovery re-adopts live OpenCode V2 stage sessions (wait,
read back the result, continue the graph), restarts unadmitted visits,
continues runs between stages, and falls back to interrupted with a reason
when the harness is gone. Chats with live work are recorded on shutdown and
reattached on start; turns that ended during the downtime are folded in
from history. The server-spawned V2 chat serve is now a detached owned
service. PIPELINES_SHUTDOWN_SESSIONS=stop restores the old behaviour.
2026-11-04 16:17:08 +00:00
alan 724de27561 feat(stage): honor per-stage reasoning effort across native harnesses 2026-10-20 23:51:14 +01:00
alan d3c7e6e784 fix: allow cold managed service discovery only at stage admission 2026-10-20 21:53:52 +01:00
alan bb98bde4fc fix: pin native exports to stage execution service 2026-10-20 21:47:04 +01:00
alan cb57da5dfc fix(stage): persist every V2 model generation in tracked sessions 2026-10-20 20:43:42 +01:00
alan 9130ae8c7a Add stage-exclusive resource waiting
Add durable FIFO capacity-one resource leases for executable stages, including cleanup-confirmed release, conservative restart quarantine, queue state events, and CLI/TUI/Android waiting presentation. Add scheduler, cleanup, protocol, client, and Android coverage plus a proposed Android v3 resource draft.

Checked packaged templates and authoring skill references: neither needs updating in this item. The per-item docs note records the shared documentation that docs-fold must update; no shared README or packaged reference was edited.
2026-10-18 12:30:30 +01:00
alan 31d8f4a6d5 Distinguish active stage deadlines from harness transport failures
Identify configured attempt expiry during V2 active-session idle waits and retain bounded phase, timeout and session diagnostics without claiming a lost response. Preserve genuine operation failures, error identities, redaction, cleanup and admitted-work replay protections. Add stage regressions and scratch-server CLI/websocket retention coverage.

Checked packaged templates in examples and the pipeline template generator: none needs updating because timeout configuration and contracts are unchanged. Checked packaged authoring skill references: debugging.md needs a STAGE DEADLINE playbook and clarified transport guidance. Recorded that stale shared documentation in docs/notes/stage-timeout-diagnostics.md for docs-fold; shared documentation was deliberately not edited.

Verification: go build ./...; go vet ./...; go test ./...; go test -count=1 ./internal/server ./internal/stage ./internal/pipeline ./internal/protocol; go test -count=1 -run TestStageDeadlineDiagnosticThroughCLIAndWebsocket -v ./internal/server; gofmt -l .; git diff --check. All passed. Live-provider behavior not tested; no deploy or deployed-service restart.
2026-10-18 11:17:01 +01:00
alan 7b72b8f20f stage: retry pre-admission harness reach failures and widen repair deadline
Treat a failure to reach the harness before any prompt or tool call is
admitted (connecting or creating the session) as a distinct retryable
reach failure, so a transport blip at the first stage no longer ends a
mutating run; keep post-admission readback/poll/agent failures retried
only on an explicit read-only stage and never replay admitted tool work.
Preserve the true cause when the context is already done (join ctx.Err
with the typed transport error) and report a sanitized unknown provider
error name/status instead of a blanket summary.

Make the isolated format-repair deadline proportional to the stage
timeout (floored, capped and clamped to remaining minus a margin) so a
slow-but-repairable result is recovered without ever extending the stage
deadline; the bounded ladder, value guard, deny-all formatter session,
cumulative usage accounting and no-tool-replay rule are unchanged.

Update docs/model-results.md, docs/pipeline-performance-review.md and the
packaged skill references; add stage, opencode and server fixtures.
2026-10-17 14:51:09 +01:00
alan f57457aec9 Make agent-stage retries safe and retain timeout failure evidence
Fix four stage failure-handling and observability defects (A2, A3, A6, A9).

- A2: add safeToReplay(def) as the single definition of retryable post-admission
  work - only an explicit tools block that selects neither edit nor shell. The
  agent/pi WorkStarted guards now break only when the stage is not safe to
  replay, so read-only stages genuinely retry a post-admission timeout while
  mutating and legacy-bundle stages still never replay admitted work.
- A6: re-emit a malformed final result on an explicit read-only stage up to the
  retry budget, preserving format_repairs evidence; mutating stages keep the
  immediate return and their session.
- A3: timed-out stages retain usable evidence. waitForSessionIdle records one
  bounded sanitized OperationEvidence on deadline; the agent deferred cleanup
  does a bounded best-effort latest-assistant readback and settles knowable V2
  usage; RetainSession adds a retained_session pointer to failed step
  transcripts so the session is visible beneath the BoundTranscript threshold.
- A9: a stage outcome routed to #failure now yields a non-empty run diagnostic
  ("stage <name> returned outcome <outcome>") while preferring an existing step
  diagnostic.

Adds focused stage and server tests for retry safety, malformed-result
re-emission, timeout evidence retention and run-level diagnostics; updates the
packaged skill references and docs/cli.md, docs/model-results.md.
2026-10-15 18:27:22 +01:00
alan 0f5a188c6a Record and serve per-stage artifacts
Every agent stage now writes into a per-run/step artifacts directory under
the data dir (<data>/artifacts/<run>/<step>), passed to Pi as
PIPELINES_ARTIFACTS_DIR, exposed on the OpenCode client env seam, and stated
in the agent prompt boilerplate so agents save images, screenshots, reports
and logs there.

After each step the server scans that directory with a bounded, symlink- and
traversal-safe inventory (os.OpenRoot + Lstat + O_NOFOLLOW) recording name,
size_bytes, content type (extension then 512-byte sniff fallback) and URL,
enforcing max files, per-file bytes and total bytes while counting skipped
and truncated entries. A missing, unreadable or oversized directory yields
only a best-effort diagnostic and never changes the step outcome.

The list is carried on protocol.RunStep as Artifacts []StageArtifact plus
skipped/truncated counters (omitempty, backwards compatible), persisted in a
new step_artifacts table keyed (run_id, sequence) alongside the
notifications/file-changes tables and loaded by GetRun.

GET /artifacts/{run}/{step}/{name} on the existing handler serves the bytes
with http.ServeContent, so Content-Type, Content-Length, HEAD and Range
(206/416) come from the stdlib; unknown runs/steps, absolute paths,
traversal, symlink escapes and non-regular files are refused.

The TUI lists each selected step's artifacts (name + human size) in the
retained-evidence section and adds an artifacts summary in the style of the
file-changes summary. Adds docs/stage-artifacts.md linked from README and an
artifacts subsection to the packaged operations skill reference.

Tests cover the scan records and caps, missing/oversized directories,
HTTP HEAD/Range/traversal/symlink/unknown refusals and protocol round-trips.
2026-10-15 17:51:18 +01:00
alan e6d00d4ee5 Scope agent session bookkeeping to the run for concurrent runs
The per-workspace concurrency added in 2618f0d exposed three places that assumed a
single run, and the second concurrent run failed with 'agent quiescence not
confirmed' because of them.

- ActiveAgentSessionsForRun(runID) scopes the end-of-step quiescence check to the
  run's own sessions, so another run's legitimately active agent no longer fails
  this one. A run still fails when its own session cannot be confirmed idle.
- The startup guard now uses ActiveSessionOwners (which carries the owning run id)
  and refuses only an ORPHANED session - one whose run is not active now - which is
  what restart recovery is actually about. A session owned by a currently active
  run no longer blocks new work.
- sessionActive examines only the target session: foreign entries in the
  /api/session/active map are ignored, the target's absence is inactive, and the
  target present with an unsupported state still fails closed.

Verified: go build, go vet and go test ./... pass, including ten concurrency tests
(two workspaces run concurrently, same-workspace refusal, symlink and default-cwd
equivalence, idempotent repeats, restart recovery, retry scoping, overlapping runs
not failing quiescence, and a run's own active session still failing it).

Operator verification, which a stage cannot perform because restarting the server
would abort the run doing it: the service was rebuilt and restarted on this
commit, then two runs in separate workspaces (/tmp/ws-c and /tmp/ws-d) executed
concurrently - both reported executing at the same time, a third run in ws-c was
refused naming run 46, and both finished succeeded with no diagnostic.
2026-10-15 15:38:50 +01:00
alan 05ae530442 Make Pipelines Docker-only with OpenCode and Pi
Install current verified harness releases inside Docker, preserve persistent accounts and settings, and provide separate interactive and pipeline model defaults with terminal onboarding and fuzzy pickers.

Add Docker-aware authoring transport, Pi stage execution, bounded workflow controls, reliable final-result readback and usage settlement, and finite short-lived CLI pipe draining. Keep dated installation evidence and packaged authoring guidance consistent.

Verified with a real Docker installation, both harness stages and interactive authors, the built-in plan/implement/review starter, independent host race/vet checks, installer tests, and packaged skill tests. Generated release receipts stay ignored in clean checkouts.
2026-10-11 23:00:25 +01:00
alan c42a19f7d3 Improve workflow UI, retain stage diffs, and add portable pipeline transfers
Make OpenCode authoring responsive and expand the managed skill references. Add conflict-aware import/export, real-provider reference workflows, and tracked V2 generation for reliable budget accounting.
2026-10-10 20:55:42 +01:00
alan a66d540125 Add stage execution limits and result-first workflow views 2026-10-10 03:03:46 +01:00
alan a1c0eb4a72 Handle native structured-format incompatibility and preserve failure evidence 2026-10-10 00:01:02 +01:00
alan 08b8647d5b Validate providers and retain complete run debugging context 2026-10-09 23:21:01 +01:00
alan f60599230c Retain OpenCode sessions and verify fresh retry isolation 2026-10-09 21:16:28 +01:00
alan a50e8648f6 Add service status and usage dashboard, refresh externally published edits 2026-10-09 20:20:44 +01:00
alan 67ac79765f Harden model results with lossless isolated repair and scoped permissions 2026-10-09 20:02:41 +01:00
alan accbe63e39 Simplify Go installation and support standard OpenCode releases 2026-10-08 23:17:38 +01:00
alan 0f30a001c2 Initial pipelines server, CLI, TUI, and Go installation 2026-10-08 22:22:17 +01:00