Commit Graph

8 Commits

Author SHA1 Message Date
alan 1f75e4797e Broker chat from an operator's own OpenCode endpoint and adopt existing sessions
Add an opt-in way to broker chat from an operator-supplied OpenCode service
instead of the instance the server spawns for itself, and a way to adopt a
session that already exists on that endpoint so it appears in chat_sessions and
can be continued from the app.

- Endpoint selection: the `-opencode-endpoint` server flag (defaulting to
  `PIPELINES_OPENCODE_ENDPOINT`) opts in; precedence is explicit flag >
  explicit environment > the endpoint recorded on a previous run, with the
  spawned instance as the default. opencode.OpenEndpoint validates the
  reference once (http(s), host, no userinfo/query/fragment/path), detects the
  backend from its own /api/info (v2) or /global/health (v1) contract, and
  requires a streamable text/event-stream event path. It owns no process
  (Close is a no-op) and never calls startServe; every failure names the
  endpoint and disables chat rather than silently falling back. An
  authenticated endpoint takes an optional password only through the private
  file PIPELINES_OPENCODE_ENDPOINT_PASSWORD_FILE, with
  PIPELINES_OPENCODE_ENDPOINT_USERNAME overriding the detected default.
- Persistence: the validated choice is recorded in an additive single-row
  server_settings table so a restart with no flag/environment reuses it; an
  unreachable recorded endpoint disables chat with a clear message and no
  fallback.
- chat_adopt: a new protocol request (ChatAdoptRequest) with handleChatAdopt in
  internal/server/chat.go. It refuses a nil broker/registry/payload and a
  blank or foreign-shaped id before any endpoint call, reads the endpoint via
  the shared Chat.Session (which reuses Sessions/decodeChatSessions), records
  through the existing RecordOperatorSession with the endpoint's directory and
  title verbatim, and echoes the adopted session plus its registry row like
  chat_create. A missing id is refused naming the id and records nothing; an
  endpoint record with an empty directory is refused rather than substituting
  the server's working directory or home.
- CLI: `pipelines chat-adopt <session-id> [--json]` (usage, notes, help
  inventory, dispatch) with docs/cli.md updated under Chat sessions.
- Autostart preserves the endpoint variables (and the password-file reference,
  never an inline secret) via serviceEnvironmentKeys.

Checks before committing:
- Package location: the change is user-facing chat behaviour, so the packaged
  authoring skill reference internal/author/skill/references/operations.md is
  updated in the same change (Brokered OpenCode chat section) with a dated
  verification-evidence entry and the not-deployed state. No new reference file
  is added, so internal/author/install.go's managed inventory is unchanged.
- Packaged templates: this repository packages no template set for this
  behaviour and none references the chat endpoint, so no template needs
  updating.
- Verification: `go build ./...`, `go vet` on the touched packages, `gofmt -l`
  clean and `go test ./...` all pass. The unit tests cover the endpoint
  validation/precedence, the adopt happy path and refusals, the recorded
  endpoint round-trip, the protocol round-trip and the CLI dispatch. The live
  scratch-server check against the operator's real endpoint was not run in this
  change.

Nothing unrelated is staged; docs left stale (README and other shared documents)
are not edited here and are for a single folding writer.
2026-10-17 20:14:11 +01:00
alan 58c6b9b7c6 server: export a finished stage's real session as session.md and session.html
Every OpenCode-harness model stage now writes its real session into its own
step artifacts directory as session.md and session.html, so the existing
bounded artifact listing, the /artifacts/<run>/<step>/<name> serving path and
the artifact scan find and serve them with no special casing, while the stage
keeps its bounded preview in the run record unchanged.

- internal/opencode/export.go adds the one shared native export invocation and
  the exact session-ID/messages validation used by both Connection.Messages and
  the new ExportSession, so the readback path does not diverge. The backend's
  published subcommand is used (v1 `export <id>`, v2 `session export <id>`),
  the source read is capped at 8 MiB, and a larger session is read as a prefix
  and reported truncated instead of failing. boundedOutput.ReadFrom keeps the
  cap effective so overflow and the truncation path are real.
- internal/stage/session_export.go renders markdown and html with a metadata
  header, every role and every part (unknown part types as raw JSON, every
  session text escaped in html), atomic temp+rename writes that never overwrite
  an existing file, a visible truncation banner at both ends when the source or
  the 2 MiB rendered cap cuts it, and an honest 0-message header for an empty
  session rather than fabricated text.
- protocol.SessionExport and RunStep.SessionExport are stored in a new
  step_session_exports table (CREATE TABLE IF NOT EXISTS, so an old database
  gains an empty table with no migration) loaded through the existing main
  SELECT alongside the other per-step evidence and deleted by run retention.
- internal/server/runner.go exports after the post-stage filediff capture and
  before scanArtifacts, with context.WithTimeout(context.WithoutCancel(ctx),
  10s); it never waits for session idleness and a failure or a Pi stage (no
  native export path) records a sanitized diagnostic and continues, never
  changing the stage status, outcome, target, timing or result validation.

Reflected where it belongs: the packaged skill references
(internal/author/skill/references/operations.md and delivery.md) carry the new
Session export rules and the run-artifacts bullet, and docs/stage-artifacts.md
documents the names, the <run>/<step> paths, the served URL, the caps, the
truncation labels, failure isolation and the Pi limitation. The packaged
templates (examples/*.md) only tell a stage to save its own artifacts and say
nothing about the server-written session export, so none needed updating; no
new reference file was added, so install.go's managed inventory is unchanged.

Shared documentation: README.md's per-step artifacts sentence is now stale - it
should name the exported session alongside images, screenshots, reports and
logs. It is deliberately not committed here; the shared documents are folded by
the single documentation writer.

Checks: gofmt -l on the touched packages produced no output; go vet ./...
passed; go test -race -count=1 ./... passed.
2026-10-17 19:40:43 +01:00
alan 18398dfa5b chat: give the operator their own sessions, separate from run sessions
chat_sessions used to list exactly the OpenCode sessions the pipelines own,
filtered through pipeline.SessionOwner (run_steps/run_session_owners), so an
operator-created chat was an accident of the run-ownership table and the
operator's working sessions were mixed with the internals of running pipelines.

Add a first-class operator session:

- internal/pipeline/operator_sessions.go: a dedicated operator_sessions table
  (session_id PK, directory, title, tag, created_at, last_used_at) with
  RecordOperatorSession (upsert preserving created_at), a one-statement
  TouchOperatorSession (no-op for a non-operator id), OperatorSessions bounded to
  128 rows most-recently-used first, and OperatorDirectories (bounded MRU,
  deduped). run_steps/run_session_owners and SessionOwner/OwnedSessions are
  untouched.
- internal/server/chat.go: chat_sessions returns operator sessions only - it
  reads the registry once, intersects it with one broker session-list call, fills
  directory/tag/created/updated from the registry, sets busy for a session held
  by another connection, keeps MRU order, and stays fail-closed (empty list, never
  the endpoint's whole list). A pipeline stage session and an untagged row can
  never be listed. chat_create resolves the directory (empty/'~'/'~/...' ->
  operator home; relative or missing rejected before any OpenCode call), calls
  broker.Create(title, resolved) and records the row with the operator tag.
  chat_directories returns home first then bounded (8) deduped recents. A
  successful prompt touches the operator session without affecting pipeline
  prompts.
- internal/opencode, internal/protocol, internal/server/websocket.go and
  message.go: unscoped session list, Create posts {"title":...,
  "location":{"directory":...}} to POST /api/session, additive protocol fields
  (omitempty) and the ChatDirectories request/response, plus the OperatorRegistry
  option defaulting to the first store. One OpenCode transport per server is
  unchanged.

Docs and references: docs/cli.md gains a "Chat sessions" section documenting the
operator-only fail-closed list, directory defaulting/rejection, chat_directories
(home first, deduped, bounded 8) and why a dedicated request was chosen.
internal/author/skill/references/operations.md's stale "Brokered OpenCode chat"
text (chat_create unbrokered, run_steps filtering, "listed the owned scratch
session" evidence) is rewritten to the operator-session registry, the fail-closed
filter, directory defaulting and the corrected live-test wording. No new
reference file, so internal/author/install.go's inventory is unchanged.

Checked for staleness elsewhere: README.md, the docs/ topic pages and the
packaged server/TUI templates carry no chat-session ownership claims, so none
needed updating.

Verified: gofmt -l clean; go vet and go build ./... clean; go test -count=1
./internal/pipeline/... ./internal/protocol/... ./internal/opencode/...
./internal/server/... and the full go test -count=1 ./... pass.
2026-10-17 11:14:25 +01:00
alan 30c1ed442e chat: broker OpenCode v2 chat over the owned serve SSE endpoint
The installed OpenCode 2.0.20 exposes the authenticated SSE event firehose at
/api/event and the /api HTTP surface, but Detect selects backend "v2" and Open
returned no owned process/URL, so chat stayed disabled and every chat_* request
was answered with "chat is not configured on this server".

Add opencode.OpenStream/OpenChat: for either backend they start the single owned
opencode serve process, discover the loopback URL and verify the backend health
contract (/api/info for v2, /global/health for v1). main and the live test share
OpenChat so the selection cannot drift. Chat now streams real incremental
session.text.delta/session.reasoning.delta frames via the existing
Connection.EventStream + NormalizeChatEvent, with no polling and no fake
streaming.

- Authenticate the owned endpoint with the per-backend user (opencode for v2,
  pipelines for v1); the v2 server ignores OPENCODE_SERVER_USERNAME.
- Send the V2 prompt shape {"text":...}, which the v1 translation also accepts.
- Skip firehose event types the adapter does not surface before decoding their
  data, so session.instructions.updated (delta is an object) no longer fails the
  stream.

Unit tests cover the per-backend user, the prompt shape/auth, the captured real
V2 event shapes and the selection guards. An opt-in live test drives
chat_sessions/chat_messages/a completed chat_prompt/an interrupted chat_prompt
against the installed OpenCode. Update the packaged operations reference with the
realized capability, dated evidence and deployment state.

Checks before commit: the packaged authoring reference
internal/author/skill/references/operations.md is updated in this change; no new
reference file was added, so internal/author/install.go's managed inventory is
unchanged and the internal/author tests pass. No packaged template describes the
brokered chat surface, and neither README.md nor the topic docs under docs/ (nor
web/) document it, so none of them needed updating.
2026-10-17 01:54:14 +01:00
alan 1818a06d2f Restrict chat_sessions to server-owned sessions
chat_sessions now brokers GET /api/session but filters the global list to
the sessions this server recorded in run_steps, deduped by session id, and
fails closed to an empty list when no ownership source is configured. A
session another client holds (in-server claim, released on stream end or
disconnect) or an active run is marked busy; a second client's chat_prompt
for a held session is rejected before any OpenCode call. chat_create stays
unbrokered.
2026-10-16 09:21:50 +01:00
alan 3ff5cd8838 Poll OpenCode permission requests and forward chat_reply 2026-10-16 08:06:39 +01:00
alan 915016df0b Expose OpenCode conversation history in chat_messages
decodeChatMessages only understood the native-serve {info,parts} envelope,
so records from the V2 HTTP API (GET /api/session/{id}/message) decoded
with an empty role and were silently dropped, leaving chat_messages with
empty history. Teach nativeChatMessage to read the V2 top-level shape
(id/type/agent/model/time, parts under "content", user prompt as a bare
string) while keeping the native {info,parts} path via role/id/model/agent/
created accessors and a contentParts fallback.

Add seeded-session evidence: a decoder table test covering both envelopes
(user + assistant with text, reasoning and a completed tool part carrying
input, content and status) plus handler tests that drive chat_messages
through the broker fallthrough and through the real adapter.
2026-10-16 05:24:00 +01:00
alan b8b545accd Stream brokered OpenCode chat replies over /connect
Add the server-side half of the brokered chat path (route B): the
opencode.Chat adapter forwards chat_prompt to POST
/api/session/{id}/prompt through the owned Connection.Call translation
and derives chat_event frames from the authenticated /api/event SSE
firehose (text and reasoning deltas, tool parts with their status
transitions, step and usage). Every pushed frame carries a stable
session and message id, with a last-message-id fallback so step/tool
events that omit the assistant message id stay attributable.

The /connect handler keeps per-session retained state (chatHub) capped at
1 MiB with least-recently-updated eviction at 128 turns, so a
reconnecting client re-syncs through chat_messages instead of replaying
deltas the OpenCode per-session log never resends. Prompt dispatch
subscribes before posting and buffers pre-ack frames to preserve order,
and cancels the stream on disconnect so no goroutine leaks. chat_sessions
and chat_create are declared but not yet brokered: they answer with an
explicit protocol error rather than the generic Pong.

Reuse the existing transport, validation and normalization; no second
OpenCode transport or credential path is introduced.

Tests: protocol golden round-trip, adapter normalization + v1/v2 Call
translation, and handler ack/stream/validation/resume/disconnect, plus a
real-adapter no-tool evidence harness.
2026-10-16 04:09:14 +01:00