Files
pipelines/internal/server/AGENTS.md
T
alan e37b864c40 stage turns: replace JSON results with file-based markdown turns
Each model visit now gets <run>/stages/<seq>-<stage>/ with input.md,
memory.md, output.md, private/ and a shared run-level public/. The model
reads the two input files, follows the stage's instructions verbatim and
returns complete markdown; only the # outcome section is parsed.

- add internal/turn layout, render and output parsing
- write input.md/memory.md before a visit and output.md from the final
  message; inventory all turn files as stage artifacts
- accumulate completed turns and system-written public file additions in
  memory, including across retries and resume
- remove the JSON envelope, strict field validation, line format and the
  repair ladder for model stages
- republish pipeline authoring prompts and docs in the new style

Checks: go build ./..., go vet ./..., go test -count=1 ./..., gofmt -l .
2026-10-19 22:27:44 +01:00

53 lines
3.0 KiB
Markdown

# The Pipelines server — working map
Read this before touching `internal/server/`, `internal/stage/`, `internal/pipeline/` or
`internal/protocol/`. The generic half — how a Go service is built, tested and reviewed — belongs to
the pipeline that started you.
## Build and test
```sh
go build ./... && go vet ./...
go test ./internal/server # ~10-15 s
go test ./internal/stage ./internal/pipeline ./internal/protocol
go test ./... # ~1 min for everything; `gofmt -l .` must print nothing
```
## Where things are
- `internal/server/websocket.go` — the `/connect` transport and the request dispatch. Envelope
validation lives here; every command returns its own typed reply.
- `internal/server/runner.go` — run execution: the stage loop, stage outcomes, budgets, retries,
ownership and settlement. This is where a run's state changes.
- `internal/server/runevents.go` — the push hub: subscriptions scoped per workspace, per-run frames,
coalescing. Live clients fold these instead of polling.
- `internal/server/chat_*.go` — the chat broker: `chat_sessions`, claims, the stream, permissions,
session registry.
- `internal/stage/` — one client per harness and stage type: `opencode.go` (native sessions),
`pi.go` (RPC), `model.go`, `agent.go`, `human.go` (decision stages), `ntfy.go` (notifications),
`model_prompt.go` (file-based turn prompt), `budget.go` (usage accounting). Model stages return
markdown for `output.md`; only its `# outcome` section is parsed (`internal/turn`).
- `internal/pipeline/` — the store: definitions and immutable versions, runs and steps, the
operator-session registry, retention, search.
- `internal/protocol/` — every wire shape. A new command means a message field here, documentation in
`docs/cli.md`, and tests both sides.
- `cmd/pipelines-server/main.go` — flags and wiring, including which OpenCode endpoint chat brokers
from (`-opencode-endpoint`, `PIPELINES_OPENCODE_ENDPOINT`, or the recorded choice).
## Checking behaviour without disturbing the live server
`scripts/run-local-server.sh` starts a scratch server, and the binary takes `-addr` and `-db`, so a
live check runs against a **scratch database and a free loopback port**. The operator's deployment is
`~/.local/share/pipelines` with the service on `0.0.0.0:8080` and ntfy on `8090`.
## Hard rules
- **Never restart, rebuild or redeploy the deployed server, ntfy or a running run** to test a change:
a restart kills every run in flight. Deploying is the operator's step.
- Commit locally only, never push.
- One run per workspace at a time — the server refuses a second one rather than sharing a tree.
- The service is durable: sessions, runs and definitions live in SQLite, so a migration must open an
existing database cleanly (see the `migrateRun*`/`addColumnIfMissing` pattern).
- Artifacts are inventoried with bounds (`internal/server/artifacts.go`: file count, per-file and total
size); a large file is served by direct URL, not by growing the inventory.