e37b864c40
Each model visit now gets <run>/stages/<seq>-<stage>/ with input.md, memory.md, output.md, private/ and a shared run-level public/. The model reads the two input files, follows the stage's instructions verbatim and returns complete markdown; only the # outcome section is parsed. - add internal/turn layout, render and output parsing - write input.md/memory.md before a visit and output.md from the final message; inventory all turn files as stage artifacts - accumulate completed turns and system-written public file additions in memory, including across retries and resume - remove the JSON envelope, strict field validation, line format and the repair ladder for model stages - republish pipeline authoring prompts and docs in the new style Checks: go build ./..., go vet ./..., go test -count=1 ./..., gofmt -l .
53 lines
3.0 KiB
Markdown
53 lines
3.0 KiB
Markdown
# The Pipelines server — working map
|
|
|
|
Read this before touching `internal/server/`, `internal/stage/`, `internal/pipeline/` or
|
|
`internal/protocol/`. The generic half — how a Go service is built, tested and reviewed — belongs to
|
|
the pipeline that started you.
|
|
|
|
## Build and test
|
|
|
|
```sh
|
|
go build ./... && go vet ./...
|
|
go test ./internal/server # ~10-15 s
|
|
go test ./internal/stage ./internal/pipeline ./internal/protocol
|
|
go test ./... # ~1 min for everything; `gofmt -l .` must print nothing
|
|
```
|
|
|
|
## Where things are
|
|
|
|
- `internal/server/websocket.go` — the `/connect` transport and the request dispatch. Envelope
|
|
validation lives here; every command returns its own typed reply.
|
|
- `internal/server/runner.go` — run execution: the stage loop, stage outcomes, budgets, retries,
|
|
ownership and settlement. This is where a run's state changes.
|
|
- `internal/server/runevents.go` — the push hub: subscriptions scoped per workspace, per-run frames,
|
|
coalescing. Live clients fold these instead of polling.
|
|
- `internal/server/chat_*.go` — the chat broker: `chat_sessions`, claims, the stream, permissions,
|
|
session registry.
|
|
- `internal/stage/` — one client per harness and stage type: `opencode.go` (native sessions),
|
|
`pi.go` (RPC), `model.go`, `agent.go`, `human.go` (decision stages), `ntfy.go` (notifications),
|
|
`model_prompt.go` (file-based turn prompt), `budget.go` (usage accounting). Model stages return
|
|
markdown for `output.md`; only its `# outcome` section is parsed (`internal/turn`).
|
|
- `internal/pipeline/` — the store: definitions and immutable versions, runs and steps, the
|
|
operator-session registry, retention, search.
|
|
- `internal/protocol/` — every wire shape. A new command means a message field here, documentation in
|
|
`docs/cli.md`, and tests both sides.
|
|
- `cmd/pipelines-server/main.go` — flags and wiring, including which OpenCode endpoint chat brokers
|
|
from (`-opencode-endpoint`, `PIPELINES_OPENCODE_ENDPOINT`, or the recorded choice).
|
|
|
|
## Checking behaviour without disturbing the live server
|
|
|
|
`scripts/run-local-server.sh` starts a scratch server, and the binary takes `-addr` and `-db`, so a
|
|
live check runs against a **scratch database and a free loopback port**. The operator's deployment is
|
|
`~/.local/share/pipelines` with the service on `0.0.0.0:8080` and ntfy on `8090`.
|
|
|
|
## Hard rules
|
|
|
|
- **Never restart, rebuild or redeploy the deployed server, ntfy or a running run** to test a change:
|
|
a restart kills every run in flight. Deploying is the operator's step.
|
|
- Commit locally only, never push.
|
|
- One run per workspace at a time — the server refuses a second one rather than sharing a tree.
|
|
- The service is durable: sessions, runs and definitions live in SQLite, so a migration must open an
|
|
existing database cleanly (see the `migrateRun*`/`addColumnIfMissing` pattern).
|
|
- Artifacts are inventoried with bounds (`internal/server/artifacts.go`: file count, per-file and total
|
|
size); a large file is served by direct URL, not by growing the inventory.
|