Files
pipelines/docs/first-version-testing.md
T
alan c9b904186a Update templates and authoring skill to the current pipeline contract
Bring the packaged starters and the pipelines skill up to the guidelines the
project now works to.

Templates:
- task and three-stage now end in a committer stage that stages the union of the
  runner's captured input.server_changed_files and the producer's declared
  changed_files, runs the repository's checks, commits locally and never pushes;
  their producers no longer commit.
- Every model/agent prompt states the line-oriented result contract (outcome on the
  first non-empty line, "# output" JSON map, other "# <heading>" sections kept
  verbatim as evidence) and an "# evidence" section for exact commands and results,
  with proportionate verification and no fabricated checks.
- Stages that can write save their reports, logs and screenshots into the runtime
  artifacts directory.
- Remove the summary starter: a tool-free single-prompt text transform with no
  repository interaction, artifact or commit is a format demonstration, not a
  reusable workflow. No starter invents a pretend approval stage; a decision point
  is a type=human stage.

Skill:
- references/format.md gains a Result format section (line form, legacy JSON
  precedence, result_parse, raw-output.md, artifacts) and a Committer stages
  section with the pre-commit checklist; the role-split and envelope notes mention
  the line form and the committer.
- orchestration.md points at the checklist and records that a genuine gate is a
  human stage; operations.md and the SKILL.md index drop the removed starter and
  name the result format and committer. No new reference, so the managed inventory
  is unchanged.

Docs/tests: examples/README, README, docs/cli, docs/first-version-testing, the CLI
help example and the smoke script track the four starters; catalog, tool-role,
server, tui and pipeline tests are updated, with a new test pinning the committer
ending and server-captured change set.

Commands run: go build ./...; go vet ./...; go test ./...; python3 -m unittest
discover -s scripts -p 'test_*.py'; python3 scripts/smoke-first-install.py. All
passed. Templates inspected through an isolated CLI server (pipelines templates;
pipelines template KEY --source). No server, service or device was restarted, and
nothing was pushed.
2026-10-17 01:09:42 +01:00

153 lines
8.6 KiB
Markdown

# First-version tester walkthrough
Pipelines is a **single-user preview**, like other personal terminal tools—not a
multi-user service. The goal is useful, supervised workflows with inspectable
history. It is not a guarantee that a model completes every task correctly.
## Before installing
- Use a Git checkout (including `.git`) so Docker setup can stamp its build identity.
- **Native host installation is the default** (`./install`/`--native`); the
[Docker installation](install.md) is explicit with `./install --docker` and needs
Docker Compose and Python 3. Docker setup builds the Go CLI/TUI clients on the host
only when Go 1.27+ is available.
- Docker setup starts services, asks for ntfy passwords, opens provider sign-in,
and then saves **two separate model defaults** (the interactive harness model and
the new-pipeline model) in two fuzzy pickers. Signing in to a provider is
separate from those saved defaults. Setup makes a small billable
provider-validation call and sends a setup alert. Docker provider sign-in is
separate from your host OpenCode login. Reopen login or the saved defaults later
with `./install --opencode` or `./install --pi`.
- Start with a **disposable repository with no valuable files or secrets**. Stage
tools can read/edit files and execute commands. Permissions and snapshots are not
an OS sandbox; an automatically mounted home directory is not isolation.
- The Pipelines API is unauthenticated. Use your own machine/trusted network only;
do not expose it to the Internet. No multi-user setup is required.
## Install and identify the build
From the checkout, run `./install` for the native host installation, or
`./install --docker` for the Docker server + ntfy installation, then restart any
existing TUI. The installer requires a Git checkout; a standalone source archive
cannot self-install.
Follow [installation](install.md) for PATH and provider setup.
Do not start a native server alongside your Docker server.
Host Go is **optional** and only needed for native host CLI/TUI binaries. Without
host Go you can still run the Docker-contained TUI/CLI against the running server.
Run Compose from the checkout with `--project-directory` (so it finds
`compose.yaml`) and `--workdir "$PWD"` (so the container keeps the actual
disposable workspace):
```sh
# TUI (allocate a TTY)
docker compose --project-directory /absolute/path/to/pipelines exec --workdir "$PWD" pipelines pipelines
# Plain CLI command (no TTY)
docker compose --project-directory /absolute/path/to/pipelines exec -T --workdir "$PWD" pipelines pipelines <command>
```
Any host client targets the mapped host port with
`--url ws://127.0.0.1:<actual-mapped-port>/connect`, reading the port from
`docker compose --project-directory /absolute/path/to/pipelines port pipelines 8080`.
The server listens on container port **8080**; only the default host mapping is
**8081**. Prefer the installer-saved configured endpoint when present; never
replace it with a guessed default.
Configuration, provider credentials, drafts, run history and ntfy data live in
Docker volumes and are reused across reinstalls/reconfigures; the visible flow
remains draft → validate → explicit publish → run → inspect. Developer/legacy
native-server instructions stay in the [installation guide](install.md) rather
than competing with first use.
```sh
pipelines version --json
pipelines ping --json
pipelines status --json
```
`version` identifies the client; `pong.build` identifies the running server.
The full commit is the primary identity. `-dirty` means uncommitted source was
included; two dirty builds at the same commit need not contain the same changes.
Missing `build`, `commit: "unknown"` or `dirty: null` is not an identifiable release.
Capture this information before reporting issues. `--url` and `PIPELINES_URL`
override the saved endpoint; check that you are connected to the intended server.
## First supervised run
1. Create a disposable repository, then launch `pipelines` **from that directory**.
Docker must see the same absolute path; repositories on another computer are
not uploaded automatically.
2. Use `:templates` and create the **review** starter (read-only, no repository
change). Review its name and publish explicitly. A fresh database is empty;
templates are never auto-published.
3. Open the pipeline and use `:run` with a small task, such as reviewing one short
file in the disposable repository. Review the execution confirmation.
4. Read the retained report and recorded visits. For an editing workflow, inspect
File changes as well as actual file contents and any verification commands.
Model-written claims alone are not verification.
5. Create/edit a draft with `:new` or `:drafts`; validate it, then choose whether to
publish. Editing alone must not change the published definition.
6. Optionally exercise `:export` and `:import`. Identical definitions skip; differing
existing names offer keep existing, keep both, or use imported as a new version.
7. Quit with `:quit`; `q` intentionally does not quit. Reopen to inspect history.
Phone reception is a separate manual check. Install/subscribe in the ntfy app and
confirm receipt before depending on it. The installer confirms publishing, not
delivery to your phone. A review starter does not send alerts automatically.
## Known boundaries
- Sequential stages within a run; no parallel branches. `type=human` decision
stages pause a run for an operator's choice. Independent runs in different
workspaces do execute concurrently.
- Cancellation/retry is **not rollback**. Retry starts from the entry stage with
fresh sessions and the pinned definition; earlier side effects can repeat.
- Diffs are bounded stage-start/stage-end observations, not full backups or proof
of exclusive authorship. Partial/missing captures must not mean “no edits.”
- ZIP bundles transfer definitions only. They do not back up history, drafts,
repositories, OpenCode credentials/sessions or ntfy storage. Preserve persistent
volumes/data; never use `docker compose down -v` as a routine reset/update.
- Token/USD budgets use reported usage and may overshoot in-flight. Prefer provider
account spending limits too; do not interpret these as a guaranteed bill ceiling.
- Live reference evidence is dated and narrow. An existing real run7
(2026-10-11) executed both configured harnesses, OpenCode 2.0.26 and Pi 1.1.0,
with `opencode-go/deepseek-v4.1-flash` stages, and both steps succeeded
(retained captures under `tmp/friend-ui-verification`); a current-source TUI
inspection of that run recorded actual 80x30 and 120x36 renderings. The earlier
[2026-10-10 OpenCode v2.0.20 trial](reference-workflow-results.md) remains the
previous dated evidence. The dated 2026-10-11 final install then completed the
new-source installer and installed-authoring verification: the runtime was
actually installed and used, with run17 executing both real harness stages (see
[installation verification](installation-verification.md)). Other providers, real
outages/long-stream interruption, fresh OAuth and phone delivery are still
pending and are not established by these runs. See
[dated results](reference-workflow-results.md), not just test pass counts.
- Windows-native installation, screen-reader accessibility and every Linux/macOS
terminal combination are not tested. This is not a stability promise for all hosts.
## Report a useful problem
Include client `version --json`, server `ping --json`, OS/architecture, install mode,
the exact command or key sequence, expected/actual behavior, and a minimal sanitized
definition/input. For a run failure include its ID and relevant inspect evidence.
Describe whether it reproduced in a disposable repository. Redact tokens, credentials,
private paths/prompts, file contents and patches before sharing; inspection output
is not automatically safe to post publicly. Do not export your database or OpenCode
credential/session storage as a bug report.
## Maintainer checks without inference or deployment
```sh
go test -race -count=1 ./...
go vet ./...
python3 -m unittest discover -s scripts -p 'test_*.py'
python3 scripts/smoke-first-install.py
```
The developer-only smoke check builds source binaries and installs the skill into a disposable directory,
starts an isolated loopback server/database, checks build identity, empty initial
state, four starters, draft/validate/publish, identical ZIP import and restart
persistence, then shuts it down and removes only its temporary directory. It does
not call models, send alerts, update your installed clients or deploy Docker services.
It does not replace real-provider, terminal interaction or fresh Docker-install tests.
Live inference remains [explicitly opt-in](reference-workflows.md).