Files
pipelines/docs/first-version-testing.md
T

105 lines
4.7 KiB
Markdown

# Alpha tester walkthrough
Pipelines is a **single-user alpha** for supervised workflows, not a multi-user
service or a guarantee that a model's answer is correct.
## Setup
1. Use macOS or Linux with an existing configured harness and the prerequisites
listed in [installation](install.md).
2. From a Git checkout, run `./install`. Native host installation is the default;
Docker is optional with `./install --docker`.
3. Open the web address reported by setup. For terminal use, put the installation's
`bin` directory on PATH and run `pipelines` from your project directory.
4. Setup also starts the authenticated web proxy, server, and ntfy and enables
login startup. See [installation](install.md) for the private web credential
location. The installed CLI's `pipelines service enable` / `service status`
manage that same service; do not start another server manually.
Native setup reuses host harness configuration. Docker has separate persistent
credentials. Never start both installations on the same ports. Do not run the
installer against a shared instance while work is in flight.
The browser proxy requires authentication. The underlying Pipelines/harness
control APIs do not: keep them private and never expose them to the Internet.
Use trusted networks and TLS/VPN as appropriate; authentication is not an OS
sandbox for model tools.
## First supervised run
1. Pick a disposable project without secrets or valuable uncommitted changes.
A remote browser selects a directory on the **server**, not its own computer.
2. Open **Settings → Templates** in the web app or `:templates` in the TUI.
Create the read-only **review** starter first. Fresh databases have no
published pipelines; choosing a template creates one explicitly.
3. Ask it to review one short file. Watch the graph and inspect the retained
report and the stage's **Audit → Session** evidence.
4. Try a research or document starter. Check citations and quoted source content
yourself; a successful stage is not proof of factual accuracy.
5. Only then try an editing starter. Inspect actual changes and test evidence,
not just the model's summary. Keep version control/backups.
Editing a draft does not publish it. Publication creates an immutable version;
runs retain the version they used. Export/import bundles contain **definitions
only**, not credentials, projects, sessions, drafts, or history.
## Sessions and stopping
Model stages settle their exact native session and verify its export before
handoff. An export failure retains computed output and waits for export-only
recovery. **Retry export only** reads that session again; it does not rerun the
stage. A genuinely missing historical conversation cannot be reconstructed by
inventing messages.
Cancellation/retry is **not rollback**. Earlier file edits or commands remain;
a new execution can repeat side effects. Never delete data/volumes to fix a run.
## Notifications
Browser alerts require the page to be open; closed-app web push is not supported.
ntfy is a separate optional channel and can notify your phone without the
Pipelines Android app. Installation verifies publishing, not reception. Check
your subscription and device reception before relying on it. See [ntfy](ntfy.md).
## Known limits
- Sequential stages within a run; independent workspaces can run concurrently.
- Tools can read/edit files and execute commands. Permissions are not an OS sandbox.
- Diffs, inventories, and session exports have explicit bounds. Partial or missing
evidence must not be interpreted as “nothing changed” or a complete capture.
- Usage budgets depend on provider-reported usage and can overshoot in flight.
Set provider spending limits too.
- Native Windows, every terminal/browser combination, fresh provider OAuth, and
phone dispatch are not established by automated macOS checks. Linux installer
mocks are not a substitute for a real Linux host trial.
## Report a problem
Include OS/architecture, native or Docker install, exact reproduction steps,
expected/actual behavior, and the affected run ID. Capture:
```sh
pipelines version --json
pipelines ping --json
pipelines status --json
```
The client and server can be different builds. Redact tokens, private paths,
prompts, files, diffs, and session content before sharing. Do not post your
database or harness credential/session storage.
## Maintainer checks (no live deployment)
```sh
go test ./...
go vet ./...
node --test web/test/*.test.mjs
python3 -m unittest discover -s scripts -p 'test_*.py'
python3 scripts/smoke-first-install.py
```
Use isolated prefixes, loopback ports, and scratch databases. The smoke check
does not run models or send alerts. Browser capture instructions are in
[web verification](../web/README.md). Do not restart the operator's services to
test source changes.