From a40898f79a85df37c815bece6ca4bacd47ca9ca2 Mon Sep 17 00:00:00 2001 From: Alan Silva Date: Tue, 20 Oct 2026 21:42:34 +0100 Subject: [PATCH] docs(alpha): simplify native setup and verify generic starter creation --- README.md | 471 ++++++---------------------------- docs/first-version-testing.md | 210 ++++++--------- docs/notes/alpha-readiness.md | 47 ++++ examples/catalog_test.go | 2 +- examples/task.md | 6 +- web/PARITY.md | 23 +- web/README.md | 21 +- web/lists.css | 2 +- web/test/browser.mjs | 6 +- web/test/ui-smoke.mjs | 13 + 10 files changed, 272 insertions(+), 529 deletions(-) create mode 100644 docs/notes/alpha-readiness.md diff --git a/README.md b/README.md index c85bfc6f..6c1decd0 100644 --- a/README.md +++ b/README.md @@ -1,418 +1,115 @@ # Pipelines -Configured OpenCode/Pi harnesses, separate model defaults, runner-enforced review -limits and selected run-local context: [guide](docs/harnesses-and-control.md). +Small, reusable AI workflows for coding, research, and everyday project work. +Run them in your browser or terminal using your existing configured harnesses. +Inspect each stage's output, file changes, artifacts, and recorded session. -Reusable, sequential workflows for your repositories, with a Go CLI/TUI, -OpenCode model stages, retained run history, and optional phone notifications. +**Alpha:** single-user, supervised work. Models can make mistakes and editing +workflows can change files and run commands. Start in a disposable project; +review results before trusting them. Provider charges may apply. -**First-version preview:** a single-user tool for supervised work. Start with a -disposable repository, review edits and command evidence, and expect rough edges. -See the [tester walkthrough and known limits](docs/first-version-testing.md). +## Install on macOS or Linux -Stage results include [retained, highlighted workspace diffs](docs/stage-file-diffs.md) -for newly executed visits, alongside their reports and audit evidence. Stages can -also [record and serve their own artifacts](docs/stage-artifacts.md) - images, -screenshots, reports and logs - per step. - -Use `:export` / `:import` to [select and transfer pipeline definitions](docs/pipeline-transfer.md) -between servers as portable ZIP bundles. - -[Real-provider reference workflows](docs/reference-workflows.md) exercise structured -handoffs, file edits/verification, failure routing and reported-usage budgets with -independent acceptance checks and retained evidence. Live inference is opt-in. - -## In action - -Real terminal captures from a live Pipelines instance on 12 October 2026. -The execution views show retained runs from 11 October 2026. - -**Workflow:** plan, implement, review, and UI review, with rejection routes back -to implementation. - -![Workflow graph with planning, implementation, review, and UI review stages](docs/images/workflow.png) - -**Execution:** a completed plan → implement → review run, with the review result -and the executed route highlighted. - -![Successful three-stage execution with retained review output](docs/images/execution.png) - -**Harnesses:** OpenCode and Pi stages in the same successful Docker-backed run. - -![Successful OpenCode and Pi execution with its retained result](docs/images/opencode-pi-execution.png) - -## Install - -`./install` installs Pipelines **natively on the host by default** (macOS and -Linux); `./install --docker` installs the **Docker server + ntfy** explicitly. -Native setup detects (but never installs) Go, python3, git, curl and a C compiler -(`cc`/`clang`/`gcc`), plus an existing `opencode`/`pi`; it builds `pipelines-server`, -`pipelines` and ntfy from source into `~/.local/share/pipelines` (honouring -`PIPELINES_PREFIX`) and generates an ntfy configuration mirroring the Docker -settings. ntfy is compiled locally because upstream publishes no macOS server -binary, and the C compiler is required because its SQLite driver is cgo. Docker -setup needs Docker Compose and Python 3 and includes OpenCode/Pi in the persistent -server container. -For the host CLI/TUI walkthrough below, also install Go 1.27+; the Docker -installer installs/updates those clients when host Go is available. +Use a Git checkout and an already configured OpenCode installation. You also +need Go 1.27+, Python 3.9+, Git, curl, a C compiler, and Node.js 18+. +The installer checks prerequisites; it does not replace your harness or provider +credentials. Pi can also be used when configured. ```sh git clone https://git.a-wan.net/alan/pipelines.git cd pipelines -./install # native host install (Go, python3, git, curl, cc, opencode) -./install --docker # explicit Docker services + container provider login +./install ``` -Native setup reuses your existing host OpenCode login and never installs, -upgrades or downloads OpenCode/Pi. It writes host harness profiles (no Docker -link) for the harnesses it detects, so authoring works immediately, and clears a -stale Docker link when migrating a Docker install. After building, it warns if a -**different** `pipelines` resolves earlier on PATH than `/bin/pipelines`, -naming that path and both versions, so a stale CLI is not left in front of a newer -server (it never modifies or upgrades that other binary). Start it with -`PIPELINES_PREFIX=~/.local/share/pipelines ~/.local/share/pipelines/bin/pipelines start`. -When Docker volumes are present it migrates the existing database and ntfy auth -store (never overwriting newer native data; use `--replace-data` to opt in). See -[installation](docs/install.md) for details. +Native installation is the default: Pipelines runs on your host and reuses your +existing harness setup. Follow the installer's connection details to open the +web app (default `http://localhost:8082/`). Setup creates or reuses its private +web sign-in credentials and enables startup at login. Run as your normal user +in a working macOS launchd or Linux systemd user session. -The installer requires this Git checkout (including `.git`); a standalone source -archive cannot self-install. Without host Go you can still use the -Docker-contained TUI/CLI. Run Compose from the checkout with `--project-directory` -so it finds `compose.yaml`, and pass `--workdir "$PWD"` so the container keeps the -actual (disposable) workspace rather than its own image directory: - -```sh -# TUI (allocate a TTY) -docker compose --project-directory /absolute/path/to/pipelines exec --workdir "$PWD" pipelines pipelines -# Plain CLI command (no TTY) -docker compose --project-directory /absolute/path/to/pipelines exec -T --workdir "$PWD" pipelines pipelines -``` - -Point a host client at the mapped host port with -`--url ws://127.0.0.1:/connect`, reading the port from -`docker compose --project-directory /absolute/path/to/pipelines port pipelines 8080`. -The container listens on port **8080**; only the default host mapping is 8081. -Prefer the installer-saved configured endpoint when present; never replace it with -a guessed default. - -Go uses `GOBIN` or its normal `GOPATH/bin` default; put that directory on PATH. -Running `pipelines` also compares its build/protocol with the server and warns (or -fails on a newer protocol) instead of dropping fields it does not model. The -Docker installer automatically detects/configures LAN networking for ntfy; -you do not need to look up your IP or supply a URL/listen address. -Both Docker services bind `0.0.0.0`; reinstall normalizes those bind settings -without replacing your URLs, timezone, ports or credentials. -The installer builds only CLI/TUI clients on the host when Go is available; -the server and stage harnesses run in Docker. It configures ntfy accounts/token, -checks publishing/health, and opens container OpenCode provider login on first setup; -existing settings, credentials and volumes are retained on reinstall. -Docker provider credentials live in its persistent container volume, separately -from your host OpenCode login; reinstalling or reconfiguring reuses the existing -container accounts and volumes rather than replacing them. Setup asks for ntfy -account passwords, then your model-provider login and models. It selects separate -saved defaults in order: first the interactive harness model, then the -new-pipeline model, in two fuzzy-searchable pickers (the saved value is -preselected and marked `(current)`; typing filters, arrow keys or Tab navigate, -Enter confirms, and Esc cancels without changing anything). Model selection -performs no inference; only the OpenCode path validates the new-pipeline model -with a small real generation (provider charges may apply). Fresh -`./install --docker --harness pi` skips OpenCode login and that generation check. Pi has no -standalone `auth login`: setup opens a fresh `/login` session (regular TUI mode) -where you choose an authentication method, reuse existing volume credentials by -exiting, or sign in with an API key. Each -setup resolves the current supported OpenCode V2 release from the single -documented complete `git/matching-refs/tags/v2` response (no pagination; never the -moving `latest` tag) and the current Pi release from npm, builds `FROM` the -OpenCode image's immutable `sha256` digest, and verifies the built image reports -exactly those executable versions (and a Node-compatible Pi engine). It stops with -a clear error instead of silently substituting a stale pin when discovery, build -or compatibility fails. Use `./install --opencode` or `./install --pi` to -reconfigure login and models inside the existing container without restarting -services: an image matching the release identity needs no rebuild, while a dirty -source always invokes the Docker build and relies on layer caching. The built -image includes python3 so the installer/verification tests can also run inside -the container. -Configuration and data remain per-user. The native default installer is described -above and in [installation](docs/install.md). - -Docker setup starts the server and saves its client endpoint (the container port -is **8080**; only the default host mapping is **8081**). That saved endpoint is -authoritative: prefer it and never replace it with a guessed default. Do **not** -also start `pipelines serve` for this setup. Explicit `--url` and `PIPELINES_URL` -override the saved endpoint; restart existing TUIs after setup. -Docker mounts your home and installation checkout at their original absolute paths -and uses your UID/GID, so the checkout supplies `compose.yaml` while -`--workdir "$PWD"` preserves the actual disposable workspace. Other host -repositories need a bind mount at the same absolute path; repositories on another -client computer are not automatically transferred. -These unauthenticated server ports are for trusted networks only—do not expose -them to the public Internet. - -### Developer/legacy local server (unsupported) - -For developing Pipelines itself or running a previous Python-supervised legacy -host setup, an explicit foreground server remains available. This is not the -native installer and is not part of Docker setup; fresh databases contain no -pipelines: - -```sh -pipelines serve # foreground developer/legacy server -# In another terminal: -pipelines -``` - -Optional developer/legacy autostart at login (starts the server now): +For optional terminal commands, select the installation and add its binaries +to PATH (use your custom prefix if you changed it): ```sh +export PATH="$HOME/.local/share/pipelines/bin:$PATH" pipelines service enable pipelines service status +pipelines doctor ``` -Uses user-level systemd on Linux or launchd on macOS; no sudo. Runs use the -directory you invoke `pipelines run` from. Preview with `--print`. Stop any existing -foreground/legacy server first. Disable with `pipelines service disable` before -uninstalling. See [installation](docs/install.md) for environment and boot options. +See [installation](docs/install.md) for autostart controls, custom ports, data +locations, and troubleshooting. No sudo is needed. Keep the raw +Pipelines and harness APIs private. Use the authenticated web proxy for browser +access; do not expose the control APIs to the Internet. -## Your first pipeline and run - -Start in the repository you want the workflow to operate on: +### Prefer Docker? ```sh -cd /path/to/your/repository +./install --docker +``` + +Docker remains optional, not the default. It uses its own persistent harness +credentials and needs Docker Compose and Python 3. Do not start a second native +server alongside it. [Docker setup and migration](docs/install.md). + +## Try a starter + +In the web app, open **Settings → Templates**, select a starter, and create a +pipeline. In the terminal, run `pipelines` and use `:templates`. + +Choose a small code change, a read-only review, research, or a document utility. +Starters use your task and selected project—not this repository's paths, stack, +or accounts. They inherit configured model defaults. Templates are available +immediately but are never published or run without your choice. + +1. Select the project directory **on the server machine**. +2. Give the pipeline a specific task and run it. +3. Inspect the report, stage visits, and any changed files. + +For terminal use, launch from your project directory: + +```sh +cd /path/to/your/project pipelines -``` - -1. **Create:** press **n** or use **:new**. Choose the configured **OpenCode** or - **Pi** harness to author a workflow together, or **nvim** to edit the Markdown - yourself. Harness authoring runs in the Docker server; only the **nvim** editor - is required on your client computer. Describe the stages, their inputs/outputs, - and the tools they need. Templates (**b** on Home) are optional; a fresh - installation has no published pipelines. - To skip authoring, use **:templates**, choose one of the four generic starters, - and press **s** or Enter on **Create pipeline**. Review the prefilled name - (edit it if needed), then press Enter to create—no separate naming or editor - step. **e Customize** edits it first. Templates include a smaller graph preview; - **Tab** switches focus and space between the description and graph. Use **hjkl** - to pan a focused graph and **n/p** to browse templates. -2. **Keep a draft:** finish/exit the editor. Nothing is published just because you - edited it. Press **d** / **:drafts** to find retained drafts; **e** edits and - **v** validates the selected draft. -3. **Create/update:** press **s** in Drafts. If it has no name, type one into the name - prompt and press Enter. Then review and explicitly confirm submission. The - action is **Create pipeline** for a new definition or **Update pipeline** for - a new version. The server validates it and retains immutable versions; the draft stays. -4. **Run:** select the published pipeline, press Enter to open it, then **:run** to - start a run. Supply your task and review the confirmation. The run uses the - directory from which you launched the client, not the installer checkout. -5. **Inspect:** watch the full graph and stage progress. **Tab** expands/focuses - panes; **n/p** select recorded stage visits; **hjkl** pan a focused graph. - On a failed run, **D** / **:debug** opens a fresh OpenCode debugging conversation; - retry requires confirmation and restarts the pinned workflow, so earlier side - effects may happen again. - -Home shows only **/** search and **:** commands as action hints; **/** opens the omni -popup, which finds pipelines, versions, runs, stages/visits, artifact names and the -text of stage inputs, outputs and evidence, and Enter opens the chosen result. Use -the command palette for creation, drafts, templates, history, status, logs, and -quitting. - -Quit with **:quit**; **q** does not exit. For a terminal-only run after publication: - -```sh -pipelines run "your pipeline name" "Describe the task" -``` - -### Phone notifications - -Open **:status** for the server and ntfy connection details. In the ntfy phone -app, add the displayed ntfy server, sign in as **phone-reader** with the password -you set during installation, and subscribe to the displayed topic. A run that -waits for a decision or fails notifies automatically; to notify on a stage's own -result, add a `notify` block. See [ntfy setup](docs/ntfy.md). Installation checks -publishing, not reception on your phone. - -For Tailscale, both devices must be connected; use -`http://:8090`. Binding to `0.0.0.0` does not bypass firewalls: -UFW/Docker forwarding rules can allow LAN sources but drop Tailscale sources. -See [Tailscale troubleshooting](docs/ntfy.md#tailscale-address-times-out-while-lan-works). - -## Uninstall - -```sh -pipelines uninstall # remove Go binaries; keep history, drafts, and ntfy data -``` - -## A four-stage pipeline - -**Plan → Implement → Review → Commit**. The first stage receives your task prompt -in `input.md`. Every later stage also gets `memory.md`, the accumulated markdown -of completed turns, and `input.md` carries the previous turn's output body (the -free-form section after `# outcome`). One -outcome selects the next stage—no parallel branches. Definitions use Markdown -`prompt`, `tools`, `configs`, and `edges` blocks. The model returns free-form -`output.md` markdown; only its `# outcome` section is interpreted. For example: - -````markdown -# stage planner - -```prompt -Turn input.md into a focused plan. Discover repository conventions and -reuse targets, and research official sources only when external knowledge is -actually required. State why when no research is needed. Return the complete -output.md markdown; the only required section is `# outcome`. -``` - -```tools -read -glob -grep -websearch -webfetch -``` - -```configs -provider=opencode-go -model=deepseek-v4.1-flash -retries=2 -timeout=1h -``` - -```edges -success -> implementer -blocked -> #failure -error -> #failure -``` -```` - -The [complete three-stage example](examples/three-stage.md) adds the implementer, -reviewer and committer: the planner may search/fetch official sources, the -implementer has no web tools and edits and runs checks from the planner's research, -the reviewer independently inspects the real changes with its own shell access, -receiving the original task and the planner-owned acceptance checks rather than the -implementer's summary, and the committer stages the runner's captured changed paths -and commits locally without ever pushing. The reviewer approves only when the -applicable acceptance checks were independently observed passing and no unresolved -substantive issue remains, and otherwise rejects with concrete findings. Review and -explicitly submit it: - -```sh -pipelines validate examples/three-stage.md -pipelines submit examples/three-stage.md --name "three stage" --create-only -pipelines run "three stage" -``` - -Every submission is validated and creates an immutable version. Examples are -optional; nothing is imported automatically. The four stages give a visible -plan → implement → review → commit path using your own accounts and repository; no -maintainer accounts or published definitions are assumed. Run from your -repository directory; stage tools are enabled automatically. Edits support any -language/file type and new files. No file list or typed RUN confirmation is -required. - -## Useful commands - -```sh -pipelines # dashboard; b browses templates -pipelines list -pipelines run # fuzzy selection + multiline prompt -pipelines run "bug fix" "Fix the layout" # run in the current repository -pipelines new "daily digest" "Summarize changes" -pipelines edit "bug fix" "Add a review step" -pipelines drafts --json -pipelines validate /path/to/draft.md -pipelines submit /path/to/draft.md # explicit publication +# After creating a pipeline: +pipelines run "your pipeline name" "Describe a small task" pipelines runs pipelines inspect 1 -pipelines serve # foreground server; Ctrl+C stops it -pipelines ping -pipelines version --json # local client identity (offline) -pipelines ping --json # running server identity -pipelines retention report --keep 200 # show what run-history retention would remove -pipelines retention apply --keep 200 --yes # remove it (artifacts go with their run) -pipelines help edit ``` -Run-history retention is explicit and never automatic: nothing prunes runs or -their artifacts until you run `pipelines retention apply`, protected runs (active, -waiting on a person, or referenced by a retained retry) are kept, and pipeline -definitions and versions are never touched. +A browser on another machine does not upload that machine's project files. +For editing work, use version control and keep a backup. Cancellation and retries +are **not rollback**; a new execution can repeat side effects. -CLI `new`/`edit` open the configured harness (**OpenCode** or **Pi**, native host -by default; a profile is Docker-linked only when a checkout is explicitly linked) -directly with the Pipelines skill. The installed skill asks -missing questions; `pipelines new` needs no preliminary questionnaire. Changes stay -as drafts by default; CLI and TUI share validation and draft storage. Most -commands support `--json` for agents and scripts. +## Sessions and notifications -In the TUI, press **d** (or use **:drafts**) to browse retained drafts, including -CLI-created ones. **e** reopens a draft, **v** validates it, and **s** opens explicit -submission confirmation, asking for a name first if needed. Drafts remain retained -after submission. Drafts are portable: submit to the currently connected server. -When that differs from the origin, submission creates a new pipeline rather than -applying the origin's version number to an existing destination pipeline, and a -notice names both server URLs instead of silently retargeting the draft. +Pipelines records and exports stage sessions itself. A model stage cannot hand +its result to the next stage until its session is settled and its export is +verified. If export fails, the computed output is retained while export-only +recovery waits; this is not a request to rerun the agent or make a human decision. +See [session capture](docs/notes/session-export-handoff-gate.md). -From a run, press **P** / **:pipeline** to open the pipeline version it used. -Initial run input also records `pwd`. Press **r** / **:retry** on a failed run -(or `pipelines retry `) to start a new attempt with the same original -input, directory, and pipeline version. Completed side effects are not undone. -Retries restart at the entry stage with fresh OpenCode sessions, never the previous -conversation or intermediate stage output. Pipelines does not automatically delete -generation, agent, or formatting-repair sessions, including on failure/cancellation. -Primary session IDs are retained in run steps (`pipelines inspect --json`). -Every run also records a short, single-line title: `pipelines run ... --title "…"` -sets one explicitly, otherwise it is generated from the prompt. `pipelines runs` -and `pipelines inspect` show it beside the pipeline name and version, and -`pipelines title "…"` renames it afterwards. -A pipeline may include a `type=human` decision stage: the run pauses as -**waiting** with the stage's instructions and options visible in `inspect` and the -run inspector. Answer it with `pipelines decide [stage] --option KEY -[--feedback "…"]` or by pressing **c** in the run inspector to pick an option and -type feedback. The choice and feedback are recorded as the step's evidence and the -run resumes along that edge. The wait is persisted, so it survives a server -restart; an unknown option is refused with the valid list, a stage that is not -waiting is refused, and a repeated decision does not double-apply. **cancel** and -**retry** keep working while waiting and after a decision. -On Home, **r** opens global run history; **/** opens the omni search popup over -pipelines, versions, runs, stages/visits, artifacts and stage input/output/evidence -text. Inside a pipeline, **r** / **:runs** opens only that pipeline's history. -Quit the TUI with **:quit**; **q** does not exit. **Ctrl+C** remains an interrupt. -Starting/retrying/canceling runs and deleting drafts require confirmation. CLI -publication also prompts; use `--yes` only for explicitly approved automation. -**n / :new** and creating from a template offer a Tab-navigable, fuzzy-searchable -picker: **nvim** opens the draft Markdown file; **OpenCode**/**Pi** author it with -you. Canceling the picker creates no draft and publishes nothing. -Wide terminals show runs and pipelines side by side; narrow terminals stack them. -The TUI uses soft light/dark-adaptive cards, muted metadata, subtle selections, -and local loading animations instead of heavy character borders and inverted bars. -The dashboard includes all-history usage, success rate, average completed duration, -per-pipeline usage, and a 14-day activity chart. CLI: `pipelines stats [--json]`. -Use **:status** for two panels: server addresses and ntfy phone connection details. -CLI: `pipelines status [--json]`. Candidate URLs are not -verified reachable; publishing endpoints and credentials are not disclosed. -Retry is offered only after the failed run and its agent have fully stopped. -Graphs use content-sized boxes/columns and tightly spaced branch lanes while -retaining every stage and labeled box-and-arrow route. Incoming route IDs (left) -and outgoing outcomes (right) share rows to reduce box height. Both pipeline and -run views center the graph horizontally and vertically when it fits. -The run inspector shows the entire pinned graph above the details, dimming stages -and routes not executed. **g** focuses the graph; **hjkl** or arrows pan it. -The path and elapsed duration update while running. **a** opens the active/latest -OpenCode session's recorded tool evidence; this does not attach a live OpenCode UI. -Press **D** / **:debug** (or `pipelines debug `) for a fresh OpenCode -conversation about that run. It uses CLI queries rather than resuming or retrying -the implementing agent. Pipeline **Edit** offers a fuzzy-searchable **nvim / -OpenCode** picker; Tab and Shift+Tab navigate it. -In the draft browser, **x** then **y** deletes the selected retained draft and -metadata. CLI: `pipelines draft-delete `. Published history is untouched. +Notifications have separate channels: -Model results are file-based turn markdown: the model writes free-form `output.md` -and only its `# outcome` section is parsed. There is no JSON envelope, output -schema, field selection or format-repair pass. Retained failures distinguish -invalid turn output from harness transport. Optional stage permissions narrow -file/command access without imposing a language-specific default. See -[model turns and permissions](docs/model-results.md). -The run graph header shows start time and elapsed/final duration, plus finish -time for completed runs. +- **Web alerts:** require the page to be open and browser permission where + supported. Closed-app browser push is not implemented. +- **ntfy:** optional phone notifications, independent of the Pipelines Android + app. Removing that app does not unsubscribe an ntfy client. -More: [CLI](docs/cli.md) · [installation](docs/install.md) · [pipeline format](internal/author/skill/references/format.md). +See [notification setup](docs/ntfy.md). The installer checks publishing, not +actual delivery to your phone. + +## More + +- [Alpha walkthrough and known limits](docs/first-version-testing.md) +- [Installation and autostart](docs/install.md) +- [Configured harnesses and models](docs/harnesses-and-control.md) +- [CLI commands](docs/cli.md) +- [Pipeline format](internal/author/skill/references/format.md) +- [Web client and verification](web/README.md) +- [Artifacts](docs/stage-artifacts.md) · [File diffs](docs/stage-file-diffs.md) +- [Transfer pipeline definitions](docs/pipeline-transfer.md) + +When reporting a problem, include your OS, install mode, reproduction steps, +`pipelines version --json`, and `pipelines ping --json`. Redact credentials and +private task/file content; do not post your database or native session storage. diff --git a/docs/first-version-testing.md b/docs/first-version-testing.md index 25cbdfa4..408f286f 100644 --- a/docs/first-version-testing.md +++ b/docs/first-version-testing.md @@ -1,62 +1,82 @@ -# First-version tester walkthrough +# Alpha tester walkthrough -Pipelines is a **single-user preview**, like other personal terminal tools—not a -multi-user service. The goal is useful, supervised workflows with inspectable -history. It is not a guarantee that a model completes every task correctly. +Pipelines is a **single-user alpha** for supervised workflows, not a multi-user +service or a guarantee that a model's answer is correct. -## Before installing +## Setup -- Use a Git checkout (including `.git`) so Docker setup can stamp its build identity. -- **Native host installation is the default** (`./install`/`--native`); the - [Docker installation](install.md) is explicit with `./install --docker` and needs - Docker Compose and Python 3. Docker setup builds the Go CLI/TUI clients on the host - only when Go 1.27+ is available. -- Docker setup starts services, asks for ntfy passwords, opens provider sign-in, - and then saves **two separate model defaults** (the interactive harness model and - the new-pipeline model) in two fuzzy pickers. Signing in to a provider is - separate from those saved defaults. Setup makes a small billable - provider-validation call and sends a setup alert. Docker provider sign-in is - separate from your host OpenCode login. Reopen login or the saved defaults later - with `./install --opencode` or `./install --pi`. -- Start with a **disposable repository with no valuable files or secrets**. Stage - tools can read/edit files and execute commands. Permissions and snapshots are not - an OS sandbox; an automatically mounted home directory is not isolation. -- The Pipelines API is unauthenticated. Use your own machine/trusted network only; - do not expose it to the Internet. No multi-user setup is required. +1. Use macOS or Linux with an existing configured harness and the prerequisites + listed in [installation](install.md). +2. From a Git checkout, run `./install`. Native host installation is the default; + Docker is optional with `./install --docker`. +3. Open the web address reported by setup. For terminal use, put the installation's + `bin` directory on PATH and run `pipelines` from your project directory. +4. Setup also starts the authenticated web proxy, server, and ntfy and enables + login startup. See [installation](install.md) for the private web credential + location. The installed CLI's `pipelines service enable` / `service status` + manage that same service; do not start another server manually. -## Install and identify the build +Native setup reuses host harness configuration. Docker has separate persistent +credentials. Never start both installations on the same ports. Do not run the +installer against a shared instance while work is in flight. -From the checkout, run `./install` for the native host installation, or -`./install --docker` for the Docker server + ntfy installation, then restart any -existing TUI. The installer requires a Git checkout; a standalone source archive -cannot self-install. -Follow [installation](install.md) for PATH and provider setup. -Do not start a native server alongside your Docker server. +The browser proxy requires authentication. The underlying Pipelines/harness +control APIs do not: keep them private and never expose them to the Internet. +Use trusted networks and TLS/VPN as appropriate; authentication is not an OS +sandbox for model tools. -Host Go is **optional** and only needed for native host CLI/TUI binaries. Without -host Go you can still run the Docker-contained TUI/CLI against the running server. -Run Compose from the checkout with `--project-directory` (so it finds -`compose.yaml`) and `--workdir "$PWD"` (so the container keeps the actual -disposable workspace): +## First supervised run -```sh -# TUI (allocate a TTY) -docker compose --project-directory /absolute/path/to/pipelines exec --workdir "$PWD" pipelines pipelines -# Plain CLI command (no TTY) -docker compose --project-directory /absolute/path/to/pipelines exec -T --workdir "$PWD" pipelines pipelines -``` +1. Pick a disposable project without secrets or valuable uncommitted changes. + A remote browser selects a directory on the **server**, not its own computer. +2. Open **Settings → Templates** in the web app or `:templates` in the TUI. + Create the read-only **review** starter first. Fresh databases have no + published pipelines; choosing a template creates one explicitly. +3. Ask it to review one short file. Watch the graph and inspect the retained + report and the stage's **Audit → Session** evidence. +4. Try a research or document starter. Check citations and quoted source content + yourself; a successful stage is not proof of factual accuracy. +5. Only then try an editing starter. Inspect actual changes and test evidence, + not just the model's summary. Keep version control/backups. -Any host client targets the mapped host port with -`--url ws://127.0.0.1:/connect`, reading the port from -`docker compose --project-directory /absolute/path/to/pipelines port pipelines 8080`. -The server listens on container port **8080**; only the default host mapping is -**8081**. Prefer the installer-saved configured endpoint when present; never -replace it with a guessed default. -Configuration, provider credentials, drafts, run history and ntfy data live in -Docker volumes and are reused across reinstalls/reconfigures; the visible flow -remains draft → validate → explicit publish → run → inspect. Developer/legacy -native-server instructions stay in the [installation guide](install.md) rather -than competing with first use. +Editing a draft does not publish it. Publication creates an immutable version; +runs retain the version they used. Export/import bundles contain **definitions +only**, not credentials, projects, sessions, drafts, or history. + +## Sessions and stopping + +Model stages settle their exact native session and verify its export before +handoff. An export failure retains computed output and waits for export-only +recovery. **Retry export only** reads that session again; it does not rerun the +stage. A genuinely missing historical conversation cannot be reconstructed by +inventing messages. + +Cancellation/retry is **not rollback**. Earlier file edits or commands remain; +a new execution can repeat side effects. Never delete data/volumes to fix a run. + +## Notifications + +Browser alerts require the page to be open; closed-app web push is not supported. +ntfy is a separate optional channel and can notify your phone without the +Pipelines Android app. Installation verifies publishing, not reception. Check +your subscription and device reception before relying on it. See [ntfy](ntfy.md). + +## Known limits + +- Sequential stages within a run; independent workspaces can run concurrently. +- Tools can read/edit files and execute commands. Permissions are not an OS sandbox. +- Diffs, inventories, and session exports have explicit bounds. Partial or missing + evidence must not be interpreted as “nothing changed” or a complete capture. +- Usage budgets depend on provider-reported usage and can overshoot in flight. + Set provider spending limits too. +- Native Windows, every terminal/browser combination, fresh provider OAuth, and + phone dispatch are not established by automated macOS checks. Linux installer + mocks are not a substitute for a real Linux host trial. + +## Report a problem + +Include OS/architecture, native or Docker install, exact reproduction steps, +expected/actual behavior, and the affected run ID. Capture: ```sh pipelines version --json @@ -64,89 +84,21 @@ pipelines ping --json pipelines status --json ``` -`version` identifies the client; `pong.build` identifies the running server. -The full commit is the primary identity. `-dirty` means uncommitted source was -included; two dirty builds at the same commit need not contain the same changes. -Missing `build`, `commit: "unknown"` or `dirty: null` is not an identifiable release. -Capture this information before reporting issues. `--url` and `PIPELINES_URL` -override the saved endpoint; check that you are connected to the intended server. +The client and server can be different builds. Redact tokens, private paths, +prompts, files, diffs, and session content before sharing. Do not post your +database or harness credential/session storage. -## First supervised run - -1. Create a disposable repository, then launch `pipelines` **from that directory**. - Docker must see the same absolute path; repositories on another computer are - not uploaded automatically. -2. Use `:templates` and create the **review** starter (read-only, no repository - change). Review its name and publish explicitly. A fresh database is empty; - templates are never auto-published. -3. Open the pipeline and use `:run` with a small task, such as reviewing one short - file in the disposable repository. Review the execution confirmation. -4. Read the retained report and recorded visits. For an editing workflow, inspect - File changes as well as actual file contents and any verification commands. - Model-written claims alone are not verification. -5. Create/edit a draft with `:new` or `:drafts`; validate it, then choose whether to - publish. Editing alone must not change the published definition. -6. Optionally exercise `:export` and `:import`. Identical definitions skip; differing - existing names offer keep existing, keep both, or use imported as a new version. -7. Quit with `:quit`; `q` intentionally does not quit. Reopen to inspect history. - -Phone reception is a separate manual check. Install/subscribe in the ntfy app and -confirm receipt before depending on it. The installer confirms publishing, not -delivery to your phone. A review starter does not send alerts automatically. - -## Known boundaries - -- Sequential stages within a run; no parallel branches. `type=human` decision - stages pause a run for an operator's choice. Independent runs in different - workspaces do execute concurrently. -- Cancellation/retry is **not rollback**. Retry starts from the entry stage with - fresh sessions and the pinned definition; earlier side effects can repeat. -- Diffs are bounded stage-start/stage-end observations, not full backups or proof - of exclusive authorship. Partial/missing captures must not mean “no edits.” -- ZIP bundles transfer definitions only. They do not back up history, drafts, - repositories, OpenCode credentials/sessions or ntfy storage. Preserve persistent - volumes/data; never use `docker compose down -v` as a routine reset/update. -- Token/USD budgets use reported usage and may overshoot in-flight. Prefer provider - account spending limits too; do not interpret these as a guaranteed bill ceiling. -- Live reference evidence is dated and narrow. An existing real run7 - (2026-10-11) executed both configured harnesses, OpenCode 2.0.26 and Pi 1.1.0, - with `opencode-go/deepseek-v4.1-flash` stages, and both steps succeeded - (retained captures under `tmp/friend-ui-verification`); a current-source TUI - inspection of that run recorded actual 80x30 and 120x36 renderings. The earlier - [2026-10-10 OpenCode v2.0.20 trial](reference-workflow-results.md) remains the - previous dated evidence. The dated 2026-10-11 final install then completed the - new-source installer and installed-authoring verification: the runtime was - actually installed and used, with run17 executing both real harness stages (see - [installation verification](installation-verification.md)). Other providers, real - outages/long-stream interruption, fresh OAuth and phone delivery are still - pending and are not established by these runs. See - [dated results](reference-workflow-results.md), not just test pass counts. -- Windows-native installation, screen-reader accessibility and every Linux/macOS - terminal combination are not tested. This is not a stability promise for all hosts. - -## Report a useful problem - -Include client `version --json`, server `ping --json`, OS/architecture, install mode, -the exact command or key sequence, expected/actual behavior, and a minimal sanitized -definition/input. For a run failure include its ID and relevant inspect evidence. -Describe whether it reproduced in a disposable repository. Redact tokens, credentials, -private paths/prompts, file contents and patches before sharing; inspection output -is not automatically safe to post publicly. Do not export your database or OpenCode -credential/session storage as a bug report. - -## Maintainer checks without inference or deployment +## Maintainer checks (no live deployment) ```sh -go test -race -count=1 ./... +go test ./... go vet ./... +node --test web/test/*.test.mjs python3 -m unittest discover -s scripts -p 'test_*.py' python3 scripts/smoke-first-install.py ``` -The developer-only smoke check builds source binaries and installs the skill into a disposable directory, -starts an isolated loopback server/database, checks build identity, empty initial -state, four starters, draft/validate/publish, identical ZIP import and restart -persistence, then shuts it down and removes only its temporary directory. It does -not call models, send alerts, update your installed clients or deploy Docker services. -It does not replace real-provider, terminal interaction or fresh Docker-install tests. -Live inference remains [explicitly opt-in](reference-workflows.md). +Use isolated prefixes, loopback ports, and scratch databases. The smoke check +does not run models or send alerts. Browser capture instructions are in +[web verification](../web/README.md). Do not restart the operator's services to +test source changes. diff --git a/docs/notes/alpha-readiness.md b/docs/notes/alpha-readiness.md new file mode 100644 index 00000000..91f2b753 --- /dev/null +++ b/docs/notes/alpha-readiness.md @@ -0,0 +1,47 @@ +# Alpha readiness pass — 2026-10-20 + +## Source changes + +- Native macOS/Linux remains the default installer; Docker remains opt-in. +- Native installation now includes the authenticated, dependency-free web client, + Go server and ntfy under one per-user supervisor. Raw API defaults to loopback. +- Native service enable/status/disable select the existing owned installation, + not a separate developer server/database. Existing credentials and valid host + harness defaults are retained; no harness is installed or upgraded. +- Fresh native installs explicitly enable pipeline authoring. Existing missing or + false roles remain execution-only unless the operator chooses + `./install --allow-authoring`. No live role or definition was changed. +- Five primary generic starters: small code change/review, research/fact-check, + summarize documents, compare options, plan/checklist. All five earlier keys + remain available (eight entries total). Models bind from configured defaults. +- README and tester walkthrough now prioritize native browser setup and explain + separate browser/ntfy channels. Closed-app browser push remains unsupported. + +## Evidence + +The scratch lifecycle smoke validates all primary drafts, creates a definition +explicitly, checks definition transfer and verifies restart persistence. It does +not call providers or mutate the operator's database. + +`web/test/ui-smoke.mjs --scratch` exercised actual catalogue reads and template +validation/publication against a current-source loopback server and scratch +database with explicit authoring enabled. All eight entries were present. +Desktop and 390px captures include create dialog, success and resulting list; +card geometry checks catch clipped stage counts and document overflow. +Captures are in the approved OpenCode temporary +`alpha-template-browser-captures/` directory, not committed private evidence. + +Native installer and service tests use isolated prefixes, fake dependencies and +mock launchd/systemd. Docker setup regressions remain separate and opt-in. +Go build/vet, source tests, Python installer tests and browser tests are verification +of source behavior, not proof of a successful operating-system installation. + +## Release boundaries + +No production deployment, service restart, provider login, phone/device test, +or live definition publication was performed. Original histories and branches +were preserved. Native Linux installation, real login autostart, fresh-provider +OAuth and real starter inference still need an operator/tester trial. A Linux +cross-build or systemd mock must not be reported as those checks passing. + +Deployment remains the operator's step under `internal/server/AGENTS.md`. diff --git a/examples/catalog_test.go b/examples/catalog_test.go index dde51cdf..1266263f 100644 --- a/examples/catalog_test.go +++ b/examples/catalog_test.go @@ -93,7 +93,7 @@ func stageTools(t *testing.T, graph pipeline.Graph, key string) ([]string, bool) var starterTools = map[string]map[string][]string{ "task": { "worker": {"read", "glob", "grep", "edit", "shell"}, - "review": {"read", "glob", "grep"}, + "review": {"read", "glob", "grep", "shell"}, }, "review": {"review": {"read", "glob", "grep", "websearch", "webfetch"}}, "research": { diff --git a/examples/task.md b/examples/task.md index ea2d755f..1d7b8db2 100644 --- a/examples/task.md +++ b/examples/task.md @@ -36,8 +36,9 @@ error -> #failure ```prompt Independently review the actual changed files and full diff against the request and project conventions, not just the worker's summary. Check correctness, -regressions, test coverage and unrelated changes. Use read-only inspection; -do not edit, commit, push or deploy. Return a concise Markdown review with +regressions, test coverage and unrelated changes. Inspect status and diffs and +run relevant existing project checks when safe; never install dependencies or +modify source, commit, push or deploy. Return a concise Markdown review with findings, paths, verification evidence and any unverified behavior. Return blocked if there are material findings or insufficient evidence to approve the change. ``` @@ -46,6 +47,7 @@ if there are material findings or insufficient evidence to approve the change. read glob grep +shell ``` ```configs diff --git a/web/PARITY.md b/web/PARITY.md index 80d70092..dbf8b6e8 100644 --- a/web/PARITY.md +++ b/web/PARITY.md @@ -8,6 +8,26 @@ test, or a code change without a capture, is `partial` at best. Status legend: `todo` · `partial` (some of it works) · `done` (working + captured). +### Generic alpha starters (2026-10-20) + +The embedded catalogue now presents small code change/review, research/fact-check, +document summary, option comparison, and plan/checklist starters, alongside the +retained legacy keys. No project path, stack, provider account or model is baked +into these starter definitions. No existing published definitions are changed. + +`test/ui-smoke.mjs --scratch` verified all eight entries and the real +validate/create/list flow against a current-source Go server on a free loopback +port with a scratch database and explicitly enabled authoring. Desktop and 390px +gallery captures check overflow and card metadata containment; grid rows now +keep their content height rather than clipping the stage counts. The browser +driver scrolls instantly before measuring click targets, avoiding stale positions +under CSS smooth scrolling. Captures: approved OpenCode temporary +`alpha-template-browser-captures/51-template-create-dialog.png`, +`52-template-created.png`, `53-pipeline-created-list.png`, +`54-alpha-starter-gallery-desktop.png`, `55-alpha-starter-gallery-mobile.png`. +These checks create no live definitions and perform no inference. Real research +tools, provider outcomes and deployment are not established by this check. + ### Session capture and export recovery (2026-10-20) **Partial against the live-server gate.** The Session tab now distinguishes a @@ -22,7 +42,8 @@ Export-gated waits are nonterminal and show no human decision controls. successful export-only retry at 390px, with sandboxed iframe and unchanged-frame checks. Captures: approved OpenCode temporary `session-audit-captures/`, `01-recorded-identity-export-failed.png` and -`02-complete-repair-refreshes-open-audit.png`. These use fixture session data, +`02-complete-repair-refreshes-open-audit.png`, plus +`03-export-wait-is-not-a-human-decision.png`. These use fixture session data, not a deployed-service claim. Separately, read-only API verification exported all five original run #19 sessions completely (14/11/10/16/11 messages), without rerunning stages. No deployment or live evidence mutation was performed. diff --git a/web/README.md b/web/README.md index 3c3692e1..2509d097 100644 --- a/web/README.md +++ b/web/README.md @@ -13,11 +13,14 @@ what is done; this file covers how to run, test and verify it. headless Chrome against a live server (`test/ui-smoke.mjs`); a unit test alone never marks a parity row done. -## Safety (unchanged from the Android client) +## Safety -- **Trusted LAN/VPN only.** The Pipelines API has **no authentication** and is - **not internet-safe**. Do not expose the server or this page to untrusted - networks. +- **Authenticated browser proxy.** Pages, assets, artifacts, and `/connect` + require a signed session cookie. The proxy refuses to start without its + configured password/secret file. Never disable authentication on a real service. +- **Private control APIs.** Pipelines and harness APIs have **no authentication**. + Keep them on loopback; expose only the authenticated proxy on a trusted LAN/VPN. + Use TLS/VPN for remote access, not an Internet-facing raw API. - **No secrets.** The page stores only the server URL and its own small UI preferences (sort/filter choices, attention dismissals, device-local chat overrides) in `localStorage`; nothing is sent anywhere but the configured @@ -25,7 +28,12 @@ what is done; this file covers how to run, test and verify it. - **No hardcoded server.** The URL is entered by the user, prefilled from the page origin, and can be overridden. -## Run it +## Run it for development + +For normal installation, use [`./install`](../docs/install.md). The following +commands are for a separate development process with an existing web-auth file +(`PIPELINES_WEB_AUTH_FILE`, default `~/.config/pipelines/web-auth.json`). They do +not create an account or configure autostart. ```sh cd web @@ -40,7 +48,8 @@ first time; the endpoint is then remembered and re-opened on later loads. Nothin is baked in. On a deployment where ntfy already uses 8090, pick another port for the page -(e.g. `--port 8095`). The page is a foreground Node process, not a service. +(e.g. `--port 8095`). These commands are foreground processes; managed startup is +described in the [installation guide](../docs/install.md). ## Test it diff --git a/web/lists.css b/web/lists.css index dd91d60b..fe22205f 100644 --- a/web/lists.css +++ b/web/lists.css @@ -226,7 +226,7 @@ .pane-head-compact .pane-head-actions { margin-left: auto; } .gallery { padding-bottom: 8px; } -.cards { display: grid; gap: var(--space-xs); padding: var(--space-s); align-content: start; } +.cards { display: grid; grid-auto-rows: max-content; gap: var(--space-xs); padding: var(--space-s); align-content: start; } .card { display: block; width: 100%; diff --git a/web/test/browser.mjs b/web/test/browser.mjs index 18ff394a..f9cc2e11 100644 --- a/web/test/browser.mjs +++ b/web/test/browser.mjs @@ -246,7 +246,9 @@ export async function launchChrome({ width = 1440, height = 900, chromePath } = return api.evaluate(`(() => { const el = document.querySelector(${JSON.stringify(selector)}); if (!el) return null; - el.scrollIntoView({ block: "center", inline: "center" }); + // CSS smooth scrolling is asynchronous; measuring immediately after + // it can click outside the viewport instead of the target control. + el.scrollIntoView({ block: "center", inline: "center", behavior: "instant" }); const r = el.getBoundingClientRect(); return { x: r.x + r.width / 2, y: r.y + r.height / 2 }; })()`); @@ -267,7 +269,7 @@ export async function launchChrome({ width = 1440, height = 900, chromePath } = const all = [...document.querySelectorAll(${JSON.stringify(selector)})]; const el = all.find((node) => (node.textContent ?? "").includes(wanted)); if (!el) return false; - el.scrollIntoView({ block: "center" }); + el.scrollIntoView({ block: "center", behavior: "instant" }); el.setAttribute("data-test-click", "1"); return true; })()`); diff --git a/web/test/ui-smoke.mjs b/web/test/ui-smoke.mjs index c1084f83..09a34819 100644 --- a/web/test/ui-smoke.mjs +++ b/web/test/ui-smoke.mjs @@ -1453,6 +1453,19 @@ async function runScratchCreate() { await ensureConnected(); await openTemplates(); await page.waitForSelector(".cards button.card", { timeout: 25000 }); + const starterKeys = await page.evaluate(`Array.from(document.querySelectorAll('[data-template-key]'), node => node.dataset.templateKey)`); + for (const key of ["task", "research", "summarize", "compare", "plan", "three-stage", "review", "debug"]) { + if (!starterKeys.includes(key)) throw new Error(`Missing alpha starter ${key}`); + } + await shot("54-alpha-starter-gallery-desktop.png"); + await page.resize(390, 844); + await settle(); + if (await page.evaluate("document.documentElement.scrollWidth > innerWidth")) throw new Error("Starter gallery overflows at 390px"); + const clipped = await page.evaluate(`Array.from(document.querySelectorAll('[data-template-key]')).filter(card => card.querySelector('.card-meta')?.getBoundingClientRect().bottom > card.getBoundingClientRect().bottom).map(card => card.dataset.templateKey)`); + if (clipped.length) throw new Error(`Starter metadata outside cards: ${clipped.join(", ")}`); + await shot("55-alpha-starter-gallery-mobile.png"); + await page.resize(1440, 900); + await settle(); await page.click(".cards button.card"); await page.waitForSelector("#detailTitle", { timeout: 25000 }); await page.waitForSelector(".template-create-open", { timeout: 15000 });