Files
pipelines/README.md
T

195 lines
8.6 KiB
Markdown

# Pipelines
Small, reusable AI workflows for coding, research and everyday project work,
plus chats with the coding agents you already use.
Pipelines is one per-user server with two clients:
- **The app** (Flutter): served as a web app behind a sign-in, and as an
**Android APK**.
- **CLI and TUI** (`pipelines`) for scripting, inspection and authoring.
A **pipeline** is a versioned Markdown definition of stages (model, human
decision, notification). A **run** executes it in a project directory on the
server machine. A **chat** is a conversation with an agent harness (OpenCode,
Claude Code or Codex) that the server keeps going while you are away.
**Alpha:** single-user, supervised work. Models make mistakes and editing
stages change files and run commands. Start in a disposable project, review
results, and expect provider charges.
![Workflow](docs/images/workflow.png)
## Install
```sh
git clone https://git.a-wan.net/alan/pipelines.git
cd pipelines
./install # server + CLI + web app, login autostart
./install --web # rebuild only the web app after app changes
./install --apk # optional: Android app, offered to phones for update
./install --linux # Linux x64 client archive, built in Docker
./install --macos --flutter /path/to/flutter # native macOS client archive
```
Needs Go (version in `go.mod`), Node.js 18+, Python 3.9+, git, curl, a C
compiler, a **running Docker** (the web app is built in Docker; the install
refuses without it) and an existing OpenCode on `PATH`. The installer never
installs these for you. See [Installation](docs/install.md).
Then open the URL the installer prints (default `http://localhost:8082/`)
and sign in with the password you chose. Put the CLI on your `PATH` as in the
[terminal quick start](#terminal-quick-start).
### Linux quick start (Debian/Ubuntu)
```sh
# Prerequisites: git, curl, a C compiler, Python 3, Node.js 18+ and Docker.
sudo apt update
sudo apt install -y git curl build-essential python3 nodejs npm docker.io
sudo usermod -aG docker "$USER" # then log out and back in (or: newgrp docker)
# Go: the version in go.mod is newer than most distro packages; use go.dev.
curl -fsSL "https://go.dev/dl/$(curl -fsSL 'https://go.dev/VERSION?m=text' | head -1).linux-amd64.tar.gz" \
| sudo tar -C /usr/local -xz
export PATH="/usr/local/go/bin:$PATH"
# OpenCode (required: it runs pipeline stages and chat by default).
curl -fsSL https://opencode.ai/install | bash
git clone https://git.a-wan.net/alan/pipelines.git && cd pipelines
./install
# Keep the server running after you log out (systemd --user service).
sudo loginctl enable-linger "$USER"
```
Check `node --version` is 18 or newer (older distros: install Node from
nodejs.org or NodeSource). On ARM machines use `linux-arm64` in the Go URL.
### Set up harnesses
One command per harness (native or Docker install). Each checks the CLI is
installed (printing the install command if not), signs in with the harness's
own login when needed, picks default models for OpenCode/Pi, and verifies it
the way the server does. No restart needed.
```sh
pipelines harness setup opencode # stages + chat (required; ./install configured it)
pipelines harness setup pi # pipeline stages
pipelines harness setup claude # chat
pipelines harness setup codex # chat
pipelines harness list # status of all four
```
## What you can do
**Runs**
- Runs list with filters, sorting (including largest on disk and most tokens)
and multi-select delete. Each row shows the run's size and reported token
total (`≥` when some stages did not report).
- Run detail: Summary (with size, tokens split fresh versus cached, and the
reported cost), Timeline, and Artifacts and Changes when the run has any.
- Artifacts viewer for files a stage produced, including the recorded session
export (`session.html`), with a full-screen view where HTML pages scroll,
zoom and open their links.
- Changes: the file diffs each stage made in the workspace.
- Human decision stages: a run waits for you; decide in the app or with
`pipelines decide`.
- Retry from the failed stage, cancel, rename, per-run model overrides.
**Pipelines**
- A new install starts with four [example pipelines](examples/README.md),
from one stage to a review loop and a human decision; they are also the
template set.
- Draft-first authoring with a harness, versioned publishing, import/export
as ZIP.
- [Schedules](docs/schedules.md): recurring runs of an existing pipeline.
- Search across pipelines, versions, runs, steps, fields and artifacts.
**Chat**
- Chat with **OpenCode**, **Claude Code** or **Codex** in any folder on the
server, picked with the folder picker.
- Send messages while the agent is working: Claude Code and Codex receive them
mid-turn; otherwise the server queues them durably and delivers them in order
when the turn ends, even with no client attached.
- Approvals, questions, uploads, model choice and interrupt in the app.
- Adopt an existing harness session with `pipelines chat-adopt`.
**Notifications**
- The Android app keeps its own connection and shows notifications in the
background (no ntfy app needed).
- Settings: any run ended, chat needs you, chat reply finished, and
**Also send to ntfy** for devices that still use ntfy.
Pi runs pipeline stages (`harness=pi`) but is not offered as a chat harness.
See [Harnesses](docs/harnesses.md).
## Architecture
```
browser / Android app ──HTTPS (tailscale serve) or HTTP──┐
▼
web proxy (node, 0.0.0.0:8082) ── sign-in, web app, /updates/
│ same-origin /connect, /artifacts/
▼
pipelines CLI/TUI ──ws://127.0.0.1:8080/connect──► pipelines-server (SQLite)
│
┌────────────────────┬───────────────┼───────────────┐
▼ ▼ ▼ ▼
opencode serve pi (RPC) claude (stdio) Codex App Server
stages + chat stages chat chat (socket)
│
▼
ntfy (8090, optional fallback)
```
- **One server process** (`pipelines-server`) owns the database, runs, chats
and notifications. Its API has no authentication and binds to
`127.0.0.1:8080` by default.
- **One web app.** The authenticated Node proxy (`web/serve.mjs`) serves the
installed web build and proxies to the server. Every address (localhost,
LAN, tailnet) reaches this same proxy.
- **Tailnet HTTPS.** When `PIPELINES_WEB_PUBLIC_URL` is an `https://*.ts.net`
address, `./install` runs `tailscale serve` to point it at the proxy.
- **ntfy** is built from source and runs alongside, for optional fallback
delivery and stage notification hooks.
- A login service (launchd on macOS, `systemd --user` on Linux) supervises the
server, web proxy and ntfy together.
## Terminal quick start
```sh
export PATH="$HOME/.local/share/pipelines/bin:$PATH" # add to ~/.bashrc or ~/.zshrc
pipelines status
cd /path/to/your/project
pipelines # TUI
pipelines run "pipeline name" "Describe a small task"
pipelines runs
pipelines inspect 1
pipelines doctor
```
Runs use the directory you start them from, on the server machine. A browser
on another computer does not upload its files.
## Documentation
- [Installation and lifecycle](docs/install.md)
- [Harnesses: OpenCode, Claude Code, Codex, Pi](docs/harnesses.md)
- [Notifications](docs/notifications.md) · [ntfy reference](docs/ntfy.md)
- [CLI and authoring](docs/cli.md)
- [Schedules](docs/schedules.md)
- [Pipeline format](internal/author/skill/references/format.md)
- [Reasoning effort and stage limits](docs/harnesses-and-control.md)
- [Chat backend details](docs/chat-backends.md)
- [Artifacts](docs/stage-artifacts.md) · [File diffs](docs/stage-file-diffs.md)
- [Transfer pipeline definitions](docs/pipeline-transfer.md)
- [Docker installation](docs/docker.md)
- [Web proxy](web/README.md)
- [Alpha walkthrough and known limits](docs/first-version-testing.md)
When reporting a problem, include your OS, install mode, reproduction steps,
`pipelines version --json` and `pipelines ping --json`. Redact credentials and
private content; do not post your database or harness session storage.