Merge commit 'a6d1492ad8bb88d78bf710f88dc10fa367e1dd7a' into pipelines/run-20261112-0053-ctrl-tab-cycles-floating-chat-windows-4658

This commit is contained in:
2026-11-12 01:05:51 +00:00
9 changed files with 279 additions and 225 deletions
+18 -176
View File
@@ -1,198 +1,40 @@
# Pipelines
Small, reusable AI workflows for coding, research and everyday project work,
plus chats with the coding agents you already use.
Pipelines is one per-user server with two clients:
- **The app** (Flutter): served as a web app behind a sign-in, and as an
**Android APK**.
- **CLI and TUI** (`pipelines`) for scripting, inspection and authoring.
A **pipeline** is a versioned Markdown definition of stages (model, human
decision, notification). A **run** executes it in a project directory on the
server machine. A **chat** is a conversation with an agent harness (OpenCode,
Claude Code or Codex) that the server keeps going while you are away.
**Alpha:** single-user, supervised work. Models make mistakes and editing
stages change files and run commands. Start in a disposable project, review
results, and expect provider charges.
![Workflow](docs/images/workflow.png)
Pipelines is a per-user server and CLI for running reusable AI workflows and chatting with coding agents.
## Install
```sh
git clone https://git.a-wan.net/alan/pipelines.git
cd pipelines
./install # server + CLI + web app, login autostart
./install --web # rebuild only the web app after app changes
./install --apk # optional: Android app, offered to phones for update
./install --linux # Linux x64 client archive, built in Docker
./install --macos --flutter /path/to/flutter # native macOS client archive
```
Needs Go (version in `go.mod`), Node.js 18+, Python 3.9+, git, curl, a C
compiler, a **running Docker** (the web app is built in Docker; the install
refuses without it) and an existing OpenCode on `PATH`. The installer never
installs these for you. See [Installation](docs/install.md).
Then open the URL the installer prints (default `http://localhost:8082/`)
and sign in with the password you chose. Put the CLI on your `PATH` as in the
[terminal quick start](#terminal-quick-start).
### Linux quick start (Debian/Ubuntu)
From this checkout, run:
```sh
# Prerequisites: git, curl, a C compiler, Python 3, Node.js 18+ and Docker.
sudo apt update
sudo apt install -y git curl build-essential python3 nodejs npm docker.io
sudo usermod -aG docker "$USER" # then log out and back in (or: newgrp docker)
# Go: the version in go.mod is newer than most distro packages; use go.dev.
curl -fsSL "https://go.dev/dl/$(curl -fsSL 'https://go.dev/VERSION?m=text' | head -1).linux-amd64.tar.gz" \
| sudo tar -C /usr/local -xz
export PATH="/usr/local/go/bin:$PATH"
# OpenCode (required: it runs pipeline stages and chat by default).
curl -fsSL https://opencode.ai/install | bash
git clone https://git.a-wan.net/alan/pipelines.git && cd pipelines
./install
# Keep the server running after you log out (systemd --user service).
sudo loginctl enable-linger "$USER"
```
Check `node --version` is 18 or newer (older distros: install Node from
nodejs.org or NodeSource). On ARM machines use `linux-arm64` in the Go URL.
### Set up harnesses
One command per harness (native or Docker install). Each checks the CLI is
installed (printing the install command if not), signs in with the harness's
own login when needed, picks default models for OpenCode/Pi, and verifies it
the way the server does. No restart needed.
If `pipelines` is not already on your `PATH`, add it:
```sh
pipelines harness setup opencode # stages + chat (required; ./install configured it)
pipelines harness setup pi # pipeline stages
pipelines harness setup claude # chat
pipelines harness setup codex # chat
pipelines harness list # status of all four
export PATH="$HOME/.local/share/pipelines/bin:$PATH"
```
## What you can do
## Set up harnesses
**Runs**
- Runs list with filters, sorting (including largest on disk and most tokens)
and multi-select delete. Each row shows the run's size and reported token
total (`≥` when some stages did not report).
- Run detail: Summary (with size, tokens split fresh versus cached, and the
reported cost), Timeline, and Artifacts and Changes when the run has any.
- Artifacts viewer for files a stage produced, including the recorded session
export (`session.html`), with a full-screen view where HTML pages scroll,
zoom and open their links.
- Changes: the file diffs each stage made in the workspace.
- Human decision stages: a run waits for you; decide in the app or with
`pipelines decide`.
- Retry from the failed stage, cancel, rename, per-run model overrides.
**Pipelines**
- A new install starts with four [example pipelines](examples/README.md),
from one stage to a review loop and a human decision; they are also the
template set.
- Draft-first authoring with a harness, versioned publishing, import/export
as ZIP.
- [Schedules](docs/schedules.md): recurring runs of an existing pipeline.
- Search across pipelines, versions, runs, steps, fields and artifacts.
**Chat**
- Chat with **OpenCode**, **Claude Code** or **Codex** in any folder on the
server, picked with the folder picker.
- Send messages while the agent is working: Claude Code and Codex receive them
mid-turn; otherwise the server queues them durably and delivers them in order
when the turn ends, even with no client attached.
- Approvals, questions, uploads, model choice and interrupt in the app.
- Adopt an existing harness session with `pipelines chat-adopt`.
**Notifications**
- The Android app keeps its own connection and shows notifications in the
background (no ntfy app needed).
- Settings: any run ended, chat needs you, chat reply finished, and
**Also send to ntfy** for devices that still use ntfy.
Pi runs pipeline stages (`harness=pi`) but is not offered as a chat harness.
Claude Code runs chats and pipeline stages (`harness=claude`).
See [Harnesses](docs/harnesses.md).
## Architecture
```
browser / Android app ──HTTPS (tailscale serve) or HTTP──┐
▼
web proxy (node, 0.0.0.0:8082) ── sign-in, web app, /updates/
│ same-origin /connect, /artifacts/
▼
pipelines CLI/TUI ──ws://127.0.0.1:8080/connect──► pipelines-server (SQLite)
│
┌────────────────────┬───────────────┼───────────────┐
▼ ▼ ▼ ▼
opencode serve pi (RPC) claude (stdio) Codex App Server
stages + chat stages chat chat (socket)
│
▼
ntfy (8090, optional fallback)
```
- **One server process** (`pipelines-server`) owns the database, runs, chats
and notifications. Its API has no authentication and binds to
`127.0.0.1:8080` by default.
- **One web app.** The authenticated Node proxy (`web/serve.mjs`) serves the
installed web build and proxies to the server. Every address (localhost,
LAN, tailnet) reaches this same proxy.
- **Tailnet HTTPS.** When `PIPELINES_WEB_PUBLIC_URL` is an `https://*.ts.net`
address, `./install` runs `tailscale serve` to point it at the proxy.
- **ntfy** is built from source and runs alongside, for optional fallback
delivery and stage notification hooks.
- A login service (launchd on macOS, `systemd --user` on Linux) supervises the
server, web proxy and ntfy together.
## Terminal quick start
Run the command for each harness you want to use:
```sh
export PATH="$HOME/.local/share/pipelines/bin:$PATH" # add to ~/.bashrc or ~/.zshrc
pipelines status
cd /path/to/your/project
pipelines # TUI
pipelines run "pipeline name" "Describe a small task"
pipelines runs
pipelines inspect 1
pipelines doctor
pipelines harness setup claude
pipelines harness setup codex
pipelines harness setup opencode
pipelines harness setup pi
```
Runs use the directory you start them from, on the server machine. A browser
on another computer does not upload its files.
## Update
## Documentation
```sh
pipelines update
```
- [Installation and lifecycle](docs/install.md)
- [Harnesses: OpenCode, Claude Code, Codex, Pi](docs/harnesses.md)
- [Notifications](docs/notifications.md) · [ntfy reference](docs/ntfy.md)
- [CLI and authoring](docs/cli.md)
- [Schedules](docs/schedules.md)
- [Pipeline format](internal/author/skill/references/format.md)
- [Reasoning effort and stage limits](docs/harnesses-and-control.md)
- [Chat backend details](docs/chat-backends.md)
- [Artifacts](docs/stage-artifacts.md) · [File diffs](docs/stage-file-diffs.md)
- [Transfer pipeline definitions](docs/pipeline-transfer.md)
- [Docker installation](docs/docker.md)
- [Web proxy](web/README.md)
- [Alpha walkthrough and known limits](docs/first-version-testing.md)
## Uninstall
When reporting a problem, include your OS, install mode, reproduction steps,
`pipelines version --json` and `pipelines ping --json`. Redact credentials and
private content; do not post your database or harness session storage.
To completely reset Pipelines, including its password and data, run `./uninstall`
from this checkout, then `./install` for a fresh setup. The checkout is retained.
```sh
./uninstall
```
@@ -0,0 +1,48 @@
import 'package:flutter/material.dart';
import 'chat_center_stage_layout.dart';
/// Hairline thickness of a center-stage edge.
const double chatCenterStageEdgeThickness = 1;
/// Where the stage's two hairlines sit: each in the gap beside the stage, or on
/// the workspace edge when the stage reaches it. Returns (left, right).
(Rect, Rect) chatCenterStageEdgeRects(Rect stage, Size size) {
final half = chatCenterStageGap / 2;
const t = chatCenterStageEdgeThickness;
final maxX = size.width - t;
final leftX = stage.left <= 0
? 0.0
: (stage.left - half - t / 2).clamp(0.0, maxX).toDouble();
final rightX = stage.right >= size.width
? maxX
: (stage.right + half - t / 2).clamp(0.0, maxX).toDouble();
return (
Rect.fromLTWH(leftX, 0, t, size.height),
Rect.fromLTWH(rightX, 0, t, size.height),
);
}
/// The stage's left and right hairlines in the workspace's stack, drawn in the
/// theme's outlineVariant so the centred chat column has a visible boundary.
/// Only center stage calls this; the split layout keeps its own dividers.
List<Widget> chatCenterStageEdges({
required BuildContext context,
required Rect rect,
required Size size,
}) {
final color = Theme.of(context).colorScheme.outlineVariant;
final (left, right) = chatCenterStageEdgeRects(rect, size);
return [
Positioned.fromRect(
key: const ValueKey('chat-center-stage-edge-left'),
rect: left,
child: ColoredBox(color: color),
),
Positioned.fromRect(
key: const ValueKey('chat-center-stage-edge-right'),
rect: right,
child: ColoredBox(color: color),
),
];
}
@@ -1,4 +1,5 @@
import 'package:flutter/material.dart';
import 'chat_center_stage_edges.dart';
import 'chat_center_stage_motion.dart';
import 'chat_center_stage_strip.dart';
import 'chat_mobile_panes.dart';
@@ -138,6 +139,12 @@ Widget chatWorkspaceSurface({
for (final pane in livePanes)
if (pane.key == ValueKey('chat-workspace-${motion.topPaneId}'))
pane,
if (stage.stage case final layout?)
...chatCenterStageEdges(
context: context,
rect: stage.rectFor(layout.stageId)!,
size: size,
),
if (mobile?.peekId != null && !motion.isAnimating)
Positioned(
top: mobile!.contentHeight - 12,
+40
View File
@@ -0,0 +1,40 @@
# Center stage column edges
## Change
- In center stage (Alt+E / Option+E), the centred chat column had no visible
boundary: its neighbours are strips or the workspace background, so the
column's end read as unfinished.
- Added two 1px `outlineVariant` hairlines (`ChatCenterStageEdges` in
`app/lib/features/chat/workspace/chat_center_stage_edges.dart`). Each sits
centred in the 12px gap beside the stage, or on the workspace edge when the
stage reaches it.
- Painted from `chat_workspace_surface.dart`, only while a stage layout exists.
The split layout, phone/collapsed layouts and single-chat workspaces draw
nothing new.
- Reading width is unchanged (stage stays `chatCenterStageWidth`, 950 at the
test size). The lines travel with the stage's animated rectangle.
## Focused results
- `tool/memwatch/full.sh 'test/features/chat/workspace/chat_center_stage_test.dart'`
→ PASS: 21 passed, 0 failed, 0 skipped; peak 893 MB; 8s; exit 0.
New tests among them:
- `the stage edges sit in the gaps beside the stage, or on the workspace edge`
- `center stage draws the column edges in the light theme, only on stage`
- `center stage draws the column edges in the dark theme, only on stage`
- `tool/analyze.sh` → clean, exit 0.
- Not run: other `test/features/chat/workspace/*` files use `ChatWorkspace`
without `centerStage` on stage, so the new branch is not reached there. The
land gate runs the suites.
## Affected states (code-level only)
- Center stage on, 3 panes, light and dark: both hairlines present, colour is
the theme's `outlineVariant`.
- Center stage off: no hairlines (asserted in both themes).
- Stage with a neighbour on each side (gaps) and stage at a workspace edge
(line on the edge) are both covered by the geometry test.
## Limitations
- No browser or device capture was made; `ui-review` owns visual evidence.
Visual weight of a 1px `outlineVariant` line in light and dark is unverified.
- Hairlines hide at once when leaving center stage (same as the split
dividers); they do not fade.
@@ -3,6 +3,7 @@ import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:pipelines_app/features/chat/workspace/chat_center_stage_controller.dart';
import 'package:pipelines_app/features/chat/workspace/chat_center_stage_edges.dart';
import 'package:pipelines_app/features/chat/workspace/chat_center_stage_keys.dart';
import 'package:pipelines_app/features/chat/workspace/chat_center_stage_layout.dart';
import 'package:pipelines_app/features/chat/workspace/chat_center_stage_strip.dart';
@@ -82,7 +83,10 @@ Widget host(
List<KeyEvent>? fieldKeys,
bool actionBars = false,
bool filesProbe = false,
ThemeMode themeMode = ThemeMode.light,
}) => MaterialApp(
themeMode: themeMode,
darkTheme: ThemeData.dark(),
home: MediaQuery(
data: MediaQueryData(disableAnimations: reduceMotion),
child: Scaffold(
@@ -410,6 +414,71 @@ void main() {
},
);
test(
'the stage edges sit in the gaps beside the stage, or on the workspace edge',
() {
final (left, right) = chatCenterStageEdgeRects(
const Rect.fromLTWH(418, 0, 950, 800),
const Size(1368, 800),
);
expect(left, const Rect.fromLTWH(411.5, 0, 1, 800));
expect(right, const Rect.fromLTWH(1367, 0, 1, 800));
// A stage that reaches the workspace's right edge keeps its hairline inside.
final (edgeLeft, edgeRight) = chatCenterStageEdgeRects(
const Rect.fromLTWH(650, 0, 950, 800),
const Size(1600, 800),
);
expect(edgeLeft, const Rect.fromLTWH(643.5, 0, 1, 800));
expect(edgeRight, const Rect.fromLTWH(1599, 0, 1, 800));
},
);
for (final mode in [ThemeMode.light, ThemeMode.dark]) {
testWidgets(
'center stage draws the column edges in the ${mode.name} theme, only on stage',
(tester) async {
wide(tester);
final workspace = threePanes();
final stage = ChatCenterStageController(store: _MemoryStore());
addTearDown(workspace.dispose);
addTearDown(stage.dispose);
await tester.pumpWidget(host(workspace, stage, themeMode: mode));
await tester.pumpAndSettle();
const leftKey = ValueKey('chat-center-stage-edge-left');
const rightKey = ValueKey('chat-center-stage-edge-right');
expect(find.byKey(leftKey), findsNothing);
expect(find.byKey(rightKey), findsNothing);
stage.value = true;
await tester.pumpAndSettle();
final outline = Theme.of(
tester.element(find.byType(ChatWorkspace)),
).colorScheme.outlineVariant;
for (final key in [leftKey, rightKey]) {
expect(
tester
.widget<ColoredBox>(
find.descendant(
of: find.byKey(key),
matching: find.byType(ColoredBox),
),
)
.color,
outline,
);
}
// The hairlines frame the stage; the reading width is unchanged.
expect(widthOf(tester, 'three'), 950);
stage.value = false;
await tester.pumpAndSettle();
expect(find.byKey(leftKey), findsNothing);
expect(find.byKey(rightKey), findsNothing);
},
);
}
testWidgets('reduced motion stages at once', (tester) async {
wide(tester);
final workspace = threePanes();
Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

+5
View File
@@ -48,6 +48,7 @@ SKIPPED = {"build", ".dart_tool", ".gradle", ".cxx", "local.properties", ".DS_St
# generated cannot be mistaken for complete web releases.
WEB_CACHE_SCHEMA = "web-release-metadata-v1"
WEB_RELEASE_SCHEMA = 1
VERBOSE = False
def flutter_version(app=APP):
@@ -103,6 +104,7 @@ def web_cache_complete(entry):
def docker(runner, *args, quiet=False):
quiet = quiet or not VERBOSE
result = runner(["docker", *args], capture_output=quiet, text=True)
if result.returncode:
detail = (result.stderr or "").strip().splitlines()[-1:] if quiet else []
@@ -410,12 +412,15 @@ def main(argv=None):
parser.add_argument("--publish", action="store_true", help="Publish desktop archive and update feed to this installation")
parser.add_argument("--output", type=Path, help="Desktop archive output directory (default: build cache)")
parser.add_argument("--flutter", help="Native Flutter executable for macOS and Android (or PIPELINES_FLUTTER/PATH)")
parser.add_argument("--verbose", action="store_true", help="Show full build and tool output")
build_mode = parser.add_mutually_exclusive_group()
build_mode.add_argument("--native", dest="native", action="store_true", default=None,
help="APK: build on this machine with the host Flutter and Android SDK")
build_mode.add_argument("--docker", dest="native", action="store_false",
help="APK: build in Docker even when a host toolchain exists")
args = parser.parse_args(argv)
global VERBOSE
VERBOSE = args.verbose
if args.kind in ("linux", "macos"):
spec = importlib.util.spec_from_file_location("pipelines_desktop_build", Path(__file__).with_name("app_desktop_build.py"))
desktop = importlib.util.module_from_spec(spec)
+71 -49
View File
@@ -72,6 +72,29 @@ class Setup:
self.popen = popen
self.resolver = resolver or release_lookup.resolve_ntfy
self.environ = environ if environ is not None else os.environ
self.verbose = False
self.install_log = None
def command(self, arguments, **kwargs):
"""Run an installer command, keeping tool output out of normal output."""
if self.verbose:
return self.runner(arguments, **kwargs)
kwargs = dict(kwargs)
kwargs.update(capture_output=True, text=True)
result = self.runner(arguments, **kwargs)
if self.install_log:
self.install_log.parent.mkdir(parents=True, exist_ok=True)
with self.install_log.open("a") as log:
log.write(f"$ {shlex.join(map(str, arguments))}\n")
if result.stdout:
log.write(result.stdout)
if result.stderr:
log.write(result.stderr)
log.write(f"[exit {result.returncode}]\n")
return result
def step(self, text):
print(f"• {text}")
# --- detection -------------------------------------------------------
@@ -261,7 +284,7 @@ class Setup:
for field, flag in (("model", "--model"), ("pipeline_model", "--pipeline-model")):
if self._valid_model(current.get(field)):
command.extend([flag, current[field]])
result = self.runner(command, cwd=str(Path.home()), env=dict(self.environ))
result = self.command(command, cwd=str(Path.home()), env=dict(self.environ))
if result.returncode:
raise ValueError("Model selection did not complete; native setup is not ready. Rerun ./install or supply --model PROVIDER/MODEL.")
return self.configure_harnesses(harness, model, allow_picker=False)
@@ -340,7 +363,7 @@ class Setup:
# Tool-manager shims dispatch by their invoked name (for example Volta).
# Resolving the symlink would run volta-shim instead of node.
node = str(Path(shutil.which("node")).absolute())
version = self.runner([node, "--version"], capture_output=True, text=True)
version = self.command([node, "--version"], capture_output=True, text=True)
try:
major = int(version.stdout.strip().removeprefix("v").split(".")[0])
except (ValueError, AttributeError):
@@ -425,7 +448,7 @@ class Setup:
raise ValueError("Docker is not running, so the web app cannot be built. The server is never "
"installed without it; start Docker and rerun ./install.")
try:
built = app_build.build("web", runner=self.runner,
built = app_build.build("web", runner=self.command,
arguments=app_build.update_signing.dart_defines(self.environ))
except RuntimeError as error:
raise ValueError(f"The web app build failed ({error}); nothing was installed") from None
@@ -453,7 +476,7 @@ class Setup:
plist = Path.home() / f"Library/LaunchAgents/{self.LEGACY_WEB_LABEL}.plist"
if not plist.is_file():
return
self.runner(["launchctl", "bootout", f"gui/{os.getuid()}/{self.LEGACY_WEB_LABEL}"],
self.command(["launchctl", "bootout", f"gui/{os.getuid()}/{self.LEGACY_WEB_LABEL}"],
capture_output=True, text=True)
plist.rename(plist.with_name(plist.name + ".disabled"))
print(f"Retired the old standalone web agent {self.LEGACY_WEB_LABEL} (kept as {plist.name}.disabled).")
@@ -468,7 +491,7 @@ class Setup:
if not tailscale:
raise ValueError(f"The public web URL {public_url} is a tailnet address but tailscale is not on PATH")
port = parsed.port or 443
result = self.runner([tailscale, "serve", "--bg", f"--https={port}", f"http://127.0.0.1:{web_port}"],
result = self.command([tailscale, "serve", "--bg", f"--https={port}", f"http://127.0.0.1:{web_port}"],
capture_output=True, text=True)
if result.returncode:
raise ValueError(f"tailscale serve could not point {public_url} at the web app: "
@@ -488,7 +511,7 @@ class Setup:
shutil.rmtree(staging, ignore_errors=True)
staging.mkdir(parents=True)
for command in self.COMMANDS:
result = self.runner(
result = self.command(
["go", "build", "-trimpath", "-o", str(staging / command), "./cmd/" + command],
cwd=str(self.directory))
if result.returncode:
@@ -557,7 +580,7 @@ class Setup:
return f"{version} (commit {commit})" if commit else str(version)
def install_skill(self, prefix):
result = self.runner([str(prefix / "bin/pipelines"), "skill", "install"], cwd=str(self.directory))
result = self.command([str(prefix / "bin/pipelines"), "skill", "install"], cwd=str(self.directory))
if result.returncode:
raise ValueError("pipelines skill install failed; the native install is incomplete")
@@ -588,10 +611,10 @@ class Setup:
return path
self.require_compiler()
pinned = f"{module}@{version}"
result = self.runner(["go", "mod", "download", pinned], cwd=str(self.directory))
result = self.command(["go", "mod", "download", pinned], cwd=str(self.directory))
if result.returncode:
raise ValueError(f"go mod download {pinned} failed; the native install is incomplete")
cache = self.runner(["go", "env", "GOMODCACHE"], cwd=str(self.directory),
cache = self.command(["go", "env", "GOMODCACHE"], cwd=str(self.directory),
capture_output=True, text=True)
if cache.returncode or not (cache.stdout or "").strip():
raise ValueError("Could not locate the Go module cache to build ntfy from")
@@ -606,7 +629,7 @@ class Setup:
shutil.copytree(source, checkout, symlinks=True)
# The module cache is read-only and carries no built web assets; the
# upstream Makefile creates the same placeholders for local builds.
self.runner(["chmod", "-R", "u+w", str(checkout)])
self.command(["chmod", "-R", "u+w", str(checkout)])
(checkout / "server/docs").mkdir(parents=True, exist_ok=True)
(checkout / "server/site").mkdir(parents=True, exist_ok=True)
(checkout / "server/docs/index.html").touch()
@@ -614,7 +637,7 @@ class Setup:
built = Path(staging) / "ntfy"
env = dict(self.environ)
env["CGO_ENABLED"] = "1"
build = self.runner(
build = self.command(
["go", "build", "-trimpath", "-tags", NTFY_BUILD_TAGS,
"-ldflags", f"-s -w -X main.version={version}", "-o", str(built), "."],
cwd=str(checkout), env=env)
@@ -670,7 +693,7 @@ class Setup:
if password is not None:
# `ntfy user add` takes the password from here instead of a prompt.
env["NTFY_PASSWORD"] = password
result = self.runner([str(prefix / "bin/ntfy"), *args], capture_output=capture, text=True, env=env)
result = self.command([str(prefix / "bin/ntfy"), *args], capture_output=capture, text=True, env=env)
if result.returncode:
step = " ".join(args[:2])
raise ValueError(f"ntfy {step} failed; the native install is incomplete")
@@ -726,7 +749,7 @@ class Setup:
def docker_volumes(self):
try:
result = self.runner(["docker", "volume", "ls", "--format", "{{.Name}}"],
result = self.command(["docker", "volume", "ls", "--format", "{{.Name}}"],
capture_output=True, text=True)
except FileNotFoundError:
# No Docker CLI at all: there is no Docker installation to migrate.
@@ -743,7 +766,7 @@ class Setup:
return tuple(name for name in ("pipelines_pipelines-data", "pipelines_ntfy-data") if name in names)
def _migration_image(self):
result = self.runner(["docker", "image", "ls", "--format", "{{.Repository}}:{{.Tag}}"],
result = self.command(["docker", "image", "ls", "--format", "{{.Repository}}:{{.Tag}}"],
capture_output=True, text=True)
if result.returncode:
return None
@@ -759,12 +782,12 @@ class Setup:
print(f"Keeping existing native {destination.name}; not overwriting it during migration.")
return False
name = f"pipelines-migrate-{os.getpid()}"
created = self.runner(["docker", "create", "--name", name, "--volume", f"{volume}:/mnt:ro", image],
created = self.command(["docker", "create", "--name", name, "--volume", f"{volume}:/mnt:ro", image],
capture_output=True, text=True)
if created.returncode:
raise ValueError(f"Could not open Docker volume {volume} for migration; install aborted")
try:
result = self.runner(["docker", "cp", f"{name}:/mnt/{inner}", str(destination)],
result = self.command(["docker", "cp", f"{name}:/mnt/{inner}", str(destination)],
capture_output=True, text=True)
if result.returncode:
raise ValueError(f"Could not copy {inner} from Docker volume {volume}; install aborted")
@@ -774,12 +797,12 @@ class Setup:
for suffix in ("-wal",):
sidecar = Path(str(destination) + suffix)
sidecar.unlink(missing_ok=True)
copy = self.runner(["docker", "cp", f"{name}:/mnt/{inner}{suffix}", str(sidecar)],
copy = self.command(["docker", "cp", f"{name}:/mnt/{inner}{suffix}", str(sidecar)],
capture_output=True, text=True)
if copy.returncode or not sidecar.exists() or sidecar.stat().st_size == 0:
sidecar.unlink(missing_ok=True)
finally:
self.runner(["docker", "rm", "-f", name], capture_output=True, text=True, check=False)
self.command(["docker", "rm", "-f", name], capture_output=True, text=True, check=False)
return True
def migrate_docker(self, prefix, replace=False):
@@ -871,9 +894,12 @@ class Setup:
resolution = self.resolve_ntfy()
# All or nothing: the web app and the server are both built before
# either is installed, and a failed build leaves the old pair running.
self.step("Building the web app")
web_build = self.build_app_web()
self.prepare_layout(prefix)
self.step("Building Pipelines")
self.build(prefix)
self.step("Configuring the installation")
web = self.configure_web(prefix, args, web_build)
self.install_binaries(prefix)
self.check_path_shadow(prefix)
@@ -909,7 +935,7 @@ class Setup:
if self.environ.get(key):
opencode_env[key] = self.environ[key]
opencode_env["PIPELINES_DEFAULT_MODEL"] = args.model or pipeline_model
web_url = f"http://{web_access.default_url_host(web, lan_ip, web_access.tailnet_ipv4(self.runner))}:{args.web_port}"
web_url = f"http://{web_access.default_url_host(web, lan_ip, web_access.tailnet_ipv4(self.command))}:{args.web_port}"
opencode_env["PIPELINES_WEB_PUBLIC_URL"] = web_public_url or web_url
path_value = str(prefix / "bin") + os.pathsep + self.environ.get("PATH", "")
manifest = {
@@ -949,14 +975,9 @@ class Setup:
# Docker address.
endpoint = f"ws://127.0.0.1:{args.port}/connect"
client_path = configure.configure_client(endpoint, self.environ)
print(f"Saved native client default: {endpoint} ({client_path}). Restart the TUI.")
override = self.environ.get("PIPELINES_URL", "")
if override and override != endpoint:
print("WARNING: PIPELINES_URL overrides the saved native address; unset it or pass --url explicitly.")
print(f"Native Pipelines installed at {prefix}."
f" ntfy {resolution.get('tag') or resolution.get('version')} was built from source.")
self.step("Starting the services")
# Parity with the Docker installer: leave the server and ntfy running.
started = self.runner([python, str(prefix / "lib/manage.py"), "start", "--prefix", str(prefix)])
started = self.command([python, str(prefix / "lib/manage.py"), "start", "--prefix", str(prefix)])
if started.returncode:
raise ValueError(
"The native services were installed but could not be started; run "
@@ -965,29 +986,18 @@ class Setup:
f"{args.port}, {args.ntfy_port} and {args.web_port} are free (a running Docker installation may use them).")
self.retire_legacy_web()
self.serve_on_tailnet(opencode_env["PIPELINES_WEB_PUBLIC_URL"], args.web_port)
print(f"ntfy is configured for {base_url}; server, web proxy and ntfy have login autostart ({service_path}).")
print("Phone notifications: subscribe to the 'pipelines' topic as phone-reader with the web password.")
print(f"Web: {web_url}/ (sign-in password in {web['auth_file']}; existing credentials retained).")
print("Web proxy access: " + ("loopback + tailnet" + (" + LAN" if args.web_lan else "")
if args.web_host == web_access.TAILNET else f"bound to {args.web_host}"))
for note in (*web_warnings, web_access.public_url_warning(opencode_env["PIPELINES_WEB_PUBLIC_URL"], web),
web_access.firewall_note(self.runner)):
if note:
print(note)
print("Login autostart is already enabled. To enable/start it again, run:\n "
+ shlex.join([str(prefix / "bin/pipelines"), "service", "enable"]))
print("Android app (optional, slower): ./install --apk builds it in Docker and offers it at "
f"{opencode_env['PIPELINES_WEB_PUBLIC_URL']}/updates/.")
print("Configured server role: " + ("single-user authoring enabled (web pipeline creation/publication allowed)."
if allow_authoring else "execution-only (pipeline publication disabled)."))
if args.bind_address == "0.0.0.0":
print(f"Phone client server URL: ws://{lan_ip}:{args.port}/connect "
"(use a workspace path that exists on this server).")
else:
print(f"Unauthenticated API bound to {args.bind_address}:{args.port}; use the authenticated web URL.")
print("\nPipelines installed")
print(f"Install location: {prefix}")
app_build.print_web_urls(manifest, lan_host=lan_ip)
print("OpenCode is required (default stage and chat harness) and is set up by ./install.")
print("Next: set up your agent harnesses: pipelines harness setup claude|codex|opencode|pi")
bin_dir = prefix / "bin"
path_entries = self.environ.get("PATH", "").split(os.pathsep)
if str(bin_dir) not in path_entries:
shell = Path(self.environ.get("SHELL", "")).name
rc = "~/.bashrc" if shell == "bash" else "~/.zshrc" if shell == "zsh" else "~/.profile"
displayed_bin = "$HOME/.local/share/pipelines/bin" if bin_dir == Path.home() / ".local/share/pipelines/bin" else str(bin_dir)
print(f"Add commands to PATH: echo 'export PATH=\"{displayed_bin}:$PATH\"' >> {rc}")
print(f"Update later: {prefix / 'bin/pipelines'} update")
print(f"Set up harnesses: {prefix / 'bin/pipelines'} harness setup claude|codex|opencode|pi")
def arguments(argv=None):
@@ -1020,17 +1030,29 @@ def arguments(argv=None):
help="Start with empty data even when an earlier Docker installation's volumes exist")
parser.add_argument("--replace-data", action="store_true",
help="Allow Docker migration to replace an existing native database/auth store")
parser.add_argument("--verbose", action="store_true", help="Show full build and tool output")
parsed = parser.parse_args(argv)
parsed.argv = list(sys.argv[1:] if argv is None else argv)
return parsed
if __name__ == "__main__":
installer = Setup(Path(__file__).resolve().parents[1])
try:
Setup(Path(__file__).resolve().parents[1]).run(arguments())
parsed = arguments()
installer.verbose = parsed.verbose
app_build.VERBOSE = parsed.verbose
prefix = parsed.prefix or Path(os.environ.get("PIPELINES_PREFIX") or DEFAULT_PREFIX).expanduser().resolve()
(prefix / "logs").mkdir(parents=True, exist_ok=True)
installer.install_log = prefix / "logs/install.log"
installer.run(parsed)
except KeyboardInterrupt:
print("\nNative setup canceled; existing configuration and data were retained.", file=sys.stderr)
sys.exit(130)
except (ValueError, OSError, json.JSONDecodeError, subprocess.SubprocessError,
ZoneInfoNotFoundError) as error:
if installer.install_log and installer.install_log.is_file() and not installer.verbose:
tail = installer.install_log.read_text(errors="replace").splitlines()[-8:]
detail = "\n".join(tail)
sys.exit(f"Native setup incomplete: {error}\nBuild/tool log: {installer.install_log}\n{detail}")
sys.exit(f"Native setup incomplete: {error}")
+21
View File
@@ -26,6 +26,27 @@ class FakeProcess:
class ManageNativeTests(unittest.TestCase):
def test_native_command_logs_tool_output_when_not_verbose(self):
with tempfile.TemporaryDirectory() as directory:
log = Path(directory) / "logs/install.log"
calls = []
def runner(command, **kwargs):
calls.append((command, kwargs))
return subprocess.CompletedProcess(command, 1, stdout="build output\n", stderr="failure\n")
installer = setup_native.Setup(directory, runner=runner)
installer.install_log = log
result = installer.command(["go", "build", "./cmd/pipelines"])
self.assertEqual(result.returncode, 1)
self.assertIn("build output", log.read_text())
self.assertIn("failure", log.read_text())
self.assertTrue(calls[0][1]["capture_output"])
def test_native_arguments_accept_verbose(self):
parsed = setup_native.arguments(["--verbose"])
self.assertTrue(parsed.verbose)
def service_config(self, home, system):
config = self.config(home / "prefix")
config["opencode_env"] = {"XDG_CONFIG_HOME": str(home / ".config")}