Commit Graph

58 Commits

Author SHA1 Message Date
alan a0f2cfec16 Merge branch 'worktree-agent-ac54c8919e940163a'
# Conflicts:
#	scripts/setup-native.py
2026-11-10 01:07:12 +00:00
alan d35ae9b65a Merge branch 'worktree-agent-a5eba3669bc0e09ad' into worktree-agent-ac54c8919e940163a 2026-11-10 01:02:08 +00:00
alan ad6cdab418 Listen on loopback and tailnet by default; make the plain-HTTP LAN opt-in
The web proxy binds the wildcard and drops connections that are not
loopback or Tailscale at both ends. A LAN listener is opt-in (web.lan) and
refuses sign-in and signed-in pages over plain HTTP unless
web.lan_plain_login is set. Existing installs keep their bind address with
a warning. Without the LAN the mDNS advert carries only the tailnet and
public routes (lan=0).
2026-11-10 01:02:02 +00:00
alan 12a74247ae Create the server token at install and document the security boundary 2026-11-10 00:59:39 +00:00
alan 9f38e38f6a Sign the app update feed with Ed25519 and verify it in the app
Each installation generates a private Ed25519 key once (0600, under
~/.config/pipelines), signs version.json with a detached version.json.sig
when publishing APK and desktop builds, and embeds its own and listed
trusted public keys in app builds. The app verifies the feed signature
before trusting the artifact sha256, refuses unsigned or foreign feeds when
it has keys, and requires the sha256 and Android signer in signed feeds.
2026-11-10 00:51:16 +00:00
alan 8ca5a1fdda Add a per-user Linux app installer script
scripts/install-linux-app.sh installs a release archive into
~/.local/share/pipelines-app (writable, so in-app updates can swap it), with
a ~/.local/bin launcher, icon and desktop entry, and checks for missing
system libraries first.
2026-11-09 19:45:17 +00:00
alan 701fb98ef0 Fix native APK homes; add a native discovery probe
The native APK build no longer sets ANDROID_SDK_HOME beside
ANDROID_USER_HOME (AGP rejects the conflict). integration_test/
discovery_probe_test.dart runs the real Bonsoir discovery on a desktop
host and prints what the platform delivers.
2026-11-09 12:42:05 +00:00
alan 0e16e6c9fe Republish mDNS on address change; build the APK natively when the host can
The server compares its LAN and tailnet addresses every 30 seconds and
republishes the advert when they change, so a machine that moves networks
stops advertising the address it had at startup.

./install --apk builds on this machine with the host Flutter and Android SDK
when both exist (--docker forces the container build), signing with the same
shared debug keystore so phones keep accepting updates.
2026-11-09 12:38:51 +00:00
alan 543e3f063a Add interactive password change command without service restart 2026-11-08 21:48:00 +00:00
alan 828227f549 Add complete uninstall reset for fresh native installations 2026-11-08 19:16:46 +00:00
alan e17bc2ca4f Exercise reset failure paths and scoped Docker resource cleanup 2026-11-08 19:13:39 +00:00
alan bcd534c456 Cover full Pipelines reset and deletion boundaries 2026-11-08 19:10:42 +00:00
alan b67ae34ca1 Stage ntfy output on installation filesystem before atomic replacement 2026-11-08 19:02:26 +00:00
alan 8ddf893de9 Publish APK certificate identity for multi-server compatibility 2026-11-08 16:58:47 +00:00
alan bd9509b744 Preserve Node tool-manager shim names during native installation 2026-11-08 16:54:17 +00:00
alan 761ff021fe Keep generated Linux plugin registries out of build identity 2026-11-08 16:50:15 +00:00
alan 36f73bb89d Build Linux and macOS clients with platform update feeds 2026-11-08 16:45:42 +00:00
alan f6ca558302 Add stable web release metadata 2026-11-08 00:10:31 +00:00
alan c2a8f5ff16 Retain Codex interaction callbacks across server restarts 2026-11-05 14:32:59 +00:00
alan 6b22389cae Merge: Claude Code chats survive server restarts via a detachable host 2026-11-04 16:15:06 +00:00
alan c810437aa5 Claude Code chats survive server restarts in a detachable host
A pipelines-claude-host process owns each claude -p CLI and its pipes,
spools its stream-json output with offsets and serves a private Unix
socket. The server detaches on shutdown and reattaches on start,
replaying the spool to rebuild turns, approvals, questions, background
tasks and the live model. Explicit close, idle release and CLI exit stop
the host; a crashed host falls back to --resume with a notice.
2026-11-04 16:09:56 +00:00
alan a48e4a5b6a Replace starters with four graded examples seeded into new databases
The template catalog is now Example 1-4: one stage, two stages (handoff),
feedback loop (runner-owned limits, retries, timeout) and human decision
(type=human approve/revise/abort with notify hooks). All read-only or leave
changes uncommitted; none commits, pushes or deploys.

A brand-new database (schema created by this Open) gets the examples once at
server start, bound to the configured default model and harness. A
server_settings marker records the decision, so removed examples never come
back and existing installs are marked without any pipeline being added or
rewritten.
2026-11-03 19:35:47 +00:00
alan 7ecc12b747 LAN discovery: server advertises _pipelines._tcp over mDNS; app finds servers on the connect screen
Server: internal/discovery publishes a DNS-SD record (hashicorp/mdns) with
the port LAN clients dial (the web proxy via -mdns-port, which the
supervisor passes, or a non-loopback -addr) and TXT v, name, path, tls, auth.
On by default; -mdns=false or PIPELINES_MDNS=off disables it. Failures are
logged and never block startup; the advert is withdrawn on shutdown.

App: the Connect to Pipelines screen gets 'Find servers on this network'
behind a ServerDiscovery interface (Bonsoir: Android NSD with multicast lock,
iOS Bonjour with NSLocalNetworkUsageDescription and NSBonjourServices).
Tapping a server fills the address and connects; manual entry stays. Web has
no scan.
2026-11-03 14:54:54 +00:00
alan 108c29f9eb One harness setup path: pipelines harness setup <claude|codex|opencode|pi>
Checks the CLI on the server's PATH (install command when missing), signs in
with the harness's own login only when needed (--login forces it), keeps the
OpenCode/Pi model pickers, and verifies with the server's check. Works on
native and Docker installs (Docker delegates to the container setup); --host
is a hidden compatibility alias. harness list shows all four harnesses'
installed/signed-in/configured/default status. ./install --opencode|--pi now
point to the command; ./install ends with the next-step hint.
2026-11-02 16:11:04 +00:00
alan 62b56e66d0 Serve one web app: route the tailnet address to the installed proxy
./install retires the pre-native standalone web agent (net.awan.pipelines.web,
kept as .disabled) and, when the public web URL is a tailnet HTTPS address,
points tailscale serve at the installed web proxy, failing the install if it
cannot.
2026-10-31 20:26:16 +00:00
alan 53be2e5334 Install and start the server only together with the web app
./install builds the web app and the server before replacing either, and
fails without Docker or on a failed build, leaving the old pair in place.
The supervisor and start refuse to run the server when the web app is not
installed.
2026-10-31 19:08:25 +00:00
alan 41a3be8715 Keep web Material styling and preserve visible installer access URLs 2026-10-26 18:37:13 +00:00
alan afba02899d fix(install): update installs a checkout without upstream as it is; the startup report no longer asks for a Claude token 2026-10-23 23:40:20 +01:00
alan 3d0c034394 fix(install): a fresh install creates ntfy accounts without prompts (database created first; phone uses the web password) 2026-10-23 23:38:51 +01:00
alan 9bad1bd824 feat(install): pipelines update re-runs the recorded install from the source checkout; --fresh skips Docker migration 2026-10-23 23:37:35 +01:00
alan 00d8714411 fix(install): stamp each APK build with a rising build number 2026-10-23 22:07:49 +01:00
alan 7958d1830e feat(install): build the Flutter web app in Docker, optional ./install --apk, chosen web password, startup report 2026-10-23 21:45:14 +01:00
alan 9ee2e003bf fix(notify): open configured web run pages from ntfy 2026-10-21 13:05:10 +01:00
alan da6daa9db5 Resolve configured host models during native installation
Read OpenCode V2's resolved default Model.Info through its authenticated CLI API from HOME when saved profile fields are missing. Preserve valid saved models, explicit --model and unrelated settings. Record the selected stage model in the native server environment so web starters have an actual default.

Run the existing read-only model picker inside ./install when a terminal needs a choice; fail noninteractive or cancelled/incomplete setup before claiming ready or starting services. Do not parse JSONC, open login flows, create sessions or run inference. Document the one-step configuration path.

Checks: 61 mocked native installer/lifecycle tests, 8 entrypoint tests, sh -n install install.sh, git diff --check. API/schema checked against official OpenCode V2 documentation. No operator CLI/service/model calls, live install, restart, deployment or native Linux verification.
2026-10-20 21:56:32 +01:00
alan bb32383bbe Route native service commands and configure fresh authoring role
Select native installs only via explicit PIPELINES_PREFIX or the matching owned prefix/bin/pipelines executable. Route enable/status/disable to the existing native supervisor; validate registration ownership, label and command, keep preview read-only, and disable login registration before stopping. Never fall back to a standalone service for an invalid explicit prefix.

Fresh native installs explicitly enable authoring, while reinstalls preserve false or missing legacy role settings. Add explicit --allow-authoring/--no-authoring overrides and pass the server flag only for manifest true. Print a copyable native enable command and update installation help/docs. No Docker configuration changes.

Checks: 54 mocked native installer/lifecycle tests; 8 entrypoint tests; offline Go installation/CLI tests and vet; gofmt clean; shell syntax and diff checks. No live install, restart, deployment, permission/config changes, or native Linux verification.
2026-10-20 21:41:28 +01:00
alan d58b3f49fc Add generic checked starter workflows and isolated lifecycle coverage 2026-10-20 21:31:36 +01:00
alan ae6ba6ae10 Install authenticated native web proxy with user autostart
Register the existing native supervisor with launchd/systemd, include the dependency-free authenticated web proxy, detect Node, and keep the API loopback by default. Preserve existing web credentials and configured host harness defaults. Clarify single-step setup, lifecycle selection, and optional ntfy limits.

Checks: 44 mocked native installer/lifecycle tests; 8 entrypoint tests; 42 Docker installer tests; GOPROXY=off GOTOOLCHAIN=local go test ./internal/installation ./cmd/pipelines; sh -n install install.sh; git diff --check. No live install, deployment, restart, downloads, real autostart/login verification or native Linux verification.
2026-10-20 21:31:36 +01:00
alan a3f571d672 docs: make docs-fold the single writer of shared documentation
Every item wrote the shared, append-only docs (android/README.md, docs/cli.md and
the packaged skill references), so every merge conflicted on prose. Keep the duty
to notice stale docs; move the writing to one deliberate place.

- Per-item note, not a shared edit. An item that makes a shared document stale
  writes docs/notes/<slug>.md - unique per item, so two items never touch the same
  file - and commits that note with its change. It never edits a README, topic doc
  or packaged skill reference itself.
- One writer. The manual-only docs-fold pipeline, bundled as the fifth starter
  examples/docs-fold.md, folds every note under docs/notes/ into the shared
  document it names and deletes the consumed notes in its own commit; an empty
  notes directory is a success no-op. Operator trigger:
  pipelines new "Docs fold" --template docs-fold --no-open; pipelines submit
  <draft.md>; pipelines run "Docs fold".
- Implemented where the convention lives: orchestration.md (brief item 5, the
  hard-constraint bullet, a "Single documentation writer" section and the
  "Pipeline choice" committer bullet), the canonical committer checklist in
  format.md (new step 6), the committer prompts in examples/task.md and
  examples/three-stage.md, the new examples/docs-fold.md with its catalog and
  README entries, and the repository guideline AGENTS.md. Test-expectation counts
  (examples/catalog_test.go, internal/server/templates_test.go,
  scripts/smoke-first-install.py) follow the fifth starter.

Reflected where it belongs: the packaged templates (examples/task.md,
examples/three-stage.md, the new examples/docs-fold.md), the packaged skill
references (orchestration.md, format.md) and the repo guideline AGENTS.md all carry
the new convention. No new reference file was added, so install.go's managed
inventory is unchanged. The shared descriptive docs were deliberately not
rewritten: this change makes README.md:191, docs/cli.md:170 and
internal/author/skill/references/operations.md:13 name four starters where there are
now five. Those stay as they are - correcting them now is exactly the per-item
shared edit this change removes - and the stale-doc note
docs/notes/docs-fold-single-writer.md records each one for the docs-fold writer to
fold. Nothing else needed updating. No new dependency; no Android, Gradle, server,
installer, Docker or published-definition change; no definition published over the
running pipelines.

Evidence: go test -count=1 ./... passed; go build ./... and go vet on the touched
packages clean; ./internal/author/... covers embedding, reference links, the managed
inventory and install/upgrade/protection; ./examples/... validates the fifth starter
and its manual-only, committer-terminated graph; ./internal/server/... covers the
served catalog. Not run: the docs-fold writer against the shared docs (nothing to
fold yet); no install or deploy.

(cherry picked from commit f29656cf084d7eee2e033f2a37a9d231d7acaaf0)
2026-10-17 23:59:59 +01:00
alan c9b904186a Update templates and authoring skill to the current pipeline contract
Bring the packaged starters and the pipelines skill up to the guidelines the
project now works to.

Templates:
- task and three-stage now end in a committer stage that stages the union of the
  runner's captured input.server_changed_files and the producer's declared
  changed_files, runs the repository's checks, commits locally and never pushes;
  their producers no longer commit.
- Every model/agent prompt states the line-oriented result contract (outcome on the
  first non-empty line, "# output" JSON map, other "# <heading>" sections kept
  verbatim as evidence) and an "# evidence" section for exact commands and results,
  with proportionate verification and no fabricated checks.
- Stages that can write save their reports, logs and screenshots into the runtime
  artifacts directory.
- Remove the summary starter: a tool-free single-prompt text transform with no
  repository interaction, artifact or commit is a format demonstration, not a
  reusable workflow. No starter invents a pretend approval stage; a decision point
  is a type=human stage.

Skill:
- references/format.md gains a Result format section (line form, legacy JSON
  precedence, result_parse, raw-output.md, artifacts) and a Committer stages
  section with the pre-commit checklist; the role-split and envelope notes mention
  the line form and the committer.
- orchestration.md points at the checklist and records that a genuine gate is a
  human stage; operations.md and the SKILL.md index drop the removed starter and
  name the result format and committer. No new reference, so the managed inventory
  is unchanged.

Docs/tests: examples/README, README, docs/cli, docs/first-version-testing, the CLI
help example and the smoke script track the four starters; catalog, tool-role,
server, tui and pipeline tests are updated, with a new test pinning the committer
ending and server-captured change set.

Commands run: go build ./...; go vet ./...; go test ./...; python3 -m unittest
discover -s scripts -p 'test_*.py'; python3 scripts/smoke-first-install.py. All
passed. Templates inspected through an isolated CLI server (pipelines templates;
pipelines template KEY --source). No server, service or device was restarted, and
nothing was pushed.
2026-10-17 01:09:42 +01:00
alan 8c1eea53a9 Report a stale client and refuse an unreadable newer server
Exchange a build/protocol identity so an older CLI can no longer talk to a
newer native server silently:

- protocol carries ProtocolVersion and every server frame (including the
  unsolicited welcome) is stamped with the server build and protocol, so a
  client that never pings still learns the server identity.
- buildinfo.Compare classifies the relationship as InSync/Warn/Refuse and
  names both builds; the client Watch reports one warning per connection and
  returns a hard error when the server speaks a newer, uninterpretable
  protocol so the caller never acts on the frame.
- client.ReceiveNotice decodes the raw frame so an unmodeled top-level field
  produces a one-time "unrecognized field(s)" notice instead of being dropped
  silently (the case that hid the artifacts field).
- CLI routes warnings to stderr, keeps machine JSON stdout clean and fails
  non-zero on a refused mismatch; ping/connection print the client/server
  relationship. The TUI surfaces the same notices through the shared log.
- scripts/setup-native.py now warns when an older `pipelines` earlier on PATH
  shadows prefix/bin/pipelines, naming both paths and versions, without
  installing or modifying anything else.
- Docs, the packaged debugging skill reference and README no longer imply the
  installed CLI always matches the server.

Tests cover the comparison table, unknown-field detection and caching, the
server identity stamping on pong and welcome, the CLI/TUI reporting paths and
the installer PATH shadowing warning.
2026-10-16 18:37:01 +01:00
alan 914345cb31 Move harness profiles between host and Docker and stop silent draft retargeting
Native installs left Docker-era harness profiles in harnesses.json, so every
authoring launch failed trying to reach a Docker service, and a retained draft
bound to the old Docker server was silently projected onto the configured one.

- harness: add Profile.Mode and VerifyDockerLink as the single mode/verification
  helpers. `harness setup --host` configures or converts a host profile and
  clears docker_directory; reconfiguring a Docker-linked profile re-verifies the
  recorded directory still owns compose.yaml and fails naming it instead of
  saving a link that cannot launch. Setup's summary and `harness list` (via the
  new List/ListView shape) report the resulting mode and, for Docker, the
  directory, without changing the stored Config schema.
- setup-native.py: after build/install-skill it writes host (no docker_directory)
  profiles for the detected harnesses, defaulting to the detected OpenCode profile
  rather than a stale pi default, taking models from --model or the existing valid
  profile and never inventing one; it clears a stale Docker link and preserves
  unrelated keys/profiles. Detection-only: no harness install, upgrade or download.
- draft: add Draft.Resolve, the shared mismatch helper, which projects a draft
  onto the configured server and returns a dual-URL notice. CLI validate/submit
  and the TUI draft browser use it so a foreign draft is never silently
  retargeted; the TUI pre-wraps the notice to the panel width.
- Tests cover the Docker->host and host->host transitions, the missing-directory
  failure, installer write/migration (Python), and the CLI/TUI notice, using the
  real config file path in a temp dir.
- Update help text, README, docs and the packaged authoring skill reference to
  describe host as the default and the Docker-linked case.
2026-10-15 19:53:10 +01:00
alan 8adc6b8080 Bind the native server to the LAN so the phone client can reach it
The native install hardcoded -addr 127.0.0.1, so the server was unreachable from
the phone: the emulator worked because it proxies to the host loopback, but a
real device could not connect even though ntfy (bound to 0.0.0.0:8090) worked.

The bind address is now a manifest setting (server_address() in manage.py, with
older manifests still defaulting to loopback) and setup-native.py records it,
defaulting to 0.0.0.0 like the Docker install did and offering
--bind-address 127.0.0.1 for a loopback-only server. An invalid address is
rejected, and setup prints the phone client URL.

Verified: scripts test suite 116 tests OK, and after reinstalling the server
listens on *:8080 and answers {"healthy":true} on 127.0.0.1, 10.0.0.75,
10.0.0.249 and the tailnet address.
2026-10-15 01:57:27 +01:00
alan a70e3aef4b Install Pipelines natively by default on macOS and Linux
Add scripts/setup-native.py and make ./install (and --native) build the server, the CLI
and ntfy from source into ~/.local/share/pipelines, detect an existing host OpenCode
and Pi instead of installing them, install the packaged authoring skill, migrate an
existing Docker database and ntfy auth store, save the host client default and start
both services. ./install --docker keeps the previous Docker server + ntfy
installation unchanged.

ntfy is compiled from its pinned Go module on every host because upstream publishes
no macOS server binary (its darwin asset is built with -tags noserver); its cgo
SQLite driver is why a C compiler is now required natively. The ntfy credential
bootstrap and the client default writer are shared by both installers,
installation.Load resolves symlinked prefixes, the Docker migration copies the
SQLite write-ahead log so the newest definitions and runs are not lost, and the
account subcommands read the generated configuration through NTFY_CONFIG_FILE.

Verified on macOS: 115 Python tests, go build/vet/test, and scratch installs that
migrate 9 pipelines and 31 runs, build ntfy v2.30.0 and start both services healthy
on spare ports before stopping them.
2026-10-13 17:20:14 +01:00
alan 72dc1f1078 Document the cloneable preview and clean up test formatting 2026-10-11 23:07:11 +01:00
alan 05ae530442 Make Pipelines Docker-only with OpenCode and Pi
Install current verified harness releases inside Docker, preserve persistent accounts and settings, and provide separate interactive and pipeline model defaults with terminal onboarding and fuzzy pickers.

Add Docker-aware authoring transport, Pi stage execution, bounded workflow controls, reliable final-result readback and usage settlement, and finite short-lived CLI pipe draining. Keep dated installation evidence and packaged authoring guidance consistent.

Verified with a real Docker installation, both harness stages and interactive authors, the built-in plan/implement/review starter, independent host race/vet checks, installer tests, and packaged skill tests. Generated release receipts stay ignored in clean checkouts.
2026-10-11 23:00:25 +01:00
alan 7b10e3889e Replace niche templates with simple starters and direct template submission 2026-10-10 02:37:31 +01:00
alan 68042aad5d Mount host repositories for Docker execution with host user ownership 2026-10-10 01:22:02 +01:00
alan 7565693957 Expand pipeline panes and select the verified Docker client endpoint 2026-10-10 00:56:55 +01:00
alan 16389b8270 Prime OpenCode catalog before container login and model validation 2026-10-09 23:34:01 +01:00
alan 08b8647d5b Validate providers and retain complete run debugging context 2026-10-09 23:21:01 +01:00