Merge branch 'worktree-agent-ac54c8919e940163a'

# Conflicts:
#	scripts/setup-native.py
This commit is contained in:
2026-11-10 01:07:12 +00:00
34 changed files with 1825 additions and 66 deletions
+248
View File
@@ -0,0 +1,248 @@
# Network defaults and update-feed signing
## Where the web proxy listens
Before this change `./install` bound the web proxy to `0.0.0.0` over plain
HTTP. With the macOS firewall off, anyone on the same café or hotel Wi-Fi could
reach the sign-in page, and the password and the 30-day session cookie crossed
that network unencrypted. The operator reaches the Mac over Tailscale
(`https://<host>.ts.net:8443` through `tailscale serve` to `127.0.0.1:8083`,
and `ws://100.x:8083` from tank), so the LAN listener was not needed.
### Defaults and opt-ins
`web.host` in `pipelines-install.json` takes one of two forms:
| `web.host` | Proxy flags (`manage.web_listen_args`) | Who can connect |
| --- | --- | --- |
| `tailnet` (new installs) | `--host :: --allow loopback,tailnet` | loopback and this machine's Tailscale IPv4/IPv6 addresses |
| `tailnet` with `web.lan: true` | `... --allow loopback,tailnet,lan` | also the LAN, but no sign-in or signed-in pages over plain HTTP |
| `tailnet` with `web.lan: true`, `web.lan_plain_login: true` | `... --lan-plain-login` | the LAN, with plain-HTTP sign-in (the old behaviour) |
| an IPv4 address, e.g. `0.0.0.0` | `--host 0.0.0.0` | whatever the bind reaches, unfiltered (the old behaviour) |
Installer flags: `--web-host tailnet|IPv4`, `--web-lan` / `--no-web-lan`,
`--web-lan-plain-login` / `--no-web-lan-plain-login`. A reinstall keeps the
saved values unless you pass the flag again. `--web-lan` on an install that
still has a literal host moves it to `tailnet`. `--web-lan` together with an
explicit IPv4 `--web-host` is refused.
On an opted-in LAN without `lan_plain_login`, a plain-HTTP LAN connection gets
`403` with an explanation for `POST /login`, `GET /login`, every signed-in
page, `/updates/`, `/api/*` and websocket upgrades. Only the public icons and
manifest are served. **Risk of `web.lan_plain_login`:** anyone on that network
who can watch the traffic sees the password and session cookie, and the cookie
works for 30 days. Use it only on a network you trust. `X-Forwarded-Proto` is
not trusted from the LAN, because a client can set it to anything.
### Why the proxy binds the wildcard and filters each connection
Tailscale often starts after the proxy (both start at login), and its address
can change. Binding the tailnet IP itself would fail at boot, and keeping it
working would need retry and rebind logic. Instead the proxy binds `::`, which
on macOS and Linux also accepts IPv4 (it falls back to `0.0.0.0` on a host
without IPv6). On every `connection` it drops the socket, before any HTTP is
read, unless **both** ends are loopback (`127/8`, `::1`) or **both** are
tailnet (`100.64.0.0/10`, `fd7a:115c:a1e0::/48`); see
`web/src/listenPolicy.js`.
Both ends are checked because macOS uses the weak host model: it accepts a
packet addressed to its tailnet IP on any interface. A LAN host that spoofs a
tailnet source address cannot finish the TCP handshake, because the reply is
routed into the tunnel.
The cost: a LAN scan sees the port open and immediately reset, and no HTTP is
served. One edge case is not handled: Tailscale subnet routes without SNAT
deliver peers with non-tailnet source addresses, and those are treated as LAN.
### Discovery (mDNS)
`manage.mdns_args` passes `-mdns-lan=false` when `web.host` is `tailnet`
without `web.lan`. `discovery.Plan` with `NoLAN` then makes these changes:
- The A records are the tailnet IPv4 addresses, not the LAN ones.
- TXT gains `lan=0`, and `ts=` and `public=` stay.
- The port is still the proxy's, which answers on the tailnet.
- With no tailnet address, nothing is published (`ErrNotReachable`).
The advert is still multicast on the LAN, so a tailnet client on the same Wi-Fi
finds the machine. `Browse` (Go, also behind `/api/discovery`) and the app's
`DiscoveredServer.fromRecord` drop the LAN route when `lan=0`, so the endpoint
becomes the first tailnet or public route. An older client still sees a "LAN"
route, but it points at the tailnet IP, which works from the tailnet.
### Firewall
The installer turns nothing on at the OS level. It only reads
`socketfilterfw --getglobalstate` and prints a recommendation when the firewall
is off. On this Mac on 2026-11-10 it reported `Firewall is disabled.
(State = 0)`. Recommendation: turn it on (System Settings > Network >
Firewall). The proxy no longer depends on it, but other local services
(ntfy on `:8090`, development servers) do.
### Migration
- **Live Mac** (`web.host: 0.0.0.0`, port 8083, public URL
`https://alans-macbook-pro.tail6bf4dc.ts.net:8443`): a plain `./install`
keeps `0.0.0.0` and prints
`WARNING: the web proxy listens on 0.0.0.0 over plain HTTP ... Recommended: ./install --web-host tailnet ...`.
To switch, run `./install --web-host tailnet` while no run or chat is active
(the install restarts the services). `tailscale serve` still reaches
`127.0.0.1:8083`, and tank's `ws://100.85.25.33:8083` is a tailnet
connection, so both keep working. Phones on the home Wi-Fi that saved
`ws://10.0.0.x:8083` must switch to the tailnet route (rediscover, or pick
the Tailscale address).
- **tank:** the same `./install --web-host tailnet`. Check first that every
client reaches it through Tailscale.
- An install whose recorded `PIPELINES_WEB_PUBLIC_URL` is a plain-HTTP LAN
address gets a warning to set it to the tailnet HTTPS address.
### Verification (network)
- `cd web && node --test test/`: `tests 24, pass 24, fail 0`. This includes
`listenPolicy.test.mjs`, which uses real sockets on this Mac's LAN (10.0.0.x)
and tailnet (100.85.25.33) addresses.
- `umask 022 && go test ./internal/discovery ./cmd/pipelines-server ./internal/server`:
all `ok`. This includes `TestPlanWithoutLANAdvertisesOnlyTailnetAndPublicRoutes`
and `TestBrowseOmitsTheLANRouteWhenTheAdvertSaysLan0`.
- `cd scripts && python3 -m unittest`: `Ran 229 tests ... OK (skipped=1)`.
This includes `test_web_access` and
`test_upgrade_keeps_an_all_interfaces_web_host_with_a_warning_and_lan_is_opt_in`.
- App: `tool/memwatch/full.sh 'test/features/updates test/features/onboarding test/features/settings'`
gave `PASS: 212 passed, 0 failed`, and `tool/analyze.sh` was clean.
- Scratch end-to-end: an isolated `HOME`, an OS-chosen port, real sign-in, and
the proxy started with `manage.web_listen_args` output:
```
== default loopback + tailnet: serve.mjs --port 53780 --host :: --allow loopback,tailnet
GET http://127.0.0.1:53780/login -> 200 POST -> 303
GET http://100.85.25.33:53780/login -> 200 POST -> 303
GET http://10.0.0.249:53780/login -> 000 (connection reset, curl exit 56)
== LAN opted in, no plain login: ... --allow loopback,tailnet,lan
GET/POST via 127.0.0.1 and 100.85.25.33 -> 200/303; via 10.0.0.249 -> 403/403
== LAN with lan_plain_login: ... --lan-plain-login
GET/POST via 10.0.0.249 -> 200/303
```
The live mDNS advert was not exercised, because a scratch server would publish
the same `Pipelines-<host>` instance name as the live one. The records are
covered by `Plan`/`Zone` tests.
## Update feed signing
Before this change the app trusted whatever `/updates/` served: the Linux
self-update installed and relaunched an archive whose sha256 came from the same
`version.json`, Android skipped its signer check when the feed omitted the
field, and the macOS zip had no check beyond that sha256. A server (or anyone
able to write its update directory, or answer for its address) could hand every
client another build.
### What is signed
- Publishing (`./install --apk`, `./install --linux|--macos --publish`) writes
`version.json` and, beside it, a detached `version.json.sig`:
`{"algorithm":"ed25519","publicKey":"<base64 32 bytes>","signature":"<base64 64 bytes>"}`.
The signature covers the exact bytes of `version.json`, which carry the
artifact `sha256` (and, for Android, `signingCertificateSha256`). The
signature is written first, then the feed, both atomically; a check racing a
publish fails once and succeeds on the next check.
- Code: `scripts/update_signing.py` (keys, signing, `publish_feed`),
`scripts/app_build.py` (`build_apk`, web and desktop defines),
`scripts/app_desktop_build.py` (`publish_desktop(..., signer=...)`).
### Key management
- Each installation has one Ed25519 key in
`~/.config/pipelines/update-signing-key.pem` (mode 0600, directory 0700),
generated once on the first `./install` or app build/publish and never
replaced. `PIPELINES_UPDATE_SIGNING_KEY` names another path. A key file that
is a symlink or readable by others is refused. It is never in the repository.
`scripts/uninstall.py` (full reset) removes `~/.config/pipelines`, and so
the key; see "Lost or replaced key" below.
- Cryptography is the OpenSSL 3 command line (`genpkey`, `pkey`,
`pkeyutl -sign -rawin`). The installer's Python has no `cryptography`
package, and macOS's `/usr/bin/openssl` is LibreSSL 3.3.6, which has no
Ed25519 (`unable to load Private Key`). The tool is chosen as
`PIPELINES_OPENSSL`, then `/opt/homebrew/opt/openssl@3/bin/openssl`,
`/usr/local/opt/openssl@3/bin/openssl`, then `openssl` on PATH, taking the
first that can generate an Ed25519 key. Without one the install stops with
`Update signing needs OpenSSL 3 with Ed25519 ...` (`brew install openssl@3`).
On this Mac `openssl` on PATH is Homebrew OpenSSL 3.6.4.
- Print this installation's public key:
`python3 scripts/update_signing.py`.
### Trust model
- Every app build embeds the keys it trusts as
`--dart-define=PIPELINES_UPDATE_PUBLIC_KEY=<key>[,<key>...]` (apk, linux,
macos and web builds; the define is part of the build cache key, so a key
change rebuilds). The list is this installation's own key first, then each
base64 key in `~/.config/pipelines/update-trusted-keys` (one per line, `#`
comments allowed; a malformed line stops the build).
- Each server signs its own feed with its own key. A client accepts a feed
only when `version.json.sig` names an embedded key and the signature over
the served bytes verifies with it; the client checks the signature before it
believes anything in the feed. So an app built on the Mac updates from tank
only if tank's key was listed in the Mac's `update-trusted-keys` when that
app was built (and vice versa). There is no trust-on-first-use pinning:
trust comes only from the build, which keeps the model to one rule.
- With keys embedded, the app refuses: an unsigned feed (no `.sig`), an
unreadable signature, a key it does not embed, a bad signature, a feed
without `sha256`, and an Android feed without `signingCertificateSha256`
(or when the installed signature is unknown). The Android certificate must
still match the installed app's, as before.
- A build with no embedded key (a developer `flutter run`/`flutter build`
outside the installer) keeps the previous unsigned behaviour. Installer
builds always embed a key.
- Verification in the app is pure Dart (`app/lib/features/updates/ed25519.dart`,
RFC 8032 verify only, BigInt arithmetic, checked against RFC 8032 test
vectors 1-3; no new package) and
`app/lib/features/updates/update_feed_signature.dart`; the reader is
`fetchLatestScoped` in `app_updates_io.dart`. The web Settings "Android
download" link is a browser download the operator installs by hand and is
unchanged.
### Migration (the live Mac install and tank)
Not done by this change; nothing was deployed. In order, on each machine:
1. `./install` (or the next `./install --apk` / desktop publish) generates
`~/.config/pipelines/update-signing-key.pem`. Check:
`ls -l ~/.config/pipelines/update-signing-key.pem` shows `-rw-------`.
2. To let one machine's apps update from the other, exchange keys before
building: on tank run `python3 scripts/update_signing.py` and append the
line to the Mac's `~/.config/pipelines/update-trusted-keys`; do the
reverse on tank. Skip this if each machine's apps only update from it.
3. Rebuild and publish: `./install --apk`, `./install --macos --publish
--flutter <flutter>`, `./install --linux --publish`. Each feed directory
then holds `version.json.sig`.
4. Installed apps from before this change do not check signatures; they take
the next signed release normally (the `.sig` is ignored), and from then on
they refuse unsigned or foreign feeds. Phones whose feed came from the other
machine need step 2 before step 3.
5. Lost or replaced key (for example after a full uninstall): apps built with
the old key refuse the new feeds. Install the new build once by hand (APK
download from Settings in the browser, or the desktop archive), or restore
the old key file from backup.
### Verification output
- `cd scripts && python3 -m unittest test_update_signing test_app_build
test_app_desktop_build test_setup_native`: `Ran 81 tests ... OK`. Covers key
generation once at 0600 in a scratch HOME, refusal of readable/symlinked
keys, LibreSSL-style tools skipped, signatures checked independently with
`openssl pkeyutl -verify` (and a tampered message refused), feed plus `.sig`
publication, trusted-key list parsing, APK publication signed with the
generated key, desktop publication passing the exact feed bytes to the
signer, and the committed Dart fixture being a valid OpenSSL signature.
- `cd app && tool/memwatch/full.sh 'test/features/updates test/features/settings'`:
`memwatch: PASS: 187 passed, 0 failed, 0 skipped`. New:
`test/features/updates/ed25519_test.dart` (RFC 8032 vectors; changed
message, signature, key; short inputs; S >= L; non-canonical key) and
`test/features/updates/update_feed_signature_test.dart` (the
Python-signed fixture `test/features/updates/fixtures/signed_feed/`
verifies; tampered feed and signature refused; missing signature refused
with keys, accepted without; untrusted and swapped keys refused; signed
feeds require sha256 and the Android signer; and the HTTP reader
`fetchLatestScoped` against a loopback server refuses an unsigned and a
tampered feed and lets the genuine one past the signature check).
`test/core/session`: `PASS: 18 passed`. `tool/analyze.sh`: `analyze: clean`.