Merge branch 'worktree-agent-ac54c8919e940163a'
# Conflicts: # scripts/setup-native.py
This commit is contained in:
@@ -0,0 +1,248 @@
|
||||
# Network defaults and update-feed signing
|
||||
|
||||
## Where the web proxy listens
|
||||
|
||||
Before this change `./install` bound the web proxy to `0.0.0.0` over plain
|
||||
HTTP. With the macOS firewall off, anyone on the same café or hotel Wi-Fi could
|
||||
reach the sign-in page, and the password and the 30-day session cookie crossed
|
||||
that network unencrypted. The operator reaches the Mac over Tailscale
|
||||
(`https://<host>.ts.net:8443` through `tailscale serve` to `127.0.0.1:8083`,
|
||||
and `ws://100.x:8083` from tank), so the LAN listener was not needed.
|
||||
|
||||
### Defaults and opt-ins
|
||||
|
||||
`web.host` in `pipelines-install.json` takes one of two forms:
|
||||
|
||||
| `web.host` | Proxy flags (`manage.web_listen_args`) | Who can connect |
|
||||
| --- | --- | --- |
|
||||
| `tailnet` (new installs) | `--host :: --allow loopback,tailnet` | loopback and this machine's Tailscale IPv4/IPv6 addresses |
|
||||
| `tailnet` with `web.lan: true` | `... --allow loopback,tailnet,lan` | also the LAN, but no sign-in or signed-in pages over plain HTTP |
|
||||
| `tailnet` with `web.lan: true`, `web.lan_plain_login: true` | `... --lan-plain-login` | the LAN, with plain-HTTP sign-in (the old behaviour) |
|
||||
| an IPv4 address, e.g. `0.0.0.0` | `--host 0.0.0.0` | whatever the bind reaches, unfiltered (the old behaviour) |
|
||||
|
||||
Installer flags: `--web-host tailnet|IPv4`, `--web-lan` / `--no-web-lan`,
|
||||
`--web-lan-plain-login` / `--no-web-lan-plain-login`. A reinstall keeps the
|
||||
saved values unless you pass the flag again. `--web-lan` on an install that
|
||||
still has a literal host moves it to `tailnet`. `--web-lan` together with an
|
||||
explicit IPv4 `--web-host` is refused.
|
||||
|
||||
On an opted-in LAN without `lan_plain_login`, a plain-HTTP LAN connection gets
|
||||
`403` with an explanation for `POST /login`, `GET /login`, every signed-in
|
||||
page, `/updates/`, `/api/*` and websocket upgrades. Only the public icons and
|
||||
manifest are served. **Risk of `web.lan_plain_login`:** anyone on that network
|
||||
who can watch the traffic sees the password and session cookie, and the cookie
|
||||
works for 30 days. Use it only on a network you trust. `X-Forwarded-Proto` is
|
||||
not trusted from the LAN, because a client can set it to anything.
|
||||
|
||||
### Why the proxy binds the wildcard and filters each connection
|
||||
|
||||
Tailscale often starts after the proxy (both start at login), and its address
|
||||
can change. Binding the tailnet IP itself would fail at boot, and keeping it
|
||||
working would need retry and rebind logic. Instead the proxy binds `::`, which
|
||||
on macOS and Linux also accepts IPv4 (it falls back to `0.0.0.0` on a host
|
||||
without IPv6). On every `connection` it drops the socket, before any HTTP is
|
||||
read, unless **both** ends are loopback (`127/8`, `::1`) or **both** are
|
||||
tailnet (`100.64.0.0/10`, `fd7a:115c:a1e0::/48`); see
|
||||
`web/src/listenPolicy.js`.
|
||||
|
||||
Both ends are checked because macOS uses the weak host model: it accepts a
|
||||
packet addressed to its tailnet IP on any interface. A LAN host that spoofs a
|
||||
tailnet source address cannot finish the TCP handshake, because the reply is
|
||||
routed into the tunnel.
|
||||
|
||||
The cost: a LAN scan sees the port open and immediately reset, and no HTTP is
|
||||
served. One edge case is not handled: Tailscale subnet routes without SNAT
|
||||
deliver peers with non-tailnet source addresses, and those are treated as LAN.
|
||||
|
||||
### Discovery (mDNS)
|
||||
|
||||
`manage.mdns_args` passes `-mdns-lan=false` when `web.host` is `tailnet`
|
||||
without `web.lan`. `discovery.Plan` with `NoLAN` then makes these changes:
|
||||
|
||||
- The A records are the tailnet IPv4 addresses, not the LAN ones.
|
||||
- TXT gains `lan=0`, and `ts=` and `public=` stay.
|
||||
- The port is still the proxy's, which answers on the tailnet.
|
||||
- With no tailnet address, nothing is published (`ErrNotReachable`).
|
||||
|
||||
The advert is still multicast on the LAN, so a tailnet client on the same Wi-Fi
|
||||
finds the machine. `Browse` (Go, also behind `/api/discovery`) and the app's
|
||||
`DiscoveredServer.fromRecord` drop the LAN route when `lan=0`, so the endpoint
|
||||
becomes the first tailnet or public route. An older client still sees a "LAN"
|
||||
route, but it points at the tailnet IP, which works from the tailnet.
|
||||
|
||||
### Firewall
|
||||
|
||||
The installer turns nothing on at the OS level. It only reads
|
||||
`socketfilterfw --getglobalstate` and prints a recommendation when the firewall
|
||||
is off. On this Mac on 2026-11-10 it reported `Firewall is disabled.
|
||||
(State = 0)`. Recommendation: turn it on (System Settings > Network >
|
||||
Firewall). The proxy no longer depends on it, but other local services
|
||||
(ntfy on `:8090`, development servers) do.
|
||||
|
||||
### Migration
|
||||
|
||||
- **Live Mac** (`web.host: 0.0.0.0`, port 8083, public URL
|
||||
`https://alans-macbook-pro.tail6bf4dc.ts.net:8443`): a plain `./install`
|
||||
keeps `0.0.0.0` and prints
|
||||
`WARNING: the web proxy listens on 0.0.0.0 over plain HTTP ... Recommended: ./install --web-host tailnet ...`.
|
||||
To switch, run `./install --web-host tailnet` while no run or chat is active
|
||||
(the install restarts the services). `tailscale serve` still reaches
|
||||
`127.0.0.1:8083`, and tank's `ws://100.85.25.33:8083` is a tailnet
|
||||
connection, so both keep working. Phones on the home Wi-Fi that saved
|
||||
`ws://10.0.0.x:8083` must switch to the tailnet route (rediscover, or pick
|
||||
the Tailscale address).
|
||||
- **tank:** the same `./install --web-host tailnet`. Check first that every
|
||||
client reaches it through Tailscale.
|
||||
- An install whose recorded `PIPELINES_WEB_PUBLIC_URL` is a plain-HTTP LAN
|
||||
address gets a warning to set it to the tailnet HTTPS address.
|
||||
|
||||
### Verification (network)
|
||||
|
||||
- `cd web && node --test test/`: `tests 24, pass 24, fail 0`. This includes
|
||||
`listenPolicy.test.mjs`, which uses real sockets on this Mac's LAN (10.0.0.x)
|
||||
and tailnet (100.85.25.33) addresses.
|
||||
- `umask 022 && go test ./internal/discovery ./cmd/pipelines-server ./internal/server`:
|
||||
all `ok`. This includes `TestPlanWithoutLANAdvertisesOnlyTailnetAndPublicRoutes`
|
||||
and `TestBrowseOmitsTheLANRouteWhenTheAdvertSaysLan0`.
|
||||
- `cd scripts && python3 -m unittest`: `Ran 229 tests ... OK (skipped=1)`.
|
||||
This includes `test_web_access` and
|
||||
`test_upgrade_keeps_an_all_interfaces_web_host_with_a_warning_and_lan_is_opt_in`.
|
||||
- App: `tool/memwatch/full.sh 'test/features/updates test/features/onboarding test/features/settings'`
|
||||
gave `PASS: 212 passed, 0 failed`, and `tool/analyze.sh` was clean.
|
||||
- Scratch end-to-end: an isolated `HOME`, an OS-chosen port, real sign-in, and
|
||||
the proxy started with `manage.web_listen_args` output:
|
||||
|
||||
```
|
||||
== default loopback + tailnet: serve.mjs --port 53780 --host :: --allow loopback,tailnet
|
||||
GET http://127.0.0.1:53780/login -> 200 POST -> 303
|
||||
GET http://100.85.25.33:53780/login -> 200 POST -> 303
|
||||
GET http://10.0.0.249:53780/login -> 000 (connection reset, curl exit 56)
|
||||
== LAN opted in, no plain login: ... --allow loopback,tailnet,lan
|
||||
GET/POST via 127.0.0.1 and 100.85.25.33 -> 200/303; via 10.0.0.249 -> 403/403
|
||||
== LAN with lan_plain_login: ... --lan-plain-login
|
||||
GET/POST via 10.0.0.249 -> 200/303
|
||||
```
|
||||
|
||||
The live mDNS advert was not exercised, because a scratch server would publish
|
||||
the same `Pipelines-<host>` instance name as the live one. The records are
|
||||
covered by `Plan`/`Zone` tests.
|
||||
|
||||
## Update feed signing
|
||||
|
||||
Before this change the app trusted whatever `/updates/` served: the Linux
|
||||
self-update installed and relaunched an archive whose sha256 came from the same
|
||||
`version.json`, Android skipped its signer check when the feed omitted the
|
||||
field, and the macOS zip had no check beyond that sha256. A server (or anyone
|
||||
able to write its update directory, or answer for its address) could hand every
|
||||
client another build.
|
||||
|
||||
### What is signed
|
||||
|
||||
- Publishing (`./install --apk`, `./install --linux|--macos --publish`) writes
|
||||
`version.json` and, beside it, a detached `version.json.sig`:
|
||||
`{"algorithm":"ed25519","publicKey":"<base64 32 bytes>","signature":"<base64 64 bytes>"}`.
|
||||
The signature covers the exact bytes of `version.json`, which carry the
|
||||
artifact `sha256` (and, for Android, `signingCertificateSha256`). The
|
||||
signature is written first, then the feed, both atomically; a check racing a
|
||||
publish fails once and succeeds on the next check.
|
||||
- Code: `scripts/update_signing.py` (keys, signing, `publish_feed`),
|
||||
`scripts/app_build.py` (`build_apk`, web and desktop defines),
|
||||
`scripts/app_desktop_build.py` (`publish_desktop(..., signer=...)`).
|
||||
|
||||
### Key management
|
||||
|
||||
- Each installation has one Ed25519 key in
|
||||
`~/.config/pipelines/update-signing-key.pem` (mode 0600, directory 0700),
|
||||
generated once on the first `./install` or app build/publish and never
|
||||
replaced. `PIPELINES_UPDATE_SIGNING_KEY` names another path. A key file that
|
||||
is a symlink or readable by others is refused. It is never in the repository.
|
||||
`scripts/uninstall.py` (full reset) removes `~/.config/pipelines`, and so
|
||||
the key; see "Lost or replaced key" below.
|
||||
- Cryptography is the OpenSSL 3 command line (`genpkey`, `pkey`,
|
||||
`pkeyutl -sign -rawin`). The installer's Python has no `cryptography`
|
||||
package, and macOS's `/usr/bin/openssl` is LibreSSL 3.3.6, which has no
|
||||
Ed25519 (`unable to load Private Key`). The tool is chosen as
|
||||
`PIPELINES_OPENSSL`, then `/opt/homebrew/opt/openssl@3/bin/openssl`,
|
||||
`/usr/local/opt/openssl@3/bin/openssl`, then `openssl` on PATH, taking the
|
||||
first that can generate an Ed25519 key. Without one the install stops with
|
||||
`Update signing needs OpenSSL 3 with Ed25519 ...` (`brew install openssl@3`).
|
||||
On this Mac `openssl` on PATH is Homebrew OpenSSL 3.6.4.
|
||||
- Print this installation's public key:
|
||||
`python3 scripts/update_signing.py`.
|
||||
|
||||
### Trust model
|
||||
|
||||
- Every app build embeds the keys it trusts as
|
||||
`--dart-define=PIPELINES_UPDATE_PUBLIC_KEY=<key>[,<key>...]` (apk, linux,
|
||||
macos and web builds; the define is part of the build cache key, so a key
|
||||
change rebuilds). The list is this installation's own key first, then each
|
||||
base64 key in `~/.config/pipelines/update-trusted-keys` (one per line, `#`
|
||||
comments allowed; a malformed line stops the build).
|
||||
- Each server signs its own feed with its own key. A client accepts a feed
|
||||
only when `version.json.sig` names an embedded key and the signature over
|
||||
the served bytes verifies with it; the client checks the signature before it
|
||||
believes anything in the feed. So an app built on the Mac updates from tank
|
||||
only if tank's key was listed in the Mac's `update-trusted-keys` when that
|
||||
app was built (and vice versa). There is no trust-on-first-use pinning:
|
||||
trust comes only from the build, which keeps the model to one rule.
|
||||
- With keys embedded, the app refuses: an unsigned feed (no `.sig`), an
|
||||
unreadable signature, a key it does not embed, a bad signature, a feed
|
||||
without `sha256`, and an Android feed without `signingCertificateSha256`
|
||||
(or when the installed signature is unknown). The Android certificate must
|
||||
still match the installed app's, as before.
|
||||
- A build with no embedded key (a developer `flutter run`/`flutter build`
|
||||
outside the installer) keeps the previous unsigned behaviour. Installer
|
||||
builds always embed a key.
|
||||
- Verification in the app is pure Dart (`app/lib/features/updates/ed25519.dart`,
|
||||
RFC 8032 verify only, BigInt arithmetic, checked against RFC 8032 test
|
||||
vectors 1-3; no new package) and
|
||||
`app/lib/features/updates/update_feed_signature.dart`; the reader is
|
||||
`fetchLatestScoped` in `app_updates_io.dart`. The web Settings "Android
|
||||
download" link is a browser download the operator installs by hand and is
|
||||
unchanged.
|
||||
|
||||
### Migration (the live Mac install and tank)
|
||||
|
||||
Not done by this change; nothing was deployed. In order, on each machine:
|
||||
|
||||
1. `./install` (or the next `./install --apk` / desktop publish) generates
|
||||
`~/.config/pipelines/update-signing-key.pem`. Check:
|
||||
`ls -l ~/.config/pipelines/update-signing-key.pem` shows `-rw-------`.
|
||||
2. To let one machine's apps update from the other, exchange keys before
|
||||
building: on tank run `python3 scripts/update_signing.py` and append the
|
||||
line to the Mac's `~/.config/pipelines/update-trusted-keys`; do the
|
||||
reverse on tank. Skip this if each machine's apps only update from it.
|
||||
3. Rebuild and publish: `./install --apk`, `./install --macos --publish
|
||||
--flutter <flutter>`, `./install --linux --publish`. Each feed directory
|
||||
then holds `version.json.sig`.
|
||||
4. Installed apps from before this change do not check signatures; they take
|
||||
the next signed release normally (the `.sig` is ignored), and from then on
|
||||
they refuse unsigned or foreign feeds. Phones whose feed came from the other
|
||||
machine need step 2 before step 3.
|
||||
5. Lost or replaced key (for example after a full uninstall): apps built with
|
||||
the old key refuse the new feeds. Install the new build once by hand (APK
|
||||
download from Settings in the browser, or the desktop archive), or restore
|
||||
the old key file from backup.
|
||||
|
||||
### Verification output
|
||||
|
||||
- `cd scripts && python3 -m unittest test_update_signing test_app_build
|
||||
test_app_desktop_build test_setup_native`: `Ran 81 tests ... OK`. Covers key
|
||||
generation once at 0600 in a scratch HOME, refusal of readable/symlinked
|
||||
keys, LibreSSL-style tools skipped, signatures checked independently with
|
||||
`openssl pkeyutl -verify` (and a tampered message refused), feed plus `.sig`
|
||||
publication, trusted-key list parsing, APK publication signed with the
|
||||
generated key, desktop publication passing the exact feed bytes to the
|
||||
signer, and the committed Dart fixture being a valid OpenSSL signature.
|
||||
- `cd app && tool/memwatch/full.sh 'test/features/updates test/features/settings'`:
|
||||
`memwatch: PASS: 187 passed, 0 failed, 0 skipped`. New:
|
||||
`test/features/updates/ed25519_test.dart` (RFC 8032 vectors; changed
|
||||
message, signature, key; short inputs; S >= L; non-canonical key) and
|
||||
`test/features/updates/update_feed_signature_test.dart` (the
|
||||
Python-signed fixture `test/features/updates/fixtures/signed_feed/`
|
||||
verifies; tampered feed and signature refused; missing signature refused
|
||||
with keys, accepted without; untrusted and swapped keys refused; signed
|
||||
feeds require sha256 and the Android signer; and the HTTP reader
|
||||
`fetchLatestScoped` against a loopback server refuses an unsigned and a
|
||||
tampered feed and lets the genuine one past the signature check).
|
||||
`test/core/session`: `PASS: 18 passed`. `tool/analyze.sh`: `analyze: clean`.
|
||||
Reference in New Issue
Block a user