Files
pipelines/docs/notes/network-and-update-signing.md

14 KiB

Network defaults and update-feed signing

Where the web proxy listens

Before this change ./install bound the web proxy to 0.0.0.0 over plain HTTP. With the macOS firewall off, anyone on the same café or hotel Wi-Fi could reach the sign-in page, and the password and the 30-day session cookie crossed that network unencrypted. The operator reaches the Mac over Tailscale (https://<host>.ts.net:8443 through tailscale serve to 127.0.0.1:8083, and ws://100.x:8083 from tank), so the LAN listener was not needed.

Defaults and opt-ins

web.host in pipelines-install.json takes one of two forms:

web.host Proxy flags (manage.web_listen_args) Who can connect
tailnet (new installs) --host :: --allow loopback,tailnet loopback and this machine's Tailscale IPv4/IPv6 addresses
tailnet with web.lan: true ... --allow loopback,tailnet,lan also the LAN, but no sign-in or signed-in pages over plain HTTP
tailnet with web.lan: true, web.lan_plain_login: true ... --lan-plain-login the LAN, with plain-HTTP sign-in (the old behaviour)
an IPv4 address, e.g. 0.0.0.0 --host 0.0.0.0 whatever the bind reaches, unfiltered (the old behaviour)

Installer flags: --web-host tailnet|IPv4, --web-lan / --no-web-lan, --web-lan-plain-login / --no-web-lan-plain-login. A reinstall keeps the saved values unless you pass the flag again. --web-lan on an install that still has a literal host moves it to tailnet. --web-lan together with an explicit IPv4 --web-host is refused.

On an opted-in LAN without lan_plain_login, a plain-HTTP LAN connection gets 403 with an explanation for POST /login, GET /login, every signed-in page, /updates/, /api/* and websocket upgrades. Only the public icons and manifest are served. Risk of web.lan_plain_login: anyone on that network who can watch the traffic sees the password and session cookie, and the cookie works for 30 days. Use it only on a network you trust. X-Forwarded-Proto is not trusted from the LAN, because a client can set it to anything.

Why the proxy binds the wildcard and filters each connection

Tailscale often starts after the proxy (both start at login), and its address can change. Binding the tailnet IP itself would fail at boot, and keeping it working would need retry and rebind logic. Instead the proxy binds ::, which on macOS and Linux also accepts IPv4 (it falls back to 0.0.0.0 on a host without IPv6). On every connection it drops the socket, before any HTTP is read, unless both ends are loopback (127/8, ::1) or both are tailnet (100.64.0.0/10, fd7a:115c:a1e0::/48); see web/src/listenPolicy.js.

Both ends are checked because macOS uses the weak host model: it accepts a packet addressed to its tailnet IP on any interface. A LAN host that spoofs a tailnet source address cannot finish the TCP handshake, because the reply is routed into the tunnel.

The cost: a LAN scan sees the port open and immediately reset, and no HTTP is served. One edge case is not handled: Tailscale subnet routes without SNAT deliver peers with non-tailnet source addresses, and those are treated as LAN.

Discovery (mDNS)

manage.mdns_args passes -mdns-lan=false when web.host is tailnet without web.lan. discovery.Plan with NoLAN then makes these changes:

  • The A records are the tailnet IPv4 addresses, not the LAN ones.
  • TXT gains lan=0, and ts= and public= stay.
  • The port is still the proxy's, which answers on the tailnet.
  • With no tailnet address, nothing is published (ErrNotReachable).

The advert is still multicast on the LAN, so a tailnet client on the same Wi-Fi finds the machine. Browse (Go, also behind /api/discovery) and the app's DiscoveredServer.fromRecord drop the LAN route when lan=0, so the endpoint becomes the first tailnet or public route. An older client still sees a "LAN" route, but it points at the tailnet IP, which works from the tailnet.

Firewall

The installer turns nothing on at the OS level. It only reads socketfilterfw --getglobalstate and prints a recommendation when the firewall is off. On this Mac on 2026-11-10 it reported Firewall is disabled. (State = 0). Recommendation: turn it on (System Settings > Network > Firewall). The proxy no longer depends on it, but other local services (ntfy on :8090, development servers) do.

Migration

  • Live Mac (web.host: 0.0.0.0, port 8083, public URL https://alans-macbook-pro.tail6bf4dc.ts.net:8443): a plain ./install keeps 0.0.0.0 and prints WARNING: the web proxy listens on 0.0.0.0 over plain HTTP ... Recommended: ./install --web-host tailnet .... To switch, run ./install --web-host tailnet while no run or chat is active (the install restarts the services). tailscale serve still reaches 127.0.0.1:8083, and tank's ws://100.85.25.33:8083 is a tailnet connection, so both keep working. Phones on the home Wi-Fi that saved ws://10.0.0.x:8083 must switch to the tailnet route (rediscover, or pick the Tailscale address).
  • tank: the same ./install --web-host tailnet. Check first that every client reaches it through Tailscale.
  • An install whose recorded PIPELINES_WEB_PUBLIC_URL is a plain-HTTP LAN address gets a warning to set it to the tailnet HTTPS address.

Verification (network)

  • cd web && node --test test/: tests 24, pass 24, fail 0. This includes listenPolicy.test.mjs, which uses real sockets on this Mac's LAN (10.0.0.x) and tailnet (100.85.25.33) addresses.
  • umask 022 && go test ./internal/discovery ./cmd/pipelines-server ./internal/server: all ok. This includes TestPlanWithoutLANAdvertisesOnlyTailnetAndPublicRoutes and TestBrowseOmitsTheLANRouteWhenTheAdvertSaysLan0.
  • cd scripts && python3 -m unittest: Ran 229 tests ... OK (skipped=1). This includes test_web_access and test_upgrade_keeps_an_all_interfaces_web_host_with_a_warning_and_lan_is_opt_in.
  • App: tool/memwatch/full.sh 'test/features/updates test/features/onboarding test/features/settings' gave PASS: 212 passed, 0 failed, and tool/analyze.sh was clean.
  • Scratch end-to-end: an isolated HOME, an OS-chosen port, real sign-in, and the proxy started with manage.web_listen_args output:
== default loopback + tailnet: serve.mjs --port 53780 --host :: --allow loopback,tailnet
GET  http://127.0.0.1:53780/login -> 200    POST -> 303
GET  http://100.85.25.33:53780/login -> 200 POST -> 303
GET  http://10.0.0.249:53780/login -> 000 (connection reset, curl exit 56)
== LAN opted in, no plain login: ... --allow loopback,tailnet,lan
GET/POST via 127.0.0.1 and 100.85.25.33 -> 200/303; via 10.0.0.249 -> 403/403
== LAN with lan_plain_login: ... --lan-plain-login
GET/POST via 10.0.0.249 -> 200/303

The live mDNS advert was not exercised, because a scratch server would publish the same Pipelines-<host> instance name as the live one. The records are covered by Plan/Zone tests.

Update feed signing

Before this change the app trusted whatever /updates/ served: the Linux self-update installed and relaunched an archive whose sha256 came from the same version.json, Android skipped its signer check when the feed omitted the field, and the macOS zip had no check beyond that sha256. A server (or anyone able to write its update directory, or answer for its address) could hand every client another build.

What is signed

  • Publishing (./install --apk, ./install --linux|--macos --publish) writes version.json and, beside it, a detached version.json.sig: {"algorithm":"ed25519","publicKey":"<base64 32 bytes>","signature":"<base64 64 bytes>"}. The signature covers the exact bytes of version.json, which carry the artifact sha256 (and, for Android, signingCertificateSha256). The signature is written first, then the feed, both atomically; a check racing a publish fails once and succeeds on the next check.
  • Code: scripts/update_signing.py (keys, signing, publish_feed), scripts/app_build.py (build_apk, web and desktop defines), scripts/app_desktop_build.py (publish_desktop(..., signer=...)).

Key management

  • Each installation has one Ed25519 key in ~/.config/pipelines/update-signing-key.pem (mode 0600, directory 0700), generated once on the first ./install or app build/publish and never replaced. PIPELINES_UPDATE_SIGNING_KEY names another path. A key file that is a symlink or readable by others is refused. It is never in the repository. scripts/uninstall.py (full reset) removes ~/.config/pipelines, and so the key; see "Lost or replaced key" below.
  • Cryptography is the OpenSSL 3 command line (genpkey, pkey, pkeyutl -sign -rawin). The installer's Python has no cryptography package, and macOS's /usr/bin/openssl is LibreSSL 3.3.6, which has no Ed25519 (unable to load Private Key). The tool is chosen as PIPELINES_OPENSSL, then /opt/homebrew/opt/openssl@3/bin/openssl, /usr/local/opt/openssl@3/bin/openssl, then openssl on PATH, taking the first that can generate an Ed25519 key. Without one the install stops with Update signing needs OpenSSL 3 with Ed25519 ... (brew install openssl@3). On this Mac openssl on PATH is Homebrew OpenSSL 3.6.4.
  • Print this installation's public key: python3 scripts/update_signing.py.

Trust model

  • Every app build embeds the keys it trusts as --dart-define=PIPELINES_UPDATE_PUBLIC_KEY=<key>[,<key>...] (apk, linux, macos and web builds; the define is part of the build cache key, so a key change rebuilds). The list is this installation's own key first, then each base64 key in ~/.config/pipelines/update-trusted-keys (one per line, # comments allowed; a malformed line stops the build).
  • Each server signs its own feed with its own key. A client accepts a feed only when version.json.sig names an embedded key and the signature over the served bytes verifies with it; the client checks the signature before it believes anything in the feed. So an app built on the Mac updates from tank only if tank's key was listed in the Mac's update-trusted-keys when that app was built (and vice versa). There is no trust-on-first-use pinning: trust comes only from the build, which keeps the model to one rule.
  • With keys embedded, the app refuses: an unsigned feed (no .sig), an unreadable signature, a key it does not embed, a bad signature, a feed without sha256, and an Android feed without signingCertificateSha256 (or when the installed signature is unknown). The Android certificate must still match the installed app's, as before.
  • A build with no embedded key (a developer flutter run/flutter build outside the installer) keeps the previous unsigned behaviour. Installer builds always embed a key.
  • Verification in the app is pure Dart (app/lib/features/updates/ed25519.dart, RFC 8032 verify only, BigInt arithmetic, checked against RFC 8032 test vectors 1-3; no new package) and app/lib/features/updates/update_feed_signature.dart; the reader is fetchLatestScoped in app_updates_io.dart. The web Settings "Android download" link is a browser download the operator installs by hand and is unchanged.

Migration (the live Mac install and tank)

Not done by this change; nothing was deployed. In order, on each machine:

  1. ./install (or the next ./install --apk / desktop publish) generates ~/.config/pipelines/update-signing-key.pem. Check: ls -l ~/.config/pipelines/update-signing-key.pem shows -rw-------.
  2. To let one machine's apps update from the other, exchange keys before building: on tank run python3 scripts/update_signing.py and append the line to the Mac's ~/.config/pipelines/update-trusted-keys; do the reverse on tank. Skip this if each machine's apps only update from it.
  3. Rebuild and publish: ./install --apk, ./install --macos --publish --flutter <flutter>, ./install --linux --publish. Each feed directory then holds version.json.sig.
  4. Installed apps from before this change do not check signatures; they take the next signed release normally (the .sig is ignored), and from then on they refuse unsigned or foreign feeds. Phones whose feed came from the other machine need step 2 before step 3.
  5. Lost or replaced key (for example after a full uninstall): apps built with the old key refuse the new feeds. Install the new build once by hand (APK download from Settings in the browser, or the desktop archive), or restore the old key file from backup.

Verification output

  • cd scripts && python3 -m unittest test_update_signing test_app_build test_app_desktop_build test_setup_native: Ran 81 tests ... OK. Covers key generation once at 0600 in a scratch HOME, refusal of readable/symlinked keys, LibreSSL-style tools skipped, signatures checked independently with openssl pkeyutl -verify (and a tampered message refused), feed plus .sig publication, trusted-key list parsing, APK publication signed with the generated key, desktop publication passing the exact feed bytes to the signer, and the committed Dart fixture being a valid OpenSSL signature.
  • cd app && tool/memwatch/full.sh 'test/features/updates test/features/settings': memwatch: PASS: 187 passed, 0 failed, 0 skipped. New: test/features/updates/ed25519_test.dart (RFC 8032 vectors; changed message, signature, key; short inputs; S >= L; non-canonical key) and test/features/updates/update_feed_signature_test.dart (the Python-signed fixture test/features/updates/fixtures/signed_feed/ verifies; tampered feed and signature refused; missing signature refused with keys, accepted without; untrusted and swapped keys refused; signed feeds require sha256 and the Android signer; and the HTTP reader fetchLatestScoped against a loopback server refuses an unsigned and a tampered feed and lets the genuine one past the signature check). test/core/session: PASS: 18 passed. tool/analyze.sh: analyze: clean.