Git-native secrets vault (SOPS + age) (#34)

2026-07-22 03:21:46 +00:00 · 2026-07-22 03:21:46 +00:00 · 792e2c5d3b
commit 792e2c5d3b
parent c7503cf5a0
14 changed files with 776 additions and 7 deletions
--- a/.claude/skills/key-gaps/SKILL.md
+++ b/.claude/skills/key-gaps/SKILL.md
@ -0,0 +1,71 @@
+---
+name: key-gaps
+description: Audit which secret keys are missing, partially configured, or drifting across Thermograph environments (prod, beta, dev) — comparing the live /etc/thermograph.env on each box and/or the SOPS vault files (deploy/secrets/*.yaml). Use when asked about "key gaps", "missing keys/secrets", "what's not set", cross-environment secret drift, or as a pre-deploy / pre-cutover safety check. Reads key NAMES only, never values.
+---
+
+# Key-gap audit
+
+Reports, per environment and across them, where secret **keys** are missing or
+inconsistent — with special weight on the failure modes that bite silently here.
+
+Run the deterministic script; don't eyeball it:
+
+```
+python3 .claude/skills/key-gaps/key_gaps.py <env>=<source> [<env>=<source> ...]
+```
+
+`<source>` is either a **SOPS file** (`deploy/secrets/prod.yaml`) or a **key-list
+file** (one `KEY` per line, or `KEY=`/`KEY:` lines). Keys are readable in a SOPS file
+even encrypted, so **no age key or decryption is needed**. Exit status is non-zero if
+any CRITICAL or required gap is found (so it also works as a CI/pre-deploy gate).
+
+## Audit the SOPS vault (offline, no SSH)
+
+```
+python3 .claude/skills/key-gaps/key_gaps.py \
+  prod=deploy/secrets/prod.yaml beta=deploy/secrets/beta.yaml
+```
+
+## Audit the LIVE boxes (read-only, key names only)
+
+Gather the key names over SSH (never the values), then audit. Hosts/keys per INFRA.md:
+
+```sh
+K=~/.ssh/thermograph_agent_ed25519
+# sudo: /etc/thermograph.env is root-owned (0640). On prod `agent` can read it
+# directly too, but sudo works uniformly on both boxes.
+ssh -i $K agent@169.58.46.181 'sudo grep -oE "^[A-Z_]+=" /etc/thermograph.env' > /tmp/prod.keys  # prod
+ssh -i $K agent@75.119.132.91  'sudo grep -oE "^[A-Z_]+=" /etc/thermograph.env' > /tmp/beta.keys  # beta
+python3 .claude/skills/key-gaps/key_gaps.py prod=/tmp/prod.keys beta=/tmp/beta.keys
+```
+
+## Verify a SOPS cutover matches live
+
+Before flipping a host onto the vault, confirm the rendered set equals the live set —
+mix the two source types for the same box:
+
+```
+python3 .claude/skills/key-gaps/key_gaps.py live=/tmp/prod.keys vault=deploy/secrets/prod.yaml
+```
+The `Cross-environment drift` section should report **none** for tracked keys.
+
+## Reading the output
+
+- **CRITICAL** — a required, *self-generating* secret is missing (`THERMOGRAPH_AUTH_SECRET`,
+  `THERMOGRAPH_VAPID_PRIVATE_KEY/_PUBLIC_KEY`). Missing here is the worst case: the app
+  silently mints a new value on boot, invalidating every login session and push
+  subscription. Fix before deploying.
+- **MISSING** — a required secret is absent (`POSTGRES_PASSWORD`, `THERMOGRAPH_METRICS_TOKEN`,
+  `REGISTRY_TOKEN`).
+- **Feature GAP** — a feature is *partially* configured (e.g. `discord-account-linking`
+  has the app id but not the client secret), so it's silently broken. A fully-unset
+  feature is "off", not a gap.
+- **drift** — a tracked key is in some environments but not others; often intentional,
+  but worth a glance.
+
+## Keeping it current
+
+The secret manifest (which keys are required / self-generating / grouped into features)
+lives at the top of `key_gaps.py` in `REQUIRED`, `OPTIONAL_SELF_GEN`, `FEATURE_GROUPS`,
+and `STANDALONE_OPTIONAL`. When a new `THERMOGRAPH_*` credential is added to
+`deploy/thermograph.env.example` / `deploy/secrets/`, add it there too.
--- a/.claude/skills/key-gaps/key_gaps.py
+++ b/.claude/skills/key-gaps/key_gaps.py
@ -0,0 +1,185 @@
+#!/usr/bin/env python3
+"""Audit secret-key coverage across Thermograph environments.
+
+Reports, per environment and across them:
+  - CRITICAL gaps: a required, *self-generating* secret is missing. Missing here is
+    the worst case — the app silently mints a new value on boot (AUTH_SECRET, VAPID,
+    INDEXNOW), invalidating every session / push subscription.
+  - Required gaps: a required secret is missing.
+  - Feature gaps: a feature is *partially* configured (some keys present, some not),
+    so it's silently broken. Fully-unset features are "off", not a gap.
+  - Drift: a key present in one environment but absent in another.
+
+It reads only KEY NAMES, never values — safe to run anywhere. Sources per environment:
+  * a SOPS-encrypted YAML (deploy/secrets/<env>.yaml) — keys are plaintext even when
+    encrypted, so no age key or decryption is needed; or
+  * a plain key-list file (one KEY per line, or KEY=... lines) — e.g. the output of
+    `ssh <box> 'grep -oE "^[A-Z_]+=" /etc/thermograph.env'` for a live audit.
+
+Usage:
+  key_gaps.py prod=deploy/secrets/prod.yaml beta=deploy/secrets/beta.yaml
+  key_gaps.py prod=/tmp/prod.keys           # a live key-list gathered over SSH
+Exit status is non-zero if any CRITICAL or required gap is found (usable in CI).
+"""
+from __future__ import annotations
+
+import re
+import sys
+
+# --- what counts as a secret/key, and how it behaves --------------------------
+# category: "required" (app needs it) | "optional" (fine to omit)
+# self_gen: missing => the app silently generates a replacement (data-loss hazard)
+REQUIRED = {
+    # key: self_generating?
+    "POSTGRES_PASSWORD": False,
+    "THERMOGRAPH_AUTH_SECRET": True,
+    "THERMOGRAPH_VAPID_PRIVATE_KEY": True,
+    "THERMOGRAPH_VAPID_PUBLIC_KEY": True,
+    "THERMOGRAPH_METRICS_TOKEN": False,
+    "REGISTRY_TOKEN": False,
+}
+OPTIONAL_SELF_GEN = {
+    "THERMOGRAPH_INDEXNOW_KEY": True,   # regenerates, but persisted to a file — low risk
+}
+# Features that only work if ALL their keys are set together; some-but-not-all = broken.
+FEATURE_GROUPS = {
+    "discord-account-linking": ["THERMOGRAPH_DISCORD_APP_ID", "THERMOGRAPH_DISCORD_CLIENT_SECRET"],
+    "mail-smtp": ["THERMOGRAPH_SMTP_USER", "THERMOGRAPH_SMTP_PASSWORD"],
+}
+# Standalone optional keys: reported present/absent, never a "gap" on their own.
+STANDALONE_OPTIONAL = [
+    "THERMOGRAPH_DISCORD_WEBHOOK",      # daily post
+    "THERMOGRAPH_DISCORD_PUBLIC_KEY",   # slash-command interactions
+    "THERMOGRAPH_DISCORD_BOT_TOKEN",    # bot DMs / gateway bot
+]
+
+C = {"red": "\033[31m", "yellow": "\033[33m", "green": "\033[32m",
+     "dim": "\033[2m", "bold": "\033[1m", "off": "\033[0m"}
+
+
+def keys_from_source(path: str) -> set[str]:
+    """Extract KEY names from a SOPS yaml or a plain key-list file (values ignored)."""
+    with open(path, encoding="utf-8", errors="replace") as fh:
+        text = fh.read()
+    keys: set[str] = set()
+    in_sops_block = False
+    for raw in text.splitlines():
+        line = raw.rstrip("\n")
+        if line.startswith("sops:"):          # SOPS metadata block — skip its subkeys
+            in_sops_block = True
+            continue
+        if in_sops_block:
+            if line and not line[0].isspace():
+                in_sops_block = False           # dedented back to a real key
+            else:
+                continue
+        s = line.strip()
+        if not s or s.startswith("#"):
+            continue
+        # Accept "KEY: ..." (yaml), "KEY=..." (dotenv), or a bare "KEY" (key-list).
+        # The whole stripped line must be just an identifier optionally followed by
+        # a : or = — so prose/nested lines don't get mistaken for keys.
+        m = re.match(r"^([A-Za-z_][A-Za-z0-9_]*)\s*(?:[:=].*)?$", s)
+        if m:
+            keys.add(m.group(1))
+    return keys
+
+
+def audit(envs: dict[str, set[str]]) -> int:
+    names = list(envs)
+    print(f"{C['bold']}Key-gap audit — {', '.join(names)}{C['off']}")
+    for n in names:
+        print(f"  {C['dim']}{n}: {len(envs[n])} keys{C['off']}")
+    print()
+    critical = required = feature = 0
+
+    def present(env, key):  # noqa: ANN001
+        return key in envs[env]
+
+    # Required + self-generating
+    print(f"{C['bold']}Required secrets{C['off']}")
+    for key, self_gen in {**REQUIRED, **OPTIONAL_SELF_GEN}.items():
+        req = key in REQUIRED
+        missing = [n for n in names if not present(n, key)]
+        if not missing:
+            print(f"  {C['green']}OK{C['off']}   {key}  (all environments)")
+            continue
+        if req and self_gen:
+            critical += 1
+            tag = f"{C['red']}CRITICAL{C['off']}"
+            note = " — self-generates on boot: MISSING = every session/subscription invalidated"
+        elif req:
+            required += 1
+            tag = f"{C['red']}MISSING {C['off']}"
+            note = ""
+        else:
+            tag = f"{C['yellow']}note    {C['off']}"
+            note = " — self-generates (persisted to a file; low risk)"
+        print(f"  {tag} {key}  absent in: {', '.join(missing)}{C['dim']}{note}{C['off']}")
+    print()
+
+    # Feature groups — partial config = broken
+    print(f"{C['bold']}Feature groups (all-or-nothing){C['off']}")
+    for feat, keys in FEATURE_GROUPS.items():
+        for n in names:
+            have = [k for k in keys if present(n, k)]
+            if not have:
+                print(f"  {C['dim']}off  {feat} @ {n} (none set){C['off']}")
+            elif len(have) == len(keys):
+                print(f"  {C['green']}OK{C['off']}   {feat} @ {n} (enabled)")
+            else:
+                feature += 1
+                miss = [k for k in keys if not present(n, k)]
+                print(f"  {C['red']}GAP {C['off']} {feat} @ {n}: has {have}, MISSING {miss}")
+    print()
+
+    # Standalone optional
+    print(f"{C['bold']}Optional keys{C['off']}")
+    for key in STANDALONE_OPTIONAL:
+        where = [n for n in names if present(n, key)]
+        state = f"set in: {', '.join(where)}" if where else "unset everywhere"
+        print(f"  {C['dim']}{key}: {state}{C['off']}")
+    print()
+
+    # Cross-env drift
+    if len(names) > 1:
+        print(f"{C['bold']}Cross-environment drift{C['off']}")
+        union = set().union(*envs.values())
+        drift = False
+        for key in sorted(union):
+            where = [n for n in names if key in envs[n]]
+            if len(where) != len(names) and (key in REQUIRED or key in OPTIONAL_SELF_GEN
+                                             or any(key in g for g in FEATURE_GROUPS.values())
+                                             or key in STANDALONE_OPTIONAL):
+                drift = True
+                absent = [n for n in names if n not in where]
+                print(f"  {C['yellow']}drift{C['off']} {key}: in {', '.join(where)}; not in {', '.join(absent)}")
+        if not drift:
+            print(f"  {C['green']}none{C['off']} (tracked keys are consistent)")
+        print()
+
+    total = critical + required + feature
+    color = C["red"] if total else C["green"]
+    print(f"{color}{C['bold']}Summary: {critical} critical, {required} required, "
+          f"{feature} feature gaps{C['off']}")
+    return 1 if (critical or required) else 0
+
+
+def main(argv: list[str]) -> int:
+    args = [a for a in argv[1:] if "=" in a]
+    if not args:
+        print(__doc__)
+        return 2
+    envs: dict[str, set[str]] = {}
+    for a in args:
+        name, _, path = a.partition("=")
+        try:
+            envs[name] = keys_from_source(path)
+        except OSError as e:
+            print(f"!! {name}: cannot read {path}: {e}", file=sys.stderr)
+            return 2
+    return audit(envs)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv))
--- a/.forgejo/workflows/secrets-guard.yml
+++ b/.forgejo/workflows/secrets-guard.yml
@ -0,0 +1,36 @@
+name: secrets-guard
+
+# Fail the build if any deploy/secrets/*.yaml is committed UNENCRYPTED. SOPS stores a
+# `sops:` metadata block and wraps every value in ENC[...]; a plaintext leak has
+# neither. This is the backstop against the classic "committed the decrypted file by
+# accident" incident. .sops.yaml (the plaintext config) and README.md are not *.yaml
+# under deploy/secrets/, so they're correctly ignored.
+on:
+  pull_request:
+  push:
+    branches: [dev, main, release]
+
+jobs:
+  encrypted:
+    runs-on: docker
+    steps:
+      - uses: actions/checkout@v4
+      - name: Assert deploy/secrets/*.yaml are SOPS-encrypted
+        run: |
+          set -euo pipefail
+          shopt -s nullglob
+          files=(deploy/secrets/*.yaml)
+          if [ ${#files[@]} -eq 0 ]; then
+            echo "no deploy/secrets/*.yaml yet — nothing to check"
+            exit 0
+          fi
+          fail=0
+          for f in "${files[@]}"; do
+            if grep -q '^sops:' "$f" && grep -q 'ENC\[AES256_GCM' "$f"; then
+              echo "ok: $f is SOPS-encrypted"
+            else
+              echo "::error file=$f::NOT SOPS-encrypted — refusing to accept a plaintext secrets file"
+              fail=1
+            fi
+          done
+          exit $fail
--- a/.sops.yaml
+++ b/.sops.yaml
@ -0,0 +1,14 @@
+# SOPS creation rules — how `sops <file>` encrypts secrets in this repo.
+#
+# Every deploy/secrets/*.yaml is encrypted to the age recipient below (values only;
+# keys stay readable for clean diffs). The matching PRIVATE key is NEVER in the repo
+# — it lives at ~/.config/sops/age/keys.txt on the operator's machine and at
+# /etc/thermograph/age.key on each host that renders secrets at deploy time.
+#
+# Rotating the age identity itself: add the new recipient here, run
+#   sops updatekeys deploy/secrets/*.yaml
+# to re-encrypt to both, distribute the new private key, then drop the old recipient
+# and updatekeys again. See deploy/secrets/README.md.
+creation_rules:
+  - path_regex: ^deploy/secrets/.*\.yaml$
+    age: age1xx4dzs0dxlwvkv9sjuqzsphl7lfrxannkfken374yu2qvvcte9sqzktqt2
--- a/deploy/deploy.sh
+++ b/deploy/deploy.sh
@ -13,7 +13,20 @@ cd "$APP_DIR"

 # Secrets (POSTGRES_PASSWORD, VAPID keys, AUTH_SECRET, ...) drive compose
 # interpolation and are also loaded into the backend container via env_file.
-# Source them here so a by-hand run interpolates the same as the systemd unit does.
+#
+# When this host is configured for SOPS (an age key + /etc/thermograph/secrets-env),
+# first render /etc/thermograph.env from the committed encrypted source of truth
+# (deploy/secrets/*.yaml) so a key rotation is just an edit+commit+deploy. The guard
+# on the helper's existence keeps the very deploy that INTRODUCES this file safe: on
+# the first pass the checkout may predate it (it arrives with the git reset below,
+# after which deploy.sh re-execs), so a missing helper simply falls back to the
+# existing /etc/thermograph.env. Then source it so a by-hand run interpolates the
+# same as the systemd unit does. See deploy/render-secrets.sh + deploy/secrets/.
+if [ -f "$APP_DIR/deploy/render-secrets.sh" ]; then
+  # shellcheck source=deploy/render-secrets.sh
+  . "$APP_DIR/deploy/render-secrets.sh"
+  render_thermograph_secrets "$APP_DIR"
+fi
 set -a; . /etc/thermograph.env 2>/dev/null || true; set +a

 # Pre-warm the ~750 city-page archives so /climate pages serve from cache and a
--- a/deploy/provision-secrets.sh
+++ b/deploy/provision-secrets.sh
@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+# Provision a host to render secrets from the SOPS vault at deploy time. Idempotent;
+# run once per box (prod/beta) as a sudo-capable user.
+#
+#   SECRETS_ENV=prod AGE_KEY_SRC=/path/to/age.key bash deploy/provision-secrets.sh
+#
+# Installs `sops` + `age`, then installs the age PRIVATE key at /etc/thermograph/age.key
+# (0400) and writes /etc/thermograph/secrets-env. After this, deploy.sh renders
+# /etc/thermograph.env from deploy/secrets/*.yaml. See deploy/secrets/README.md.
+set -euo pipefail
+
+SECRETS_ENV="${SECRETS_ENV:?set SECRETS_ENV=prod|beta}"
+AGE_KEY_SRC="${AGE_KEY_SRC:?set AGE_KEY_SRC=/path/to/the/age/private/key}"
+SOPS_VERSION="${SOPS_VERSION:-v3.13.2}"
+AGE_VERSION="${AGE_VERSION:-v1.3.1}"
+BIN="${BIN:-/usr/local/bin}"
+ARCH="$(uname -m)"; case "$ARCH" in x86_64) ARCH=amd64;; aarch64|arm64) ARCH=arm64;; esac
+
+install_sops() {
+  command -v sops >/dev/null 2>&1 && { echo "==> sops already installed"; return; }
+  echo "==> Installing sops ${SOPS_VERSION}"
+  sudo curl -fsSL "https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux.${ARCH}" -o "${BIN}/sops"
+  sudo chmod +x "${BIN}/sops"
+}
+
+install_age() {
+  command -v age >/dev/null 2>&1 && { echo "==> age already installed"; return; }
+  echo "==> Installing age ${AGE_VERSION}"
+  tmp="$(mktemp -d)"
+  curl -fsSL "https://github.com/FiloSottile/age/releases/download/${AGE_VERSION}/age-${AGE_VERSION}-linux-${ARCH}.tar.gz" | tar -xz -C "$tmp"
+  sudo install -m 0755 "$tmp/age/age" "$tmp/age/age-keygen" "$BIN/"
+  rm -rf "$tmp"
+}
+
+install_sops
+install_age
+
+echo "==> Installing age key -> /etc/thermograph/age.key (0400) and marker (${SECRETS_ENV})"
+sudo mkdir -p /etc/thermograph
+sudo install -m 0400 "$AGE_KEY_SRC" /etc/thermograph/age.key
+printf '%s\n' "$SECRETS_ENV" | sudo tee /etc/thermograph/secrets-env >/dev/null
+sudo chmod 0644 /etc/thermograph/secrets-env
+
+echo "==> Done. Verify:  sops --version && ls -l /etc/thermograph/"
+echo "    Next: seed deploy/secrets/*.yaml, dry-run render, then deploy (see README)."
--- a/deploy/render-secrets.sh
+++ b/deploy/render-secrets.sh
@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+# Render /etc/thermograph.env from the SOPS-encrypted source of truth
+# (deploy/secrets/<env>.yaml, committed encrypted) — sourced by deploy.sh and
+# deploy-dev.sh, not run directly.
+#
+# The encrypted files are the single source of truth; /etc/thermograph.env becomes a
+# rendered artifact that the existing seams (docker-compose `env_file:`, the systemd
+# unit's EnvironmentFile, and entrypoint.sh's /run/secrets shim) keep consuming
+# unchanged. Common secrets + the per-host overrides are concatenated with the host
+# file LAST, so a host value wins (env_file and `source` both take the last
+# occurrence of a duplicate key).
+#
+# Presence-detected: a host is "configured for SOPS" only when it has an age key at
+# /etc/thermograph/age.key AND an environment name in /etc/thermograph/secrets-env
+# (prod|beta|dev). A host without them keeps whatever /etc/thermograph.env it already
+# has — so merging this is a safe no-op until a host is deliberately migrated. See
+# deploy/secrets/README.md.
+render_thermograph_secrets() {
+  local repo="${1:-.}"                                   # repo root holding deploy/secrets
+  local key="${THERMOGRAPH_AGE_KEY:-/etc/thermograph/age.key}"
+  local marker="${THERMOGRAPH_SECRETS_ENV_FILE:-/etc/thermograph/secrets-env}"
+  local env_name
+  env_name=$(cat "$marker" 2>/dev/null || true)
+
+  # The per-host file is required; common.yaml is optional so the initial cutover can
+  # seed each host as an exact copy of its live env (byte-identical render, no value
+  # changes) and factor out shared secrets into common.yaml later.
+  if [ -z "$env_name" ] || [ ! -f "$key" ] \
+     || [ ! -f "$repo/deploy/secrets/${env_name}.yaml" ]; then
+    echo "==> SOPS secrets not configured here; using existing /etc/thermograph.env"
+    return 0
+  fi
+  if ! command -v sops >/dev/null 2>&1; then
+    echo "!! sops not installed but this host is configured for SOPS secrets" >&2
+    return 1
+  fi
+
+  echo "==> Rendering /etc/thermograph.env from deploy/secrets (common + ${env_name})"
+  local tmp; tmp=$(mktemp)
+  : > "$tmp"
+  # set -e in the caller makes a decrypt failure fatal here (no partial env). common
+  # first, host second, so a host value overrides a shared one (last-wins).
+  if [ -f "$repo/deploy/secrets/common.yaml" ]; then
+    SOPS_AGE_KEY_FILE="$key" sops -d --input-type yaml --output-type dotenv \
+      "$repo/deploy/secrets/common.yaml" >> "$tmp"
+  fi
+  SOPS_AGE_KEY_FILE="$key" sops -d --input-type yaml --output-type dotenv \
+    "$repo/deploy/secrets/${env_name}.yaml" >> "$tmp"
+
+  # /etc/thermograph.env is in the root-owned /etc: write it directly if we can, else
+  # via sudo. Fail loudly rather than deploy against stale secrets.
+  if install -m 0640 "$tmp" /etc/thermograph.env 2>/dev/null; then :
+  elif sudo install -m 0640 "$tmp" /etc/thermograph.env 2>/dev/null; then :
+  else
+    rm -f "$tmp"
+    echo "!! cannot write /etc/thermograph.env (need write access or passwordless sudo)" >&2
+    return 1
+  fi
+  rm -f "$tmp"
+}
--- a/deploy/secrets/README.md
+++ b/deploy/secrets/README.md
@ -0,0 +1,107 @@
+# Secrets — the git-native vault (SOPS + age)
+
+The **single source of truth** for Thermograph's secrets is the set of
+SOPS-encrypted YAML files in this directory. They're committed to the repo
+**encrypted** (values only — keys stay readable so diffs are meaningful) and
+**rendered into `/etc/thermograph.env` at deploy time** by
+[`deploy/render-secrets.sh`](../render-secrets.sh), which the prod/beta
+`deploy.sh` invokes.
+
+Cycling a key is: **edit → commit → deploy**. No SSHing in to hand-edit a root file,
+no per-host duplication.
+
+## Files
+
+| File | Holds | Encrypted? |
+|------|-------|------------|
+| `common.yaml` | Secrets shared by every host — Discord tokens/secret, VAPID keypair, `THERMOGRAPH_AUTH_SECRET`, metrics token, indexnow key, SMTP creds | ✅ |
+| `prod.yaml` | Prod-only overrides — `POSTGRES_PASSWORD`, `REGISTRY_TOKEN` | ✅ |
+| `beta.yaml` | Beta-only overrides | ✅ |
+| `example.yaml` | Format reference / CI fixture (fake values) | ✅ |
+| `../../.sops.yaml` | Which age recipient files are encrypted to (plaintext config) | — |
+
+At deploy the renderer concatenates `common.yaml` then `<env>.yaml`, so a **host
+value wins** (env_file / `source` take the last occurrence of a duplicate key).
+`<env>` comes from `/etc/thermograph/secrets-env` on the box (`prod`/`beta`/`dev`).
+`dev` has no real secrets and is intentionally not wired in — see `deploy-dev.sh`.
+
+## The age key (the one thing that matters)
+
+- The **private key is the single recovery root.** Lose it and every secret here is
+  unrecoverable; leak it and every secret is compromised.
+- It is **never in the repo.** It lives at:
+  - `~/.config/sops/age/keys.txt` on the operator's machine (for editing), and
+  - `/etc/thermograph/age.key` (`0400`) on each host that renders at deploy time.
+- **Back it up** in the password manager. The public recipient is in `.sops.yaml`.
+
+## Prerequisites (once per machine)
+
+Install `sops` + `age` (single static binaries). On a host, use
+[`deploy/provision-secrets.sh`](../provision-secrets.sh); on your laptop:
+
+```sh
+# see the pinned URLs in provision-secrets.sh, or:
+go install github.com/getsops/sops/v3/cmd/sops@latest   # if you have Go
+```
+
+## Everyday: rotate a key
+
+```sh
+sops deploy/secrets/common.yaml        # opens DECRYPTED in $EDITOR; re-encrypts on save
+git commit -am "rotate <thing>" && git push forgejo
+# beta auto-deploys on push to main. prod (no CI) — one command:
+ssh agent@169.58.46.181 'cd /opt/thermograph && git pull && deploy/deploy.sh'
+```
+
+A host-specific value (e.g. `POSTGRES_PASSWORD`) is the same, editing `prod.yaml` /
+`beta.yaml` instead. **`POSTGRES_PASSWORD` is special**: the database only reads it on
+a *fresh* volume, so also `ALTER ROLE thermograph PASSWORD '…'` inside the running
+Postgres to match — the env change alone won't re-key an initialized DB.
+
+## Add a new secret
+
+1. `sops deploy/secrets/common.yaml`, add `THERMOGRAPH_NEW_THING: value`.
+2. Add the app reader (`os.environ.get("THERMOGRAPH_NEW_THING")`).
+3. Commit + deploy. It flows into `/etc/thermograph.env` automatically.
+
+## Rotate the age identity itself (re-key)
+
+```sh
+age-keygen -o new.key                              # new identity
+# add the new PUBLIC recipient alongside the old in .sops.yaml, then:
+sops updatekeys deploy/secrets/*.yaml              # re-encrypt to BOTH keys
+# distribute new.key to /etc/thermograph/age.key on every host + ~/.config/sops/age,
+# deploy once so every host can decrypt with the new key, then remove the OLD
+# recipient from .sops.yaml and:
+sops updatekeys deploy/secrets/*.yaml              # drop the old key
+git commit -am "rotate age identity" && push + deploy
+```
+
+## First-time bootstrap (seed from the live boxes)
+
+Order matters — values must match the live env exactly so `AUTH_SECRET` / VAPID /
+`POSTGRES_PASSWORD` don't rotate unintentionally:
+
+1. Install `sops`+`age` and drop `/etc/thermograph/age.key` (0400) +
+   `/etc/thermograph/secrets-env` on prod & beta (`provision-secrets.sh`).
+2. Seed each host's file from its live env with the helper (pulls over SSH, encrypts,
+   and verifies the render round-trips — run it on your own machine, it reads live
+   secrets):
+   ```sh
+   deploy/secrets/seed-from-live.sh prod agent@169.58.46.181
+   deploy/secrets/seed-from-live.sh beta agent@75.119.132.91
+   ```
+   That writes exact per-host copies (`prod.yaml`/`beta.yaml`); factor shared values
+   into `common.yaml` later. Commit.
+3. **Dry-run:** on the box, render to a temp file and `diff` against the live
+   `/etc/thermograph.env` — must be byte-identical before cutover.
+4. Merge the `deploy.sh` wiring; run one deploy; confirm the app is healthy.
+
+## Safety
+
+- CI (`.forgejo/workflows/secrets-guard.yml`) fails if any `*.yaml` here is committed
+  unencrypted.
+- Rendering is **presence-detected**: a host without the age key + marker keeps its
+  existing `/etc/thermograph.env`, so nothing breaks before a host is migrated.
+- Rendering happens **on the target box**, so CI/runners never see the age key or the
+  plaintext.
--- a/deploy/secrets/beta.yaml
+++ b/deploy/secrets/beta.yaml
@ -0,0 +1,33 @@
+PORT: ENC[AES256_GCM,data:/knJSQ==,iv:g8vSX20vD9Hz16lMdj46nToKYzLKzNskxzG/VHVVPbM=,tag:+X7H/JnC2n2Ie7RGzsaM8Q==,type:str]
+POSTGRES_PASSWORD: ENC[AES256_GCM,data:Vz/qCb50A3BLoQw+sDn7J7lvqld/urPj96qtKziR57c=,iv:ZjZi34GZZmMOTUKbSKdDV9KDNTrpSZTWww70LMV75ns=,tag:GuISiND97ThKgPl0wCywQg==,type:str]
+THERMOGRAPH_DATABASE_URL: ENC[AES256_GCM,data:o7rHapWWNbgMCNPz15NrpUbTbo7mzXgc4a2gGhXrcy7oNTAqA2ExV/+kvKv8CIvIKtgaMUrEmeh38XSd/ARVYauLRTjzCiAJIWtkXeN8fD8E4eGpfw==,iv:d8uZDon4Ub9g+I8MlgXMT+dWYrJUGCGrWdYfRNJ7kro=,tag:r/vR1zzGjZa7/pTOjBwk6g==,type:str]
+WORKERS: ENC[AES256_GCM,data:/A==,iv:kvFqDjAGq50aXhNrXmCxwYQXT24Tczh5GNPagGj28jc=,tag:oz0r4maN94LcBbzWDVOc1g==,type:str]
+APP_CPUS: ENC[AES256_GCM,data:Og==,iv:7UGacuCWIJlXu2B3WDjT6Z5BxFxUKxOlpUH8Xi+Xsgc=,tag:GOV5xS6B8u766kM6UBuwOQ==,type:str]
+DB_CPUS: ENC[AES256_GCM,data:Eg==,iv:rlZ5BjkagQR+T9BmOMIZz6R51x4boGYu12cxdG2ztvc=,tag:MQXbDXRCYlm8FFvXWwEzpg==,type:str]
+DB_MEMORY: ENC[AES256_GCM,data:5FQ=,iv:7FHgihO5DQ1se1Ng9KjKJHnY2NyoKJ+yWHt2I/jbTmU=,tag:CcsNGnpBFELh3IOrGrVjPA==,type:str]
+TIMESCALEDB_TAG: ENC[AES256_GCM,data:CWzVuxA7bbcAOb0=,iv:hRRNh11ETDTbIx7kkhumAc0QBaTV9SW8CLiq6McRqKA=,tag:3hBURxt2/BJpAUSgZyZ99g==,type:str]
+THERMOGRAPH_BASE: ENC[AES256_GCM,data:lw==,iv:kZOXeO841rEIQEnVnstm+u9BHKolnM0vvhNukx7RC8o=,tag:EMxBP63Zn1uIC8M5L7fmsg==,type:str]
+THERMOGRAPH_BASE_URL: ENC[AES256_GCM,data:DO2KCmfqs2zdxVuJivPVNn1YeFF8DED0tzET/g==,iv:Mrsrb6ToITqyatu5heC5s55SL8/Y4GMXqoYREMkOVd4=,tag:net8qhJ4ptXo3QcWYlbv6g==,type:str]
+THERMOGRAPH_COOKIE_SECURE: ENC[AES256_GCM,data:ow==,iv:U/I93B7gjYSSZjqudHd6QDQIRATx+l5QZ1MRWYD3aGk=,tag:f6Bs47hby1WQxAaDQ/fZ1w==,type:str]
+THERMOGRAPH_AUTH_SECRET: ENC[AES256_GCM,data:KsQO8ZxRp8n9AGL/z9kLxpO7vsmTBtM5H6ZpXzhm1VLVtc4CV+evEs4sAA==,iv:pMC8QruPjTRc6cXIwPBWHLY4eVomNkf4MYfWSeG/6qM=,tag:4W6Px9Xz8uvew95DIHDWVg==,type:str]
+THERMOGRAPH_METRICS_TOKEN: ENC[AES256_GCM,data:Ba1kxhSUNxq+oVHCskmQHhPuIcZPQQXTx2ht57D8Lyg=,iv:vT+aerexKF5ZXddQoXtBuzqAlzYQ8jT2wbef7HfXy9U=,tag:VkZb+s0C8UjnQnJKMC4HHQ==,type:str]
+THERMOGRAPH_INDEXNOW_KEY: ENC[AES256_GCM,data:GPytN7OsoKVO0n3VsiKERr0V5CwfXId3dCr4rMHNuHg=,iv:DlFEG3BiSMPh0jlB5YgSn6GrYgT0nxmoQGCDt1jUt28=,tag:/jclaZE6hd4EKpL0/7/WbA==,type:str]
+THERMOGRAPH_VAPID_PRIVATE_KEY: ENC[AES256_GCM,data:sTtkAbMVUtwvRssCZJa2PWUjlfnYSahciq9okgZj+DaqU4/mzNO3EDinMA==,iv:+sS0E+OP8qlBw+BEaSggEaaS2SCSBTrjaMzks8tdrJQ=,tag:pRN32rNjN4dp9OCzTHNkcQ==,type:str]
+THERMOGRAPH_VAPID_PUBLIC_KEY: ENC[AES256_GCM,data:SnFw+pwNgZlxMEhMUH/1daMZlRB2QFNJHtL+4C3S1eFcJJdT2uNGkY7Ym9FPfx1V7cwt/3urHg1oVgdDzYaIal2wye4l5HhKBP5PjtQkDW+Pck24kxul,iv:Ku4MhsrAfyA2Y3CqhGIsYKrjhKQ4cqAYjCqjFeEvBWE=,tag:hw42iS+79D7gNf+BPTYM0g==,type:str]
+THERMOGRAPH_VAPID_CONTACT: ENC[AES256_GCM,data:ox+hHiFMTPZqYwICDSwPwcT13ZDSKiGcAVHfVVMNYQ==,iv:DssV/olPXERpdmXNMmzjVYvhn/0zmuSp2XlTs42pIi0=,tag:4YHXI1BFbGZ6JsdXqJuf3w==,type:str]
+REGISTRY_TOKEN: ENC[AES256_GCM,data:pC1Zzs56la8qjzWcbkuKyeVHwS5WfIe+mrLm5Sdi/LKYziAvRU5cOA==,iv:57PlJEBdgwNJsfFdNOGqAnKk8l6zpmMSWYFEKS6pn+4=,tag:pPftNHUVU6+CY2nWMaXO6w==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUbjg4ZmQ2bXRzc1FoSjRj
+            RzYwTXlzYkwyYjNBbUZoU1hlY0ludjNlamlJClBQK0NRRHFzcGpqZmIvTkNBMXNZ
+            VmFRaXVwbTJPTHY1dzI1UGdOb0QvSzgKLS0tIGs1Q2NQUHJXZFd4WnpUaFVpY0NP
+            TmhhMEN6ekg4Q0F1aUtuaDdOTVJEZHcKeCAW+Gqt6IRHizb0cOpKinyJzTkWORX5
+            0+PQdtDxom72BqrAoj5lhepxW8YWPqQIRXsVS/XYQEQfPJuuuiRMKw==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1xx4dzs0dxlwvkv9sjuqzsphl7lfrxannkfken374yu2qvvcte9sqzktqt2
+    lastmodified: "2026-07-22T03:17:28Z"
+    mac: ENC[AES256_GCM,data:npF+ngF4Khj3NJ2PNFcQjQYIrZVpdPM0drk43pAI6HfFaynX8JOqRFZRsH0sCfMoPkXB/ffCJxla1Tt3Mu3zgKTNSWcn+mjzl+dYvlruY1VylHLS+GnTO9zKeQcwbrYzTfZNWzQhm6/mw8KCL3791U/xNHwJc6g5VxRUWl6TBdo=,iv:JLzY8rm33NLG8CqyAlU6vBhlODUiRipKu9daoRvgyzg=,tag:yIsxVZAnw8FD7Bt/x55wMQ==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
--- a/deploy/secrets/example.yaml
+++ b/deploy/secrets/example.yaml
@ -0,0 +1,21 @@
+#ENC[AES256_GCM,data:mVl42QkTGozrOS8g1ufphjX65wDrrQ6CwB2ZkeliOUmvQ9TvaGe7M9i702gDBOkrqmwYRYdcPDLiLzJXt/mqpi8n0t0rnhduZxQG9WDO8ny0E+OHSO4=,iv:mr6GaRYJ8BKtrUz04MRevaPwfevwEFInKXs3MTFtY3w=,tag:gO/T8sy8Q9VLpYl2G9lXhg==,type:comment]
+#ENC[AES256_GCM,data:r+UY39rGucBlaPg9l2ce9KMzVXbZ6pv/GeuMmkfdUjiYNE9b+oH0NUNP/7kQAUrLG/ggjFAZ3sIGkIGTqi3iPuMDWW1fYv4Xs5fbe7ZfYoOI+A4=,iv:RfpoTanyXjCl03zMX4eZP/6BUhcEUyjQIPRnuCt9Bm4=,tag:AsFmRmjvnJ8eMRXrqLO78g==,type:comment]
+#ENC[AES256_GCM,data:bRsSXptMUyQ0i0JohyBhxUgn17Ozd4qltPf+fVF7vcP8IcuIln5DieCzW2TcsYQjTlhgXPPzjQA8gq4bXw8p3h2QUR9g4YLm2THRg85dKqdobQDnbIU=,iv:YbPFoIZXxpC4gpRh3spoQfYdEOpsKszqlNanPuPCXVI=,tag:VW2sCR74BlUosbxUj5P9tQ==,type:comment]
+#ENC[AES256_GCM,data:EMLogDZpYWRvdI2t7N95d8UP4a5v0qt+eS6FwHt/xRU2hUO008Ybm9z61bULqNm9iL/j6p4rPye7SNbckFU7oT/U4CvC0QitKNYf,iv:TAaCF/0f3Iv1cRbkKT7ZwnDamMOS1ec6jWiHOm/3Rgk=,tag:wC5mHKw5Q032zS2k4V6DvA==,type:comment]
+THERMOGRAPH_EXAMPLE_TOKEN: ENC[AES256_GCM,data:azyOkty43gxbFZvFfOK92aBcEwgsNAa0JuxOoQ==,iv:JI+M8AJ1XXyviaB7mz68ocv/VKIoxFyC7mD/k335LSQ=,tag:Tbglfpg6nw3EAzU2Cv4aUw==,type:str]
+THERMOGRAPH_EXAMPLE_API_KEY: ENC[AES256_GCM,data:LNDu2eHHIHykWBetNMn7wZM=,iv:dD+XmzW2xI2LkZ3DbJ6AVqOm/bmQ9e3Qrvls9dZo1T8=,tag:v1JHnPNRHXuvYlF+xHlR7w==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA1VGlocmFpZFVqUzRsRVRR
+            LzRHUWxydkFBdHhoQ0xnWnhYQ0h3Y1dSY1JFCm9ESnUwKzVxNk9PUzgwUzdpWFk0
+            a1dtTVNlNkJhZFJxOVBHZDMrTFZUYzQKLS0tIGl5SzMxdE90bm9sNThFWkxvSXFl
+            eWlDUFY5RlMvd1BqMExwaGU0ekE5bjQKdQV93BjiY+9aMRzWF+Fce9qMQokEaj8g
+            V7fziLeEW1mXJFXiq7KhL2ryV+chZOtgGFn2sfzQlWOTu2LxO/+0ug==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1xx4dzs0dxlwvkv9sjuqzsphl7lfrxannkfken374yu2qvvcte9sqzktqt2
+    lastmodified: "2026-07-22T02:20:08Z"
+    mac: ENC[AES256_GCM,data:pObgYq9Tu7EcfHmLIq2Q5hPF7RA3yR6L2xEIOI+d2zTU+t0gLKdOE3l2kpiHf568PLcbkdZbzF057H7MW1AQd192y5Ph2OidA05D0eZmWaykmFHHY9ZyjbTondXSbTWmbVMYDXZGkZnvPnaJaivEmfFM7n0xeOf6ecI6C/tBmG8=,iv:fWN+C1MryDXzCpMcuKddsRqEt7StW6dc+K9K3+JIGxs=,tag:+wGtLFw+yBNa8HQltUrCkg==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
--- a/deploy/secrets/prod.yaml
+++ b/deploy/secrets/prod.yaml
@ -0,0 +1,39 @@
+PORT: ENC[AES256_GCM,data:3VXjXA==,iv:9tU5b/Ept1xDFVi5oRq/rimdXYxQj5qCQkbtYKQLT5Y=,tag:1gPFP4oUwp66aNP2bbDOLg==,type:str]
+POSTGRES_PASSWORD: ENC[AES256_GCM,data:Hct/SZ4mFqb3Vm4a3OgCSbKhdLLTxOOSb/kHBiAZ9pM=,iv:fURYQkJ4biEDruWhfMqjEESF8Y64JR+Tjf429Rrt7J4=,tag:h1ZAoUImYINVzLcvTbs4dQ==,type:str]
+THERMOGRAPH_DATABASE_URL: ENC[AES256_GCM,data:gfmGcH19GDNjKCffMADnKrViei2LuGJae06MsNS7biOFcBr23oY76gCEZVRa4YAaoUfNjr5HyyS3wiIUB+qAz/XvYXilzl9cYONVY2wXWplmQ0sW4Q==,iv:4h6LxDucrJ6Ma13z23KJlzz4lDFXaz9fg134kwpLeX0=,tag:BqMQbndCngN6I0UIhjsE6w==,type:str]
+WORKERS: ENC[AES256_GCM,data:6g==,iv:R0vRawz9uTPFL/GoVQTOFybau/QHVSmXU2N4x/xPjcQ=,tag:rZ2GnEYo4EBISmgSMQdjXA==,type:str]
+APP_CPUS: ENC[AES256_GCM,data:AQ==,iv:DILyG0LefEnnVHJF7PUshbAX+wYmPqTlduOnec7lMWY=,tag:U7vQJ5l1C4vTWvf1bVSglg==,type:str]
+DB_CPUS: ENC[AES256_GCM,data:7Q==,iv:6Tc5DYEPSBeJMpdPj+zS8B8S5DEWl28/3YqnFlFHTRs=,tag:9AOgfaEkcvs4jRTo6mKYLA==,type:str]
+DB_MEMORY: ENC[AES256_GCM,data:/FIY,iv:jvVm7C9CTnCFudGDOcpCn3VgZlvHmSq27eFV/1/PMuA=,tag:xfljFaZ6ImG3c03gtjC32A==,type:str]
+TIMESCALEDB_TAG: ENC[AES256_GCM,data:HQ3xemefK2/1bPE=,iv:yJHJac2ZTzw0lCESoqywogLhUs+EAHIQKPSCXTa+kzs=,tag:POiiKXCP2CEHAatQy9T7sg==,type:str]
+THERMOGRAPH_BASE: ENC[AES256_GCM,data:Gw==,iv:2VJRKggVu41cjnq3zdl9gBYhD+bWU1HXSpTCTwXDsPM=,tag:Sosggs9Fg4YN9eBTYSgzxw==,type:str]
+THERMOGRAPH_BASE_URL: ENC[AES256_GCM,data:dbONWMu5QUsbmS3Z27v9v5w9Q9w7V18=,iv:mywyTNJLbHueL2jMeOMQBu9os3CvldaAuVP+Irg3xOE=,tag:FB/86crdYna2TyGpys/spw==,type:str]
+THERMOGRAPH_COOKIE_SECURE: ENC[AES256_GCM,data:NQ==,iv:j6BUIBWP9ebS7xFw/2RrKo+fvmhOn9mYTR6+K7lizvw=,tag:B/PEjByAT4XsXcECOLSoPw==,type:str]
+THERMOGRAPH_AUTH_SECRET: ENC[AES256_GCM,data:8+s0NGrtJBV6nOgHfmoODukS8hSij8BoUjxYTH4s398LGABwro+MzB4Xsg==,iv:tmxMUkHND2x9RZquWKesDiPePMJA1qNLD+TMQLYR35M=,tag:iSHOu3En5rZpY+MOH4Us/A==,type:str]
+THERMOGRAPH_METRICS_TOKEN: ENC[AES256_GCM,data:PcEsEp+bAMsiXxAFHDX4bS3dnwQz5q23gMb1C5tVopI=,iv:VkbuTVkOBKlCyq6mNFAIM26VSzUbJ/Nyvg1vEWuUWJg=,tag:CR4ALs4UCXZ1ybwfkwO+1Q==,type:str]
+THERMOGRAPH_INDEXNOW_KEY: ENC[AES256_GCM,data:4NzyzEyy8khcWVBJj7Rs/lP4fuAn13aNzrQRTHGgVDw=,iv:fc9VwStsWO2gbbgqK43vFdSR3zl4+vmKUQsFY9Fu1YQ=,tag:D/aEkNBlzE1K5ldLt5SGGw==,type:str]
+THERMOGRAPH_VAPID_PRIVATE_KEY: ENC[AES256_GCM,data:v2U/CVUMOAkZG8a9IadkaSPWHhg9nOQnsIHoziE7A5GxiNTwm946+Uy2UQ==,iv:ZokatpIp58zj+KSPvA5/D7Xh6n6NSiX9z7PDmuHPrLs=,tag:FDDVU1KcBd1newEpd9PKWw==,type:str]
+THERMOGRAPH_VAPID_PUBLIC_KEY: ENC[AES256_GCM,data:0Noa0XlR3NJz9SxY/IOfBei+DbeWnv+yLSb7SkKY+hmrOnNwb6Hcz5AOVOP9nVIgHZaT9cpl0e7LVJyO4QofLzZqms9Uy8HtQh3J58vTXAh24sKX/ZBR,iv:Aa3766JRR9OLeyrkmN6/yi5UqexITXObpQwDIH6BuBI=,tag:MjIVwQoxXpEwIbKjSgXYsQ==,type:str]
+THERMOGRAPH_VAPID_CONTACT: ENC[AES256_GCM,data:4UM7/dB8BDPa/VodQkEuIi5ku2JlgJkSqUVdojTR4g==,iv:09t1CpbnOIunKSegqlKKXkEubmjgidACOmXuQNMLzfk=,tag:pPz2MUzgCTZVXX0TnlgKwg==,type:str]
+THERMOGRAPH_MAIL_BACKEND: ENC[AES256_GCM,data:qucAxA==,iv:XTBitLKgpOrWGePxe09K/76hUCmPADZpKT2l/Kp0HXc=,tag:WAwk3kW/b2aXBXjau3lMyQ==,type:str]
+THERMOGRAPH_SMTP_HOST: ENC[AES256_GCM,data:GzxyI/CRiwx4gw==,iv:VleW+mP3BQii9axQVm3zHXTZsLvS89DOzdt0rFd81z0=,tag:13+I5LEasTKYkYDu2eGGYA==,type:str]
+THERMOGRAPH_SMTP_PORT: ENC[AES256_GCM,data:yTM=,iv:ja9tBwmTgSJLcUXzrvPYZKUAuFdzn753wZ5FuQJUrNM=,tag:5t0/G9/TuHl8LJNTCW5Q5w==,type:str]
+THERMOGRAPH_DISCORD_CLIENT_SECRET: ENC[AES256_GCM,data:CrhrDfFmk7F2lWQeY9kd7yxzofz6D2beXf8Iluqf3OA=,iv:0cLNwuuCqThcp7PanztQWjzNjt/QE3Q0p+R+2sd/dvI=,tag:EM2+VmwSaDj0/IItHJA5Uw==,type:str]
+THERMOGRAPH_DISCORD_APP_ID: ENC[AES256_GCM,data:lxwB08B143dCFsobG9JOkmScIQ==,iv:TFjzSDZWptqWgswiPHxJAzG1hlVI5bUA87ZKjBTsCGs=,tag:eRgvfiSS1Ujd5T3AeZCwUw==,type:str]
+THERMOGRAPH_DISCORD_BOT_TOKEN: ENC[AES256_GCM,data:zfVJLH8ddct2Jvstl/olZLQ9SM15ccWLCIpYe/k5BUZiV50atzpQfE13FifB1jWF6C+5P95odxVFtne7YJyaDxryfIX8Gbea,iv:wDM4qFpGYGtUJfEk5IbhMg9J9gr8ambEOkSWWBA2CSU=,tag:FhtUOe5WKMH2h7qIi4yKgQ==,type:str]
+REGISTRY_TOKEN: ENC[AES256_GCM,data:3VETZR026YXvWA7DLly4/mz+hqEK7vkEMwKlDKU6CQZeZiFpbQXWig==,iv:ixZJi2VWAZak6dFPvVyErVRanp9bzqXyV3MVa4QsBuU=,tag:Pc2Mdw58S0tSdkgJ5yuVNw==,type:str]
+sops:
+    age:
+        - enc: |
+            -----BEGIN AGE ENCRYPTED FILE-----
+            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSA0THIwYmxSRnIveGlCWHZS
+            NDR1bHZ6MGNGdFJZbFRNdEJXZk8zdHBMdkMwCnNiTlVFRGpKNHB3TzYycEY5dUd1
+            RktTaWZ3dFNVQzV1OXhCSmp2VUJtaXMKLS0tIEJ0dGJXajFYcDFKaXhKZ0FaOUEz
+            eUh2VFlYeW5CQ1hnTVQvN2lBMU9TclUKC3riyd2eEFsej7ItoQ8rpsAiRms8yJgo
+            PDXCB8EqKwdC3BnKlpD9ptYcuPHVUSgve1wIZYJ7iHIEhW9Kc6RXOw==
+            -----END AGE ENCRYPTED FILE-----
+          recipient: age1xx4dzs0dxlwvkv9sjuqzsphl7lfrxannkfken374yu2qvvcte9sqzktqt2
+    lastmodified: "2026-07-22T03:17:22Z"
+    mac: ENC[AES256_GCM,data:e9mkAW/bib2yvx50FinJaDJe9dENkMmsW8sek072Gunh9+gh6rJZfYPZ+neo1a+B8T9I4WVMIraKl2fSbOCc4OPCxbxsNO/vemrlsF1RMl13aklxxAJqbl2TOL7zPv8c7gv6DaUxMYB2//UTnOurHiSa+LgEoexp8QM3C0yyICk=,iv:bsfkniozEHdZTKxNR+nrrxXXyCXKZz0Cc1D0EV0SPjY=,tag:G5YUWV0Et6X61HLVm7IJrA==,type:str]
+    unencrypted_suffix: _unencrypted
+    version: 3.13.2
--- a/deploy/secrets/seed-encrypt-on-host.sh
+++ b/deploy/secrets/seed-encrypt-on-host.sh
@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+# Seed deploy/secrets/<env>.yaml by encrypting a box's live /etc/thermograph.env
+# ENTIRELY ON THE BOX. The plaintext never leaves the box; only the already-encrypted
+# ciphertext is written back here. Encryption uses the age PUBLIC key (safe to hardcode)
+# so the machine you run this from needs nothing but SSH access + the agent key — no
+# sops, no age, no private key. sops is installed on the box on demand.
+#
+# Run from any checkout of this branch that can SSH to the target with the agent key:
+#   deploy/secrets/seed-encrypt-on-host.sh prod agent@169.58.46.181
+#   deploy/secrets/seed-encrypt-on-host.sh beta agent@75.119.132.91
+# Then commit + push the resulting deploy/secrets/<env>.yaml. Verify faithfulness with
+# the key-gaps skill after. See README.md.
+set -euo pipefail
+cd "$(dirname "$(readlink -f "$0")")/../.."
+
+ENV_NAME="${1:?usage: seed-encrypt-on-host.sh <env> <ssh-target> [ssh-key]}"
+SSH_TARGET="${2:?ssh target, e.g. agent@169.58.46.181}"
+SSH_KEY="${3:-$HOME/.ssh/thermograph_agent_ed25519}"
+OUT="deploy/secrets/${ENV_NAME}.yaml"
+
+# Public age recipient — NOT a secret (matches .sops.yaml). The private key never
+# leaves the operator's machine / the hosts' /etc/thermograph/age.key.
+AGE_PUB="age1xx4dzs0dxlwvkv9sjuqzsphl7lfrxannkfken374yu2qvvcte9sqzktqt2"
+SOPS_URL="https://github.com/getsops/sops/releases/download/v3.13.2/sops-v3.13.2.linux.amd64"
+
+echo "==> Encrypting ${SSH_TARGET}:/etc/thermograph.env on-host -> ${OUT}"
+ssh -i "$SSH_KEY" "$SSH_TARGET" "AGE_PUB='$AGE_PUB' SOPS_URL='$SOPS_URL' bash -s" > "$OUT" <<'REMOTE'
+set -euo pipefail
+exec 3>&1                       # real stdout carries ONLY the ciphertext
+{                               # setup noise -> stderr, so it can't corrupt the file
+  if ! command -v sops >/dev/null 2>&1; then
+    sudo curl -fsSL "$SOPS_URL" -o /usr/local/bin/sops
+    sudo chmod +x /usr/local/bin/sops
+  fi
+} >&2
+tmp="$(mktemp)"; tmy="$(mktemp)"; trap 'rm -f "$tmp" "$tmy"' EXIT
+sudo cat /etc/thermograph.env > "$tmp"          # plaintext stays on this box only
+python3 -c '
+import json, sys
+for line in open(sys.argv[1], encoding="utf-8"):
+    s = line.rstrip("\n").strip()
+    if not s or s.startswith("#") or "=" not in s:
+        continue
+    k, _, v = s.partition("=")
+    print(f"{k.strip()}: {json.dumps(v)}")       # JSON-escaped scalar = valid YAML
+' "$tmp" > "$tmy"
+cd /tmp                                          # no .sops.yaml here, so --age is honored
+sops -e --age "$AGE_PUB" --input-type yaml --output-type yaml "$tmy" >&3
+REMOTE
+
+if grep -q 'ENC\[AES256_GCM' "$OUT"; then
+  echo "==> OK: ${OUT} written and encrypted ($(grep -cE '^[A-Za-z_][A-Za-z0-9_]*: ' "$OUT") keys)"
+else
+  echo "!! ${OUT} is not encrypted (the on-host step failed) — removing" >&2
+  rm -f "$OUT"; exit 1
+fi
--- a/deploy/secrets/seed-from-live.sh
+++ b/deploy/secrets/seed-from-live.sh
@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+# Seed (or re-seed) an encrypted secrets file from a box's live /etc/thermograph.env.
+# Run on the operator's machine — it reads production secrets, so it is deliberately
+# NOT something the agent runs for you. Requires: sops + age on PATH, the age private
+# key at ~/.config/sops/age/keys.txt (or SOPS_AGE_KEY_FILE), and SSH access to the box.
+#
+#   deploy/secrets/seed-from-live.sh prod agent@169.58.46.181 ~/.ssh/thermograph_agent_ed25519
+#   deploy/secrets/seed-from-live.sh beta agent@75.119.132.91  ~/.ssh/thermograph_agent_ed25519
+#
+# Writes deploy/secrets/<env>.yaml ENCRYPTED (exact copy of the live env, so the
+# deploy render is faithful), then verifies the render round-trips to the same
+# KEY=VALUE set. Never prints secret values. See README.md.
+set -euo pipefail
+
+# Anchor to the repo that holds this script, so it works when invoked by absolute path
+# from any cwd (a different checkout, your home dir, etc.) — OUT and .sops.yaml resolve
+# relative to here, not to wherever you happened to run it.
+cd "$(dirname "$(readlink -f "$0")")/../.."
+
+ENV_NAME="${1:?usage: seed-from-live.sh <env> <ssh-target> [ssh-key]}"
+SSH_TARGET="${2:?ssh target, e.g. agent@169.58.46.181}"
+SSH_KEY="${3:-$HOME/.ssh/thermograph_agent_ed25519}"
+OUT="deploy/secrets/${ENV_NAME}.yaml"
+command -v sops >/dev/null || { echo "!! sops not on PATH" >&2; exit 1; }
+
+tmp="$(mktemp)"; trap 'rm -f "$tmp" "$tmp.env"' EXIT
+umask 077
+
+echo "==> Pulling ${SSH_TARGET}:/etc/thermograph.env (sudo)"
+ssh -i "$SSH_KEY" "$SSH_TARGET" 'sudo cat /etc/thermograph.env' > "$tmp"
+n=$(grep -cE '^[A-Za-z_][A-Za-z0-9_]*=' "$tmp" || true)
+[ "$n" -gt 0 ] || { echo "!! no KEY=VALUE lines read (permission? path?)" >&2; exit 1; }
+echo "    ${n} vars"
+
+echo "==> Writing ${OUT} (flat YAML) and encrypting"
+python3 - "$tmp" "$OUT" "$ENV_NAME" <<'PY'
+import json, sys
+src, dst, env = sys.argv[1:4]
+pairs = {}
+for line in open(src, encoding="utf-8"):
+    s = line.rstrip("\n").strip()
+    if not s or s.startswith("#") or "=" not in s:
+        continue
+    k, _, v = s.partition("=")
+    k = k.strip()
+    if not all(c.isalnum() or c == "_" for c in k) or not k[0].isalpha() and k[0] != "_":
+        continue
+    pairs[k] = v
+with open(dst, "w", encoding="utf-8") as fh:
+    fh.write(f"# SOPS-encrypted — {env} host secrets, seeded from the live /etc/thermograph.env.\n")
+    for k, v in pairs.items():
+        fh.write(f"{k}: {json.dumps(v)}\n")   # JSON-escaped scalar is valid YAML
+print(f"    {len(pairs)} keys")
+PY
+sops -e -i "$OUT"
+grep -q 'ENC\[AES256_GCM' "$OUT" || { echo "!! $OUT did not encrypt — refusing to keep it" >&2; rm -f "$OUT"; exit 1; }
+
+echo "==> Verifying the render round-trips to the same KEY=VALUE set"
+sops -d --input-type yaml --output-type dotenv "$OUT" > "$tmp.env"
+python3 - "$tmp" "$tmp.env" <<'PY'
+import sys
+def load(p):
+    d = {}
+    for line in open(p, encoding="utf-8"):
+        s = line.rstrip("\n").strip()
+        if not s or s.startswith("#") or "=" not in s:
+            continue
+        k, _, v = s.partition("=")
+        d[k.strip()] = v
+    return d
+live, rendered = load(sys.argv[1]), load(sys.argv[2])
+missing = sorted(set(live) - set(rendered))
+extra   = sorted(set(rendered) - set(live))
+diff    = sorted(k for k in live if k in rendered and live[k] != rendered[k])
+if missing or extra or diff:
+    if missing: print("   !! keys lost in render:", missing)
+    if extra:   print("   !! keys added in render:", extra)
+    if diff:    print("   !! values changed (keys only):", diff)
+    sys.exit(1)
+print(f"   OK — {len(live)} keys render identically")
+PY
+echo "==> ${OUT} is ready. Review with:  sops ${OUT}   (or git diff --stat)"
+echo "    Then commit + push; the agent can take it from there."
--- a/deploy/thermograph.env.example
+++ b/deploy/thermograph.env.example
@ -1,9 +1,15 @@
-# Copy to /etc/thermograph.env on the VPS and edit. On prod/beta, Terraform
-# renders this file for you (see terraform/README.md) — edit the tfvars, not the
-# host. Sourced by deploy/deploy.sh (and any `docker compose` invocation) so
-# compose can interpolate it, AND loaded into the app container (env_file in
-# docker-compose.yml). Anything secret the app needs — Postgres password, VAPID
-# keys, auth secret — belongs here.
+# This is the REFERENCE for what /etc/thermograph.env contains — the full set of
+# vars the app reads. On a host wired for the SOPS vault (an age key +
+# /etc/thermograph/secrets-env), you do NOT hand-edit /etc/thermograph.env: it is
+# rendered at deploy from the encrypted source of truth in deploy/secrets/*.yaml
+# (see deploy/secrets/README.md). To change a secret there, `sops edit` + commit +
+# deploy — not a host edit. (Legacy: Terraform can also render this file from tfvars,
+# and a from-scratch host can start by copying this example and editing it.)
+#
+# Sourced by deploy/deploy.sh (and any `docker compose` invocation) so compose can
+# interpolate it, AND loaded into the app container (env_file in docker-compose.yml).
+# Anything secret the app needs — Postgres password, VAPID keys, auth secret —
+# belongs here.

 # Port uvicorn binds inside the container. The compose stack publishes it on the
 # host loopback (127.0.0.1:8137) for Caddy to proxy to. Keep 8137.