secrets: factor shared values into common.yaml; stop the renderer failing open #75

Merged

admin_emi merged 1 commit from fix/key-gaps-digit-regex into dev

2026-07-25 00:44:53 +00:00

admin_emi commented

2026-07-25 00:42:26 +00:00

Owner

Two changes to how credentials are stored and distributed.

1. `common.yaml` now exists

The renderer has always concatenated common.yaml then <env>.yaml (host winning), and the vault README has always listed common.yaml with a ✅. The file was never created. So every value shared by prod and beta was duplicated in both vaults, free to drift apart with nothing to detect it.

16 values → common.yaml: VAPID keypair, metrics token, IndexNow key, REGISTRY_TOKEN, the S3 endpoint/bucket and both S3 keypairs, plus shared config.
5 stay per-host because their values genuinely differ: APP_CPUS, DB_CPUS, DB_MEMORY, WORKERS, THERMOGRAPH_BASE_URL.
8 exist only on prod — the Discord and mail credentials. Beta has no Discord setup at all.

Three held back deliberately

POSTGRES_PASSWORD, THERMOGRAPH_AUTH_SECRET and THERMOGRAPH_DATABASE_URL are identical today, so by the mechanical rule they belong in common.yaml. They are kept per-host anyway.

These are the credentials that let one environment act as another: with them, a foothold on beta is a foothold on prod's database and prod's session signing — and beta is the more exposed box, serving public Forgejo and Grafana. They match only because beta was seeded from prod, not because the two are meant to be one system.

Keeping them per-host costs one line each and preserves the ability to diverge, so that rotating prod's database password stops implying "and beta's too". Putting them in common.yaml would encode that equivalence as intentional and make breaking it a migration rather than an edit. The reasoning is in the vault README, because otherwise a future reader will helpfully "fix" it.

How this was done safely

No plaintext left prod. Ciphertext was shipped up, decrypted against the host's age key, recombined, re-encrypted, shipped back. The consolidation refuses to write unless it has proved merged(common + env) == the original env vault — same keys, same values — and then re-verifies from the written files. Both checks passed for prod and beta.

An earlier attempt failed at the encrypt step because SOPS's creation rule is path_regex: ^deploy/secrets/.*\.yaml$, which doesn't match a scratch directory. It failed before writing anything and left no plaintext behind.

2. `render_thermograph_secrets` no longer fails open

One return 0 covered two different situations:

"not configured for SOPS" — true of the LAN dev box, and correct to succeed.
"configured, but the vault isn't where we looked" — a failure, which returned 0 anyway. A silent no-op wearing a success code: a deploy reports success having rendered nothing, and the host keeps serving whatever /etc/thermograph.env already held — including after a rotation.

The likely cause is passing the wrong root. The function wants the directory containing deploy/secrets, which on the hosts is /opt/thermograph/infra, not /opt/thermograph. That mistake looked exactly like "not configured here", which is why it went unnoticed. It now exits 1 and names the probable cause.

Verified across all three paths: unconfigured → rc=0, configured-but-missing → rc=1 with a diagnostic, configured-and-present → proceeds.

Two changes to how credentials are stored and distributed. ## 1. `common.yaml` now exists The renderer has **always** concatenated `common.yaml` then `<env>.yaml` (host winning), and the vault README has always listed `common.yaml` with a ✅. The file was never created. So every value shared by prod and beta was **duplicated in both vaults, free to drift apart with nothing to detect it.** - **16 values → `common.yaml`**: VAPID keypair, metrics token, IndexNow key, `REGISTRY_TOKEN`, the S3 endpoint/bucket and both S3 keypairs, plus shared config. - **5 stay per-host** because their values genuinely differ: `APP_CPUS`, `DB_CPUS`, `DB_MEMORY`, `WORKERS`, `THERMOGRAPH_BASE_URL`. - **8 exist only on prod** — the Discord and mail credentials. Beta has no Discord setup at all. ### Three held back deliberately `POSTGRES_PASSWORD`, `THERMOGRAPH_AUTH_SECRET` and `THERMOGRAPH_DATABASE_URL` are **identical today**, so by the mechanical rule they belong in `common.yaml`. They are kept per-host anyway. These are the credentials that let one environment act as another: with them, a foothold on beta is a foothold on prod's database and prod's session signing — and beta is the *more* exposed box, serving public Forgejo and Grafana. They match only because beta was seeded from prod, not because the two are meant to be one system. Keeping them per-host costs one line each and preserves the ability to diverge, so that rotating prod's database password stops implying "and beta's too". Putting them in `common.yaml` would encode that equivalence as intentional and make breaking it a migration rather than an edit. The reasoning is in the vault README, because otherwise a future reader will helpfully "fix" it. ### How this was done safely No plaintext left prod. Ciphertext was shipped up, decrypted against the host's age key, recombined, re-encrypted, shipped back. The consolidation **refuses to write** unless it has proved `merged(common + env) == the original env vault` — same keys, same values — and then **re-verifies from the written files**. Both checks passed for prod and beta. An earlier attempt failed at the encrypt step because SOPS's creation rule is `path_regex: ^deploy/secrets/.*\.yaml$`, which doesn't match a scratch directory. It failed *before writing anything* and left no plaintext behind. ## 2. `render_thermograph_secrets` no longer fails open One `return 0` covered two different situations: - **"not configured for SOPS"** — true of the LAN dev box, and correct to succeed. - **"configured, but the vault isn't where we looked"** — a failure, which returned 0 anyway. A silent no-op wearing a success code: a deploy reports success having rendered nothing, and the host keeps serving whatever `/etc/thermograph.env` already held — **including after a rotation**. The likely cause is passing the wrong root. The function wants the directory *containing* `deploy/secrets`, which on the hosts is `/opt/thermograph/infra`, not `/opt/thermograph`. That mistake looked exactly like "not configured here", which is why it went unnoticed. It now exits 1 and names the probable cause. Verified across all three paths: unconfigured → `rc=0`, configured-but-missing → `rc=1` with a diagnostic, configured-and-present → proceeds.

admin_emi added 1 commit 2026-07-25 00:42:26 +00:00

secrets: factor shared values into common.yaml; stop the renderer failing open

PR build (required check) / changes (pull_request) Successful in 7s

Details

secrets-guard / encrypted (pull_request) Successful in 5s

Details

shell-lint / shellcheck (pull_request) Successful in 6s

Details

PR build (required check) / validate-observability (pull_request) Has been skipped

Details

PR build (required check) / build-frontend (pull_request) Successful in 1m11s

Details

PR build (required check) / build-backend (pull_request) Successful in 1m25s

Details

PR build (required check) / gate (pull_request) Successful in 2s

Details

cb69a22c0f

Two changes to how credentials are stored and distributed.

1. common.yaml now exists.

The renderer has always concatenated common.yaml then <env>.yaml, host winning,
and the README has always documented common.yaml with a checkmark. The file was
never created. So every value shared by prod and beta was duplicated in both
vaults, free to drift apart with nothing to detect it.

16 values move to common.yaml: VAPID keypair, metrics token, IndexNow key,
REGISTRY_TOKEN, the S3 endpoint/bucket and both S3 keypairs, plus shared config.
5 stay per-host because their values genuinely differ (APP_CPUS, DB_CPUS,
DB_MEMORY, WORKERS, THERMOGRAPH_BASE_URL). 8 exist only on prod — the Discord
and mail credentials, which beta does not have at all.

Three more are held back deliberately despite being identical today:
POSTGRES_PASSWORD, THERMOGRAPH_AUTH_SECRET, THERMOGRAPH_DATABASE_URL. These are
the credentials that let one environment act as another, and beta is the more
exposed box — it serves public Forgejo and Grafana. They match only because beta
was seeded from prod. Keeping them per-host costs one line each and preserves
the ability to diverge; putting them in common.yaml would encode the equivalence
as intentional and make breaking it a migration rather than an edit. Reasoning
recorded in the vault README, since a future reader will otherwise "fix" it.

Done without any plaintext leaving prod: ciphertext was shipped up, decrypted
against the host's age key, recombined, re-encrypted, and shipped back. The
consolidation refuses to write unless it has proved merged(common + env) equals
the original env vault exactly — same keys, same values — and re-verifies from
the written files afterwards. Both checks passed for prod and beta.

2. render_thermograph_secrets no longer fails open.

One `return 0` covered two different situations: "this host is not configured
for SOPS" (true of the LAN dev box, and correct to succeed) and "this host IS
configured but its vault is not where we looked". The second is a failure, and
returning 0 made it a silent no-op wearing a success code — a deploy would
report success having rendered nothing, and the host would keep serving whatever
/etc/thermograph.env already held, including after a rotation.

The likely cause is passing the wrong root: the function wants the directory
containing deploy/secrets, which on the hosts is /opt/thermograph/infra, not
/opt/thermograph. That mistake looked exactly like "not configured here", which
is why it went unnoticed. It now exits 1 and names the probable cause.

Claude-Session: https://claude.ai/code/session_0182KTMrsTHJc3TcewCatJFY