All checks were successful
PR build (required check) / changes (pull_request) Successful in 7s
secrets-guard / encrypted (pull_request) Successful in 5s
shell-lint / shellcheck (pull_request) Successful in 6s
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-frontend (pull_request) Successful in 1m11s
PR build (required check) / build-backend (pull_request) Successful in 1m25s
PR build (required check) / gate (pull_request) Successful in 2s
Two changes to how credentials are stored and distributed. 1. common.yaml now exists. The renderer has always concatenated common.yaml then <env>.yaml, host winning, and the README has always documented common.yaml with a checkmark. The file was never created. So every value shared by prod and beta was duplicated in both vaults, free to drift apart with nothing to detect it. 16 values move to common.yaml: VAPID keypair, metrics token, IndexNow key, REGISTRY_TOKEN, the S3 endpoint/bucket and both S3 keypairs, plus shared config. 5 stay per-host because their values genuinely differ (APP_CPUS, DB_CPUS, DB_MEMORY, WORKERS, THERMOGRAPH_BASE_URL). 8 exist only on prod — the Discord and mail credentials, which beta does not have at all. Three more are held back deliberately despite being identical today: POSTGRES_PASSWORD, THERMOGRAPH_AUTH_SECRET, THERMOGRAPH_DATABASE_URL. These are the credentials that let one environment act as another, and beta is the more exposed box — it serves public Forgejo and Grafana. They match only because beta was seeded from prod. Keeping them per-host costs one line each and preserves the ability to diverge; putting them in common.yaml would encode the equivalence as intentional and make breaking it a migration rather than an edit. Reasoning recorded in the vault README, since a future reader will otherwise "fix" it. Done without any plaintext leaving prod: ciphertext was shipped up, decrypted against the host's age key, recombined, re-encrypted, and shipped back. The consolidation refuses to write unless it has proved merged(common + env) equals the original env vault exactly — same keys, same values — and re-verifies from the written files afterwards. Both checks passed for prod and beta. 2. render_thermograph_secrets no longer fails open. One `return 0` covered two different situations: "this host is not configured for SOPS" (true of the LAN dev box, and correct to succeed) and "this host IS configured but its vault is not where we looked". The second is a failure, and returning 0 made it a silent no-op wearing a success code — a deploy would report success having rendered nothing, and the host would keep serving whatever /etc/thermograph.env already held, including after a rotation. The likely cause is passing the wrong root: the function wants the directory containing deploy/secrets, which on the hosts is /opt/thermograph/infra, not /opt/thermograph. That mistake looked exactly like "not configured here", which is why it went unnoticed. It now exits 1 and names the probable cause. Claude-Session: https://claude.ai/code/session_0182KTMrsTHJc3TcewCatJFY
118 lines
6.6 KiB
Bash
Executable file
118 lines
6.6 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Render /etc/thermograph.env from the SOPS-encrypted source of truth
|
|
# (deploy/secrets/<env>.yaml, committed encrypted) — sourced by deploy.sh and
|
|
# deploy-dev.sh, not run directly.
|
|
#
|
|
# The encrypted files are the single source of truth; /etc/thermograph.env becomes a
|
|
# rendered artifact that the existing seams (docker-compose `env_file:`, the systemd
|
|
# unit's EnvironmentFile, and entrypoint.sh's /run/secrets shim) keep consuming
|
|
# unchanged. Common secrets + the per-host overrides are concatenated with the host
|
|
# file LAST, so a host value wins (env_file and `source` both take the last
|
|
# occurrence of a duplicate key).
|
|
#
|
|
# Presence-detected: a host is "configured for SOPS" only when it has an age key at
|
|
# /etc/thermograph/age.key AND an environment name in /etc/thermograph/secrets-env
|
|
# (prod|beta|dev). A host without them keeps whatever /etc/thermograph.env it already
|
|
# has — so merging this is a safe no-op until a host is deliberately migrated. See
|
|
# deploy/secrets/README.md.
|
|
render_thermograph_secrets() {
|
|
local repo="${1:-.}" # repo root holding deploy/secrets
|
|
local key="${THERMOGRAPH_AGE_KEY:-/etc/thermograph/age.key}"
|
|
local marker="${THERMOGRAPH_SECRETS_ENV_FILE:-/etc/thermograph/secrets-env}"
|
|
local env_name
|
|
env_name=$(cat "$marker" 2>/dev/null || true)
|
|
|
|
# The per-host file is required; common.yaml is optional so the initial cutover can
|
|
# seed each host as an exact copy of its live env (byte-identical render, no value
|
|
# changes) and factor out shared secrets into common.yaml later.
|
|
#
|
|
# These are two different situations and used to share one `return 0`.
|
|
#
|
|
# (a) Not configured for SOPS at all — no env marker, or no age key. The LAN
|
|
# dev box is legitimately in this state. Saying so and succeeding is right.
|
|
if [ -z "$env_name" ] || [ ! -f "$key" ]; then
|
|
echo "==> SOPS secrets not configured here; using existing /etc/thermograph.env"
|
|
return 0
|
|
fi
|
|
|
|
# (b) Configured — there is a key and a named environment — but the vault for
|
|
# that environment is not where we looked. That is a failure, and it used
|
|
# to return 0: a silent no-op wearing a success code. A deploy would report
|
|
# success having rendered nothing, and the host would quietly keep serving
|
|
# whatever /etc/thermograph.env already held, including after a rotation.
|
|
#
|
|
# The likely cause is the caller passing the wrong root. This function
|
|
# wants the directory CONTAINING deploy/secrets — on the hosts that is
|
|
# /opt/thermograph/infra, not /opt/thermograph. Getting it wrong looked
|
|
# exactly like "not configured here", which is why it went unnoticed.
|
|
if [ ! -f "$repo/deploy/secrets/${env_name}.yaml" ]; then
|
|
echo "!! this host is configured for SOPS (env='${env_name}', age key present)" >&2
|
|
echo "!! but no vault was found at ${repo}/deploy/secrets/${env_name}.yaml" >&2
|
|
echo "!! pass the directory containing deploy/secrets — on the hosts that is" >&2
|
|
echo "!! /opt/thermograph/infra — or add ${env_name}.yaml to the vault." >&2
|
|
return 1
|
|
fi
|
|
if ! command -v sops >/dev/null 2>&1; then
|
|
echo "!! sops not installed but this host is configured for SOPS secrets" >&2
|
|
return 1
|
|
fi
|
|
|
|
echo "==> Rendering /etc/thermograph.env from deploy/secrets (common + ${env_name})"
|
|
# The age private key is root-owned (0400). Read it directly if we can, else via
|
|
# sudo into SOPS_AGE_KEY — so the key never has to be readable by the deploy user.
|
|
# (The render needs sudo to write /etc/thermograph.env below anyway.)
|
|
local key_env=()
|
|
if [ -r "$key" ]; then
|
|
key_env=("SOPS_AGE_KEY_FILE=$key")
|
|
else
|
|
local keymat; keymat=$(sudo cat "$key" 2>/dev/null | grep '^AGE-SECRET-KEY-' || true)
|
|
[ -n "$keymat" ] || { echo "!! cannot read age key at $key (need sudo)" >&2; return 1; }
|
|
key_env=("SOPS_AGE_KEY=$keymat")
|
|
fi
|
|
local tmp; tmp=$(mktemp)
|
|
# The tmp file holds DECRYPTED secrets, so every exit path below removes it.
|
|
#
|
|
# Deliberately NOT a `trap ... RETURN`, which looks like the tidier way to make
|
|
# that unconditional: a RETURN trap set inside a SOURCED function persists in
|
|
# the CALLER's shell after the function returns, and a RETURN trap also fires
|
|
# when a `.`/source completes. deploy.sh sources /etc/thermograph.env a few
|
|
# lines after calling us, which would re-fire the trap at top level where `tmp`
|
|
# (function-local) is unset -- fatal under deploy.sh's `set -u`, and silent,
|
|
# because that line already redirects stderr to /dev/null. Explicit removal on
|
|
# each path is duller and correct.
|
|
: > "$tmp" || { rm -f "$tmp"; return 1; }
|
|
# Decrypt failures return 1 explicitly, so a partial env is never written and
|
|
# correctness doesn't depend on the caller's shell options. common first, host
|
|
# second, so a host value overrides a shared one (last-wins).
|
|
if [ -f "$repo/deploy/secrets/common.yaml" ]; then
|
|
env "${key_env[@]}" sops -d --input-type yaml --output-type dotenv \
|
|
"$repo/deploy/secrets/common.yaml" >> "$tmp" || { rm -f "$tmp"; return 1; }
|
|
fi
|
|
env "${key_env[@]}" sops -d --input-type yaml --output-type dotenv \
|
|
"$repo/deploy/secrets/${env_name}.yaml" >> "$tmp" || { rm -f "$tmp"; return 1; }
|
|
|
|
# Write /etc/thermograph.env. Prefer an in-place write when the existing file is
|
|
# writable by us (e.g. a group-writable 0660 root:<deploygroup> on a box whose CI
|
|
# deploy user isn't root and has no broad sudo — beta's `deploy`), since that needs
|
|
# only file write, not /etc dir write or sudo. Else install; else sudo install (a
|
|
# root/agent deploy) -- and on that sudo path CHOWN the result to the invoking
|
|
# deploy user (-o/-g $(id -un/-gn)), or the file lands root:root 0640 and the very
|
|
# next line of deploy.sh (`. /etc/thermograph.env` as that non-root user) can't read
|
|
# it, so POSTGRES_PASSWORD never enters the env and `docker compose` dies on
|
|
# interpolation. Fail loudly rather than deploy against stale secrets.
|
|
# rc + a single cleanup point: the plaintext tmp must go whichever branch runs,
|
|
# but the old trailing `rm` was also the function's last command, so it masked a
|
|
# failed in-place `cat` write to status 0. Capture the status instead, so a
|
|
# half-written /etc/thermograph.env fails loudly rather than deploying stale.
|
|
local rc=0
|
|
if [ -f /etc/thermograph.env ] && [ -w /etc/thermograph.env ]; then
|
|
cat "$tmp" > /etc/thermograph.env || rc=1
|
|
elif install -m 0640 "$tmp" /etc/thermograph.env 2>/dev/null; then :
|
|
elif sudo install -m 0640 -o "$(id -un)" -g "$(id -gn)" "$tmp" /etc/thermograph.env 2>/dev/null; then :
|
|
else
|
|
echo "!! cannot write /etc/thermograph.env (need file write access or passwordless sudo)" >&2
|
|
rc=1
|
|
fi
|
|
rm -f "$tmp"
|
|
return "$rc"
|
|
}
|