2026-07-22 18:57:02 +00:00
|
|
|
#!/usr/bin/env bash
|
|
|
|
|
# Run ON a prod/beta box (via `ssh <box> 'bash -s' < this`), after the encrypted vault
|
|
|
|
|
# file has been scp'd to /tmp/tg-<env>.yaml. NON-DESTRUCTIVE go/no-go for the cutover:
|
|
|
|
|
# renders the vault and diffs the KEY=VALUE set against the live /etc/thermograph.env.
|
|
|
|
|
# Prints only PASS/FAIL and any differing key NAMES — never secret values. No git, no
|
|
|
|
|
# writes; touches nothing. (The encrypted file is safe to ship; only ciphertext.)
|
|
|
|
|
set -euo pipefail
|
|
|
|
|
key=/etc/thermograph/age.key
|
|
|
|
|
env_name="$(sudo cat /etc/thermograph/secrets-env)"
|
|
|
|
|
host="/tmp/tg-${env_name}.yaml"
|
|
|
|
|
common="/tmp/tg-common.yaml"
|
|
|
|
|
|
|
|
|
|
command -v sops >/dev/null || { echo "FAIL: sops not installed"; exit 1; }
|
|
|
|
|
[ -f "$host" ] || { echo "FAIL: $host not found — scp the encrypted vault file first"; exit 1; }
|
|
|
|
|
|
|
|
|
|
keymat="$(sudo grep '^AGE-SECRET-KEY-' "$key")"
|
|
|
|
|
tmp="$(mktemp)"; live="$(mktemp)"; trap 'rm -f "$tmp" "$live"' EXIT
|
|
|
|
|
: > "$tmp"
|
|
|
|
|
[ -f "$common" ] && SOPS_AGE_KEY="$keymat" sops -d --input-type yaml --output-type dotenv "$common" >> "$tmp"
|
|
|
|
|
SOPS_AGE_KEY="$keymat" sops -d --input-type yaml --output-type dotenv "$host" >> "$tmp"
|
|
|
|
|
|
shell: add shellcheck CI guard and drive the tree to zero findings
No static analysis has ever run over the ~2k lines of shell that deploy,
provision secrets, and bootstrap hosts as root over SSH. Add shell-lint.yml
(pinned shellcheck v0.11.0 + sha256, -x, default severity, fail on any
finding) and fix everything it reports, plus two defects it structurally
cannot see.
Not path-filtered, matching secrets-guard's call: a backstop that only runs
when you expect it to isn't a backstop. Scripts are discovered with find, so
new ones are covered on landing. The version is pinned to a static release
rather than apt's, so a drifted shellcheck can't fail CI on an unrelated push.
render-secrets.sh: the mktemp holding DECRYPTED vault contents was only
removed on the success path and one failure branch, so a sops decrypt failure
left plaintext POSTGRES_PASSWORD in /tmp indefinitely on a live host. A RETURN
trap makes removal unconditional, and the two sops calls now `|| return 1`
explicitly instead of relying on the caller's set -e (a bare set -e abort
skips the trap). The function stays free of `set -e` itself -- it is sourced,
and shell options would leak into the caller.
autoscale.sh: ran `set -eu` without pipefail while piping docker stats into
awk, so a failed left side was swallowed and the loop scaled on empty input.
Promoted to pipefail with avg_cpu's failure treated as a missed sample, so a
daemon hiccup can't kill the autoscaler. Verified busybox ash in docker:27-cli
supports pipefail and the script still parses there.
capture-fixtures.sh: `jq . || cat` ran cat after jq had already consumed
stdin, silently writing a truncated fixture; now a real if/else that fails
loudly. deploy.sh/deploy-stack.sh: `# shellcheck source=` paths corrected for
the monorepo layout, and /etc/thermograph.env marked unfollowable (it is
rendered at deploy time and cannot exist at lint time).
2026-07-23 21:59:47 +00:00
|
|
|
# sudo is only for the READ of the root-owned live env; the redirect target is our
|
|
|
|
|
# own mktemp file, so the redirect correctly runs as the invoking user (sudo tee
|
|
|
|
|
# would wrongly write it as root).
|
|
|
|
|
# shellcheck disable=SC2024
|
2026-07-22 18:57:02 +00:00
|
|
|
sudo cat /etc/thermograph.env > "$live"
|
|
|
|
|
python3 - "$live" "$tmp" <<'PY'
|
|
|
|
|
import sys
|
|
|
|
|
def load(p):
|
|
|
|
|
out={}
|
|
|
|
|
for line in open(p, encoding="utf-8"):
|
|
|
|
|
s=line.rstrip("\n").strip()
|
|
|
|
|
if not s or s.startswith("#") or "=" not in s: continue
|
|
|
|
|
k,_,v=s.partition("="); out[k.strip()]=v
|
|
|
|
|
return out
|
|
|
|
|
live,rend=load(sys.argv[1]),load(sys.argv[2])
|
|
|
|
|
lost=sorted(set(live)-set(rend)); added=sorted(set(rend)-set(live))
|
|
|
|
|
changed=sorted(k for k in live if k in rend and live[k]!=rend[k])
|
|
|
|
|
if lost or added or changed:
|
|
|
|
|
print("FAIL — the render would NOT match the live env")
|
|
|
|
|
if lost: print(" lost keys:", lost)
|
|
|
|
|
if added: print(" extra keys:", added)
|
|
|
|
|
if changed: print(" value-changed keys:", changed)
|
|
|
|
|
sys.exit(1)
|
|
|
|
|
print(f"PASS — all {len(live)} keys render byte-identical to the live /etc/thermograph.env")
|
|
|
|
|
PY
|