thermograph/infra/.claude/skills/key-gaps/SKILL.md

name

description

key-gaps

Audit which secret keys are missing, partially configured, or drifting across Thermograph environments (prod, beta, dev) — comparing the live /etc/thermograph.env on each box and/or the SOPS vault files (deploy/secrets/*.yaml). Use when asked about "key gaps", "missing keys/secrets", "what's not set", cross-environment secret drift, or as a pre-deploy / pre-cutover safety check. Reads key NAMES only, never values.

Key-gap audit

Reports, per environment and across them, where secret keys are missing or inconsistent — with special weight on the failure modes that bite silently here.

Run the deterministic script; don't eyeball it:

python3 .claude/skills/key-gaps/key_gaps.py <env>=<source> [<env>=<source> ...]

<source> is either a SOPS file (deploy/secrets/prod.yaml) or a key-list file (one KEY per line, or KEY=/KEY: lines). Keys are readable in a SOPS file even encrypted, so no age key or decryption is needed. Exit status is non-zero if any CRITICAL or required gap is found (so it also works as a CI/pre-deploy gate).

Audit the SOPS vault (offline, no SSH)

python3 .claude/skills/key-gaps/key_gaps.py \
  prod=deploy/secrets/prod.yaml beta=deploy/secrets/beta.yaml

Audit the LIVE boxes (read-only, key names only)

Gather the key names over SSH (never the values), then audit. Hosts per the root CLAUDE.md topology:

• vps2 (169.58.46.181) hosts BOTH prod and beta as separate Swarm stacks — TWO live env files on the same box: /etc/thermograph.env (prod) and /etc/thermograph-beta.env (beta). "The live env file per environment" is no longer one path per host; it's one path per environment, and both environments' files live on this one host. Get both in the same SSH round trip, or two separate commands against the same host — never assume one host means one file here.
• vps1 (75.119.132.91) hosts dev, at the same /etc/thermograph.env path (it's a different host, so no collision with prod's file of the same name).

K=~/.ssh/thermograph_agent_ed25519
# sudo: the env files are root-owned (0640). On these boxes `agent` can often
# read them directly too, but sudo works uniformly regardless.
#
# The character class must allow DIGITS. This was `^[A-Z_]+=` until 2026-07-24,
# which silently skipped every key whose name contains a digit — in this estate
# that is all six *_S3_* keys. Prod audited as 26 keys against 32 real ones, and
# the audit reported the S3 credentials as missing from both hosts. They were
# present the whole time, and the phantom was independently "confirmed" twice by
# people re-running the same pattern. An audit that under-reports is worse than
# no audit: the invented finding sends someone to provision a credential that
# already exists, and it was briefly recorded as a root cause of a real bug.
# The match still stops at the `=`, so no value is ever read.
ssh -i $K agent@169.58.46.181 'sudo grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph.env' > /tmp/prod.keys       # vps2: prod
ssh -i $K agent@169.58.46.181 'sudo grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph-beta.env' > /tmp/beta.keys  # vps2: beta
ssh -i $K agent@75.119.132.91  'sudo grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph.env' > /tmp/dev.keys       # vps1: dev
python3 .claude/skills/key-gaps/key_gaps.py prod=/tmp/prod.keys beta=/tmp/beta.keys dev=/tmp/dev.keys

Verify a SOPS cutover matches live

Before flipping a host onto the vault, confirm the rendered set equals the live set — mix the two source types for the same box:

python3 .claude/skills/key-gaps/key_gaps.py live=/tmp/prod.keys vault=deploy/secrets/prod.yaml

The Cross-environment drift section should report none for tracked keys.

Reading the output

• CRITICAL — a required, self-generating secret is missing (THERMOGRAPH_AUTH_SECRET, THERMOGRAPH_VAPID_PRIVATE_KEY/_PUBLIC_KEY). Missing here is the worst case: the app silently mints a new value on boot, invalidating every login session and push subscription. Fix before deploying.
• MISSING — a required secret is absent (POSTGRES_PASSWORD, REGISTRY_TOKEN).
• Feature GAP — a feature is partially configured (e.g. discord-account-linking has the app id but not the client secret), so it's silently broken. A fully-unset feature is "off", not a gap.
• Dependent-key GAP — an optional key is set but the other key it needs to do anything isn't (e.g. THERMOGRAPH_DISCORD_WEATHER_CHANNEL set with no THERMOGRAPH_DISCORD_BOT_TOKEN — discord.py gates the post on both). Asymmetric, unlike a feature group: the prerequisite key is fine set alone (the bot token alone already enables DMs).
• drift — a tracked key is in some environments but not others; often intentional, but worth a glance.

Keeping it current

The secret manifest (which keys are required / self-generating / grouped into features / dependent on another key) lives at the top of key_gaps.py in REQUIRED, OPTIONAL_SELF_GEN, FEATURE_GROUPS, DEPENDENT_OPTIONAL, and STANDALONE_OPTIONAL. When a new THERMOGRAPH_* credential is added to deploy/thermograph.env.example / deploy/secrets/, add it there too — check the code path that reads it (os.environ.get call site) to see whether it's truly required, self-generating, part of an all-or-nothing feature, or only meaningful alongside another key, rather than guessing from the name.