From 7ed375f32e249d2275c0d9afe9163346f639850f Mon Sep 17 00:00:00 2001 From: Emi Griffith Date: Fri, 24 Jul 2026 15:57:15 -0700 Subject: [PATCH 1/2] key-gaps: the key regex was blind to digits, inventing missing secrets MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `grep -oE "^[A-Z_]+="` cannot match a key name containing a digit. This estate has exactly six such names — all the *_S3_* keys — so a prod audit returned 26 keys against 32 real ones, and reported the S3 and lake credentials as missing from both live hosts. They were present the whole time. The phantom was independently reproduced twice by re-running the same pattern, which is what made it convincing, and it was briefly recorded as one of two root causes of the lake being unqueryable (#56). That issue has one cause: the missing duckdb-lake image. An audit that under-reports is worse than no audit. A missing-secret finding sends someone to provision a credential that already exists, and in a rotation tool it would justify writing over one. Fixed to `^[A-Z][A-Z0-9_]*=` in both the skill and key_gaps.py's docstring. The match still stops at the `=`, so no value is read — that property is the reason this grep exists rather than a parser. Centralis's secrets_gaps carried the same bug and is fixed separately, with a regression test. Claude-Session: https://claude.ai/code/session_0182KTMrsTHJc3TcewCatJFY --- .claude/worktrees/city-resolver | 1 + .claude/worktrees/thermograph-mentions | 1 + infra/.claude/skills/key-gaps/SKILL.md | 14 ++++++++++++-- infra/.claude/skills/key-gaps/key_gaps.py | 2 +- 4 files changed, 15 insertions(+), 3 deletions(-) create mode 160000 .claude/worktrees/city-resolver create mode 160000 .claude/worktrees/thermograph-mentions diff --git a/.claude/worktrees/city-resolver b/.claude/worktrees/city-resolver new file mode 160000 index 0000000..1b862aa --- /dev/null +++ b/.claude/worktrees/city-resolver @@ -0,0 +1 @@ +Subproject commit 1b862aa049403c02e8e9c4ef14dfc8c7a55b5aa2 diff --git a/.claude/worktrees/thermograph-mentions b/.claude/worktrees/thermograph-mentions new file mode 160000 index 0000000..ce4350c --- /dev/null +++ b/.claude/worktrees/thermograph-mentions @@ -0,0 +1 @@ +Subproject commit ce4350c64ecd6d6aaa89bdd965ea93f63b463295 diff --git a/infra/.claude/skills/key-gaps/SKILL.md b/infra/.claude/skills/key-gaps/SKILL.md index 3896d72..f243540 100644 --- a/infra/.claude/skills/key-gaps/SKILL.md +++ b/infra/.claude/skills/key-gaps/SKILL.md @@ -34,8 +34,18 @@ Gather the key names over SSH (never the values), then audit. Hosts/keys per INF K=~/.ssh/thermograph_agent_ed25519 # sudo: /etc/thermograph.env is root-owned (0640). On prod `agent` can read it # directly too, but sudo works uniformly on both boxes. -ssh -i $K agent@169.58.46.181 'sudo grep -oE "^[A-Z_]+=" /etc/thermograph.env' > /tmp/prod.keys # prod -ssh -i $K agent@75.119.132.91 'sudo grep -oE "^[A-Z_]+=" /etc/thermograph.env' > /tmp/beta.keys # beta +# +# The character class must allow DIGITS. This was `^[A-Z_]+=` until 2026-07-24, +# which silently skipped every key whose name contains a digit — in this estate +# that is all six *_S3_* keys. Prod audited as 26 keys against 32 real ones, and +# the audit reported the S3 credentials as missing from both hosts. They were +# present the whole time, and the phantom was independently "confirmed" twice by +# people re-running the same pattern. An audit that under-reports is worse than +# no audit: the invented finding sends someone to provision a credential that +# already exists, and it was briefly recorded as a root cause of a real bug. +# The match still stops at the `=`, so no value is ever read. +ssh -i $K agent@169.58.46.181 'sudo grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph.env' > /tmp/prod.keys # prod +ssh -i $K agent@75.119.132.91 'sudo grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph.env' > /tmp/beta.keys # beta python3 .claude/skills/key-gaps/key_gaps.py prod=/tmp/prod.keys beta=/tmp/beta.keys ``` diff --git a/infra/.claude/skills/key-gaps/key_gaps.py b/infra/.claude/skills/key-gaps/key_gaps.py index 775f7ab..a32df73 100755 --- a/infra/.claude/skills/key-gaps/key_gaps.py +++ b/infra/.claude/skills/key-gaps/key_gaps.py @@ -17,7 +17,7 @@ It reads only KEY NAMES, never values — safe to run anywhere. Sources per envi * a SOPS-encrypted YAML (deploy/secrets/.yaml) — keys are plaintext even when encrypted, so no age key or decryption is needed; or * a plain key-list file (one KEY per line, or KEY=... lines) — e.g. the output of - `ssh 'grep -oE "^[A-Z_]+=" /etc/thermograph.env'` for a live audit. + `ssh 'grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph.env'` for a live audit. Usage: key_gaps.py prod=deploy/secrets/prod.yaml beta=deploy/secrets/beta.yaml -- 2.45.2 From f4313b5d877f7f6991264056f6140ad68e01140a Mon Sep 17 00:00:00 2001 From: Emi Griffith Date: Fri, 24 Jul 2026 15:57:34 -0700 Subject: [PATCH 2/2] Drop accidentally-committed worktrees; ignore .claude/worktrees MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit swept in .claude/worktrees/city-resolver and .claude/worktrees/thermograph-mentions as embedded git repositories. Those are other Claude sessions' live checkouts and have no business in this tree. Ignoring the directory so `git add -A` cannot do it again — the repo already has a documented history of parallel sessions colliding through shared checkouts, and this is the same hazard wearing a different hat. Claude-Session: https://claude.ai/code/session_0182KTMrsTHJc3TcewCatJFY --- .claude/worktrees/city-resolver | 1 - .claude/worktrees/thermograph-mentions | 1 - .gitignore | 4 ++++ 3 files changed, 4 insertions(+), 2 deletions(-) delete mode 160000 .claude/worktrees/city-resolver delete mode 160000 .claude/worktrees/thermograph-mentions create mode 100644 .gitignore diff --git a/.claude/worktrees/city-resolver b/.claude/worktrees/city-resolver deleted file mode 160000 index 1b862aa..0000000 --- a/.claude/worktrees/city-resolver +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 1b862aa049403c02e8e9c4ef14dfc8c7a55b5aa2 diff --git a/.claude/worktrees/thermograph-mentions b/.claude/worktrees/thermograph-mentions deleted file mode 160000 index ce4350c..0000000 --- a/.claude/worktrees/thermograph-mentions +++ /dev/null @@ -1 +0,0 @@ -Subproject commit ce4350c64ecd6d6aaa89bdd965ea93f63b463295 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..5aee0f7 --- /dev/null +++ b/.gitignore @@ -0,0 +1,4 @@ + +# Claude Code worktrees. These are other sessions' checkouts living inside this +# repo; `git add -A` will otherwise commit them as embedded git repositories. +.claude/worktrees/ -- 2.45.2