Docs pass after the app-repo archive; sync key-gaps skill; add CLAUDE.md (#5)
This commit is contained in:
parent
5fd96552f7
commit
e929f4606f
6 changed files with 113 additions and 25 deletions
|
|
@ -55,17 +55,24 @@ The `Cross-environment drift` section should report **none** for tracked keys.
|
||||||
`THERMOGRAPH_VAPID_PRIVATE_KEY/_PUBLIC_KEY`). Missing here is the worst case: the app
|
`THERMOGRAPH_VAPID_PRIVATE_KEY/_PUBLIC_KEY`). Missing here is the worst case: the app
|
||||||
silently mints a new value on boot, invalidating every login session and push
|
silently mints a new value on boot, invalidating every login session and push
|
||||||
subscription. Fix before deploying.
|
subscription. Fix before deploying.
|
||||||
- **MISSING** — a required secret is absent (`POSTGRES_PASSWORD`, `THERMOGRAPH_METRICS_TOKEN`,
|
- **MISSING** — a required secret is absent (`POSTGRES_PASSWORD`, `REGISTRY_TOKEN`).
|
||||||
`REGISTRY_TOKEN`).
|
|
||||||
- **Feature GAP** — a feature is *partially* configured (e.g. `discord-account-linking`
|
- **Feature GAP** — a feature is *partially* configured (e.g. `discord-account-linking`
|
||||||
has the app id but not the client secret), so it's silently broken. A fully-unset
|
has the app id but not the client secret), so it's silently broken. A fully-unset
|
||||||
feature is "off", not a gap.
|
feature is "off", not a gap.
|
||||||
|
- **Dependent-key GAP** — an optional key is set but the *other* key it needs to do
|
||||||
|
anything isn't (e.g. `THERMOGRAPH_DISCORD_WEATHER_CHANNEL` set with no
|
||||||
|
`THERMOGRAPH_DISCORD_BOT_TOKEN` — discord.py gates the post on both). Asymmetric,
|
||||||
|
unlike a feature group: the prerequisite key is fine set alone (the bot token alone
|
||||||
|
already enables DMs).
|
||||||
- **drift** — a tracked key is in some environments but not others; often intentional,
|
- **drift** — a tracked key is in some environments but not others; often intentional,
|
||||||
but worth a glance.
|
but worth a glance.
|
||||||
|
|
||||||
## Keeping it current
|
## Keeping it current
|
||||||
|
|
||||||
The secret manifest (which keys are required / self-generating / grouped into features)
|
The secret manifest (which keys are required / self-generating / grouped into features /
|
||||||
lives at the top of `key_gaps.py` in `REQUIRED`, `OPTIONAL_SELF_GEN`, `FEATURE_GROUPS`,
|
dependent on another key) lives at the top of `key_gaps.py` in `REQUIRED`,
|
||||||
and `STANDALONE_OPTIONAL`. When a new `THERMOGRAPH_*` credential is added to
|
`OPTIONAL_SELF_GEN`, `FEATURE_GROUPS`, `DEPENDENT_OPTIONAL`, and `STANDALONE_OPTIONAL`.
|
||||||
`deploy/thermograph.env.example` / `deploy/secrets/`, add it there too.
|
When a new `THERMOGRAPH_*` credential is added to `deploy/thermograph.env.example` /
|
||||||
|
`deploy/secrets/`, add it there too — check the code path that reads it (`os.environ.get`
|
||||||
|
call site) to see whether it's truly required, self-generating, part of an all-or-nothing
|
||||||
|
feature, or only meaningful alongside another key, rather than guessing from the name.
|
||||||
|
|
|
||||||
|
|
@ -8,6 +8,9 @@ Reports, per environment and across them:
|
||||||
- Required gaps: a required secret is missing.
|
- Required gaps: a required secret is missing.
|
||||||
- Feature gaps: a feature is *partially* configured (some keys present, some not),
|
- Feature gaps: a feature is *partially* configured (some keys present, some not),
|
||||||
so it's silently broken. Fully-unset features are "off", not a gap.
|
so it's silently broken. Fully-unset features are "off", not a gap.
|
||||||
|
- Dependent-key gaps: an optional key is set but the OTHER key it needs to do
|
||||||
|
anything isn't (e.g. a Discord channel id with no bot token) — asymmetric,
|
||||||
|
unlike feature gaps: the prerequisite is fine set alone.
|
||||||
- Drift: a key present in one environment but absent in another.
|
- Drift: a key present in one environment but absent in another.
|
||||||
|
|
||||||
It reads only KEY NAMES, never values — safe to run anywhere. Sources per environment:
|
It reads only KEY NAMES, never values — safe to run anywhere. Sources per environment:
|
||||||
|
|
@ -35,7 +38,6 @@ REQUIRED = {
|
||||||
"THERMOGRAPH_AUTH_SECRET": True,
|
"THERMOGRAPH_AUTH_SECRET": True,
|
||||||
"THERMOGRAPH_VAPID_PRIVATE_KEY": True,
|
"THERMOGRAPH_VAPID_PRIVATE_KEY": True,
|
||||||
"THERMOGRAPH_VAPID_PUBLIC_KEY": True,
|
"THERMOGRAPH_VAPID_PUBLIC_KEY": True,
|
||||||
"THERMOGRAPH_METRICS_TOKEN": False,
|
|
||||||
"REGISTRY_TOKEN": False,
|
"REGISTRY_TOKEN": False,
|
||||||
}
|
}
|
||||||
OPTIONAL_SELF_GEN = {
|
OPTIONAL_SELF_GEN = {
|
||||||
|
|
@ -46,11 +48,22 @@ FEATURE_GROUPS = {
|
||||||
"discord-account-linking": ["THERMOGRAPH_DISCORD_APP_ID", "THERMOGRAPH_DISCORD_CLIENT_SECRET"],
|
"discord-account-linking": ["THERMOGRAPH_DISCORD_APP_ID", "THERMOGRAPH_DISCORD_CLIENT_SECRET"],
|
||||||
"mail-smtp": ["THERMOGRAPH_SMTP_USER", "THERMOGRAPH_SMTP_PASSWORD"],
|
"mail-smtp": ["THERMOGRAPH_SMTP_USER", "THERMOGRAPH_SMTP_PASSWORD"],
|
||||||
}
|
}
|
||||||
|
# Optional keys that only do anything when a specific OTHER key is also set. Unlike
|
||||||
|
# FEATURE_GROUPS the relationship is asymmetric: the prerequisite is fine set alone
|
||||||
|
# (THERMOGRAPH_DISCORD_BOT_TOKEN alone already enables DMs), but the dependent key
|
||||||
|
# alone is a silent no-op (discord.py gates both channel posts on
|
||||||
|
# `bool(BOT_TOKEN and <channel>)`) — flagging it as a symmetric feature group would
|
||||||
|
# false-positive on every environment that has the bot token for DMs only.
|
||||||
|
DEPENDENT_OPTIONAL = {
|
||||||
|
"THERMOGRAPH_DISCORD_WEATHER_CHANNEL": "THERMOGRAPH_DISCORD_BOT_TOKEN",
|
||||||
|
"THERMOGRAPH_DISCORD_SUBSCRIPTION_CHANNEL": "THERMOGRAPH_DISCORD_BOT_TOKEN",
|
||||||
|
}
|
||||||
# Standalone optional keys: reported present/absent, never a "gap" on their own.
|
# Standalone optional keys: reported present/absent, never a "gap" on their own.
|
||||||
STANDALONE_OPTIONAL = [
|
STANDALONE_OPTIONAL = [
|
||||||
|
"THERMOGRAPH_METRICS_TOKEN", # ops metrics remote access (else loopback-only, not broken)
|
||||||
"THERMOGRAPH_DISCORD_WEBHOOK", # daily post
|
"THERMOGRAPH_DISCORD_WEBHOOK", # daily post
|
||||||
"THERMOGRAPH_DISCORD_PUBLIC_KEY", # slash-command interactions
|
"THERMOGRAPH_DISCORD_PUBLIC_KEY", # slash-command interactions
|
||||||
"THERMOGRAPH_DISCORD_BOT_TOKEN", # bot DMs / gateway bot
|
"THERMOGRAPH_DISCORD_BOT_TOKEN", # bot DMs / gateway bot (prerequisite for the two below)
|
||||||
]
|
]
|
||||||
|
|
||||||
C = {"red": "\033[31m", "yellow": "\033[33m", "green": "\033[32m",
|
C = {"red": "\033[31m", "yellow": "\033[33m", "green": "\033[32m",
|
||||||
|
|
@ -91,7 +104,7 @@ def audit(envs: dict[str, set[str]]) -> int:
|
||||||
for n in names:
|
for n in names:
|
||||||
print(f" {C['dim']}{n}: {len(envs[n])} keys{C['off']}")
|
print(f" {C['dim']}{n}: {len(envs[n])} keys{C['off']}")
|
||||||
print()
|
print()
|
||||||
critical = required = feature = 0
|
critical = required = feature = dependent = 0
|
||||||
|
|
||||||
def present(env, key): # noqa: ANN001
|
def present(env, key): # noqa: ANN001
|
||||||
return key in envs[env]
|
return key in envs[env]
|
||||||
|
|
@ -133,6 +146,19 @@ def audit(envs: dict[str, set[str]]) -> int:
|
||||||
print(f" {C['red']}GAP {C['off']} {feat} @ {n}: has {have}, MISSING {miss}")
|
print(f" {C['red']}GAP {C['off']} {feat} @ {n}: has {have}, MISSING {miss}")
|
||||||
print()
|
print()
|
||||||
|
|
||||||
|
# Dependent optional — the key only does anything with its prerequisite also set
|
||||||
|
print(f"{C['bold']}Dependent optional keys{C['off']}")
|
||||||
|
for key, prereq in DEPENDENT_OPTIONAL.items():
|
||||||
|
for n in names:
|
||||||
|
if not present(n, key):
|
||||||
|
print(f" {C['dim']}off {key} @ {n} (unset){C['off']}")
|
||||||
|
elif present(n, prereq):
|
||||||
|
print(f" {C['green']}OK{C['off']} {key} @ {n} (enabled, {prereq} set)")
|
||||||
|
else:
|
||||||
|
dependent += 1
|
||||||
|
print(f" {C['red']}GAP {C['off']} {key} @ {n}: set but {prereq} is missing — silently no-ops")
|
||||||
|
print()
|
||||||
|
|
||||||
# Standalone optional
|
# Standalone optional
|
||||||
print(f"{C['bold']}Optional keys{C['off']}")
|
print(f"{C['bold']}Optional keys{C['off']}")
|
||||||
for key in STANDALONE_OPTIONAL:
|
for key in STANDALONE_OPTIONAL:
|
||||||
|
|
@ -150,6 +176,7 @@ def audit(envs: dict[str, set[str]]) -> int:
|
||||||
where = [n for n in names if key in envs[n]]
|
where = [n for n in names if key in envs[n]]
|
||||||
if len(where) != len(names) and (key in REQUIRED or key in OPTIONAL_SELF_GEN
|
if len(where) != len(names) and (key in REQUIRED or key in OPTIONAL_SELF_GEN
|
||||||
or any(key in g for g in FEATURE_GROUPS.values())
|
or any(key in g for g in FEATURE_GROUPS.values())
|
||||||
|
or key in DEPENDENT_OPTIONAL
|
||||||
or key in STANDALONE_OPTIONAL):
|
or key in STANDALONE_OPTIONAL):
|
||||||
drift = True
|
drift = True
|
||||||
absent = [n for n in names if n not in where]
|
absent = [n for n in names if n not in where]
|
||||||
|
|
@ -158,10 +185,10 @@ def audit(envs: dict[str, set[str]]) -> int:
|
||||||
print(f" {C['green']}none{C['off']} (tracked keys are consistent)")
|
print(f" {C['green']}none{C['off']} (tracked keys are consistent)")
|
||||||
print()
|
print()
|
||||||
|
|
||||||
total = critical + required + feature
|
total = critical + required + feature + dependent
|
||||||
color = C["red"] if total else C["green"]
|
color = C["red"] if total else C["green"]
|
||||||
print(f"{color}{C['bold']}Summary: {critical} critical, {required} required, "
|
print(f"{color}{C['bold']}Summary: {critical} critical, {required} required, "
|
||||||
f"{feature} feature gaps{C['off']}")
|
f"{feature} feature gaps, {dependent} dependent-key gaps{C['off']}")
|
||||||
return 1 if (critical or required) else 0
|
return 1 if (critical or required) else 0
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
42
CLAUDE.md
Normal file
42
CLAUDE.md
Normal file
|
|
@ -0,0 +1,42 @@
|
||||||
|
# thermograph-infra — agent instructions
|
||||||
|
|
||||||
|
How and where the already-built Thermograph app images run. This repo owns
|
||||||
|
Terraform, the SOPS+age secrets vault, compose files, deploy scripts, host
|
||||||
|
provisioning, and the ops cron (DB backup + IndexNow). The app repos
|
||||||
|
(`thermograph-backend`, `thermograph-frontend`) own building and testing the
|
||||||
|
images; this repo never checks out app source. The old monorepo
|
||||||
|
(`emi/thermograph`) is archived — do not point anything at it.
|
||||||
|
|
||||||
|
## The four machines
|
||||||
|
|
||||||
|
Same as ACCESS.md: **dev machine** (operator's box, LAN dev server, CI runner),
|
||||||
|
**prod** (`169.58.46.181`, thermograph.org, `agent` user, passwordless sudo),
|
||||||
|
**beta** (`75.119.132.91`, beta.thermograph.org + Forgejo, `agent` user). Hosts'
|
||||||
|
`/opt/thermograph` (and LAN's `~/thermograph-dev`) are checkouts of THIS repo.
|
||||||
|
|
||||||
|
## Deploys
|
||||||
|
|
||||||
|
- `deploy/deploy.sh` — the one deploy path (prod/beta): resets this repo's
|
||||||
|
checkout, renders secrets from the SOPS vault, pulls the per-service image
|
||||||
|
tags (`BACKEND_IMAGE_TAG`/`FRONTEND_IMAGE_TAG`, persisted in untracked
|
||||||
|
`deploy/.image-tags.env`), rolls the target service, health-checks via
|
||||||
|
container healthchecks. Invoked over SSH by the app repos' deploy workflows.
|
||||||
|
- `deploy/deploy-dev.sh` — thin LAN-dev wrapper around deploy.sh (dev compose
|
||||||
|
overlay, `~/thermograph-dev`, branch `dev`).
|
||||||
|
- Branch model: see README — infra `main` is live on prod+beta, `dev` on LAN;
|
||||||
|
`release` is currently unused. App code is env-staged via image tags; infra
|
||||||
|
is not.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
- **Never run `terraform apply` casually** — no tfstate exists anywhere (never
|
||||||
|
persisted), so an apply would attempt full re-provisioning of live hosts.
|
||||||
|
Terraform here is executable documentation until state is bootstrapped.
|
||||||
|
- Secrets: only via the SOPS vault (`deploy/secrets/*.yaml`, `sops edit` +
|
||||||
|
commit + deploy). Never hand-edit `/etc/thermograph.env` on a SOPS-enabled
|
||||||
|
host — it's a rendered artifact. `secrets-guard` CI rejects plaintext.
|
||||||
|
- The ops cron (`.forgejo/workflows/ops-cron.yml`) is THE prod backup — it uses
|
||||||
|
this repo's `PROD_SSH_*` Actions secrets. If you touch it, verify a dump
|
||||||
|
actually lands in `agent@prod:~/thermograph-backups/`.
|
||||||
|
- Commits/PRs: concise and technical; never mention AI/assistants/automated
|
||||||
|
authorship.
|
||||||
18
README.md
18
README.md
|
|
@ -26,3 +26,21 @@ The app's own source, `Dockerfile`, and build/test CI stay in the app repo —
|
||||||
this repo never checks out app source; hosts only pull tagged images from the
|
this repo never checks out app source; hosts only pull tagged images from the
|
||||||
registry. See `ACCESS.md` for host access and the Swarm/Forgejo topology, and
|
registry. See `ACCESS.md` for host access and the Swarm/Forgejo topology, and
|
||||||
`terraform/README.md` for the day-to-day `plan`/`apply` workflow.
|
`terraform/README.md` for the day-to-day `plan`/`apply` workflow.
|
||||||
|
|
||||||
|
## Branches & how changes reach each environment
|
||||||
|
|
||||||
|
- **`main`** — what **prod and beta** run: their `/opt/thermograph` checkouts
|
||||||
|
`git reset --hard origin/main` at the start of every deploy (`deploy/deploy.sh`).
|
||||||
|
A merge to `main` reaches those hosts on the next app deploy (or a by-hand
|
||||||
|
`deploy.sh` run); there is no separate infra deploy trigger.
|
||||||
|
- **`dev`** — what **LAN dev** runs: `~/thermograph-dev` resets to it via
|
||||||
|
`deploy/deploy-dev.sh`. Keep it fast-forwarded to `main` (infra changes are not
|
||||||
|
environment-staged today; the branches exist so LAN dev *can* trail or lead
|
||||||
|
when needed).
|
||||||
|
- **`release`** — currently consumed by nothing (prod tracks `main`, not
|
||||||
|
`release`). It exists to mirror the app repos' dev→main→release promotion
|
||||||
|
shape if per-environment infra staging is ever wanted; until then, treat
|
||||||
|
`main` as live-everywhere.
|
||||||
|
|
||||||
|
Note the asymmetry with the app repos: app code IS environment-staged
|
||||||
|
(dev→main→release maps to LAN→beta→prod via image tags), infra is not.
|
||||||
|
|
|
||||||
|
|
@ -2,19 +2,13 @@
|
||||||
# LAN-dev deploy: roll the per-service registry-pull stack (see deploy.sh) onto
|
# LAN-dev deploy: roll the per-service registry-pull stack (see deploy.sh) onto
|
||||||
# the ~/thermograph-dev overlay instead of prod/beta's loopback-only stack.
|
# the ~/thermograph-dev overlay instead of prod/beta's loopback-only stack.
|
||||||
#
|
#
|
||||||
# *** PREREQUISITE -- READ BEFORE RUNNING ***
|
|
||||||
# This script assumes $APP_DIR is a checkout of THIS repo (thermograph-infra),
|
# This script assumes $APP_DIR is a checkout of THIS repo (thermograph-infra),
|
||||||
# the same way /opt/thermograph is on prod/beta. As of this writing, the LAN
|
# the same way /opt/thermograph is on prod/beta. That is the live state: the LAN
|
||||||
# dev box's ~/thermograph-dev is still provisioned by deploy/provision-dev-lan.sh
|
# box's ~/thermograph-dev was reprovisioned as an infra checkout during the
|
||||||
# cloning the OLD MONOREPO (thermograph.git, see its REPO_URL default) and running
|
# 2026-07-22 cutover (the app monorepo is archived), provision-dev-lan.sh's
|
||||||
# the monorepo's build-in-place deploy/deploy-dev.sh -- an app-code checkout with
|
# REPO_URL defaults to this repo, and the app repos' deploy-dev.yml workflows
|
||||||
# Dockerfiles, not an infra checkout with compose files. Until provision-dev-lan.sh
|
# invoke this script on the thermograph-lan runner. This IS the live LAN-dev
|
||||||
# (and its REPO_URL) is repointed at thermograph-infra and ~/thermograph-dev is
|
# deploy path.
|
||||||
# reprovisioned from that, THIS SCRIPT IS INERT: pointing it at the live dev box
|
|
||||||
# today would `git fetch`/`reset` the wrong repository entirely. Do not treat this
|
|
||||||
# as a live deploy path until that reprovision happens -- it exists now so the infra
|
|
||||||
# repo carries its own dev-deploy story from day one, same as it already carries
|
|
||||||
# deploy.sh for prod/beta.
|
|
||||||
#
|
#
|
||||||
# Design: a thin wrapper around deploy.sh, not a duplicate. deploy.sh already owns
|
# Design: a thin wrapper around deploy.sh, not a duplicate. deploy.sh already owns
|
||||||
# the entire per-service registry-pull mechanism -- secrets sourcing, docker login,
|
# the entire per-service registry-pull mechanism -- secrets sourcing, docker login,
|
||||||
|
|
@ -30,7 +24,7 @@
|
||||||
# from prod/beta (not just "different files/directory"), promote this to a
|
# from prod/beta (not just "different files/directory"), promote this to a
|
||||||
# standalone script at that point rather than growing special cases into deploy.sh.
|
# standalone script at that point rather than growing special cases into deploy.sh.
|
||||||
#
|
#
|
||||||
# Usage (once the prerequisite above is satisfied), mirrors deploy.sh directly:
|
# Usage, mirrors deploy.sh directly:
|
||||||
# # roll just the backend onto a dev-tagged image:
|
# # roll just the backend onto a dev-tagged image:
|
||||||
# ssh dev-box 'SERVICE=backend BACKEND_IMAGE_TAG=sha-<12hex> deploy/deploy-dev.sh'
|
# ssh dev-box 'SERVICE=backend BACKEND_IMAGE_TAG=sha-<12hex> deploy/deploy-dev.sh'
|
||||||
# # bring the whole dev stack up (both tags required, same as deploy.sh):
|
# # bring the whole dev stack up (both tags required, same as deploy.sh):
|
||||||
|
|
|
||||||
|
|
@ -9,7 +9,7 @@
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
APP_DIR="${APP_DIR:-$HOME/thermograph-dev}"
|
APP_DIR="${APP_DIR:-$HOME/thermograph-dev}"
|
||||||
REPO_URL="${REPO_URL:-http://10.10.0.2:3080/emi/thermograph.git}"
|
REPO_URL="${REPO_URL:-http://10.10.0.2:3080/emi/thermograph-infra.git}"
|
||||||
BRANCH="${BRANCH:-dev}"
|
BRANCH="${BRANCH:-dev}"
|
||||||
here="$(cd "$(dirname "$0")" && pwd)"
|
here="$(cd "$(dirname "$0")" && pwd)"
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue