6bd07d33cb
Phase 1 of moving the secret store to OpenBao. Nothing is cut over: TG_SECRETS_BACKEND defaults to sops for all three environments, so SOPS remains authoritative until an environment is flipped in a reviewed PR. The reason for the migration is one specific failure class. infra/CLAUDE.md requires that vps1 never hold prod credentials, but that rule is currently enforced by a caller-side shell flag (THERMOGRAPH_SECRETS_SKIP_COMMON) at three call sites. A fourth call site, or one by-hand render, writes both S3 keypairs, the VAPID private key, REGISTRY_TOKEN and the IndexNow/metrics tokens onto the Forgejo CI-runner box and exits 0. policies/tg-host-dev.hcl makes that a 403 instead: dev's identity cannot read the common path at all. Shape: - vps2 only, native systemd (not a container: a Swarm service is circular, and a plain docker run is one `prune -a` from gone), raft storage (2.6.0 deprecated the file backend), static auto-unseal so a reboot cannot block deploys, mesh-only self-signed TLS (not ACME — that would put DNS in the boot chain of the secret store). - AppRole per environment, CIDR-bound, 5-minute tokens. Each environment's secret-id is owned by its own deploy user, which is a boundary that does not exist today since prod and beta both reach the same root-owned age key. - Render still produces the same raw unquoted dotenv: Centralis compares /etc/thermograph.env textually and the Swarm entrypoint parses it by line. The backend dispatch in render-secrets.sh returns early rather than refactoring the shared write path, so the SOPS path stays byte-identical. Verified by rendering dev (12 keys) and prod (32 keys) through it, matching the live hosts. The duplicated write path is noted for consolidation once prod has been on OpenBao for a release cycle. The OpenBao renderer rejects values containing a newline or a shell metacharacter. /etc/thermograph.env is sourced by bash in six places, so such a value is a command-execution path on the deploy host; the SOPS path has always had this hazard and avoids it only because every current value happens to be alphanumeric. Also records that the age keypair is NOT retired by this migration: it is the backup encryption key for every off-box dump (ops-cron.yml:142,197, 30-day retention), so deleting it with the vault files would silently destroy a month of database recoverability. verify-parity.sh is the cutover gate — it renders both backends and diffs them, reporting key names and counts only, never values. |
||
|---|---|---|
| .. | ||
| .claude/skills/key-gaps | ||
| deploy | ||
| lake-iceberg | ||
| openbao | ||
| ops | ||
| terraform | ||
| .env.example | ||
| .gitignore | ||
| .sops.yaml | ||
| ACCESS.md | ||
| CLAUDE.md | ||
| DEPLOY-DEV.md | ||
| DEPLOY.md | ||
| docker-compose.dev.yml | ||
| docker-compose.openmeteo.yml | ||
| docker-compose.yml | ||
| Makefile | ||
| README.md | ||
infra/
Infrastructure for Thermograph: Terraform host
provisioning, the SOPS+age secrets vault, WireGuard/Swarm networking, Forgejo,
Caddy, mail, and the deploy scripts that run the already-built app images on each
host. This is a domain of the emi/thermograph monorepo — a host's checkout
(/opt/thermograph, /opt/thermograph-beta, or /opt/thermograph-dev) is a
checkout of the whole monorepo, and infra/ never builds app source; it only
runs published images.
terraform/— provisions/configures hosts (SSH-driven by default; an optional GCP-creating module is scaffolded, no live resources yet). Seeterraform/README.md. No tfstate is persisted anywhere — treatapplyas executable documentation, not a routine operation.deploy/secrets/— the git-native SOPS+age secrets vault (every app secret, encrypted at rest, rendered at deploy time). Seedeploy/secrets/README.md.deploy/swarm/,deploy/forgejo/— the WireGuard/Swarm mesh spanning vps1, vps2 and the desktop, and Forgejo (git + CI + registry), pinned to vps1. SeeACCESS.md.deploy/env-topology.sh— the single source of truth for where each environment (dev/beta/prod) lives: host, checkout path, branch, deploy mode, stack/compose name, env file, LB ports, DB role/database, service-name prefix. Every deploy path sources it and derives its behavior from it rather than guessing from the host it happens to be running on — necessary since vps2 alone now runs two environments.deploy/deploy.sh— the single deploy entry point for dev, beta and prod. TakesSERVICE=backend|frontend|allplusBACKEND_IMAGE_TAG/FRONTEND_IMAGE_TAG(and, on vps2,THERMOGRAPH_ENV=beta|prodto say which of the two checkouts it's acting on), resets the host checkout, renders secrets, and routes to the right orchestrator perenv-topology.sh.deploy/deploy-dev.sh— a thin dev-specific wrapper arounddeploy.sh(dev compose overlay, dev's secrets policy). SeeDEPLOY-DEV.md.deploy/stack/— the Swarm path, live on vps2 for both prod and beta:thermograph-stack.yml(prod: db, web, worker, lake, daemon, frontend, autoscaler, autoscaler-lake) andthermograph-beta-stack.yml(beta: the same service shape minusdband the autoscalers, every service name prefixedbeta-).deploy-stack.sh,autoscale.shandlb/are shared by both. Rolling updates are start-first, health-gated, with auto-rollback.STACK_TEST=1rehearses the whole stack on throwaway volumes and ports.docker-compose*.yml— the compose path, live only on dev (vps1) (db, backend, lake, daemon, frontend).docker-compose.dev.ymlis dev's mesh-only overlay;docker-compose.openmeteo.ymlis the self-hosted Open-Meteo overlay (prod only).make dev-upalso runs this path locally as a laptop convenience — that is not an "environment", just a local rehearsal.
Which path an environment takes is decided by deploy/env-topology.sh
(TG_DEPLOY_MODE, keyed by dev/beta/prod): dev is compose, beta and prod
are both stack. The old host-wide marker /etc/thermograph/deploy-mode still
exists as a fallback for a by-hand run with no explicit environment, but it
cannot describe vps2, which runs two environments in two different checkouts —
so it is no longer the thing that decides where files go. The workflows never
need to know which mode an environment runs; they only pass THERMOGRAPH_ENV.
Branches & how changes reach each environment
dev— deploys to dev on vps1 (/opt/thermograph-dev).main— deploys to beta on vps2 (/opt/thermograph-beta).release— deploys to prod on vps2 (/opt/thermograph).
App code IS environment-staged this way (dev→main→release maps to
vps1/dev → vps2/beta → vps2/prod via image tags, one Deploy workflow keyed by
branch). Infra itself is not environment-staged the same way: infra-sync.yml
fires on a push touching infra/** — on dev it fast-forwards vps1's dev
checkout, on main it fast-forwards both of vps2's checkouts (beta and
prod) — re-rendering each environment's own env file from the vault. It
deliberately does not roll any service: image tags are the app domains'
axis, not infra's. A compose or stack change that must recreate containers
takes effect on the next app deploy, or a by-hand SERVICE=all … deploy/deploy.sh (or deploy-dev.sh) run.
Note the asymmetry this leaves: dev's infra checkout tracks dev, the same
branch its app images are staged by. Beta's and prod's infra checkouts both
track main — prod's app images are staged by release, but prod's infra
checkout follows main, same as beta's.