0f34594ce0
All checks were successful
PR build (required check) / changes (pull_request) Successful in 12s
secrets-guard / encrypted (pull_request) Successful in 11s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 1m40s
PR build (required check) / gate (pull_request) Successful in 4s
Move climate history to ERA5 served from our own object storage, with a prod-only query service in front: - data/era5lake.py: lake layout (per-point whole-record 1940+ serving files, a tile/year/month hive table, an Iceberg-style manifest) plus grid math and the read client (local dir -> lake service -> bucket). - gen_era5_lake.py: tile-aligned extractor from the Earthmover Icechunk ERA5 archive (one 86-year pull covers all 144 points of a chunk tile; resumable from the manifest; --cities / --tiles / --land). - lake_app.py + THERMOGRAPH_ROLE=lake: the lake service. /history serves a point's parquet off a disk cache (11ms cold / 3ms warm in rehearsal); /query runs SELECT-only SQL on DuckDB over the hive table (79ms pruned aggregate, 88ms full scan of a 4.5M-row tile). httpfs is baked at image build. - climate.py: history chain is now era5-lake -> NASA POWER -> Open-Meteo; the lake slice starts at START_DATE so grading windows are unchanged, and an unconfigured lake costs nothing (beta/LAN unchanged). - stack: lake service (1..2 replicas behind the VIP, own cache volume) plus a second autoscaler instance (autoscale.sh gains TARGET_SERVICE); web/worker get THERMOGRAPH_LAKE_URL. - seed_era5.py: constants corrected against the live store (icechunkV2, single/temporal, valid_time, ECMWF short names, pcodec). Bucket creds (THERMOGRAPH_LAKE_S3_ACCESS_KEY/_SECRET_KEY) go in the prod vault; until they land the lake stays healthy and everything falls through to NASA exactly as before. |
||
|---|---|---|
| .. | ||
| .claude/skills/key-gaps | ||
| deploy | ||
| terraform | ||
| .env.example | ||
| .gitignore | ||
| .sops.yaml | ||
| ACCESS.md | ||
| CLAUDE.md | ||
| DEPLOY-DEV.md | ||
| DEPLOY.md | ||
| docker-compose.dev.yml | ||
| docker-compose.openmeteo.yml | ||
| docker-compose.yml | ||
| docker-stack.yml | ||
| Makefile | ||
| README.md | ||
thermograph-infra
Infrastructure for Thermograph: Terraform host
provisioning, the SOPS+age secrets vault, Docker Swarm/WireGuard networking,
Forgejo, Caddy, and the deploy scripts that run the already-built app image on
each host. Extracted from the app monorepo (emi/thermograph) — the app repo
owns building and testing the app; this repo owns running it.
terraform/— provisions/configures hosts (SSH-driven by default; an optional GCP-creating module is scaffolded, no live resources yet) and triggers each deploy. Seeterraform/README.md.deploy/secrets/— the git-native SOPS+age secrets vault (every app secret, encrypted at rest, rendered at deploy time). Seedeploy/secrets/README.md.deploy/swarm/,deploy/forgejo/— the 3-node WireGuard/Swarm cluster that hosts Forgejo (git + CI + registry); does not run the app itself. SeeACCESS.mdand the READMEs under each directory.deploy/deploy.sh— pulls the pinned app image (IMAGE_TAG) and rolls the compose stack; invoked by Terraform and by the app repo's.forgejo/workflows/deploy.ymlover SSH.docker-compose*.yml,docker-stack.yml— how the app image runs (compose in production today;docker-stack.ymlis a design record for a possible future Swarm-based app deploy, not currently live).
The app's own source, Dockerfile, and build/test CI stay in the app repo —
this repo never checks out app source; hosts only pull tagged images from the
registry. See ACCESS.md for host access and the Swarm/Forgejo topology, and
terraform/README.md for the day-to-day plan/apply workflow.
Branches & how changes reach each environment
main— what prod and beta run: their/opt/thermographcheckoutsgit reset --hard origin/mainat the start of every deploy (deploy/deploy.sh). A merge tomainreaches those hosts on the next app deploy (or a by-handdeploy.shrun); there is no separate infra deploy trigger.dev— what LAN dev runs:~/thermograph-devresets to it viadeploy/deploy-dev.sh. Keep it fast-forwarded tomain(infra changes are not environment-staged today; the branches exist so LAN dev can trail or lead when needed).release— currently consumed by nothing (prod tracksmain, notrelease). It exists to mirror the app repos' dev→main→release promotion shape if per-environment infra staging is ever wanted; until then, treatmainas live-everywhere.
Note the asymmetry with the app repos: app code IS environment-staged (dev→main→release maps to LAN→beta→prod via image tags), infra is not.