Move climate history to ERA5 served from our own object storage, with a
prod-only query service in front:
- data/era5lake.py: lake layout (per-point whole-record 1940+ serving files,
a tile/year/month hive table, an Iceberg-style manifest) plus grid math and
the read client (local dir -> lake service -> bucket).
- gen_era5_lake.py: tile-aligned extractor from the Earthmover Icechunk ERA5
archive (one 86-year pull covers all 144 points of a chunk tile; resumable
from the manifest; --cities / --tiles / --land).
- lake_app.py + THERMOGRAPH_ROLE=lake: the lake service. /history serves a
point's parquet off a disk cache (11ms cold / 3ms warm in rehearsal);
/query runs SELECT-only SQL on DuckDB over the hive table (79ms pruned
aggregate, 88ms full scan of a 4.5M-row tile). httpfs is baked at image
build.
- climate.py: history chain is now era5-lake -> NASA POWER -> Open-Meteo; the
lake slice starts at START_DATE so grading windows are unchanged, and an
unconfigured lake costs nothing (beta/LAN unchanged).
- stack: lake service (1..2 replicas behind the VIP, own cache volume) plus a
second autoscaler instance (autoscale.sh gains TARGET_SERVICE); web/worker
get THERMOGRAPH_LAKE_URL.
- seed_era5.py: constants corrected against the live store (icechunkV2,
single/temporal, valid_time, ECMWF short names, pcodec).
Bucket creds (THERMOGRAPH_LAKE_S3_ACCESS_KEY/_SECRET_KEY) go in the prod
vault; until they land the lake stays healthy and everything falls through to
NASA exactly as before.
Steady-state usage of the Open-Meteo API is removed; it remains only as a dormant fallback. Geocoding -> local GeoNames + Nominatim; wind gusts -> Meteostat; history -> NASA POWER (curated cells seeded with keyless ERA5); recent+forecast -> NASA range + MET Norway. Plus a drift-check tool comparing NASA vs Open-Meteo. All keyless, no new infra. Backend suite green in-image (gate).
The cutover moved the host checkout's deploy tree to /opt/thermograph/infra/deploy/,
but the stack file's absolute bind sources (db init, env-entrypoint.sh, autoscale.sh)
still pointed at the old /opt/thermograph/deploy/ -- a service update kept the stale
mounts and new tasks failed with 'bind source path does not exist'.
Backend runs its full hermetic suite, frontend its unit tier, inside the
just-built image -- the dev-only feature both app repos carried in their own
build.yml (deleted here in favor of the root workflow).
/opt/thermograph becomes a monorepo checkout: deploy.sh gains INFRA_DIR (git
ops at the root, compose work cd'd into infra/), lock/tags/render paths move
under infra/deploy/, image-path defaults become emi/thermograph/backend|
frontend across compose, stack, and the tag-prune. docker-compose.yml pins
name: thermograph -- without it compose run from .../infra derives project
"infra" and recreates the whole stack beside the running one;
deploy-dev.sh pins COMPOSE_PROJECT_NAME=thermograph-dev (env wins over the
file key) to keep LAN dev's separate project.
One root workflow set replaces the four repos' copies (deleted -- root-only
is where Forgejo reads them, and dead copies are a trap): per-domain
build-push with explicit image paths (emi/thermograph/backend|frontend; the
old github.repository-derived path collides in a monorepo), path-filtered
per-domain beta/prod/dev deploys, a domain-input reusable build check, a
single always-reporting PR gate (path-filtered required checks deadlock
auto-merge), a new infra-sync pipeline (host checkout + secrets render on
infra/** pushes), and ports of secrets-guard / ops-cron /
observability-validate to monorepo paths.
main received the perf hardening line (#5, #8) and its own landing of the
Discord gateway bot (#7) while dev carried a parallel port of the same bot
(#3). The two implementations are near-identical ports of the same archived
source; main's includes one extra hardening (grading calls moved off the
gateway event loop via asyncio.to_thread) and broader tests, so bot files
resolve wholesale to main's side. dev keeps its test-runner/smoke tooling and
the run-tests-in-image CI, which main lacks.
# Conflicts:
# notifications/discord_bot.py
# tests/notifications/test_discord_bot.py
# web/app.py
