Registers the hive part files under era5/daily into an Apache Iceberg v2
table at iceberg/era5_daily via pyiceberg add_files -- the metadata points
at the existing parquet in place, no data rewrite. Incremental: each run
diffs era5/manifest.parquet (the completeness signal while extraction is
running) against the thermograph.synced-tiles table property, updated in
the same commit as the files, so runs are idempotent and resume at batch
boundaries. Partitioned by truncate[12](lat_idx), truncate[12](lon_idx),
year(date), month(date) -- order-preserving transforms add_files derives
from footer stats; queries prune on plain column predicates. Local sqlite
catalog that re-registers from the latest metadata JSON if lost;
version-hint.text refreshed per run so DuckDB can iceberg_scan the bare
table root. Per-batch retries with backoff for Contabo SLOW_DOWN and
slow-transfer timeouts. Slim pinned one-shot image; local-filesystem
pytest suite (no network).
The cutover moved the host checkout's deploy tree to /opt/thermograph/infra/deploy/,
but the stack file's absolute bind sources (db init, env-entrypoint.sh, autoscale.sh)
still pointed at the old /opt/thermograph/deploy/ -- a service update kept the stale
mounts and new tasks failed with 'bind source path does not exist'.
/opt/thermograph becomes a monorepo checkout: deploy.sh gains INFRA_DIR (git
ops at the root, compose work cd'd into infra/), lock/tags/render paths move
under infra/deploy/, image-path defaults become emi/thermograph/backend|
frontend across compose, stack, and the tag-prune. docker-compose.yml pins
name: thermograph -- without it compose run from .../infra derives project
"infra" and recreates the whole stack beside the running one;
deploy-dev.sh pins COMPOSE_PROJECT_NAME=thermograph-dev (env wins over the
file key) to keep LAN dev's separate project.
One root workflow set replaces the four repos' copies (deleted -- root-only
is where Forgejo reads them, and dead copies are a trap): per-domain
build-push with explicit image paths (emi/thermograph/backend|frontend; the
old github.repository-derived path collides in a monorepo), path-filtered
per-domain beta/prod/dev deploys, a domain-input reusable build check, a
single always-reporting PR gate (path-filtered required checks deadlock
auto-merge), a new infra-sync pipeline (host checkout + secrets render on
infra/** pushes), and ports of secrets-guard / ops-cron /
observability-validate to monorepo paths.
