thermograph/infra/deploy/db/init/10-timescaledb.sql

15 lines
859 B
MySQL
Raw Normal View History

Move the climate record from parquet to TimescaleDB hypertables (#227) Replace the per-cell parquet cache with TimescaleDB hypertables as the production backend for the raw daily climate record, and drop pg_duckdb. Parquet stays the backend whenever THERMOGRAPH_DATABASE_URL is not a Postgres URL (dev, tests, offline tooling), the same dialect switch the accounts DB and derived store already use, so CI stays Postgres-free. - data/climate_store.py: psycopg + polars bridge over climate_history (a hypertable), climate_recent, and climate_sync (per-cell freshness). Reads via pl.read_database, writes via COPY + ON CONFLICT upsert; fail-soft to a cache miss so a DB hiccup degrades to upstream refetch. - data/climate.py: route every cache/mtime touchpoint through a backend dispatch. recent_stamp becomes int(recent_synced_at) on Postgres; the stale-serve path still avoids bumping it, so derived-payload tokens invalidate on exactly the same events as before. - alembic 0002: CREATE EXTENSION timescaledb plus the hypertable schema (compression policy on year-old chunks), guarded to no-op off Postgres. - migrate_cache_to_pg.py (make migrate-cache): idempotent backfill of the parquet cache into the hypertables, preserving file mtimes as sync timestamps so recent_stamp is unchanged across cutover. - db image -> stock timescale/timescaledb:latest-pg18; drop the custom pg_duckdb Dockerfile, the read-only /parquet mount, and the duckdb tuning GUC. Docs updated for the new backend and cutover. Co-authored-by: Claude <noreply@anthropic.com>
2026-07-20 20:15:55 +00:00
-- Enable TimescaleDB so the app's climate record lives in hypertables
-- (climate_history) instead of parquet files. The stock timescale/timescaledb
-- image already preloads the extension (shared_preload_libraries=timescaledb);
-- this just runs CREATE EXTENSION on first cluster init.
--
-- This runs once, on first cluster init (empty PGDATA), from
-- /docker-entrypoint-initdb.d. Because the compose `db` service keeps its data on
-- a persistent named volume, init scripts do NOT re-run on an existing database.
-- Alembic (backend/alembic) also runs `CREATE EXTENSION IF NOT EXISTS timescaledb`
-- at app boot, so an existing volume gets it there; to enable it by hand instead:
--
-- docker compose exec db psql -U thermograph -d thermograph \
-- -c 'CREATE EXTENSION IF NOT EXISTS timescaledb;'
CREATE EXTENSION IF NOT EXISTS timescaledb;