Tune Postgres to cache + process against ~8 GB (#222)

deploy/db/init/20-tuning.sql sets the memory budget via ALTER SYSTEM (applied on a
fresh volume, effective after the post-init restart): shared_buffers 2GB,
effective_cache_size 6GB, work_mem 64MB, maintenance_work_mem 512MB, max_wal_size
4GB, plus duckdb.max_memory 4GB for parquet processing.

Compose: the db container gets an 8g memory ceiling + 1g shm_size (parallel-query
shared memory) in prod; the dev overlay resets mem_limit so dev memory is uncapped
too (the ~8 GB budget still comes from the tuning). Verified the settings take
effect and pg_duckdb still loads.

deploy/db/init/20-tuning.sql

@@ -0,0 +1,31 @@
+-- Memory / performance tuning for an ~8 GB budget. Applied once, on a fresh data
+-- volume, via ALTER SYSTEM (persists to postgresql.auto.conf); the container's
+-- post-init restart brings the restart-only settings (shared_buffers, …) into
+-- effect. To re-apply on an existing volume, run these by hand and restart:
+--   docker compose exec db psql -U thermograph -d thermograph -f /docker-entrypoint-initdb.d/20-tuning.sql
+--   docker compose restart db
+
+-- Caching: the shared page cache, and the planner's view of total cache (PG + OS).
+ALTER SYSTEM SET shared_buffers = '2GB';
+ALTER SYSTEM SET effective_cache_size = '6GB';
+
+-- Processing: per-operation sort/hash memory, and maintenance (index builds, VACUUM).
+ALTER SYSTEM SET work_mem = '64MB';
+ALTER SYSTEM SET maintenance_work_mem = '512MB';
+
+-- Write throughput: fewer, larger checkpoints.
+ALTER SYSTEM SET wal_buffers = '16MB';
+ALTER SYSTEM SET min_wal_size = '1GB';
+ALTER SYSTEM SET max_wal_size = '4GB';
+ALTER SYSTEM SET checkpoint_completion_target = 0.9;
+
+-- SSD-friendly planner + IO concurrency.
+ALTER SYSTEM SET random_page_cost = 1.1;
+ALTER SYSTEM SET effective_io_concurrency = 200;
+
+-- Some room for parallel scans/aggregates on the bigger analytic queries.
+ALTER SYSTEM SET max_parallel_workers_per_gather = 2;
+
+-- DuckDB (pg_duckdb) memory ceiling for parquet processing. pg_duckdb is
+-- preloaded, so this GUC exists at ALTER SYSTEM time.
+ALTER SYSTEM SET duckdb.max_memory = '4GB';

docker-compose.yml

@@ -36,6 +36,13 @@ services:
       interval: 5s
       timeout: 5s
       retries: 10
+    # Give Postgres room to cache + process against ~8 GB. mem_limit is the ceiling;
+    # the actual budget is tuned in deploy/db/init/20-tuning.sql (shared_buffers 2GB,
+    # effective_cache_size 6GB, work_mem/maintenance_work_mem, plus duckdb.max_memory
+    # 4GB for parquet queries). shm_size backs parallel-query shared memory (the 64MB
+    # docker default is too small once shared_buffers/parallelism grow).
+    mem_limit: 8g
+    shm_size: 1gb
     # Cap the DB at 2 CPUs. Compose v2 honors the top-level `cpus:`; the
     # deploy.resources block is the Swarm-style equivalent, kept for parity.
     cpus: 2.0
@@ -43,6 +50,8 @@ services:
       resources:
         limits:
           cpus: "2.0"
+          # Must match mem_limit above (compose rejects distinct values).
+          memory: 8G
     restart: unless-stopped
     # No host port on purpose: the app reaches Postgres as db:5432 on the
     # compose network. Nothing outside the stack should touch the database.