Jinemi/thermograph
4
0
Fork 0
Code Issues 2 Pull requests 4 Projects Releases Packages Wiki Activity Actions 2

Tune Postgres to cache + process against ~8 GB (#222)

Browse source 
deploy/db/init/20-tuning.sql sets the memory budget via ALTER SYSTEM (applied on a
fresh volume, effective after the post-init restart): shared_buffers 2GB,
effective_cache_size 6GB, work_mem 64MB, maintenance_work_mem 512MB, max_wal_size
4GB, plus duckdb.max_memory 4GB for parquet processing.

Compose: the db container gets an 8g memory ceiling + 1g shm_size (parallel-query
shared memory) in prod; the dev overlay resets mem_limit so dev memory is uncapped
too (the ~8 GB budget still comes from the tuning). Verified the settings take
effect and pg_duckdb still loads.
This commit is contained in:
Emi Griffith 2026-07-19 23:56:23 -07:00 committed by GitHub
parent 6fd2d7c981
commit 2e1c746b7c
2 changed files with 40 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

31
deploy/db/init/20-tuning.sql Normal file
View file

@ -0,0 +1,31 @@
-- Memory / performance tuning for an ~8 GB budget. Applied once, on a fresh data
-- volume, via ALTER SYSTEM (persists to postgresql.auto.conf); the container's
-- post-init restart brings the restart-only settings (shared_buffers, …) into
-- effect. To re-apply on an existing volume, run these by hand and restart:
-- docker compose exec db psql -U thermograph -d thermograph -f /docker-entrypoint-initdb.d/20-tuning.sql
-- docker compose restart db
-- Caching: the shared page cache, and the planner's view of total cache (PG + OS).
ALTER SYSTEM SET shared_buffers = '2GB';
ALTER SYSTEM SET effective_cache_size = '6GB';
-- Processing: per-operation sort/hash memory, and maintenance (index builds, VACUUM).
ALTER SYSTEM SET work_mem = '64MB';
ALTER SYSTEM SET maintenance_work_mem = '512MB';
-- Write throughput: fewer, larger checkpoints.
ALTER SYSTEM SET wal_buffers = '16MB';
ALTER SYSTEM SET min_wal_size = '1GB';
ALTER SYSTEM SET max_wal_size = '4GB';
ALTER SYSTEM SET checkpoint_completion_target = 0.9;
-- SSD-friendly planner + IO concurrency.
ALTER SYSTEM SET random_page_cost = 1.1;
ALTER SYSTEM SET effective_io_concurrency = 200;
-- Some room for parallel scans/aggregates on the bigger analytic queries.
ALTER SYSTEM SET max_parallel_workers_per_gather = 2;
-- DuckDB (pg_duckdb) memory ceiling for parquet processing. pg_duckdb is
-- preloaded, so this GUC exists at ALTER SYSTEM time.
ALTER SYSTEM SET duckdb.max_memory = '4GB';

9
docker-compose.yml
View file

@ -36,6 +36,13 @@ services:
interval: 5s
timeout: 5s
retries: 10
# Give Postgres room to cache + process against ~8 GB. mem_limit is the ceiling;
# the actual budget is tuned in deploy/db/init/20-tuning.sql (shared_buffers 2GB,
# effective_cache_size 6GB, work_mem/maintenance_work_mem, plus duckdb.max_memory
# 4GB for parquet queries). shm_size backs parallel-query shared memory (the 64MB
# docker default is too small once shared_buffers/parallelism grow).
mem_limit: 8g
shm_size: 1gb
# Cap the DB at 2 CPUs. Compose v2 honors the top-level `cpus:`; the
# deploy.resources block is the Swarm-style equivalent, kept for parity.
cpus: 2.0
@ -43,6 +50,8 @@ services:
resources:
limits:
cpus: "2.0"
# Must match mem_limit above (compose rejects distinct values).
memory: 8G
restart: unless-stopped
# No host port on purpose: the app reaches Postgres as db:5432 on the
# compose network. Nothing outside the stack should touch the database.