thermograph/alembic/env.py

59 lines
2 KiB
Python
Raw Normal View History

Containerize the app and move the databases to PostgreSQL 18 (#220) Run Thermograph as a docker-compose stack (app + Postgres 18) and standardize the data layer on Postgres, while keeping the test suite on SQLite. - accounts/db.py: DSN-driven engines. On Postgres, a per-worker read-write + read-only asyncpg pair (the RO engine pins read-only transactions, used by the pure-GET endpoints) plus a sync psycopg engine for the notifier thread; the SQLite path is preserved for tests/local (selected when THERMOGRAPH_DATABASE_URL is unset). models.py: boolean server_default -> sa.false(). - store.py / metrics.py: dialect-flexible — Postgres UNLOGGED tables via psycopg when configured, else the existing raw-sqlite3 paths byte-for-byte; sync interfaces and every fail-soft contract preserved. - Alembic (backend/alembic/) manages the accounts schema; the container entrypoint runs `alembic upgrade head` before uvicorn (4 workers). migrate_accounts_to_pg.py copies the accounts data SQLite->PG through the ORM (UUID/bool/JSON coerced), skips access_token, and resets identity sequences. - Dockerfile + docker-compose.yml: app image (uvicorn, 4 workers, loopback 8137) and a Postgres 18 db (2 CPUs) running pg_duckdb (deploy/db/) so the parquet climate cache is queryable in-DB via read_parquet('/parquet/cache/*.parquet'). - deploy.sh/thermograph.service rewired to manage the compose stack; env example, Makefile targets (up/down/db-up), and deploy/POSTGRES-MIGRATION.md cutover runbook. Tests stay on SQLite (dialect fallback) — 323 pass. The full Postgres stack was verified via docker compose: alembic migrations, register/login, the RO endpoint, store/metrics round-trips, and the accounts data migration.
2026-07-20 06:28:23 +00:00
"""Alembic environment for the accounts database.
The migration URL is derived from ``THERMOGRAPH_DATABASE_URL`` (the async app URL)
by swapping to a sync driver Alembic runs synchronously. Falls back to the
SQLite accounts file when no Postgres URL is set (local / tests). ``target_metadata``
is the accounts ``Base.metadata`` so ``alembic revision --autogenerate`` can diff
against the ORM models.
"""
import os
import sys
from logging.config import fileConfig
from alembic import context
from sqlalchemy import create_engine, pool
# Alembic runs with cwd = backend/ (WORKDIR); make the backend package importable
# regardless of where alembic is invoked from.
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from accounts.db import Base # noqa: E402
from accounts import models # noqa: E402,F401 (registers tables on Base.metadata)
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
target_metadata = Base.metadata
def _sync_url() -> str:
url = os.environ.get("THERMOGRAPH_DATABASE_URL", "").strip()
if url.startswith("postgresql"):
return url.replace("+asyncpg", "+psycopg")
from accounts.db import DB_PATH # SQLite fallback path
return f"sqlite:///{DB_PATH}"
def run_migrations_offline() -> None:
context.configure(url=_sync_url(), target_metadata=target_metadata,
literal_binds=True, dialect_opts={"paramstyle": "named"})
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
engine = create_engine(_sync_url(), poolclass=pool.NullPool, future=True)
with engine.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata,
render_as_batch=connection.dialect.name == "sqlite")
with context.begin_transaction():
context.run_migrations()
engine.dispose()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()