thermograph/backend/CLAUDE.md

thermograph-backend — agent instructions

What this repo is

The Thermograph API service: FastAPI app serving the graded-climate API (/api/v2/...), user accounts (accounts/, fastapi-users + Postgres via alembic), push/email/Discord notifications (notifications/), and the SSR-content JSON API (api/content_routes.py + api/content_payloads.py) that the separate thermograph-frontend's server-rendered pages consume. It owns the climate data pipeline (data/ — grid snapping, Open-Meteo fetch + parquet cache, percentile grading/scoring, places/cities) and the derived-payload cache/ETag store (data/store.py, api/payloads.py). It does not serve any HTML/CSS/JS itself — that moved to thermograph-frontend (repo-split Stage 7a); this process is a pure JSON API + DB-backed service, launched as app:app (a shim re-exporting web.app:app, kept stable for systemd/CI/run.sh callers that still expect that target).

Split topology

Split out of the emi/thermograph monorepo (see MONOREPO-CUTOVER-PLAN.md at the workspace root during the transition). Sibling repos:

• thermograph-frontend — the client the public sees: static JS/CSS/HTML + frontend_ssr's server-rendered climate hub/city/month pages, calling this repo's /api/v2/... and /content/... JSON endpoints.
• thermograph-infra — deploy/terraform/Swarm-Compose plumbing: docker-compose.yml, deploy/deploy.sh, secrets (SOPS-encrypted deploy/secrets/*.yaml), Terraform for prod/beta provisioning. This repo has no deploy scripts or terraform of its own — it only builds an image and hands infra a tag.
• thermograph-docs — non-code planning layer: architecture decision records and operator runbooks. Nothing here should duplicate that; if you're writing a cross-cutting decision doc or a step-by-step operator runbook, it belongs there, not in this repo's README/CLAUDE.md.
• thermograph-observability — monitoring/metrics stack, separate from this repo's own lightweight in-process core/metrics.py.

Branch, PR & deploy flow

• build-push.yml builds this repo's own image (git.thermograph.org/emi/thermograph-backend/app, tagged sha-<12hex> on every push to dev/main/release and additionally by semver on v*.*.* tags) — no more shared monorepo image. build.yml (push/PR to main) is a build-only sanity check (docker build), not a boot/health check — repeated attempts at a live boot+healthz check hit this runner's own quirks (bridge-IP reachability, non-unique $$, squatted host ports) without ever going reliably green; the image's actual boot is verified by hand (docker run + alembic + /healthz 200 + a real /api/v2/place 200) instead.
• main → beta: deploy.yml SSHes to beta and runs thermograph-infra's /opt/thermograph/deploy/deploy.sh with SERVICE=backend + BACKEND_IMAGE_TAG=sha-<12hex> (must match build-push.yml's 12-hex SHA tag exactly — Actions expressions have no substring function for the full 40-char SHA). deploy.sh rolls only the backend compose service (--no-deps) and persists the tag in deploy/.image-tags.env host-side, so a backend-only deploy never touches the frontend's running container or tag.
• release → prod: deploy-prod.yml is the same shape against a completely separate secret set (PROD_SSH_HOST/USER/KEY/PORT) so a beta credential leak can't reach prod. Also SERVICE=backend + BACKEND_IMAGE_TAG.
• The frontend deploys itself the same way, independently — that independence (a backend change ships without a frontend deploy and vice versa) is the entire point of the FE/BE CI-CD split.
• No dev-branch LAN deploy path exists in this repo (the monorepo's deploy-dev.yml/docker-compose.dev.yml never made it across the split — known gap, see the cutover plan if resurrecting it).

Run / test locally

There is no Makefile in this repo — the monorepo's app-level make targets (lan-run, test, venv, ...) haven't been given a new home yet (tracked as a gap in the cutover plan). Until that lands, drive it directly:

python3 -m venv .venv && .venv/bin/pip install -r requirements.txt -r requirements-dev.txt
.venv/bin/uvicorn app:app --host 0.0.0.0 --port 8137          # serve
.venv/bin/python -m pytest tests                              # test

The monorepo's Makefile preferred uv venv --python 3.12 .venv (falling back to plain python3 -m venv only if uv isn't installed) — prefer uv here too if it's on the box. There is no committed .venv in this repo; if one exists locally and looks broken (import errors, wrong Python), just rm -rf .venv and recreate rather than debugging it — it's gitignored, disposable state, not something the repo depends on being a specific way.

Tests are hermetic (tests/conftest.py): no real Open-Meteo/Nominatim/GeoNames calls, a throwaway SQLite for accounts + the derived-payload store, and the notifier thread disabled. THERMOGRAPH_FRONTEND_BASE_INTERNAL is set to an unreachable placeholder in conftest — web/app.py fails loud at import if it's unset in a real run (it's the internal URL back to frontend_ssr for the non-API HTML routes this process no longer serves itself).

API version contract

• Everything real lives under /api/v2/...; /api/ and /api/v1/ stay mounted as aliases of the same handlers (they always were — not new).
• GET /api/version (web/app.py) is the capability/negotiation endpoint a split-out frontend can call at boot: {backend_version, min_frontend, payload_ver}, driven by two module constants next to it — API_CONTRACT_VERSION (bump only on a breaking change to any /api/v{N} route or payload shape) and MIN_SUPPORTED_FRONTEND (oldest frontend contract version this backend still serves correctly). A genuine breaking change ships as a new v3 router mounted alongside v2 re-registering only the changed handlers, v2 kept working, API_CONTRACT_VERSION bumped in the same PR.
• PAYLOAD_VER (api/payloads.py, currently "p2") is a separate concern from URL versioning — it's the cache/ETag invalidation token baked into every derived-store validity token (history_token()). Bump it whenever a response payload's shape changes (new metric, renamed/removed field) so one bump atomically orphans every pre-upgrade cached row instead of a stale row's history_end happening to still validate under the old shape.
• Both endpoints (/healthz, /api/version) are deliberately I/O-free so they stay cheap under tight healthcheck intervals.

Live cross-repo contracts (don't break silently)

• /cell bundle + ETag/If-None-Match: every graded payload (grade, calendar, day) is cached in SQLite by (kind, cell_id, key) keyed to a validity token that only advances when the underlying history actually changes (_etag_for in web/app.py, history_token/PAYLOAD_VER in api/payloads.py). The same token doubles as a weak ETag; a client sending If-None-Match gets an empty 304 without the payload being rebuilt or even loaded. expose_headers=["ETag"] in the CORS middleware matters as much as allow_origins — without it the frontend's cache.js reading res.headers.get("ETag") cross-origin silently gets null and never revalidates correctly. Don't change the ETag derivation or the CORS expose_headers list without checking the frontend's cache layer.
• Percentile/unit parity with thermograph-frontend's static/shared.js: the frontend's percentile-to-ordinal display logic explicitly mirrors this repo's data/grading.py::pct_ordinal() (floors an empirical percentile into 1..99 — a rank against the sample can round to 0 or 100, and "100th percentile" is never shown). TEMP_BANDS/RAIN_BANDS tier boundaries in data/grading.py are the source of truth for the tier names/thresholds (Near Record / High / Above Normal / Normal / Below Normal / Low, and the five rain-intensity tiers) — changing them without a corresponding frontend update produces a client that draws differently-colored tiers than the labels the API returns.
• SSR content API (api/content_routes.py): a separate ETag'd JSON surface (/content/hub, /content/city/{slug}, /content/city/{slug}/month/{month}, /content/city/{slug}/records, /content/home, /content/sitemap, /content/indexnow-key) that frontend_ssr calls in-process-free (no shared Python import) to render pages. Same caching pattern as the main API; keep payload shape changes coordinated with whatever consumes it on the frontend side.

Layout

accounts/ (fastapi-users models/schemas/db, alembic/ migrations), api/ (versioned payload builders + SSR content routes), core/ (metrics, audit logging, a singleton helper), data/ (grid, climate fetch/cache, grading/ scoring, places/cities, the derived-payload store), notifications/ (push, email, Discord bot + interactions + linking, digest, scheduler), web/app.py (the actual FastAPI app; app.py at the repo root is a one-line re-export shim). paths.py resolves every filesystem location (climate parquet cache, accounts DB, logs, bundled cities.json/cities_flavor.json) from the repo root — don't reintroduce __file__-relative paths in a module, it breaks the moment a module moves. deploy/entrypoint.sh is the container entrypoint (alembic migrate-then-serve, with a Swarm-secrets-as-files shim); actual deploy orchestration lives in thermograph-infra, not here.