thermograph/backend/CLAUDE.md

# thermograph-backend — agent instructions

## What this repo is

The Thermograph **API service**: FastAPI app serving the graded-climate API
(`/api/v2/...`), user accounts (`accounts/`, fastapi-users + Postgres via
alembic), push/email/Discord notifications (`notifications/`), and the
SSR-content JSON API (`api/content_routes.py` + `api/content_payloads.py`) that
the separate `thermograph-frontend`'s server-rendered pages consume. It owns
the climate data pipeline (`data/` — grid snapping, Open-Meteo fetch + parquet
cache, percentile grading/scoring, places/cities) and the derived-payload
cache/ETag store (`data/store.py`, `api/payloads.py`). It does **not** serve
any HTML/CSS/JS itself — that moved to `thermograph-frontend` (repo-split
Stage 7a); this process is a pure JSON API + DB-backed service, launched as
`app:app` (a shim re-exporting `web.app:app`, kept stable for systemd/CI/`run.sh`
callers that still expect that target).

## Split topology

Split out of the `emi/thermograph` monorepo (see
`MONOREPO-CUTOVER-PLAN.md` at the workspace root during the transition). Sibling
repos:
- **`thermograph-frontend`** — the client the public sees: static
  JS/CSS/HTML + `frontend_ssr`'s server-rendered climate hub/city/month pages,
  calling this repo's `/api/v2/...` and `/content/...` JSON endpoints.
- **`thermograph-infra`** — deploy/terraform/Swarm-Compose plumbing:
  `docker-compose.yml`, `deploy/deploy.sh`, secrets (SOPS-encrypted
  `deploy/secrets/*.yaml`), Terraform for prod/beta provisioning. This repo has
  no deploy scripts or terraform of its own — it only builds an image and hands
  infra a tag.
- **`thermograph-docs`** — non-code planning layer: architecture decision
  records and operator runbooks. Nothing here should duplicate that; if you're
  writing a cross-cutting decision doc or a step-by-step operator runbook,
  it belongs there, not in this repo's README/CLAUDE.md.
- **`thermograph-observability`** — monitoring/metrics stack, separate from
  this repo's own lightweight in-process `core/metrics.py`.

## Branch, PR & deploy flow

- `build-push.yml` builds **this repo's own image**
  (`git.thermograph.org/emi/thermograph-backend/app`, tagged `sha-<12hex>` on
  every push to `dev`/`main`/`release` and additionally by semver on `v*.*.*`
  tags) — no more shared monorepo image. `build.yml` (push/PR to `main`) is a
  build-only sanity check (`docker build`), not a boot/health check — repeated
  attempts at a live boot+healthz check hit this runner's own quirks (bridge-IP
  reachability, non-unique `$$`, squatted host ports) without ever going
  reliably green; the image's actual boot is verified by hand (`docker run` +
  alembic + `/healthz` 200 + a real `/api/v2/place` 200) instead.
- **`main` → beta**: `deploy.yml` SSHes to beta and runs
  `thermograph-infra`'s `/opt/thermograph/deploy/deploy.sh` with
  `SERVICE=backend` + `BACKEND_IMAGE_TAG=sha-<12hex>` (must match
  build-push.yml's 12-hex SHA tag exactly — Actions expressions have no
  substring function for the full 40-char SHA). `deploy.sh` rolls only the
  `backend` compose service (`--no-deps`) and persists the tag in
  `deploy/.image-tags.env` host-side, so a backend-only deploy never touches
  the frontend's running container or tag.
- **`release` → prod**: `deploy-prod.yml` is the same shape against a
  completely separate secret set (`PROD_SSH_HOST/USER/KEY/PORT`) so a beta
  credential leak can't reach prod. Also `SERVICE=backend` +
  `BACKEND_IMAGE_TAG`.
- The frontend deploys itself the same way, independently — that
  independence (a backend change ships without a frontend deploy and vice
  versa) is the entire point of the FE/BE CI-CD split.
- No `dev`-branch LAN deploy path exists in this repo (the monorepo's
  `deploy-dev.yml`/`docker-compose.dev.yml` never made it across the split —
  known gap, see the cutover plan if resurrecting it).

## Run / test locally

There is **no Makefile in this repo** — the monorepo's app-level `make`
targets (`lan-run`, `test`, `venv`, ...) haven't been given a new home yet
(tracked as a gap in the cutover plan). Until that lands, drive it directly:

```bash
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt -r requirements-dev.txt
.venv/bin/uvicorn app:app --host 0.0.0.0 --port 8137          # serve
.venv/bin/python -m pytest tests                              # test
```

The monorepo's Makefile preferred `uv venv --python 3.12 .venv` (falling back
to plain `python3 -m venv` only if `uv` isn't installed) — prefer `uv` here too
if it's on the box. There is no committed `.venv` in this repo; if one exists
locally and looks broken (import errors, wrong Python), just `rm -rf .venv` and
recreate rather than debugging it — it's gitignored, disposable state, not
something the repo depends on being a specific way.

Tests are hermetic (`tests/conftest.py`): no real Open-Meteo/Nominatim/GeoNames
calls, a throwaway SQLite for accounts + the derived-payload store, and the
notifier thread disabled. `THERMOGRAPH_FRONTEND_BASE_INTERNAL` is set to an
unreachable placeholder in conftest — `web/app.py` fails loud at import if it's
unset in a real run (it's the internal URL back to `frontend_ssr` for the
non-API HTML routes this process no longer serves itself).

## API version contract

- Everything real lives under **`/api/v2/...`**; `/api/` and `/api/v1/` stay
  mounted as aliases of the same handlers (they always were — not new).
- **`GET /api/version`** (`web/app.py`) is the capability/negotiation endpoint
  a split-out frontend can call at boot: `{backend_version, min_frontend,
  payload_ver}`, driven by two module constants next to it —
  `API_CONTRACT_VERSION` (bump only on a breaking change to any `/api/v{N}`
  route or payload shape) and `MIN_SUPPORTED_FRONTEND` (oldest frontend
  contract version this backend still serves correctly). A genuine breaking
  change ships as a new `v3` router mounted alongside `v2` re-registering only
  the changed handlers, `v2` kept working, `API_CONTRACT_VERSION` bumped in the
  same PR.
- **`PAYLOAD_VER`** (`api/payloads.py`, currently `"p2"`) is a *separate*
  concern from URL versioning — it's the cache/ETag invalidation token baked
  into every derived-store validity token (`history_token()`). Bump it
  whenever a response payload's shape changes (new metric, renamed/removed
  field) so one bump atomically orphans every pre-upgrade cached row instead
  of a stale row's `history_end` happening to still validate under the old
  shape.
- Both endpoints (`/healthz`, `/api/version`) are deliberately I/O-free so they
  stay cheap under tight healthcheck intervals.

## Live cross-repo contracts (don't break silently)

- **`/cell` bundle + ETag/If-None-Match**: every graded payload (`grade`,
  `calendar`, `day`) is cached in SQLite by `(kind, cell_id, key)` keyed to a
  validity token that only advances when the underlying history actually
  changes (`_etag_for` in `web/app.py`, `history_token`/`PAYLOAD_VER` in
  `api/payloads.py`). The same token doubles as a weak ETag; a client sending
  `If-None-Match` gets an empty 304 without the payload being rebuilt or even
  loaded. `expose_headers=["ETag"]` in the CORS middleware matters as much as
  `allow_origins` — without it the frontend's `cache.js` reading
  `res.headers.get("ETag")` cross-origin silently gets `null` and never
  revalidates correctly. Don't change the ETag derivation or the CORS
  `expose_headers` list without checking the frontend's cache layer.
- **Percentile/unit parity with `thermograph-frontend`'s `static/shared.js`**:
  the frontend's percentile-to-ordinal display logic explicitly mirrors this
  repo's `data/grading.py::pct_ordinal()` (floors an empirical percentile into
  1..99 — a rank against the sample can round to 0 or 100, and "100th
  percentile" is never shown). `TEMP_BANDS`/`RAIN_BANDS` tier boundaries in
  `data/grading.py` are the source of truth for the tier names/thresholds
  (Near Record / High / Above Normal / Normal / Below Normal / Low, and the
  five rain-intensity tiers) — changing them without a corresponding frontend
  update produces a client that draws differently-colored tiers than the
  labels the API returns.
- **SSR content API** (`api/content_routes.py`): a separate ETag'd JSON surface
  (`/content/hub`, `/content/city/{slug}`, `/content/city/{slug}/month/{month}`,
  `/content/city/{slug}/records`, `/content/home`, `/content/sitemap`,
  `/content/indexnow-key`) that `frontend_ssr` calls in-process-free (no shared
  Python import) to render pages. Same caching pattern as the main API; keep
  payload shape changes coordinated with whatever consumes it on the frontend
  side.

## Layout

`accounts/` (fastapi-users models/schemas/db, `alembic/` migrations),
`api/` (versioned payload builders + SSR content routes), `core/` (metrics,
audit logging, a singleton helper), `data/` (grid, climate fetch/cache, grading/
scoring, places/cities, the derived-payload store), `notifications/` (push,
email, Discord bot + interactions + linking, digest, scheduler), `web/app.py`
(the actual FastAPI app; `app.py` at the repo root is a one-line re-export
shim). `paths.py` resolves every filesystem location (climate parquet cache,
accounts DB, logs, bundled `cities.json`/`cities_flavor.json`) from the repo
root — don't reintroduce `__file__`-relative paths in a module, it breaks the
moment a module moves. `deploy/entrypoint.sh` is the container entrypoint
(alembic migrate-then-serve, with a Swarm-secrets-as-files shim); actual deploy
orchestration lives in `thermograph-infra`, not here.