thermograph/backend
Emi Griffith af21d8e477
All checks were successful
secrets-guard / encrypted (pull_request) Successful in 5s
shell-lint / shellcheck (pull_request) Successful in 10s
PR build (required check) / changes (pull_request) Successful in 16s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Successful in 20s
PR build (required check) / build-backend (pull_request) Successful in 45s
PR build (required check) / gate (pull_request) Successful in 5s
docs: correct file references and the dev reachability claim
Audited the five CLAUDE.md files and all twenty-one README.md files against the
tree, machine-checking every in-repo path they name and verifying the testable
claims against the live hosts.

The one that matters is in the root file: dev was documented as reachable on
the mesh at 10.10.0.2:8137. It is not, and never was from anywhere but vps1 —
infra/docker-compose.yml binds the port to 127.0.0.1, and the address answers
from neither vps2 nor vps1 itself. Anyone following it gets a connection
refused with nothing to explain it.

The rest are stale paths, several from the reunification:

  * assetlinks.json moved under frontend/static/ in the subtree merge; the TWA
    README kept the pre-merge path in both places it names it. Following it
    would put the file where nothing serves it and Android app-link
    verification would fail silently.
  * push.py and notify.py now live in backend/notifications/.
  * INFRA.md and deploy/stack/README have never existed in this repo, in any
    branch.
  * the Caddyfile is at deploy/stack/lb/Caddyfile.
  * three bare relative paths that resolve for a reader but not from the
    directory the file sits in: units.js is the frontend's, deploy.sh is
    infra's, entrypoint.sh is the backend's.

Also records why mesh clients must pin the ROOT_URL host and not only the image
host: the registry's bearer-token realm follows ROOT_URL, so pinning
git.thermograph.org alone still sends the token request out the public route,
where the /v2/* matcher returns 403 and docker falls back to anonymous. That
surfaces as `unauthorized: reqPackageAccess`, indistinguishable from a bad
credential.

Verified true and left alone: the four-domain layout, both .claude runbooks,
the absence of any domain-level .forgejo directory, the pinned compose project
name, the deploy contract, prod's eight stack services, beta's five prefixed
ones with no db of its own, dev's five, and every documented make target.
2026-08-01 11:49:28 -07:00
..
accounts accounts: sign in with Google, on a shared provider-agnostic OAuth engine (#122) 2026-07-27 00:56:43 +00:00
alembic accounts: sign in with Google, on a shared provider-agnostic OAuth engine (#122) 2026-07-27 00:56:43 +00:00
api Frontend QA batch: date/TZ, https origin, date-422, VAPID rotation, trace-precip, partial-day gate (#70) 2026-07-24 23:13:36 +00:00
core Stop logging /healthz + internal SSR hop, truncate logged IPs (#35) 2026-07-24 19:28:47 +00:00
daemon daemon: move the Discord gateway and scheduler out of the web process into Go (#21) 2026-07-23 22:49:54 +00:00
data climate: stop dropping the current day from the Open-Meteo bundle (#92) 2026-07-25 16:56:21 +00:00
deploy Stop logging /healthz + internal SSR hop, truncate logged IPs (#35) 2026-07-24 19:28:47 +00:00
notifications accounts: sign in with Google, on a shared provider-agnostic OAuth engine (#122) 2026-07-27 00:56:43 +00:00
scripts registry: move image and repo references to the Jinemi namespace 2026-08-01 09:25:02 -07:00
tests accounts: sign in with Google, on a shared provider-agnostic OAuth engine (#122) 2026-07-27 00:56:43 +00:00
web accounts: sign in with Google, on a shared provider-agnostic OAuth engine (#122) 2026-07-27 00:56:43 +00:00
.dockerignore daemon: move the Discord gateway and scheduler out of the web process into Go (#21) 2026-07-23 22:49:54 +00:00
.gitignore Subtree-sync backend to split main a4d7fcd (dev->main promotion: reconciled bot + hardening, in-image CI); subtree workflow copies stay deleted (CI lives at root) 2026-07-22 22:37:21 -07:00
alembic.ini Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
app.py Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
cities.json Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
cities_flavor.json Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
CLAUDE.md docs: correct file references and the dev reachability claim 2026-08-01 11:49:28 -07:00
docker-compose.test.yml registry: move image and repo references to the Jinemi namespace 2026-08-01 09:25:02 -07:00
Dockerfile Lake /query reads era5_daily through Iceberg metadata (#44) 2026-07-24 19:09:22 +00:00
drift_check.py Migrate off the Open-Meteo weather API (Phases 0-4) 2026-07-23 14:34:51 +00:00
gen_cities.py Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
gen_era5_lake.py Survive Contabo PUT throttling in the lake extractor (#22) 2026-07-23 22:47:26 +00:00
gen_flavor.py Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
indexnow.py daemon: move the Discord gateway and scheduler out of the web process into Go (#21) 2026-07-23 22:49:54 +00:00
lake_app.py Lake /query reads era5_daily through Iceberg metadata (#44) 2026-07-24 19:09:22 +00:00
Makefile Subtree-sync backend to split main a4d7fcd (dev->main promotion: reconciled bot + hardening, in-image CI); subtree workflow copies stay deleted (CI lives at root) 2026-07-22 22:37:21 -07:00
migrate.py Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
migrate_accounts_to_pg.py Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
migrate_cache_to_pg.py Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
paths.py Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
README.md docs: correct file references and the dev reachability claim 2026-08-01 11:49:28 -07:00
requirements-dev.txt Subtree-merge thermograph-backend (origin/main) into backend/ 2026-07-22 22:01:11 -07:00
requirements-seed.txt ERA5 lake: bucket-hosted history primary + SQL indexer service (#15) 2026-07-23 21:20:35 +00:00
requirements.txt Reconcile: merge main (shellcheck guard, Go daemon) into dev 2026-07-23 17:06:52 -07:00
seed_era5.py ERA5 lake: bucket-hosted history primary + SQL indexer service (#15) 2026-07-23 21:20:35 +00:00
warm_cities.py Pre-warm the SEO content derived-store off-request (#49) 2026-07-24 19:31:08 +00:00

thermograph-backend

The Thermograph API service: grades recent local weather against ~45 years of climate history, and hosts the accounts, notification, and SSR-content back-end that the rest of the split Thermograph stack (thermograph-frontend) talks to over HTTP. Split from the Jinemi/thermograph monorepo — this repo owns the API/DB/accounts/notifications layer only; it renders no HTML/CSS/JS of its own.

See CLAUDE.md for the full split topology, deploy flow, and API version contract, and thermograph-docs (a sibling repo) for cross-cutting architecture decisions and operator runbooks.

How it works

  1. Grid — a lat/lon is snapped to a stable ~4 sq mi cell (data/grid.py); longitude spacing is scaled by cos(latitude) so cells stay roughly square at any latitude. The cell id is the cache key, so the same spot always resolves to the same data.
  2. Data (on-demand, cached to parquet) — the first request for a cell fetches the full 1980present daily record (max/min temp, precip) from the free Open-Meteo archive (ERA5) and writes it to data/cache/<cell_id>.parquet (zstd, ~200 KB for 45 years). Later requests read the parquet directly. Recent days come from Open-Meteo's forecast API (past_days), so history and grading share one source.
  3. Percentiles & grading (data/grading.py) — each day is graded against every historical day within ±7 days of it (a 15-day seasonal window, wrapping year-end), as an empirical mid-rank percentile. Temperature uses a symmetric tier ladder (TEMP_BANDS: Near Record / High / Above Normal / Normal / Below Normal / Low / Near Record); precipitation is graded separately (RAIN_BANDS) since most days are dry — a rainy day is ranked only among rain days in its window, dry days are colored by dry-streak length instead.
  4. Caching & ETags — every derived payload (grade/calendar/day/SSR content) is cached in SQLite (data/store.py) under (kind, cell_id, key), validated by a token that only advances when the cell's history actually changes. That token doubles as a weak ETag, so an unchanged request costs a 304 with no payload rebuild (api/payloads.py, web/app.py).

Layout

accounts/       fastapi-users models/schemas/db + api_accounts routes
alembic/        Postgres schema migrations (alembic upgrade head on boot)
api/            versioned payload builders (payloads.py, content_payloads.py)
                + route wiring (content_routes.py, sitemap.py, homepage.py)
core/           metrics, audit/access logging, a singleton helper
data/           grid snapping, climate fetch/cache, grading/scoring,
                places/cities, the derived-payload store
notifications/  push (VAPID), email, Discord bot (interactions + linking),
                monthly digest, the in-process scheduler (city warming, IndexNow)
web/app.py      the FastAPI app (routes, CORS, ETag/versioning, middleware)
deploy/         container entrypoint.sh (alembic migrate, then serve)
app.py          shim re-exporting web.app:app — keeps the launch target
                `app:app` stable regardless of internal package layout
scripts/        one-off admin scripts (Discord slash-command registration)
tests/          pytest suite, hermetic (see tests/conftest.py)
cities.json,
cities_flavor.json   bundled reference data (generated by gen_cities.py /
                     gen_flavor.py), not runtime state

How it fits the split

  • thermograph-frontend calls this service's GET /api/v2/... endpoints (grade, geocode, calendar) and the SSR content endpoints under /content/...; it negotiates compatibility via GET /api/version.
  • thermograph-infra owns the deploy/Compose/Terraform layer — this repo only builds and publishes its own container image (git.thermograph.org/admin_emi/thermograph-backend/app) and hands infra a tag to roll out (SERVICE=backend + BACKEND_IMAGE_TAG into infra's infra/deploy/deploy.sh).
  • thermograph-docs holds the cross-repo architecture/runbook docs; this README only covers what's local to this service.

Build & run

docker build -t thermograph-backend .
docker run -p 8137:8137 --env-file .env thermograph-backend

The image runs deploy/entrypoint.sh: alembic upgrade head against THERMOGRAPH_DATABASE_URL (retried, since a fresh Postgres volume can still be starting up), then uvicorn app:app on $PORT (default 8137) with $WORKERS workers (default 4). /healthz is an I/O-free liveness probe.

For local development without a container, see the "Run / test locally" section of CLAUDE.md — there is no Makefile in this repo yet, so it's a plain venv + pytest/uvicorn invocation.

Notifications: Discord slash commands

notifications/discord_interactions.py answers Discord's HTTP Interactions endpoint (/discord/interactions) for the /grade <city> slash command. Registering (or updating) the command definition with Discord's REST API is a one-off admin action, not part of the running app:

THERMOGRAPH_DISCORD_APP_ID=... THERMOGRAPH_DISCORD_BOT_TOKEN=... \
    python3 scripts/register_discord_commands.py

Global command changes can take up to an hour to propagate.