thermograph/docs/onboarding/03-repo-map.md

# 3. Repo map

The monorepo was reunified on **2026-07-22** by subtree-merging four split
repos with full history (380 commits). The split's *properties* were kept
deliberately: per-domain images, per-domain deploys, and an async FE/BE
contract. That's why the layout looks like four projects sharing a root.

```
thermograph/
├── CLAUDE.md                  root agent instructions — branch model, deploy contract
├── CUTOVER-NOTES.md           what is and isn't live; the reunification record
├── README.md                  domain table + how CI stays decoupled
├── .forgejo/workflows/        ALL CI. Nine files. Never add workflows elsewhere.
├── .claude/                   settings.json + enforcement hooks
├── backend/                   FastAPI API + Go daemon        → emi/thermograph/backend
├── frontend/                  Go SSR + static assets         → emi/thermograph/frontend
├── infra/                     compose, Swarm, deploy, secrets, terraform, ops
├── observability/             Loki + Grafana + Alloy
└── docs/onboarding/           you are here
```

> **CI lives only in root `.forgejo/workflows/`**, path-filtered per domain.
> Forgejo only reads root workflows — a copy under a domain's own `.forgejo/`
> is inert, and becomes a trap when someone edits it expecting an effect.

## `backend/` — the API service

Python 3.12 / FastAPI. Owns climate data, grading, the API, accounts,
notifications, and the SSR-content JSON API the frontend renders from. Serves
**no** HTML/CSS/JS.

```
app.py                 one-line re-export shim for web/app.py — keeps `app:app`
                       stable for systemd/CI/entrypoint. Don't change the name.
paths.py               every filesystem location, resolved from the repo root.
                       NEVER reintroduce __file__-relative paths in a module.
web/app.py     (1022)  the real FastAPI app: routes, CORS, ETag, middleware,
                       lifespan, role gating, the frontend proxy fallback
api/
  payloads.py    (342) pure inputs → response dict builders. One definition per
                       payload shape; PAYLOAD_VER lives here.
  content_payloads.py  the SSR content API's builders (JSON-safe, no Markup)
  content_routes.py    route wiring for /content/* + the same ETag pattern
  internal_routes.py   /internal/* — the Go daemon's control surface. Fails
                       closed: no token configured ⇒ the whole surface 404s.
  homepage.py    (367) the precomputed "unusual right now" feed
  sitemap.py
data/
  climate.py    (1248) fetch + cache the daily record. The source ladder lives
                       here. The single biggest file in the repo.
  grading.py     (532) day-of-year climatology, percentiles, TEMP/RAIN_BANDS
  grid.py         (92) lat/lon → stable ~4 sq mi cell id
  store.py       (329) derived-payload store (SQLite/Postgres). Pure accelerator.
  climate_store.py(353) raw climate record on TimescaleDB hypertables
  era5lake.py    (173) the ERA5 object-storage lake: layout, grid math, clients
  scoring.py     (293) climate-shift scoring (recent years vs full baseline)
  places.py      (355) typo-tolerant local place index for /suggest
  cities.py, city_events.py, meteostat.py
core/
  metrics.py     (805) since-start traffic counters, multi-worker via SQLite
  audit.py       (167) JSONL audit / errors / access / activity / heartbeat
  singleton.py   (129) leader election: flock (per host) or PG advisory lock
                       (per cluster). Read this before adding any timer.
accounts/              fastapi-users: models, schemas, db, users, api_accounts
notifications/
  notify.py      (542) the subscription evaluation engine (background thread)
  push.py, mailer.py, digest.py, discord.py, discord_link.py,
  discord_interactions.py
daemon/                the Go daemon — Discord gateway + cron timers
alembic/               Postgres migrations (run by the entrypoint on boot)
deploy/entrypoint.sh   container entrypoint: alembic migrate, then serve
lake_app.py    (204)   the prod-only `lake` service (DuckDB over the bucket)
tests/                 the hermetic pytest suite
cities.json, cities_flavor.json   bundled reference data (generated)
```

Standalone scripts at the root are tooling, not runtime: `warm_cities.py`,
`indexnow.py`, `seed_era5.py`, `gen_era5_lake.py`, `gen_cities.py`,
`gen_flavor.py`, `drift_check.py`, `migrate*.py`.

Deep dive: [04-backend.md](04-backend.md).

## `frontend/` — SSR + static assets

```
server/                ★ THE LIVE SERVICE (Go 1.26, module thermograph/frontend)
  main.go              config + mux + static + graceful shutdown
  internal/config/     every env var, with the same names/defaults as before
  internal/contentapi/ backend /content/* client: TTL cache, bounded LRU,
                       per-key single-flight, origin forwarding
  internal/content/    page handlers, funcmap, SEO
  internal/contentdata/ glossary.yaml + pages.yaml loader (fail-loud)
  internal/format/     unit-aware °C/°F formatting + band names
  internal/handlers/   routing, SPA shells, static
  internal/render/     html/template over embed.FS + ETag helpers
  internal/render/templates/*.tmpl   the embedded page templates
static/                every static asset — app.js, cache.js, shared.js,
                       account.js, calendar/day/score/compare .js + .html
                       shells, style.css (all design tokens), icons, sw.js
content/               glossary.yaml, pages.yaml — structured SSR copy
templates/*.html.j2    Jinja templates — used by the PYTHON service only
tests/                 Python tiers: unit (hermetic, fixtures) + integration
tests/fixtures/*.json  golden payloads — ALSO consumed by the Go tests
tools/shoot.py         the `make shots` screenshot sweep
DESIGN.md              the visual system: tokens, components, breakpoints
app.py, content.py,    ⚠ the superseded Python implementation — not deployed
api_client.py,           (see traps)
format.py, paths.py,
content_loader.py
```

Deep dive: [05-frontend.md](05-frontend.md).

## `infra/` — how and where images run

```
docker-compose.yml          the compose stack — dev's, on vps1 (and a local
                            `make dev-up`): db, backend, lake, daemon, frontend.
                            `name: thermograph` is PINNED — see below.
docker-compose.dev.yml      dev overlay: uncapped, backend bound to
                            `${DEV_BIND_ADDR:-127.0.0.1}` — vps1's mesh IP
                            (10.10.0.2) at deploy time, never 0.0.0.0
docker-compose.openmeteo.yml self-hosted Open-Meteo overlay (prod-only)
Makefile                    compose orchestration only
deploy/
  env-topology.sh           ★ single source of truth: env → host, checkout,
                            branch, deploy mode, stack name, env file, LB
                            ports, DB role/name, service prefix
  deploy.sh                 ★ the single entry point for dev, beta and prod
  deploy-dev.sh              thin wrapper: routes deploy.sh at dev's compose
                            overlay and secrets policy
  render-secrets.sh         renders the env file for a given environment from
                            the SOPS vault
  secrets/                  the vault: common.yaml, prod/beta/dev.yaml,
                            centralis.prod.yaml, example.yaml
  stack/                    the SWARM path — both live on vps2:
                            thermograph-stack.yml (prod),
                            thermograph-beta-stack.yml (beta, prefixed
                            services, no `db` of its own), deploy-stack.sh,
                            autoscale.sh, lb/ (per-environment Caddyfiles)
  db/provision-env-db.sh    creates each environment's database + role on the
                            shared TimescaleDB instance
  provision-dev.sh          one-time bootstrap of the dev environment on vps1
  swarm/, forgejo/          mesh + Forgejo/CI-runner provisioning
  Caddyfile                 the reverse proxy (path-splits FE vs BE)
  provision-*.sh            host bootstrap: agent access, mail, secrets
  migrations/               hand-run SQL migrations
ops/
  dbq.sh                    read-only psql into any environment's db container
  iceberg.sh                duckdb over the ERA5 Iceberg lake
terraform/                  host provisioning — NO STATE IS PERSISTED
ACCESS.md, DEPLOY.md, DEPLOY-DEV.md    (partly stale — see traps)
```

**Why the project name is pinned.** Compose creates volumes
`thermograph_pgdata` / `_appdata` / `_applogs`, and the Swarm stacks declare
those exact names as `external: true` at the same mount paths. Running compose
from `infra/` without `name: thermograph` derives project `infra`, which makes a
whole new stack with empty volumes next to the running one. `deploy-dev.sh`
exports `COMPOSE_PROJECT_NAME=thermograph-dev` (env wins over the file key) to
keep dev separate on purpose. Keep both halves.

Deep dive: [08-infra-secrets.md](08-infra-secrets.md).

## `observability/` — the logging stack

```
docker-compose.yml            Loki + Grafana (runs on vps1)
loki/config.yml               mesh-only, filesystem storage, 30-day retention
grafana/provisioning/         datasource + dashboard provider (auto-loaded)
grafana/provisioning/alerting/ rules, the Discord contact point, the policy
grafana/dashboards/*.json     the fleet-logs dashboard
alloy/config.alloy            the per-node shipper
alloy/docker-compose.agent.yml runs Alloy on a node (ALLOY_NODE per host)
caddy-grafana.conf            vps1's Grafana vhost, kept for reference
```

No build, no deploy automation — it ships by hand. **The repo is the only
durable path**: Grafana is provisioned from this directory at startup, so UI
edits are overwritten. Grafana is at **`dashboard.thermograph.org`** (Google
SSO, pre-provisioned users only) — never `grafana.thermograph.org`.

Deep dive: [09-observability.md](09-observability.md).

## The nine CI workflows

| Workflow | Trigger | Does |
|---|---|---|
| `pr-build.yml` | PR → `dev`/`main` | The `gate` required check. Diffs the PR, builds only touched domains. Deliberately **not** path-filtered. |
| `build.yml` | `workflow_call` | Build one domain's image; run the backend suite **inside** the built image. |
| `build-push.yml` | push to `dev`/`main`/`release` touching an app domain, or a `v*.*.*` tag | Builds + pushes `sha-<12hex>` images. A version tag builds **both**. |
| `deploy.yml` | push to `dev`/`main`/`release` touching an app domain | Branch selects environment — three legs: `dev`→vps1, `main`→beta on vps2, `release`→prod on vps2; matrix covers services; SSHes and runs `deploy.sh` with `THERMOGRAPH_ENV` set. |
| `infra-sync.yml` | push to `main` touching `infra/**` | Fast-forwards beta's and prod's checkouts (both on vps2) and re-renders their secrets. Rolls **no** service. |
| `observability-validate.yml` | push touching `observability/**`, or `workflow_call` | Parses every artifact; validates Alloy config with the pinned binary; strict alerting checks. |
| `ops-cron.yml` | daily 03:00 UTC | **THE prod backup** (pg_dump) + IndexNow, against prod on vps2. Uses `VPS2_SSH_*`. |
| `secrets-guard.yml` | every PR and push | Fails if any `infra/deploy/secrets/*.yaml` isn't SOPS-encrypted. |
| `shell-lint.yml` | every PR and push | shellcheck (pinned v0.11.0 + sha256) over every `*.sh`. |

Deep dive: [07-ci-and-release.md](07-ci-and-release.md).

Next: [Backend deep dive](04-backend.md).