thermograph/CUTOVER-NOTES.md

Monorepo cutover notes (assembled 2026-07-22)

Scratch assembly for inspection — nothing live points here yet. Built by subtree-merging each split repo's origin/main (full history, 380 commits):

• backend/ ← thermograph-backend 83c2e05 (Discord gateway bot landed, #7)
• frontend/ ← thermograph-frontend 3a98146 (responsiveness hardening, #4)
• infra/ ← thermograph-infra 99b4b3f (mail docs, #12)
• observability/ ← thermograph-observability 19e74af (validator CI, #3)

What changed vs. the split repos

CI (all in root .forgejo/workflows/; the subtree copies were deleted — Forgejo only reads root workflows, but dead copies are a trap):

Superseded 2026-07-25. The per-domain workflow names below describe the cutover as it landed, and are kept for that history. They no longer exist: the six *-deploy*.yml files were collapsed into a single deploy.yml (branch selects environment, matrix covers the services) and the two *-build-push.yml into a single build-push.yml. The two LAN-dev deploy workflows were deleted outright rather than ported, having been inert since cutover. Everything described below about behaviour — path filtering, the domain-keyed 12-hex tag, per-domain concurrency, the v*.*.* both-images exception, separate SSH_*/PROD_SSH_* secret sets — is unchanged.

• backend-build-push.yml / frontend-build-push.yml — the old per-repo build-push, path-filtered (paths: ['backend/**'] etc.), image paths now explicit: emi/thermograph/backend|frontend (the old ${{ github.repository }}/app derivation collides in a monorepo). Per-domain concurrency groups. Caveat: v*.*.* tag pushes carry no paths context, so a version tag builds both images (intended: a release names the set).
• backend|frontend-deploy[.yml|-prod.yml|-dev.yml] — ports of the SSH deploy workflows, path-filtered, script path now /opt/thermograph/infra/deploy/deploy.sh. Same secret names (SSH_* beta, PROD_SSH_* prod) — but now one secret set on one repo.
• build.yml — the build-only sanity check, now a single reusable workflow taking a domain input (context = the domain dir, so Dockerfiles see the same tree as in the split era).
• pr-build.yml — one changes job (git-diff domain detection) + per-domain gated builds + a single always-reporting gate required check. Deliberately not path-filtered: a required check that never fires blocks auto-merge forever.
• infra-sync.yml — new (the infra domain's own pipeline): push to main touching infra/** → git-sync /opt/thermograph on beta+prod and re-render secrets. Rolls no services (image tags are the app domains' axis).
• observability-validate.yml — port with observability/ prefixes + workflow_call so pr-build reuses it.
• secrets-guard.yml — glob is now infra/deploy/secrets/*.yaml; still runs on everything (cheap backstop).
• ops-cron.yml — compose invocations cd /opt/thermograph/infra.

Deploy scripts / compose (hosts see a monorepo checkout at /opt/thermograph):

• deploy.sh: INFRA_DIR="$APP_DIR/infra"; git ops at the checkout root, cd "$INFRA_DIR" for all compose work; lock/tags/render paths under infra/deploy/; image-path defaults renamed.
• docker-compose.yml: name: thermograph pinned. Critical: without it, running compose from …/infra derives project "infra" — a silently NEW project that would recreate the whole stack beside the running one. deploy-dev.sh exports COMPOSE_PROJECT_NAME=thermograph-dev (env wins over the file key), preserving LAN dev's separate project.
• deploy/stack/deploy-stack.sh + thermograph-stack.yml: same path/image renames (prod is stack-mode; docker stack deploy takes an explicit stack name, so no project-name concern there).

Verified here: every workflow + compose file parses (PyYAML); all three deploy scripts pass bash -n; no emi/thermograph-{backend,frontend}/app stragglers in sh/yml. Not verified (impossible locally): Forgejo actually honoring on.push.paths and workflow_call inputs on our version; any live deploy.

Before cutover (in order)

1. Run the pending prod network-recreate runbook first. Don't rebase the deploy path on a prod that wedges on its next deploy.
2. Prove Forgejo path filters + reusable-workflow inputs with a throwaway repo: one push touching only backend/** must fire only backend workflows; a uses: ./.forgejo/workflows/build.yml + with: job must pass its input. If paths aren't honored, fall back to the pr-build pattern (diff job + gated jobs) for the push workflows too.
3. Branch drift: the split repos' branch state as of 2026-07-22 evening is fully migrated (see "Branch migration status" below) — re-run the check (git merge-tree --write-tree per branch vs dev/main) if cutover slips and upstream moves again.

Branch migration status (2026-07-22)

Checked every split-repo branch by dry-run merge; migrated everything real:

• dev (this repo) = main + backend/frontend origin/dev subtrees: the in-image test CI feature (ported into root build.yml — backend full suite, frontend unit tier; their subtree build.yml edits resolved as deleted) + backend test scripts/Makefile/compose.test + frontend two-tier suite with fixtures. Faithful to upstream: the feature is dev-only, so mono main doesn't have it; promote with dev→main like the split flow.
• feature/geocode-local → feature/history-meteostat-gusts → feature/nasa-primary-flip → feature/era5-seed — the unmerged backend stack, rebuilt stacked on mono dev. ⚠️ Two conflicts were resolved here that upstream has NOT resolved (the same conflicts exist merging those branches into split-repo dev): data/climate.py (dev's hardened _request signature combined with the gust-fill wrap) and tests/data/test_climate.py (kept both appended test blocks). Unverified — tests weren't runnable locally (Docker Hub blocked); run the in-image suite on these branches before trusting them.
• Stale (absorbed upstream, nothing migrated — safe to delete in the split repos): backend ci-run-tests-in-image, perf/*, phase5/discord-gateway-bot, port-discord-gateway-bot, reconcile-dev-with-main, test-runner-and-smoke; frontend ci-run-tests-in-image, reconcile-dev-with-main; all three infra branches (ci-fix/render-secrets-owner, phase5/discord-bot-flag, terraform-two-image-tags — all landed in infra main); both observability dashboard branches.
• Dismissed, not stale-but-not-migrated: frontend perf/frontend-responsiveness (its "new" content would resurrect test files dev reorganized into the unit tier) and test-real-backend-harness (an older revision of dev's tests/unit/test_rendering.py).

Cutover runbook

1. Create emi/thermograph on Forgejo (a new repo — do not resurrect the archived monorepo) and push main.
2. Actions config, once: secrets REGISTRY_TOKEN, SSH_HOST/USER/KEY/PORT (beta), PROD_SSH_* (prod); variable REGISTRY_HOST=git.thermograph.org.
3. Branch protection on dev + main: required check gate, squash merges, auto-merge on green.
4. Hosts (beta, then prod): clone the monorepo to /opt/thermograph.new; copy deploy/.image-tags.env from the old checkout into infra/deploy/ (its path moved — losing it makes the next single-service deploy roll the sibling onto local); swap directories (running containers are untouched — they're images, not the checkout); /etc/thermograph* files stay as-is.
5. LAN box: re-point ~/thermograph-dev at the monorepo the same way.
6. First push builds the new image paths — old emi/thermograph-{backend,frontend}/app packages stay in the registry for rollback; GC after burn-in. deploy.sh's tag prune already targets the new paths only.
7. One workflow_dispatch of backend-deploy.yml after its build-push completes = the live end-to-end validation, same as the split-era wiring test.
8. After burn-in: archive the four split repos (leave thermograph-docs alone), update its runbooks to the monorepo paths.

Known gaps / deferred

• Discord/notifier extraction (phase 2): discord still ships inside the backend image (backend/notifications/); a discord tweak = a backend deploy. True notifier/ domain (own image + compose service, singleton lock retired) is a refactor for after the monorepo settles.
• Domain CLAUDE.mds still speak repo-era ("this repo", sibling-repo links) — correct them as touched; root CLAUDE.md points the right way.
• backend/MONOREPO-CUTOVER-PLAN.md references (the split plan) are now historical.
• Infra Makefile local-image targets still assume sibling checkouts — update when first needed locally.

vps1/vps2 host-topology re-architecture (landed 2026-07-25, NOT YET EXECUTED)

This section documents a second re-architecture, layered on top of the monorepo cutover above. It is landed IN THE REPO — code, stack files, deploy scripts, docs — but has NOT been run against the live estate. The boxes described elsewhere in this file (prod at 169.58.46.181, beta at 75.119.132.91, a desktop LAN dev box) are still what is actually running today. Do not treat anything below as live until the cutover runbook has been executed and this note updated.

What changes, in one line: hosts are renamed by role instead of by environment, and beta moves onto the same box as prod.

• vps1 (mesh 10.10.0.2, public 75.119.132.91) — unchanged box, new job. Keeps Forgejo, Grafana/Loki/Alloy and the emigriffith.dev portfolio; gains the dev environment (its own Postgres, plain compose, mesh-only — 10.10.0.2:8137, no DNS record, no Caddy site, no TLS). This is the box that used to be called "beta".
• vps2 (mesh 10.10.0.1, public 169.58.46.181) — unchanged box, new job. Keeps prod; gains beta as a second, co-resident Docker Swarm stack (prefixed services, own checkout, own env file, own loopback LB ports). One TimescaleDB instance now serves both prod and beta, on separate databases and separate roles. This is the box that used to be called "prod".
• desktop (mesh 10.10.0.3) — loses the LAN dev stack and the Forgejo Actions runner entirely; becomes AI-model hosting plus flex Swarm capacity. make dev-up still works there as a laptop convenience, but it is no longer an environment.

The mesh IPs did not move — only which environment lives on which box did. See infra/deploy/env-topology.sh for the single source of truth on per-environment checkout paths, branches, stack names, ports and DB roles, and infra/deploy/RUNBOOK-vps1-vps2-cutover.md for the execution runbook. Until that runbook has been run, CLAUDE.md, README.md and docs/onboarding/ describe the target state, not today's.