From e9df43e6c0d211e495b0369115ae5dc8c0c74671 Mon Sep 17 00:00:00 2001 From: emi Date: Tue, 21 Jul 2026 15:11:21 +0000 Subject: [PATCH] Sync docs to the 3-node / release-branch / Forgejo architecture (#7) --- INFRA.md | 86 ++++++++++++++++------------------ deploy/thermograph.env.example | 10 ++-- terraform/README.md | 6 +-- 3 files changed, 50 insertions(+), 52 deletions(-) diff --git a/INFRA.md b/INFRA.md index ba026e3..8b8c2e5 100644 --- a/INFRA.md +++ b/INFRA.md @@ -41,9 +41,9 @@ Both VPS boxes are provisioned and reachable as of this writing: | Host | Role | Public IP | Login | |------|------|-----------|-------| -| prod | Swarm manager, Thermograph's eventual home | `169.58.46.181` | `agent` | -| beta | Swarm worker, old VPS | `75.119.132.91` | `agent` | -| desktop | Swarm worker, LAN dev machine + Actions runner | (local) | (already have access) | +| prod | Swarm manager; Thermograph's live prod home (`release`, thermograph.org) | `169.58.46.181` | `agent` | +| beta | Swarm worker; `main`/beta.thermograph.org + hosts Forgejo | `75.119.132.91` | `agent` | +| desktop | Swarm worker, LAN dev machine + Forgejo Actions runner | (local) | (already have access) | ``` ssh -i ~/.ssh/thermograph_agent_ed25519 agent@169.58.46.181 # prod @@ -66,11 +66,13 @@ directory that gets cleaned up, not anywhere-in-repo. Losing it just means regenerating and re-running the provisioning script; it does not lock either box, since your own login is untouched. -As of this writing, neither VPS has Docker installed yet and Terraform has -not been applied to either (`prod` is a bare OS install; `beta` has an old -`/opt/thermograph` checkout but no Docker) — Track 2 is paused on that. The -desktop already has Docker (used for the LAN dev containerized stack) but -hasn't joined anything yet (`docker info` shows `Swarm: inactive`). +**Live as of 2026-07-21** (was: "no Docker / Terraform not applied / Swarm +inactive" — all now done): Docker is installed on all three nodes; the +WireGuard mesh and Docker Swarm are up with all three nodes `Ready` (prod +manager, beta + desktop workers); Terraform has been applied to both VPS +boxes (prod stood up fresh on `release`, beta rebuilt on `main`); Forgejo is +serving at `git.thermograph.org`; and the desktop runs the Forgejo Actions +runner. See the verification checklist at the end. ## Track 2 — Swarm @@ -112,23 +114,19 @@ service anymore. `actions/setup-python`, which resolve unchanged). - **The custom auto-merge workflow step is gone.** It existed only because GitHub's native branch protection/auto-merge is paywalled on private - free-tier repos. Forgejo has no such tier — configure it directly: - **Settings → Branches → protect `dev` → Enable Status Check → list `build` - → merge style: squash.** Then a ready PR gets a real "Auto merge when checks - succeed" button. -- One knock-on simplification: a native auto-merge is an ordinary git push - (unlike GitHub's token-based merge commit, which doesn't re-trigger `push`), - so `deploy-dev.yml`'s push trigger fires naturally after auto-merge — no - more "direct push vs. PR-merge" double-trigger logic to maintain. -- `.forgejo/workflows/deploy.yml` ("Deploy to beta VPS") triggers on `main`, - same as it always did. This was flagged as a possible mismatch against - `terraform.tfvars.example`'s `release` branch for prod, but turned out not - to be one: this workflow predates the prod/beta split, so its secrets almost - certainly already point at beta (`main` is beta's branch per Terraform) — - see the file's own header comment. A separate `release` → prod deploy path - doesn't exist anywhere yet; don't invent one speculatively, confirm with - whoever owns Terraform whether prod even wants a push-triggered deploy or - is meant to go through `terraform apply` only. + free-tier repos. Forgejo has no such tier: `dev` is protected with the + `build` status check required (Settings → Branches). Forgejo does **not** + auto-merge on green by itself — a ready, green, mergeable PR is merged + **explicitly** (`POST .../pulls/{n}/merge`, `{"Do": "squash"}`), per + `CLAUDE.md`. That merge is an ordinary push, so `deploy-dev.yml`'s push + trigger fires naturally afterward. +- **Branch/deploy mapping (settled):** `.forgejo/workflows/deploy.yml` + ("Deploy to beta VPS") triggers on `main` and SSHes to **beta** + (`75.119.132.91`) running `deploy/deploy.sh`. **Prod is not deployed by a + push-triggered workflow** — it's deployed with `terraform apply` + (`terraform/README.md`) on the `release` branch. So the promotion chain is + `dev` → `main` (this workflow deploys beta) → `release` (Terraform deploys + prod). There is deliberately no `release`-triggered workflow. A second, independent set of Forgejo workflows (`.forgejo/workflows/{ci,build-push,ops-cron}.yml`) was added by a parallel @@ -139,27 +137,25 @@ runner-label and ssh-action guesses to match the real infrastructure here). reconciliation; `build-push.yml` (image → Forgejo's built-in registry) and `ops-cron.yml` (backup + IndexNow) are novel and still present. -### 3d. Cut over -Only after 3a–3c are verified: -1. Move the deploy secrets (`SSH_HOST`, `SSH_USER`, `SSH_KEY`, `SSH_PORT`) into - Forgejo's repo Secrets (Settings → Actions → Secrets). -2. Re-point the LAN dev runner: `bash deploy/forgejo/register-lan-runner.sh - ` — stops/disables the old GitHub runner service on - that same machine and registers `forgejo-runner` in its place, same - `systemd --user` pattern as before (see `DEPLOY-DEV.md` for the original; - this supersedes it for the runner specifically). -3. Open a test PR against `dev` in Forgejo, confirm the required check runs, - auto-merge fires, and the LAN deploy lands. -4. Only once that's solid: disable/archive the `.github/workflows/*.yml` - triggers (or the whole GitHub repo) — GitHub was the fallback during this - whole migration; retire it deliberately, not by accident. +### 3d. Cut over — DONE (2026-07-21) +The GitHub → Forgejo cutover is complete; this records what was done: +1. Deploy secrets (`SSH_HOST`, `SSH_USER`, `SSH_KEY`, `SSH_PORT`) live in + Forgejo's repo Secrets, pointing at beta with a dedicated CI deploy key. +2. The LAN dev runner was re-pointed to Forgejo via + `deploy/forgejo/register-lan-runner.sh` (systemd --user, on the desktop). +3. The repo was migrated into Forgejo; `dev`/`main`/`release` all promoted + through Forgejo and deploys verified live. +4. GitHub is retired as git host and CI (PR "Retire GitHub as the git host and + CI platform") — `origin` remains only as a read-only mirror of history. -## Verification checklist +## Verification checklist — all met (2026-07-21) - [x] `ssh agent@` and `ssh agent@` both work; `sudo whoami` → `root` - [x] Password SSH auth confirmed dead on both boxes (tested with an actual failed login attempt, not just config inspection) -- [ ] `docker node ls` (from prod) shows both nodes `Ready` -- [ ] Swarm ports (2377/7946/4789) unreachable from outside the WireGuard tunnel -- [ ] `https://git.thermograph.org` (or your chosen domain) serves Forgejo over TLS -- [ ] A test PR: required check runs → auto-merges → LAN dev deploy lands -- [ ] GitHub retired only after all of the above hold +- [x] `docker node ls` (from prod) shows all three nodes `Ready` +- [x] Swarm ports (2377/7946/4789) unreachable from outside the WireGuard tunnel +- [x] `https://git.thermograph.org` serves Forgejo over TLS; `/v2/` registry + API is 403 from off-mesh (firewalled to the WireGuard CIDR) +- [x] A test PR flow verified: required `build` check runs → explicit squash + merge → LAN dev deploy lands +- [x] GitHub retired as git host + CI; kept only as a read-only history mirror diff --git a/deploy/thermograph.env.example b/deploy/thermograph.env.example index 94cb2e8..a2d5ef0 100644 --- a/deploy/thermograph.env.example +++ b/deploy/thermograph.env.example @@ -1,7 +1,9 @@ -# Copy to /etc/thermograph.env on the VPS and edit. -# Read by the systemd unit (EnvironmentFile) so `docker compose up` can interpolate -# it, AND loaded into the app container (env_file in docker-compose.yml). Anything -# secret the app needs — Postgres password, VAPID keys, auth secret — belongs here. +# Copy to /etc/thermograph.env on the VPS and edit. On prod/beta, Terraform +# renders this file for you (see terraform/README.md) — edit the tfvars, not the +# host. Sourced by deploy/deploy.sh (and any `docker compose` invocation) so +# compose can interpolate it, AND loaded into the app container (env_file in +# docker-compose.yml). Anything secret the app needs — Postgres password, VAPID +# keys, auth secret — belongs here. # Port uvicorn binds inside the container. The compose stack publishes it on the # host loopback (127.0.0.1:8137) for Caddy to proxy to. Keep 8137. diff --git a/terraform/README.md b/terraform/README.md index 62fc69a..e4e7ef1 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -12,11 +12,11 @@ One reusable module (`modules/thermograph-host`) is instantiated per host via | key | role | VPS | branch | domain | notes | |--------|--------|-------------------------|-----------|-------------------|------------------------------------| -| `prod` | prod | NEW 48 GB / 12-core box | `release` | `thermograph.org` | Caddy TLS; sized up (8/4/16g) | -| `beta` | beta | old box `75.119.132.91` | `main` | *(none)* | testing/beta; no public domain yet | +| `prod` | prod | NEW 48 GB / 12-core box (`169.58.46.181`) | `release` | `thermograph.org` | Caddy TLS; sized up (8/8/4/16g) | +| `beta` | beta | old box `75.119.132.91` | `main` | `beta.thermograph.org` | Caddy TLS; also hosts Forgejo | The `dev` branch is **out of scope here** — it deploys to the LAN dev server via -`deploy/deploy-dev.sh` (a self-hosted GitHub Actions runner), not Terraform. +`deploy/deploy-dev.sh` (a self-hosted **Forgejo** Actions runner), not Terraform. Per host, over SSH provisioners, Terraform: