thermograph/INFRA.md

# Agent access, Docker Swarm, and Forgejo CI/CD

This covers three additive infrastructure layers on top of the two VPS boxes
Terraform already provisions (see `terraform/README.md` and
`terraform.tfvars.example`, where they're named **prod** — the new 48 GB /
12-core box, `thermograph.org` — and **beta** — the old VPS,
`75.119.132.91`). **None of this touches `backend/`, `Dockerfile`,
`docker-compose*.yml`, `terraform/`, or `deploy/db/`** — those stay owned by
the app-containerization work; this is strictly the layer above it.

```
Track 1: deploy/provision-agent-access.sh   — a dedicated full-root login for me
Track 2: deploy/swarm/                      — Swarm cluster spanning both boxes
Track 3: deploy/forgejo/ + .forgejo/         — Forgejo + Forgejo Actions, replacing GitHub
```

## Track 1 — Agent access

Run `sudo bash deploy/provision-agent-access.sh` on **both** boxes. Creates a
dedicated `agent` user (not raw root login — a distinct name gives a clean
audit trail) with passwordless sudo, installs the agent's public key, disables
SSH password auth fleet-wide, and turns on `auditd` logging of every
root-effective command. See the script's header comment for the full
rationale and revocation steps (one line to delete, or delete the whole
account — your own access is never affected).

## Track 2 — Swarm

See `deploy/swarm/README.md` for the exact order of operations (WireGuard
tunnel first, then swarm init/join, then lock the Swarm ports down to the
tunnel interface, then label beta for Forgejo placement). This cluster's only
job is hosting Forgejo — it does not orchestrate the Terraform-managed app
deploys.

## Track 3 — Forgejo, replacing GitHub

### 3a. Stand up Forgejo
See `deploy/forgejo/README.md` — deploys the stack, walks through minting the
two Swarm secrets it needs (`forgejo_db_password`, `forgejo_runner_token`).

### 3b. Migrate the repo (mirror first, verify, then cut over)
1. In Forgejo: **+ New Migration → GitHub**. Point it at
   `https://github.com/griffemi/thermograph`, pull code + issues + PRs +
   releases + labels. This is non-destructive to GitHub — do this as a mirror
   and leave GitHub live and untouched until you're confident.
2. Spot-check: branch list matches, a few PRs/issues render correctly, LFS (if
   any) came across.
3. **Don't retarget any secret or disable a GitHub workflow yet** — see 3d.

### 3c. Workflows
`.forgejo/workflows/*.yml` already mirror `.github/workflows/*.yml` in this
PR — copied with the mechanical changes Forgejo needs:
- `runs-on: ubuntu-latest` → `runs-on: docker` (the label the Swarm-hosted
  runner in `deploy/forgejo/docker-stack.yml` registers under).
- `appleboy/ssh-action` referenced by full GitHub URL (not mirrored in
  Forgejo's default action registry, unlike `actions/checkout` /
  `actions/setup-python`, which resolve unchanged).
- **The custom auto-merge workflow step is gone.** It existed only because
  GitHub's native branch protection/auto-merge is paywalled on private
  free-tier repos. Forgejo has no such tier — configure it directly:
  **Settings → Branches → protect `dev`  → Enable Status Check → list `build`
  → merge style: squash.** Then a ready PR gets a real "Auto merge when checks
  succeed" button.
- One knock-on simplification: a native auto-merge is an ordinary git push
  (unlike GitHub's token-based merge commit, which doesn't re-trigger `push`),
  so `deploy-dev.yml`'s push trigger fires naturally after auto-merge — no
  more "direct push vs. PR-merge" double-trigger logic to maintain.
- `.forgejo/workflows/deploy.yml` still targets `main` faithfully mirroring
  the current GitHub workflow, but `terraform.tfvars.example` describes prod's
  branch as `release`. That mismatch is flagged in the file's own header
  comment — reconcile with whoever's driving the Terraform/deploy side rather
  than guessing here.

### 3d. Cut over
Only after 3a–3c are verified:
1. Move the deploy secrets (`SSH_HOST`, `SSH_USER`, `SSH_KEY`, `SSH_PORT`) into
   Forgejo's repo Secrets (Settings → Actions → Secrets).
2. Re-point the LAN dev runner: `bash deploy/forgejo/register-lan-runner.sh
   <forgejo_url> <token>` — stops/disables the old GitHub runner service on
   that same machine and registers `forgejo-runner` in its place, same
   `systemd --user` pattern as before (see `DEPLOY-DEV.md` for the original;
   this supersedes it for the runner specifically).
3. Open a test PR against `dev` in Forgejo, confirm the required check runs,
   auto-merge fires, and the LAN deploy lands.
4. Only once that's solid: disable/archive the `.github/workflows/*.yml`
   triggers (or the whole GitHub repo) — GitHub was the fallback during this
   whole migration; retire it deliberately, not by accident.

## Verification checklist
- [ ] `ssh agent@<prod>` and `ssh agent@<beta>` both work; `sudo whoami` → `root`
- [ ] Password SSH auth confirmed dead on both boxes
- [ ] `docker node ls` (from prod) shows both nodes `Ready`
- [ ] Swarm ports (2377/7946/4789) unreachable from outside the WireGuard tunnel
- [ ] `https://git.thermograph.org` (or your chosen domain) serves Forgejo over TLS
- [ ] A test PR: required check runs → auto-merges → LAN dev deploy lands
- [ ] GitHub retired only after all of the above hold