0d8fc9f4d0
Three additive infrastructure layers on top of the two VPS boxes Terraform already provisions (prod: new 48 GB/12-core box, thermograph.org; beta: old VPS, 75.119.132.91). None of this touches backend/, Dockerfile, docker-compose*.yml, terraform/, or deploy/db/ — that stays owned by the app-containerization work in flight elsewhere; this is strictly the layer on top. See INFRA.md for the full runbook and order of operations. - deploy/provision-agent-access.sh: a dedicated, auditable full-sudo login (not raw root) for agent-driven ops — passwordless sudo under a distinct username, sshd hardened to key-only auth, auditd logging every root-effective command. One line to revoke. - deploy/swarm/: a 2-node Swarm (prod=manager, beta=worker) joined over a WireGuard tunnel rather than trusting the public internet with the overlay data plane, which Docker's own guidance says should never face it directly. Swarm ports locked to the tunnel interface once joined. This cluster's only workload is Forgejo — it does not orchestrate the Terraform-managed app deploys, so nothing here can strand the app's single-writer database. - deploy/forgejo/ + .forgejo/workflows/: Forgejo + Traefik + a Docker-in-Docker-sandboxed runner as a Swarm stack pinned to beta, plus Forgejo Actions workflows mirroring .github/workflows/*.yml. The custom auto-merge workflow step is dropped — it existed only to work around GitHub's paywalled branch protection on private free-tier repos, which Forgejo has no such tier for; native "auto merge when checks succeed" replaces it, and as a real git push (unlike GitHub's non-triggering token-merge) it fires the LAN deploy naturally with no double-trigger logic needed. appleboy/ssh-action is referenced by full URL (not mirrored on Forgejo's default action registry); actions/checkout and actions/setup-python resolve unchanged. Migration is mirror-first: the GitHub repo import and workflow files land here, but cutting deploy secrets over and retiring GitHub happens only after verification (INFRA.md 3d) — GitHub stays live as a fallback throughout. One flagged, unresolved mismatch: deploy.yml still triggers on `main`, but terraform.tfvars.example names prod's deploy branch `release`. Left as a faithful mirror rather than guessed at — reconcile with whoever's driving Terraform/deploy.
96 lines
5.2 KiB
Markdown
96 lines
5.2 KiB
Markdown
# Agent access, Docker Swarm, and Forgejo CI/CD
|
||
|
||
This covers three additive infrastructure layers on top of the two VPS boxes
|
||
Terraform already provisions (see `terraform/README.md` and
|
||
`terraform.tfvars.example`, where they're named **prod** — the new 48 GB /
|
||
12-core box, `thermograph.org` — and **beta** — the old VPS,
|
||
`75.119.132.91`). **None of this touches `backend/`, `Dockerfile`,
|
||
`docker-compose*.yml`, `terraform/`, or `deploy/db/`** — those stay owned by
|
||
the app-containerization work; this is strictly the layer above it.
|
||
|
||
```
|
||
Track 1: deploy/provision-agent-access.sh — a dedicated full-root login for me
|
||
Track 2: deploy/swarm/ — Swarm cluster spanning both boxes
|
||
Track 3: deploy/forgejo/ + .forgejo/ — Forgejo + Forgejo Actions, replacing GitHub
|
||
```
|
||
|
||
## Track 1 — Agent access
|
||
|
||
Run `sudo bash deploy/provision-agent-access.sh` on **both** boxes. Creates a
|
||
dedicated `agent` user (not raw root login — a distinct name gives a clean
|
||
audit trail) with passwordless sudo, installs the agent's public key, disables
|
||
SSH password auth fleet-wide, and turns on `auditd` logging of every
|
||
root-effective command. See the script's header comment for the full
|
||
rationale and revocation steps (one line to delete, or delete the whole
|
||
account — your own access is never affected).
|
||
|
||
## Track 2 — Swarm
|
||
|
||
See `deploy/swarm/README.md` for the exact order of operations (WireGuard
|
||
tunnel first, then swarm init/join, then lock the Swarm ports down to the
|
||
tunnel interface, then label beta for Forgejo placement). This cluster's only
|
||
job is hosting Forgejo — it does not orchestrate the Terraform-managed app
|
||
deploys.
|
||
|
||
## Track 3 — Forgejo, replacing GitHub
|
||
|
||
### 3a. Stand up Forgejo
|
||
See `deploy/forgejo/README.md` — deploys the stack, walks through minting the
|
||
two Swarm secrets it needs (`forgejo_db_password`, `forgejo_runner_token`).
|
||
|
||
### 3b. Migrate the repo (mirror first, verify, then cut over)
|
||
1. In Forgejo: **+ New Migration → GitHub**. Point it at
|
||
`https://github.com/griffemi/thermograph`, pull code + issues + PRs +
|
||
releases + labels. This is non-destructive to GitHub — do this as a mirror
|
||
and leave GitHub live and untouched until you're confident.
|
||
2. Spot-check: branch list matches, a few PRs/issues render correctly, LFS (if
|
||
any) came across.
|
||
3. **Don't retarget any secret or disable a GitHub workflow yet** — see 3d.
|
||
|
||
### 3c. Workflows
|
||
`.forgejo/workflows/*.yml` already mirror `.github/workflows/*.yml` in this
|
||
PR — copied with the mechanical changes Forgejo needs:
|
||
- `runs-on: ubuntu-latest` → `runs-on: docker` (the label the Swarm-hosted
|
||
runner in `deploy/forgejo/docker-stack.yml` registers under).
|
||
- `appleboy/ssh-action` referenced by full GitHub URL (not mirrored in
|
||
Forgejo's default action registry, unlike `actions/checkout` /
|
||
`actions/setup-python`, which resolve unchanged).
|
||
- **The custom auto-merge workflow step is gone.** It existed only because
|
||
GitHub's native branch protection/auto-merge is paywalled on private
|
||
free-tier repos. Forgejo has no such tier — configure it directly:
|
||
**Settings → Branches → protect `dev` → Enable Status Check → list `build`
|
||
→ merge style: squash.** Then a ready PR gets a real "Auto merge when checks
|
||
succeed" button.
|
||
- One knock-on simplification: a native auto-merge is an ordinary git push
|
||
(unlike GitHub's token-based merge commit, which doesn't re-trigger `push`),
|
||
so `deploy-dev.yml`'s push trigger fires naturally after auto-merge — no
|
||
more "direct push vs. PR-merge" double-trigger logic to maintain.
|
||
- `.forgejo/workflows/deploy.yml` still targets `main` faithfully mirroring
|
||
the current GitHub workflow, but `terraform.tfvars.example` describes prod's
|
||
branch as `release`. That mismatch is flagged in the file's own header
|
||
comment — reconcile with whoever's driving the Terraform/deploy side rather
|
||
than guessing here.
|
||
|
||
### 3d. Cut over
|
||
Only after 3a–3c are verified:
|
||
1. Move the deploy secrets (`SSH_HOST`, `SSH_USER`, `SSH_KEY`, `SSH_PORT`) into
|
||
Forgejo's repo Secrets (Settings → Actions → Secrets).
|
||
2. Re-point the LAN dev runner: `bash deploy/forgejo/register-lan-runner.sh
|
||
<forgejo_url> <token>` — stops/disables the old GitHub runner service on
|
||
that same machine and registers `forgejo-runner` in its place, same
|
||
`systemd --user` pattern as before (see `DEPLOY-DEV.md` for the original;
|
||
this supersedes it for the runner specifically).
|
||
3. Open a test PR against `dev` in Forgejo, confirm the required check runs,
|
||
auto-merge fires, and the LAN deploy lands.
|
||
4. Only once that's solid: disable/archive the `.github/workflows/*.yml`
|
||
triggers (or the whole GitHub repo) — GitHub was the fallback during this
|
||
whole migration; retire it deliberately, not by accident.
|
||
|
||
## Verification checklist
|
||
- [ ] `ssh agent@<prod>` and `ssh agent@<beta>` both work; `sudo whoami` → `root`
|
||
- [ ] Password SSH auth confirmed dead on both boxes
|
||
- [ ] `docker node ls` (from prod) shows both nodes `Ready`
|
||
- [ ] Swarm ports (2377/7946/4789) unreachable from outside the WireGuard tunnel
|
||
- [ ] `https://git.thermograph.org` (or your chosen domain) serves Forgejo over TLS
|
||
- [ ] A test PR: required check runs → auto-merges → LAN dev deploy lands
|
||
- [ ] GitHub retired only after all of the above hold
|