0a848e39af
Records the concrete, live state of Track 1 rather than just the setup procedure: both boxes' roles/IPs, the ssh invocation, where the private key lives (~/.ssh/thermograph_agent_ed25519, same convention DEPLOY.md already uses for the CI deploy key), and how to rotate it. Also notes that neither box has Docker yet and Terraform hasn't been applied to either, which is why Track 2 is currently paused. Checks off the two verification-checklist items that are actually confirmed now (SSH access, password auth dead) rather than leaving them unchecked.
131 lines
6.9 KiB
Markdown
131 lines
6.9 KiB
Markdown
# Agent access, Docker Swarm, and Forgejo CI/CD
|
||
|
||
This covers three additive infrastructure layers on top of the two VPS boxes
|
||
Terraform already provisions (see `terraform/README.md` and
|
||
`terraform.tfvars.example`, where they're named **prod** — the new 48 GB /
|
||
12-core box, `thermograph.org` — and **beta** — the old VPS,
|
||
`75.119.132.91`). **None of this touches `backend/`, `Dockerfile`,
|
||
`docker-compose*.yml`, `terraform/`, or `deploy/db/`** — those stay owned by
|
||
the app-containerization work; this is strictly the layer above it.
|
||
|
||
```
|
||
Track 1: deploy/provision-agent-access.sh — a dedicated full-root login for me
|
||
Track 2: deploy/swarm/ — Swarm cluster spanning both boxes
|
||
Track 3: deploy/forgejo/ + .forgejo/ — Forgejo + Forgejo Actions, replacing GitHub
|
||
```
|
||
|
||
## Track 1 — Agent access
|
||
|
||
Run `sudo bash deploy/provision-agent-access.sh` on **both** boxes. Creates a
|
||
dedicated `agent` user (not raw root login — a distinct name gives a clean
|
||
audit trail) with passwordless sudo, installs the agent's public key, disables
|
||
SSH password auth fleet-wide, and turns on `auditd` logging of every
|
||
root-effective command. See the script's header comment for the full
|
||
rationale and revocation steps (one line to delete, or delete the whole
|
||
account — your own access is never affected).
|
||
|
||
### Current access state (live)
|
||
|
||
Both boxes are provisioned and reachable as of this writing:
|
||
|
||
| Host | Role | Public IP | Login |
|
||
|------|------|-----------|-------|
|
||
| prod | manager (Swarm), Thermograph's eventual home | `169.58.46.181` | `agent` |
|
||
| beta | worker (Swarm), old VPS | `75.119.132.91` | `agent` |
|
||
|
||
```
|
||
ssh -i ~/.ssh/thermograph_agent_ed25519 agent@169.58.46.181 # prod
|
||
ssh -i ~/.ssh/thermograph_agent_ed25519 agent@75.119.132.91 # beta
|
||
```
|
||
|
||
The private half of the agent's dedicated keypair lives at
|
||
`~/.ssh/thermograph_agent_ed25519` on the operator's own machine (never in
|
||
this repo, never in a CI secret) — same convention `DEPLOY.md` already uses
|
||
for the separate CI deploy key (`~/.ssh/thermograph_ci`). If you're setting
|
||
this up fresh elsewhere, generate a new pair the same way
|
||
(`ssh-keygen -t ed25519 -a 100 -C "claude-agent@thermograph-infra" -f
|
||
~/.ssh/thermograph_agent_ed25519 -N ""`), embed the new public half in
|
||
`AGENT_PUBKEY` at the top of `provision-agent-access.sh`, and re-run it on
|
||
both boxes — that replaces the authorized key wholesale rather than
|
||
appending, so an old key stops working the moment the new one lands.
|
||
|
||
**Keep the private key somewhere that survives** — not a scratch/temp
|
||
directory that gets cleaned up, not anywhere-in-repo. Losing it just means
|
||
regenerating and re-running the provisioning script; it does not lock either
|
||
box, since your own login is untouched.
|
||
|
||
As of this writing, neither box has Docker installed yet and Terraform has
|
||
not been applied to either (`prod` is a bare OS install; `beta` has an old
|
||
`/opt/thermograph` checkout but no Docker) — Track 2 is paused on that.
|
||
|
||
## Track 2 — Swarm
|
||
|
||
See `deploy/swarm/README.md` for the exact order of operations (WireGuard
|
||
tunnel first, then swarm init/join, then lock the Swarm ports down to the
|
||
tunnel interface, then label beta for Forgejo placement). This cluster's only
|
||
job is hosting Forgejo — it does not orchestrate the Terraform-managed app
|
||
deploys.
|
||
|
||
## Track 3 — Forgejo, replacing GitHub
|
||
|
||
### 3a. Stand up Forgejo
|
||
See `deploy/forgejo/README.md` — deploys the stack, walks through minting the
|
||
two Swarm secrets it needs (`forgejo_db_password`, `forgejo_runner_token`).
|
||
|
||
### 3b. Migrate the repo (mirror first, verify, then cut over)
|
||
1. In Forgejo: **+ New Migration → GitHub**. Point it at
|
||
`https://github.com/griffemi/thermograph`, pull code + issues + PRs +
|
||
releases + labels. This is non-destructive to GitHub — do this as a mirror
|
||
and leave GitHub live and untouched until you're confident.
|
||
2. Spot-check: branch list matches, a few PRs/issues render correctly, LFS (if
|
||
any) came across.
|
||
3. **Don't retarget any secret or disable a GitHub workflow yet** — see 3d.
|
||
|
||
### 3c. Workflows
|
||
`.forgejo/workflows/*.yml` already mirror `.github/workflows/*.yml` in this
|
||
PR — copied with the mechanical changes Forgejo needs:
|
||
- `runs-on: ubuntu-latest` → `runs-on: docker` (the label the Swarm-hosted
|
||
runner in `deploy/forgejo/docker-stack.yml` registers under).
|
||
- `appleboy/ssh-action` referenced by full GitHub URL (not mirrored in
|
||
Forgejo's default action registry, unlike `actions/checkout` /
|
||
`actions/setup-python`, which resolve unchanged).
|
||
- **The custom auto-merge workflow step is gone.** It existed only because
|
||
GitHub's native branch protection/auto-merge is paywalled on private
|
||
free-tier repos. Forgejo has no such tier — configure it directly:
|
||
**Settings → Branches → protect `dev` → Enable Status Check → list `build`
|
||
→ merge style: squash.** Then a ready PR gets a real "Auto merge when checks
|
||
succeed" button.
|
||
- One knock-on simplification: a native auto-merge is an ordinary git push
|
||
(unlike GitHub's token-based merge commit, which doesn't re-trigger `push`),
|
||
so `deploy-dev.yml`'s push trigger fires naturally after auto-merge — no
|
||
more "direct push vs. PR-merge" double-trigger logic to maintain.
|
||
- `.forgejo/workflows/deploy.yml` still targets `main` faithfully mirroring
|
||
the current GitHub workflow, but `terraform.tfvars.example` describes prod's
|
||
branch as `release`. That mismatch is flagged in the file's own header
|
||
comment — reconcile with whoever's driving the Terraform/deploy side rather
|
||
than guessing here.
|
||
|
||
### 3d. Cut over
|
||
Only after 3a–3c are verified:
|
||
1. Move the deploy secrets (`SSH_HOST`, `SSH_USER`, `SSH_KEY`, `SSH_PORT`) into
|
||
Forgejo's repo Secrets (Settings → Actions → Secrets).
|
||
2. Re-point the LAN dev runner: `bash deploy/forgejo/register-lan-runner.sh
|
||
<forgejo_url> <token>` — stops/disables the old GitHub runner service on
|
||
that same machine and registers `forgejo-runner` in its place, same
|
||
`systemd --user` pattern as before (see `DEPLOY-DEV.md` for the original;
|
||
this supersedes it for the runner specifically).
|
||
3. Open a test PR against `dev` in Forgejo, confirm the required check runs,
|
||
auto-merge fires, and the LAN deploy lands.
|
||
4. Only once that's solid: disable/archive the `.github/workflows/*.yml`
|
||
triggers (or the whole GitHub repo) — GitHub was the fallback during this
|
||
whole migration; retire it deliberately, not by accident.
|
||
|
||
## Verification checklist
|
||
- [x] `ssh agent@<prod>` and `ssh agent@<beta>` both work; `sudo whoami` → `root`
|
||
- [x] Password SSH auth confirmed dead on both boxes (tested with an actual
|
||
failed login attempt, not just config inspection)
|
||
- [ ] `docker node ls` (from prod) shows both nodes `Ready`
|
||
- [ ] Swarm ports (2377/7946/4789) unreachable from outside the WireGuard tunnel
|
||
- [ ] `https://git.thermograph.org` (or your chosen domain) serves Forgejo over TLS
|
||
- [ ] A test PR: required check runs → auto-merges → LAN dev deploy lands
|
||
- [ ] GitHub retired only after all of the above hold
|