thermograph/infra/deploy/forgejo
Emi Griffith ae1d9bb534 Subtree-merge thermograph-infra (origin/main) into infra/
git-subtree-dir: infra
git-subtree-mainline: d6df04eab2
git-subtree-split: 99b4b3f78d
2026-07-22 22:01:11 -07:00
..
caddy-git.conf Subtree-merge thermograph-infra (origin/main) into infra/ 2026-07-22 22:01:11 -07:00
docker-stack.yml Subtree-merge thermograph-infra (origin/main) into infra/ 2026-07-22 22:01:11 -07:00
README.md Subtree-merge thermograph-infra (origin/main) into infra/ 2026-07-22 22:01:11 -07:00
register-lan-runner.sh Subtree-merge thermograph-infra (origin/main) into infra/ 2026-07-22 22:01:11 -07:00

Forgejo on the Swarm cluster

Runs as deploy/forgejo/docker-stack.yml — the only Swarm-scheduled workload this cluster carries (the Thermograph app itself stays on the Terraform-managed docker compose deploys; see terraform/README.md). Pinned to the beta node (old VPS) via the role=forge label from deploy/swarm/label-forge-node.sh.

The Actions runner is deliberately not part of this stack — it runs on the desktop as a plain systemd service (register-lan-runner.sh below), per thermograph-docs/runbooks/implementation-handoff.md Track B step 5. That's the canonical placement; an earlier revision of this stack ran the runner as a Swarm-scheduled Docker-in-Docker sidecar pinned to beta, which is gone now.

Prerequisites

  1. All three nodes have joined the swarm (deploy/swarm/) and beta is labeled role=forge.
  2. docker node ls (from the manager) shows all three Ready.

One-time setup: Swarm secret

One secret the stack expects to already exist (a Swarm secret, not a file — external: true in the stack file, so docker stack deploy never creates or sees the value, only references it):

# A strong random password for Forgejo's own Postgres (NOT related to
# Thermograph's app database — entirely separate instance/network).
openssl rand -base64 32 | docker secret create forgejo_db_password -

Deploy / update

docker stack deploy -c deploy/forgejo/docker-stack.yml forgejo

Re-running is safe — Swarm only touches services whose spec actually changed. Do this before the DNS + Caddy step below — Caddy's reverse_proxy target (127.0.0.1:3080) needs the forgejo service actually listening first, or its first health check just fails harmlessly until it is.

DNS + TLS: reusing beta's existing Caddy, not a second reverse proxy

Forgejo is pinned to beta (role=forge) — but beta is also today's live thermograph.org host, and its Caddy already owns ports 80/443 (/etc/caddy/Caddyfile on that box). A second ingress (Traefik) trying to bind the same ports would collide with it. So there's no Traefik in this stack: forgejo's web port publishes to 127.0.0.1:3080 only (host-local), and beta's existing Caddy gets one more site block reverse-proxying to it — same pattern as its thermograph.org block, same automatic-HTTPS.

  1. Point the Forgejo domain (default git.thermograph.org; override with FORGEJO_DOMAIN=... before docker stack deploy) at beta's public IP — that's where the task actually runs, not prod's or the desktop's.
  2. Append deploy/forgejo/caddy-git.conf to beta's /etc/caddy/Caddyfile, adjusting the domain if you didn't use the default, then systemctl reload caddy.
  3. That file also resolves the registry-exposure hazard (#15 in thermograph-docs/runbooks/hop1-forgejo-registry-cutover.md): /v2/* (the registry API) is blocked to everything except the WireGuard mesh CIDR (10.10.0.0/24); the git/web UI stays public. CI runners and Swarm nodes reach the registry over the mesh, not the public internet — see "Registry access from mesh clients" below.

Registry access from mesh clients

Any node that needs docker login/push/pull against the registry (the desktop's CI runner building/pushing images, later any Swarm node pulling them) must reach git.thermograph.org over the WireGuard tunnel, not beta's public IP — otherwise Caddy's /v2/* block above refuses the connection. Public DNS resolves the domain to beta's public IP, so add a /etc/hosts override on each such node pinning it to beta's WireGuard address instead:

echo "10.10.0.2 git.thermograph.org" | sudo tee -a /etc/hosts

(10.10.0.2 is beta's WG address per deploy/swarm/README.md's peer numbering — adjust if you assigned it differently.) The git/web UI keeps working normally for everyone else since only /v2/* is restricted.

Register the Actions runner (on the desktop, not through Swarm)

Once Forgejo answers at its domain:

# On the desktop:
#   Forgejo web UI -> Site Administration -> Actions -> Runners -> Create new Runner
#   (or, for a repo-scoped runner: <repo> -> Settings -> Actions -> Runners)
#   copy the registration token, then:
bash deploy/forgejo/register-lan-runner.sh https://<forgejo-domain> <token>

See that script's header for exactly what it replaces (the pre-Forgejo GitHub self-hosted runner on this same machine) and why it registers with two labels where there used to be two separate runners.

Why Postgres here and not the Thermograph app's TimescaleDB

Separate instance, separate network (forgejo_net, not the app's compose network), separate volume. Forgejo is a distinct product with its own schema and its own backup/restore lifecycle — sharing a database with the app would couple two things that should be able to fail, migrate, and restore independently.

Verifying

docker service ls                       # forgejo_db, forgejo_forgejo both Running, 1/1
curl -I https://git.thermograph.org/    # 200, valid cert (Caddy's, not a new one)
curl -I https://git.thermograph.org/v2/ # 403 from anywhere off the WireGuard mesh
# On the desktop, after registering the runner:
systemctl --user status forgejo-runner   # active, both labels registered

Rollback / removal

docker stack rm forgejo
# volumes (forgejo_data, forgejo_db, ...) survive a stack rm — remove them
# explicitly only if you actually want to destroy the Forgejo instance's data.