forgejo: record why the runner is on vps2, rather than leaving a contradiction
All checks were successful
PR build (required check) / changes (pull_request) Successful in 6s
secrets-guard / encrypted (pull_request) Successful in 5s
PR build (required check) / build-backend (pull_request) Has been skipped
PR build (required check) / build-frontend (pull_request) Has been skipped
shell-lint / shellcheck (pull_request) Successful in 7s
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / gate (pull_request) Successful in 1s
Sync infra to hosts / sync-beta (push) Has been skipped
Sync infra to hosts / sync-prod (push) Has been skipped
Sync infra to hosts / sync-dev (push) Successful in 5s
Sync infra to hosts / sync-centralis (push) Has been skipped
secrets-guard / encrypted (push) Successful in 4s
shell-lint / shellcheck (push) Successful in 7s

This README said, in bold, that a runner must never go on prod or beta. A
runner has been registered and running on vps2 since 2026-08-01. Both
statements cannot stand: an instruction the estate visibly ignores teaches
readers to ignore the next one too.

The original objection is kept rather than deleted, because it is correct on
its merits -- docker_host: automount hands job containers the host's Docker
socket, which on vps2 is root over both the prod and beta stacks. What changed
is that the alternative proved worse. The desktop was the ONLY registered
runner in the estate; when it dropped on 2026-07-31 nothing merged, nothing
deployed and the nightly backup did not fire for 21 hours. On a repo where
every branch is protected and every change is a PR, one absent runner freezes
everything, and the backup hangs off the same path.

So the section now records the trade and the bounds actually applied --
capacity 1, --cpus=2/--memory=4g on job containers, valid_volumes empty, no
thermograph network -- and states plainly that this is defence against
accident, not against a hostile workflow author. Forgejo's database already
stored VPS2_SSH_KEY, which is root on that box; what changed is that the
material is now reachable by a job rather than only at rest.

It also keeps the original advice for the case it was actually written for:
if you are adding CAPACITY rather than REDUNDANCY, raise capacity or use a box
that hosts nothing.
This commit is contained in:
Emi Griffith 2026-08-01 09:59:14 -07:00
parent 9e0128bf01
commit 486a194086

View file

@ -177,14 +177,48 @@ parallel, and even capacity 3 (an earlier, undocumented hand-tune) leaves
`build-backend`/`build-frontend`/`validate-observability` queued behind
those three before they get a slot.
**Adding more runner capacity should mean raising this number, or adding a
second runner alongside it — not putting a runner on prod or beta (vps2).**
`container.docker_host: automount` gives job containers the *host's* Docker
socket; on vps2 that would mean any CI job has root-equivalent access to both
the prod and beta stacks running there. An earlier revision of this stack
actually ran the runner as a Swarm-hosted container on the box Forgejo was
pinned to and was deliberately reverted for this same class of reason — see
the note at the top of `docker-stack.yml`.
**This paragraph used to say a runner must never go on vps2. That was reversed
deliberately on 2026-08-01, and the reasoning is worth keeping rather than
quietly deleting.**
The original objection stands on its merits: `container.docker_host: automount`
gives job containers the *host's* Docker socket, so a runner on vps2 means any
CI job has root-equivalent access to both the prod and beta stacks running
there. An earlier revision of this stack ran the runner as a Swarm-hosted
container on the Forgejo node and was reverted for the same class of reason.
What changed is that the alternative turned out to be worse. The desktop was
the estate's **only** registered runner, and when it went offline on
2026-07-31 nothing merged, nothing deployed, and the nightly backup did not
fire for 21 hours — on a repo where every branch is protected and every change
is a PR, a single absent runner freezes the whole estate. Availability of CI
is not a convenience here; it is what the backup and the deploy path both
hang off.
So the runner on vps2 (`runner-vps2/`) is a considered trade, not an
oversight, and it is bounded rather than assumed benign:
- `capacity: 1` — one job at a time, so a CI burst cannot contend with
`thermograph_web` for the box's six cores.
- `--cpus=2 --memory=4g` on job containers, set in `container.options`. Job
containers are siblings of the runner rather than children, so the runner's
own compose limits do not reach them; that option is the only lever that
does.
- `valid_volumes: []` — a job cannot bind-mount a host path, which is the
difference between "a job can read `/etc/thermograph.env`" and "a job
cannot", on the box where that file is prod's.
- The compose project joins no `thermograph` network.
Be honest about what that buys: it is defence against **accident**, not
against a hostile workflow author. Forgejo's own database on vps1 already
stores `VPS2_SSH_KEY`, which is root on vps2, so the material was already
reachable — what changed is that it is now reachable by a *job* rather than
only at rest. On a two-person estate where both people can already SSH to that
box as root, that is the honest boundary.
**If you are adding capacity rather than redundancy, raise `capacity` or add a
runner on a box that hosts nothing.** The vps2 runner exists so that CI has a
second home, not because prod is a good place to run CI.
## Custom CI job image (`ci-runner/`)