infra/deploy/forgejo/README.md

@@ -41,6 +41,15 @@ Do this **before** the DNS + Caddy step below — Caddy's reverse_proxy target
 (`127.0.0.1:3080`) needs the `forgejo` service actually listening first, or
 its first health check just fails harmlessly until it is.
 
+This stack has **no auto-deploy trigger** — nothing in `.forgejo/workflows/`
+redeploys it on push. A change to `docker-stack.yml` only takes effect once
+someone re-runs `docker stack deploy` by hand on the manager (prod).
+
+`db`/`forgejo` both carry `resources.limits` (defaults: db 1 CPU/1g, forgejo 2
+CPU/2g — several times observed steady-state usage), overridable with
+`FORGEJO_DB_CPUS`/`FORGEJO_DB_MEMORY`/`FORGEJO_CPUS`/`FORGEJO_MEMORY` env vars
+before `docker stack deploy`, same convention as the app stack.
+
 ## DNS + TLS: reusing beta's existing Caddy, not a second reverse proxy
 
 Forgejo is pinned to beta (`role=forge`) — but beta is also **today's live
@@ -98,6 +107,39 @@ See that script's header for exactly what it replaces (the pre-Forgejo GitHub
 self-hosted runner on this same machine) and why it registers with two
 labels where there used to be two separate runners.
 
+## Custom CI job image (`ci-runner/`)
+
+`ci-runner/Dockerfile` replaces the stock `node:20-bookworm` image most
+`docker`-labeled workflow jobs run in. That base is leftover from when the
+frontend was Python/Jinja and the old GitHub runner also built JS assets for
+it — the frontend is Go now, nothing in `.forgejo/workflows/` uses npm/node.
+Every `docker`-labeled build-push job also currently re-installs the Docker
+CLI on each run (`apt-get install docker.io`), which pulls in the classic
+builder rather than BuildKit (the classic builder mishandles `COPY
+--chown=<name>` group resolution — a real bug hit during the frontend Go
+rewrite). `ci-runner` is a slim Debian base with `docker-ce-cli` +
+`docker-buildx-plugin` (BuildKit) preinstalled, plus `git`/`python3`/
+`python3-yaml` for the other jobs that need them (`shell-lint`,
+`observability-validate`).
+
+Built and verified locally; **not yet pushed** — pushing to
+`git.thermograph.org/emi/thermograph/ci-runner` needs a PAT with
+`write:package` scope (the embedded git-remote token lacks it, same
+requirement documented in `backend-build-push.yml`). Once pushed:
+
+```bash
+docker build -t git.thermograph.org/emi/thermograph/ci-runner:v1 deploy/forgejo/ci-runner
+docker push git.thermograph.org/emi/thermograph/ci-runner:v1
+```
+
+then cut the live runner over by editing the `labels` array in
+`~/forgejo-runner/.runner` on the desktop (same runner id/token, no
+re-registration) and updating `register-lan-runner.sh`'s `LABELS` default so
+future re-provisioning matches. Only after that cutover is verified working
+should the now-redundant `apt-get install docker.io` / `python3-yaml` steps be
+removed from the workflows that had them — removing them first would break
+every job still running on `node:20-bookworm`.
+
 ## Why Postgres here and not the Thermograph app's TimescaleDB
 
 Separate instance, separate network (`forgejo_net`, not the app's compose

infra/deploy/forgejo/ci-runner/Dockerfile

@@ -0,0 +1,41 @@
+# Job-container image for the LAN Forgejo Actions runner's `docker`-labeled
+# jobs (see ../register-lan-runner.sh) — used by every backend/frontend
+# build-push, deploy, and validation workflow that needs `docker build`.
+#
+# Replaces the stock node:20-bookworm image. That base is pure leftover from
+# when the frontend was a Python/Jinja app the old GitHub runner also built
+# JS assets for; the frontend is Go now (no npm/node anywhere in
+# .forgejo/workflows/) so carrying a full Node runtime here just adds weight.
+# Every workflow using the `docker` label also currently re-provisions the
+# Docker CLI itself on each run via `apt-get install docker.io` — that step
+# costs ~15-20s per job and, more importantly, installs the CLASSIC Docker
+# builder rather than BuildKit, which mishandles `COPY --chown=<name>` group
+# resolution on images without a matching system group (a real bug hit during
+# the frontend Go rewrite). Installing docker-buildx-plugin here makes
+# BuildKit the default, closing that bug class, on a much smaller base.
+#
+# Build/push (manual — see ../README.md for when to rebuild):
+#   docker build -t git.thermograph.org/emi/thermograph/ci-runner:vN \
+#     infra/deploy/forgejo/ci-runner
+#   docker push git.thermograph.org/emi/thermograph/ci-runner:vN
+#
+# Cutting over the live runner to a new tag means editing the `labels` array
+# in ~/forgejo-runner/.runner on the runner host directly (same runner
+# id/token, no re-registration needed) and updating register-lan-runner.sh's
+# LABELS default so future re-provisioning picks it up too.
+FROM debian:bookworm-slim
+
+RUN apt-get update -qq \
+    && apt-get install -y -qq --no-install-recommends \
+        ca-certificates curl gnupg git python3 python3-yaml \
+    && install -m 0755 -d /etc/apt/keyrings \
+    && curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
+    && chmod a+r /etc/apt/keyrings/docker.asc \
+    && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian bookworm stable" \
+        > /etc/apt/sources.list.d/docker.list \
+    && apt-get update -qq \
+    && apt-get install -y -qq --no-install-recommends docker-ce-cli docker-buildx-plugin \
+    && rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/docker.list
+
+RUN docker --version && docker buildx version && git --version \
+    && python3 -c "import yaml; print('PyYAML', yaml.__version__)"

infra/deploy/forgejo/docker-stack.yml

@@ -48,6 +48,10 @@ services:
     deploy:
       placement:
         constraints: [node.labels.role == forge]
+      resources:
+        limits:
+          cpus: "${FORGEJO_DB_CPUS:-1}"
+          memory: ${FORGEJO_DB_MEMORY:-1g}
       restart_policy:
         condition: on-failure
 
@@ -131,6 +135,10 @@ services:
     deploy:
       placement:
         constraints: [node.labels.role == forge]
+      resources:
+        limits:
+          cpus: "${FORGEJO_CPUS:-2}"
+          memory: ${FORGEJO_MEMORY:-2g}
       restart_policy:
         condition: on-failure