Merge pull request 'forgejo: make the desktop runner unit restart-always, fix the vps2 config path' (#157) from feat/forgejo-runners into dev
All checks were successful
Sync infra to hosts / sync-beta (push) Has been skipped
Sync infra to hosts / sync-prod (push) Has been skipped
Sync infra to hosts / sync-centralis (push) Has been skipped
Sync infra to hosts / sync-dev (push) Successful in 6s
secrets-guard / encrypted (push) Successful in 5s
shell-lint / shellcheck (push) Successful in 18s
PR build (required check) / changes (pull_request) Successful in 7s
shell-lint / shellcheck (pull_request) Successful in 7s
secrets-guard / encrypted (pull_request) Successful in 11s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 38s
PR build (required check) / validate-observability (pull_request) Successful in 45s
PR build (required check) / gate (pull_request) Successful in 1s

This commit is contained in:
Emi Griffith 2026-08-01 18:52:51 +00:00
commit 9f9c436424
2 changed files with 30 additions and 3 deletions

View file

@ -93,6 +93,13 @@ cat > "$HOME/.config/systemd/user/forgejo-runner.service" <<EOF
[Unit]
Description=Forgejo Actions runner (docker + thermograph-lan)
After=network-online.target
# These bound the Restart=always loop below, so a genuinely broken config (bad
# token, docker unreachable) ends as a stopped unit rather than spinning
# forever. They belong in [Unit]: systemd accepts them in [Service] without
# complaint but IGNORES them, leaving the 10s default — \`systemctl --user show
# forgejo-runner -p StartLimitIntervalUSec\` is how you tell which one you got.
StartLimitIntervalSec=300
StartLimitBurst=5
[Service]
WorkingDirectory=${RUNNER_DIR}
@ -102,7 +109,16 @@ WorkingDirectory=${RUNNER_DIR}
# SOPS-configured) fails with "not installed" even though it plainly is.
Environment=PATH=%h/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
ExecStart=${RUNNER_DIR}/forgejo-runner daemon -c config.yaml
Restart=on-failure
# \`always\`, NOT \`on-failure\` — the same distinction that took Forgejo down for
# 27 hours on 2026-07-29 (see docker-stack.yml's restart_policy comment): an
# always-on daemon that exits **0** is not "finished successfully", it is a
# daemon that stopped and must come back. \`on-failure\` cannot tell those apart,
# and forgejo-runner exits 0 on several paths — a lost instance connection it
# gives up on, or a SIGTERM from a Docker restart it interprets as a clean
# shutdown. The failure mode is silent: the unit sits \`inactive (dead)\`, the UI
# shows the runner offline, and every protected-branch merge blocks on a check
# that will never be produced.
Restart=always
RestartSec=5
[Install]

View file

@ -49,13 +49,24 @@ deploys.
So this directory is the source of truth; vps2 gets a copy:
The two files do **not** go to the same place, and getting it wrong fails in a
confusing way. `docker-compose.yml` mounts `./data:/data` and starts the daemon
with `--config /data/config.yaml`, so `/data` *is* `data/` — a `config.yaml`
sitting beside the compose file is invisible to the container. The daemon then
either refuses to start or silently runs on defaults (no `--add-host`, so every
registry push fails as if the credential were wrong).
```bash
install -d -m 0755 /opt/forgejo-runner
install -d -m 0755 /opt/forgejo-runner/data
cp /opt/thermograph/infra/deploy/forgejo/runner-vps2/docker-compose.yml \
/opt/thermograph/infra/deploy/forgejo/runner-vps2/config.yaml \
/opt/forgejo-runner/
cp /opt/thermograph/infra/deploy/forgejo/runner-vps2/config.yaml \
/opt/forgejo-runner/data/
```
`data/` is also where `register` writes `.runner`, so the runner's whole state —
config plus registration — lives in the one directory the compose file mounts.
Re-copy after changing either file here, then `docker compose up -d`. There is
no automation for that step and there should not be: a workflow that redeploys
the runner runs *on* the runner.