7 commits

Author	SHA1	Message	Date
Emi Griffith	53e2eb7c84	registry: move the registry host to dev.jinemi.com, MCP to mcp.jinemi.com ... Completes the Forgejo domain migration. ROOT_URL moved to dev.jinemi.com earlier; the registry half was deliberately deferred. Every image name, REGISTRY_HOST default, runner label and --add-host pin now names dev.jinemi.com, so the registry host and the bearer-token realm agree again. Both names address the same Forgejo, so no image needs re-pushing and a rollback to a tag pushed under the old prefix still resolves. git.thermograph.org therefore stays served off the same Caddy site block -- one block, so the /v2/* mesh-only matcher keeps covering both names -- for pre-migration tags and for runners holding it as their registered instance URL. Mesh clients now pin both names in /etc/hosts: the new one as registry host and token realm, the old one for pre-migration tags. runner-vps2/config.yaml carries both --add-host entries for the same reason. Also renames Centralis' endpoint to mcp.jinemi.com in the two places this repo names it; Centralis itself is provisioned outside this repo. Host-side steps this cannot do (documented in deploy/forgejo/README.md, "Host-side steps"): the Forgejo Actions variable REGISTRY_HOST, docker login against the new host, and the /etc/hosts pins.	2026-08-01 16:06:22 -07:00
Emi Griffith	a83a0b7d32	forgejo: make dev.jinemi.com canonical (ROOT_URL and OAuth callback) ... Forgejo derives every absolute URL from its single ROOT_URL, so this is what moves clone URLs, the Google OAuth callback, webhook payload URLs and mail links onto the new domain. Both redirect URIs are registered on the Google OAuth client and were verified against Google before the flip — login is SSO-only, so a mismatch locks everyone out of the UI including out of undoing it. README documents the probe so the check is repeatable rather than assumed. git.thermograph.org stays on the same Caddy site block and stays load-bearing: the registry host baked into image names, the mesh /etc/hosts pins that make the /v2/* matcher see a mesh source IP, and the runners' registered instance URL all still use it. Renaming the registry is a separate migration; the docs now say so where someone would otherwise retire the name as dead weight. The stack has no auto-deploy, so this takes effect on a manual `docker stack deploy` from the manager.	2026-08-01 09:32:12 -07:00
Emi Griffith	001e6b1365	ci: add an always-on Actions runner on vps2 ... The estate had exactly one registered runner, on the desktop. It went offline at 2026-07-31 16:31Z; for the next 21 hours no PR could satisfy a required check, no deploy could run, and the 03:00Z ops-cron -- the only backup for both application databases and for Forgejo -- did not fire. Forgejo queued that scheduled run rather than dropping it, so it completed on reconnect and nothing was lost. A longer outage would have meant real gaps. Three files claimed an "always-on Swarm-hosted runner" existed and that the estate therefore no longer depended on the desktop. It did not exist: an early revision of docker-stack.yml ran one as a Docker-in-Docker sidecar and it was removed. That claim is why a single point of failure sat unnoticed. Corrected in docker-stack.yml, forgejo/README.md and register-lan-runner.sh. The new runner is a plain restart:always container, not a Swarm service: a Swarm-scheduled runner cannot redeploy the Swarm that schedules it, so CI would be gone exactly when the cluster is what is broken. It runs from /opt/forgejo-runner rather than in place, because the checkout is reset on every prod deploy and one `git clean -fdx` there would destroy the registration. vps2 runs prod, so the socket mount is bounded rather than assumed benign: capacity 1, --cpus=2/--memory=4g on job containers, valid_volumes empty so no job can bind-mount /etc/thermograph.env, and no thermograph network joined. This defends against accident, not against a hostile workflow author -- stated plainly in the compose header rather than implied. The desktop runner stays registered as extra capacity. Nothing may assume it is up.	2026-08-01 07:54:28 -07:00
Emi Griffith	8a2c838663	forgejo: restart the db service on a clean exit, not just on failure ... forgejo_db has been at 0/1 replicas since 2026-07-29 01:24 UTC. Postgres hit an invalid data-directory lock file ("could not open file postmaster.pid ... performing immediate shutdown because data directory lock file is invalid") and exited 0. With restart_policy.condition=on-failure, Swarm read the zero status as successful completion, marked the task Complete, and never rescheduled it. The forgejo service itself stayed Up and kept serving its homepage, so the outage presented as every repository page, the whole API and all CI returning 500 with "dial tcp: lookup db on 127.0.0.11:53: no such host" — including the auth path, which is why API calls reported "user does not exist [uid: 0]" rather than a database error. on-failure cannot distinguish "finished successfully" from "shut itself down and should be restarted", and Postgres exits 0 on several such paths, so it is the wrong policy for an always-on stateful service. This is the durable fix; it does not restart the currently stopped task.	2026-07-30 06:30:30 -07:00
emi	d138f00a20	infra: split the estate into vps1/vps2 — beta joins prod, dev gets a home (#103)	2026-07-26 06:56:38 +00:00
emi	a4ecb51401	web/worker: add a process-level liveness heartbeat (#80)	2026-07-25 04:13:47 +00:00
Emi Griffith	ae1d9bb534	Subtree-merge thermograph-infra (origin/main) into infra/ ... git-subtree-dir: infra git-subtree-mainline: d6df04eab2 git-subtree-split: 99b4b3f78d	2026-07-22 22:01:11 -07:00