7878763b66
All checks were successful
PR build (required check) / changes (pull_request) Successful in 5s
PR build (required check) / validate-observability (pull_request) Has been skipped
secrets-guard / encrypted (pull_request) Successful in 4s
PR build (required check) / build-backend (pull_request) Has been skipped
shell-lint / shellcheck (pull_request) Successful in 6s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / gate (pull_request) Successful in 1s
forgejo_db has been at 0/1 replicas since 2026-07-29 01:24 UTC. Postgres hit
an invalid data-directory lock file ("could not open file postmaster.pid ...
performing immediate shutdown because data directory lock file is invalid")
and exited 0. With restart_policy.condition=on-failure, Swarm read the zero
status as successful completion, marked the task Complete, and never
rescheduled it.
The forgejo service itself stayed Up and kept serving its homepage, so the
outage presented as every repository page, the whole API and all CI returning
500 with "dial tcp: lookup db on 127.0.0.11:53: no such host" — including the
auth path, which is why API calls reported "user does not exist [uid: 0]"
rather than a database error.
on-failure cannot distinguish "finished successfully" from "shut itself down
and should be restarted", and Postgres exits 0 on several such paths, so it is
the wrong policy for an always-on stateful service.
This is the durable fix; it does not restart the currently stopped task.
|
||
|---|---|---|
| .. | ||
| backup | ||
| db | ||
| forgejo | ||
| migrations | ||
| openmeteo | ||
| secrets | ||
| stack | ||
| swarm | ||
| twa | ||
| Caddyfile.vps1 | ||
| Caddyfile.vps2 | ||
| deploy-dev.sh | ||
| deploy.sh | ||
| env-topology.sh | ||
| migrate-db.py | ||
| POSTGRES-MIGRATION.md | ||
| provision-agent-access.sh | ||
| provision-dev.sh | ||
| provision-mail-supervision.sh | ||
| provision-mail.sh | ||
| provision-secrets.sh | ||
| render-secrets-openbao.sh | ||
| render-secrets.sh | ||
| RUNBOOK-vps1-vps2-cutover.md | ||
| thermograph-dev.service | ||
| thermograph.env.example | ||
| thermograph.service | ||