Jinemi/thermograph
4
0
Fork 0
Code Issues 2 Pull requests 10 Projects Releases Packages Wiki Activity Actions

runbook: chown the new Caddy log before reloading
All checks were successful
PR build (required check) / changes (pull_request) Successful in 6s
Details
secrets-guard / encrypted (pull_request) Successful in 5s
Details
PR build (required check) / build-backend (pull_request) Has been skipped
Details
shell-lint / shellcheck (pull_request) Successful in 7s
Details
PR build (required check) / build-frontend (pull_request) Has been skipped
Details
PR build (required check) / validate-observability (pull_request) Has been skipped
Details
PR build (required check) / gate (pull_request) Successful in 1s
Details

Browse source 
`caddy validate` run as root creates the log files the config names, owned by
root. The caddy user then cannot open them and the reload fails with
'permission denied' on the log writer — which is exactly what happened on the
live cutover, and what the root-owned centralis.log in that directory is a
fossil of.

Caddy rejected the config atomically and kept serving prod on the old one, so
nothing broke; but the step reads as a no-op until it isn't.
This commit is contained in:
Emi Griffith 2026-07-26 00:16:36 -07:00
parent 1aeea6bdc5
commit 24b7ef4c86
1 changed files with 11 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
infra/deploy/RUNBOOK-vps1-vps2-cutover.md
View file

@ -247,12 +247,23 @@ serving; nothing has moved yet.
``` ```
ssh agent@169.58.46.181 ssh agent@169.58.46.181
sudo caddy validate --config /etc/caddy/Caddyfile # BEFORE reloading sudo caddy validate --config /etc/caddy/Caddyfile # BEFORE reloading
# `validate` runs as root and CREATES any log file the config names, owned by
# root — which the caddy user then cannot open, and the reload fails. Fix the
# ownership before reloading, or the first reload rejects the whole config:
sudo chown caddy:caddy /var/log/caddy/beta.log
sudo systemctl reload caddy sudo systemctl reload caddy
``` ```
Validate before reload, always: a malformed Caddyfile takes `thermograph.org` Validate before reload, always: a malformed Caddyfile takes `thermograph.org`
down with it, and prod is on this box now. down with it, and prod is on this box now.
The `chown` is not optional and it bit this cutover. `caddy validate` as root
left `/var/log/caddy/beta.log` as `root:root`, so the reload failed with
`permission denied` on the log writer. (The root-owned `centralis.log` sitting
in that directory is the same mistake, made earlier.) Note what did NOT happen:
Caddy rejects a bad config atomically and keeps serving the old one, so
`thermograph.org` stayed up throughout — verify it anyway.
Caddy cannot issue the certificate until DNS moves (step 7), so expect the beta Caddy cannot issue the certificate until DNS moves (step 7), so expect the beta
hostname to fail TLS until then. That is fine and expected. hostname to fail TLS until then. That is fine and expected.