No static analysis has ever run over the ~2k lines of shell that deploy,
provision secrets, and bootstrap hosts as root over SSH. Add shell-lint.yml
(pinned shellcheck v0.11.0 + sha256, -x, default severity, fail on any
finding) and fix everything it reports, plus two defects it structurally
cannot see.
Not path-filtered, matching secrets-guard's call: a backstop that only runs
when you expect it to isn't a backstop. Scripts are discovered with find, so
new ones are covered on landing. The version is pinned to a static release
rather than apt's, so a drifted shellcheck can't fail CI on an unrelated push.
render-secrets.sh: the mktemp holding DECRYPTED vault contents was only
removed on the success path and one failure branch, so a sops decrypt failure
left plaintext POSTGRES_PASSWORD in /tmp indefinitely on a live host. A RETURN
trap makes removal unconditional, and the two sops calls now `|| return 1`
explicitly instead of relying on the caller's set -e (a bare set -e abort
skips the trap). The function stays free of `set -e` itself -- it is sourced,
and shell options would leak into the caller.
autoscale.sh: ran `set -eu` without pipefail while piping docker stats into
awk, so a failed left side was swallowed and the loop scaled on empty input.
Promoted to pipefail with avg_cpu's failure treated as a missed sample, so a
daemon hiccup can't kill the autoscaler. Verified busybox ash in docker:27-cli
supports pipefail and the script still parses there.
capture-fixtures.sh: `jq . || cat` ran cat after jq had already consumed
stdin, silently writing a truncated fixture; now a real if/else that fails
loudly. deploy.sh/deploy-stack.sh: `# shellcheck source=` paths corrected for
the monorepo layout, and /etc/thermograph.env marked unfollowable (it is
rendered at deploy time and cannot exist at lint time).
The cutover moved the host checkout's deploy tree to /opt/thermograph/infra/deploy/,
but the stack file's absolute bind sources (db init, env-entrypoint.sh, autoscale.sh)
still pointed at the old /opt/thermograph/deploy/ -- a service update kept the stale
mounts and new tasks failed with 'bind source path does not exist'.
/opt/thermograph becomes a monorepo checkout: deploy.sh gains INFRA_DIR (git
ops at the root, compose work cd'd into infra/), lock/tags/render paths move
under infra/deploy/, image-path defaults become emi/thermograph/backend|
frontend across compose, stack, and the tag-prune. docker-compose.yml pins
name: thermograph -- without it compose run from .../infra derives project
"infra" and recreates the whole stack beside the running one;
deploy-dev.sh pins COMPOSE_PROJECT_NAME=thermograph-dev (env wins over the
file key) to keep LAN dev's separate project.
One root workflow set replaces the four repos' copies (deleted -- root-only
is where Forgejo reads them, and dead copies are a trap): per-domain
build-push with explicit image paths (emi/thermograph/backend|frontend; the
old github.repository-derived path collides in a monorepo), path-filtered
per-domain beta/prod/dev deploys, a domain-input reusable build check, a
single always-reporting PR gate (path-filtered required checks deadlock
auto-merge), a new infra-sync pipeline (host checkout + secrets render on
infra/** pushes), and ports of secrets-guard / ops-cron /
observability-validate to monorepo paths.
main received the perf hardening line (#5, #8) and its own landing of the
Discord gateway bot (#7) while dev carried a parallel port of the same bot
(#3). The two implementations are near-identical ports of the same archived
source; main's includes one extra hardening (grading calls moved off the
gateway event loop via asyncio.to_thread) and broader tests, so bot files
resolve wholesale to main's side. dev keeps its test-runner/smoke tooling and
the run-tests-in-image CI, which main lacks.
# Conflicts:
# notifications/discord_bot.py
# tests/notifications/test_discord_bot.py
# web/app.py
Two bugs surfaced wiring dev auto-deploy:
1. Sourcing .image-tags.env clobbered the caller's incoming *_IMAGE_TAG. The
first backend-only deploy persists FRONTEND_IMAGE_TAG=local (sibling unknown);
the next frontend deploy then sourced that and pulled :local -> 'manifest
unknown'. Now the incoming env is captured before sourcing and re-applied.
2. Health-checked services via a host-port curl, but the dev overlay leaves the
frontend port unpublished (reached via the backend proxy) -> false failure.
Now polls each container's own HEALTHCHECK status via docker inspect.
The CI/SSH deploy paths (deploy.yml/deploy-prod.yml over SSH, deploy-dev on the
LAN runner) don't pass REGISTRY_TOKEN and rely on the host already being
docker-logged-in; an unconditional login with an empty token aborts deploy.sh
under set -e. Skip the login when no token is set and trust the host cred;
pull still fails loudly on a real auth problem.
The split backend's Python package data/ lives at /app/data; mounting the
appdata runtime volume there erased data/*.py and broke import at boot. Point
THERMOGRAPH_DATA_DIR + the appdata mount + the singleton lock at /state,
outside the code tree. Pairs with thermograph-backend paths.py making
DATA_DIR/LOGS_DIR env-overridable.
After the repo split the code lives at the image root, so the Python package
data/ and the runtime data dir <root>/data are the same path (/app/data). The
deploy compose mounts the appdata volume there, shadowing data/*.py so
'import data.climate' fails at container boot. THERMOGRAPH_DATA_DIR (and
_LOGS_DIR) now let the container point runtime state at a path outside the code
tree; local dev is unchanged (defaults to <root>/data).
The split branch carried the #34-era render-secrets.sh, missing two fixes
required for prod/beta deploy: (1) sudo-read the root-owned 0400 age key into
SOPS_AGE_KEY when the deploy user can't read it directly; (2) in-place-write
/etc/thermograph.env when it's group-writable (beta's non-root 'deploy' user)
instead of only install(1). Without these, deploy.sh's render fails on both
hosts. Flagged by the terraform-layer workstream.
Ports the frontend/ portion of monorepo PR #50: a "what to expect" insight
card on the Weekly results (app.js, from data.climatology's rain_freq/
sun_freq) and a matching per-month-selection insight card on the Calendar
(calendar.js, from data.months_climate, incl. the cross-month "Nx more
likely to rain" comparison), plus the calendar.html mount point and
style.css card styling. The backend portion of #50 (climate.py/grading.py
computing rain_freq/sun_freq/months_climate) lives in the separate backend
repo and is out of scope here; this frontend code just consumes those
fields when the paired backend supports them. No fetch URLs were touched by
this change.
