The SSR frontend's _origin() (canonical, og:url, og:image, robots, sitemap
<loc>) and the backend content API's _origin() (the jsonld url folded into
each payload) both trusted x-forwarded-proto / request.url.scheme. Behind
Caddy — which terminates TLS and reverse-proxies plain HTTP — those read
"http", so on the HTTPS-only public site every absolute URL emitted http://,
which 308-redirects to https://: canonicals self-conflict and the sitemap
lists redirecting URLs.
Take the scheme from the configured public origin (THERMOGRAPH_BASE_URL, set
per-host in the deploy env) when the request arrives on that host, so prod and
beta emit https. localhost and LAN dev never match the public host and keep the
observed scheme, so plain-HTTP development is unchanged; the frontend keeps its
X-Forwarded-Host precedence for the proxy-fallback path.
Tests assert canonical/og:url/og:image, robots Sitemap and every sitemap <loc>
are https for the public host (and via X-Forwarded-Host), that a LAN host stays
http, and that the backend payload's jsonld url is likewise https.
Open-Meteo's daily high/low for the current day aggregates only the hours
elapsed so far, so grading it reads a still-unfolding day as complete and
produces spurious extremes — a cool morning served as a 1st-percentile
record-low high. The MET Norway primary already guards this with its
diurnal-coverage gate; the Open-Meteo fallback had no equivalent, so a
fleet-wide failover onto it exposed the bug.
Use the utc_offset_seconds Open-Meteo reports for a timezone=auto request to
identify the cell's local today and exclude it from the bundle. Past days are
complete and future days are whole-day forecasts, so only today is dropped;
the day lands in the record once it is over. When no offset is reported the
guard is skipped rather than guessing a date.
After a VAPID keypair rotation, subscribers minted under the old key
silently stopped receiving notifications while the UI still reported
"on", and the dead rows were never cleaned up.
Frontend (enable): a browser holding an existing PushSubscription never
re-subscribed, so it kept using the old applicationServerKey. Now the
current server VAPID key is always fetched and compared against the
subscription's baked-in key; on a mismatch the stale subscription is
unsubscribed and re-created with the new key. The matching-key path is
unchanged.
Backend (send): a rotated key makes the push service reject delivery
with 401/403, which returned "error" and left the row in place forever.
Treat 401/403 as permanently dead alongside 404/410 so the caller prunes
the row. Genuinely transient failures (rate limits, 5xx) still return
"error" and keep the row.
Also correct the default VAPID contact to mailto:admin@thermograph.org
(the .app domain was a typo); the env override is unchanged.
Extends tests/notifications/test_push.py with the send() status mapping
and the contact default.
api_grade and api_day parsed the `date` query param with an unguarded
datetime.date.fromisoformat, so a malformed or non-calendar value
(notadate, 2026-13-40, 2026-02-30) raised ValueError and surfaced as a
500. Route the parse through a _parse_target_date helper that maps the
ValueError to HTTPException(422); absent/empty and valid dates behave
exactly as before. Add a route-level test asserting 422 on bad dates.
After a cell's archive advances, the first content request recomputes a 45yr payload cold. Add warm_cities.warm_content — idempotent, cache-only, bounded — that pre-populates the content derived-store under the same (kind,key,content_token) the routes read, wired into the leader-gated notifier loop (50 cities/30-min tick) so the ~1000-city set stays warm within hours of each daily advance.
The three content handlers loaded the full ~45yr archive before checking the cache, so even hits paid the load. Restructure to resolve cell + compute content_token cheaply first, check the store, and load history only inside the miss-only build() closure. Cache hits and 304s now return without touching the archive. Keys/kinds/ETag unchanged.
Content pages keyed cache validity on the full-history hist_end, invalidating ~1-2x/day and paying a 45yr load even to compute the token. Add content_token(cell_id) = PAYLOAD_VER:CONTENT_VER:max_date, backed by an indexed MAX(date) (climate_store.history_max_date / climate.history_max_date), so it survives intra-day top-ups and needs no full-history load.
main took two direct PRs (#19 shell-lint, #21 the thermograph-daemon Go
service) while dev accumulated the ERA5 lake stack; both sides added compose/
stack services and touched the same seams. Resolutions keep both worlds:
- compose + Swarm stack: lake AND daemon are sibling services; backend env
carries THERMOGRAPH_LAKE_URL and THERMOGRAPH_INTERNAL_TOKEN.
- deploy.sh: backend deploys roll (backend lake daemon); all adds frontend.
- requirements.txt: websockets/apscheduler stay removed (moved to the Go
daemon), duckdb stays (the lake role's engine).
Merged tree: backend suite green, both compose configs validate.
The gateway bot and APScheduler were long-lived stateful I/O loops running
inside the async web app under a leader election. They move into a single Go
binary that owns ONLY that I/O -- websocket, RESUME, heartbeat, backoff, timers.
It owns no grading logic. Anything needing data calls back over a new
internal-only surface (/internal/discord/grade, /internal/jobs/*). Grading
depends on polars and the parquet cache; reimplementing it in Go would let the
bot's grades drift from the API's. The grade route returns gateway-ready JSON
and Go relays the bytes verbatim.
The binary ships in the backend image and runs as a second compose service off
the same tag, so the two ends of the /internal/* contract can never skew.
deploy.sh rolls daemon alongside backend -- without that the service would never
be created, since a single-service deploy uses --no-deps. It also probes the
image first and skips the daemon when rolling a tag that predates the binary:
infra tracks main while image tags are env-staged, so a host can legitimately be
asked to roll an older backend image, and creating the service anyway would
leave a container crash-looping on a missing binary.
replicas: 1 with order: stop-first replaces the leader election -- Discord
permits one gateway connection per bot token.
THERMOGRAPH_INTERNAL_TOKEN is optional: both ends derive it from
THERMOGRAPH_AUTH_SECRET via HMAC under a domain-separation label, so this needs
no new vault entry. The derivation is pinned to a shared cross-language test
vector asserted on both sides, so drift fails CI instead of 401ing every call.
Fail closed when neither secret is set.
Improvements over the Python: a close intended for RESUME uses 4000 rather than
1000 (Discord invalidates a session closed 1000, so the old default defeated its
own resume); MESSAGE_CREATE runs on a bounded worker pool; and a malformed HELLO
returns an error rather than a clean reconnect, which would otherwise reset
backoff and hot-loop against the gateway.
365 Python tests pass; Go build/vet/test -race clean; shellcheck 0 findings.
Adds .forgejo/workflows/shell-lint.yml (pinned shellcheck v0.11.0 + sha256, -x,
default severity, fail on any finding, not path-filtered) and drives all 26
scripts to zero findings.
Two defects shellcheck cannot see:
render-secrets.sh left DECRYPTED vault contents in /tmp whenever a sops decrypt
failed -- the caller's set -e aborted the function before either cleanup ran.
Now removed on every exit path, with `|| return 1` on both sops calls so a
decrypt failure can never write a partial /etc/thermograph.env regardless of the
caller's shell options. Explicitly not a `trap ... RETURN`: such a trap set in a
sourced function persists into the caller's shell and re-fires when the caller's
next `.`/source completes, where the function-local tmp is unset -- fatal and
silent under deploy.sh's set -u. The file now records that reasoning.
autoscale.sh ran `set -eu` without pipefail while piping docker stats into awk,
so a failed left side was swallowed and the loop autoscaled on empty input.
Promoted to pipefail with a missed sample treated as a skip; verified busybox ash
in docker:27-cli supports it.
Also: capture-fixtures.sh's `jq . || cat` ran cat after jq had consumed stdin,
silently writing truncated fixtures; deploy.sh/deploy-stack.sh `# shellcheck
source=` paths corrected for the monorepo layout.
Steady-state usage of the Open-Meteo API is removed; it remains only as a dormant fallback. Geocoding -> local GeoNames + Nominatim; wind gusts -> Meteostat; history -> NASA POWER (curated cells seeded with keyless ERA5); recent+forecast -> NASA range + MET Norway. Plus a drift-check tool comparing NASA vs Open-Meteo. All keyless, no new infra. Backend suite green in-image (gate).
One root workflow set replaces the four repos' copies (deleted -- root-only
is where Forgejo reads them, and dead copies are a trap): per-domain
build-push with explicit image paths (emi/thermograph/backend|frontend; the
old github.repository-derived path collides in a monorepo), path-filtered
per-domain beta/prod/dev deploys, a domain-input reusable build check, a
single always-reporting PR gate (path-filtered required checks deadlock
auto-merge), a new infra-sync pipeline (host checkout + secrets render on
infra/** pushes), and ports of secrets-guard / ops-cron /
observability-validate to monorepo paths.
