Promote dev to main: geocode P0 fix + ops tooling + privacy logging #52

Merged

admin_emi merged 11 commits from dev into main 2026-07-24 19:48:41 +00:00

Conversation 0 Commits 11 Files changed 20 +1392 -73

admin_emi commented 2026-07-24 19:46:13 +00:00

Owner

Copy link

Includes the geocode_nominatim NameError fix (#51, P0 — live 502s on the Nominatim-fallback geocode path for ~30h), the render-secrets RETURN-trap fix (#50), ops/iceberg.sh + ops/dbq.sh (#23, #18), and the healthz/IP-privacy logging change (#35), plus content-cache work already landed by other in-flight work (#47-#49).

Includes the geocode_nominatim NameError fix (#51, P0 — live 502s on the Nominatim-fallback geocode path for ~30h), the render-secrets RETURN-trap fix (#50), ops/iceberg.sh + ops/dbq.sh (#23, #18), and the healthz/IP-privacy logging change (#35), plus content-cache work already landed by other in-flight work (#47-#49).

admin_emi added 11 commits 2026-07-24 19:46:14 +00:00

shell: add shellcheck CI guard and drive the tree to zero findings ... 40a3ce21d9

No static analysis has ever run over the ~2k lines of shell that deploy,
provision secrets, and bootstrap hosts as root over SSH. Add shell-lint.yml
(pinned shellcheck v0.11.0 + sha256, -x, default severity, fail on any
finding) and fix everything it reports, plus two defects it structurally
cannot see.

Not path-filtered, matching secrets-guard's call: a backstop that only runs
when you expect it to isn't a backstop. Scripts are discovered with find, so
new ones are covered on landing. The version is pinned to a static release
rather than apt's, so a drifted shellcheck can't fail CI on an unrelated push.

render-secrets.sh: the mktemp holding DECRYPTED vault contents was only
removed on the success path and one failure branch, so a sops decrypt failure
left plaintext POSTGRES_PASSWORD in /tmp indefinitely on a live host. A RETURN
trap makes removal unconditional, and the two sops calls now `|| return 1`
explicitly instead of relying on the caller's set -e (a bare set -e abort
skips the trap). The function stays free of `set -e` itself -- it is sourced,
and shell options would leak into the caller.

autoscale.sh: ran `set -eu` without pipefail while piping docker stats into
awk, so a failed left side was swallowed and the loop scaled on empty input.
Promoted to pipefail with avg_cpu's failure treated as a missed sample, so a
daemon hiccup can't kill the autoscaler. Verified busybox ash in docker:27-cli
supports pipefail and the script still parses there.

capture-fixtures.sh: `jq . || cat` ran cat after jq had already consumed
stdin, silently writing a truncated fixture; now a real if/else that fails
loudly. deploy.sh/deploy-stack.sh: `# shellcheck source=` paths corrected for
the monorepo layout, and /etc/thermograph.env marked unfollowable (it is
rendered at deploy time and cannot exist at lint time).

render-secrets: drop the RETURN trap, it broke deploy silently ... 56be0dd9b7

The RETURN trap added in the previous commit leaked out of the function and
killed every deploy on a SOPS-configured host.

A RETURN trap set inside a SOURCED function is not function-scoped: it persists
in the caller's shell after the function returns, and a RETURN trap also fires
when a `.`/source completes. deploy.sh sources /etc/thermograph.env six lines
after calling render_thermograph_secrets, which re-fired the trap at top level
where `tmp` -- function-local -- is unset. Under deploy.sh's `set -u` that is
fatal, and silent: that line already sends stderr to /dev/null, so the deploy
rendered secrets and then died with no diagnostic before pulling or rolling
anything.

Replaced with explicit `rm -f "$tmp"` on each exit path, plus a comment
recording why the tidier-looking trap is wrong here so it doesn't come back.
The original defect the trap was meant to fix stays fixed: the decrypt-failure
path removes the plaintext temp file before returning 1.

The write section now captures its status in `rc` and cleans up once, rather
than ending on `rm` -- as the last command it was masking a failed in-place
`cat` write to status 0, so a half-written /etc/thermograph.env would have
deployed as if it succeeded.

Verified in a container against the real call pattern (strict-mode caller,
source lib, call, then source the rendered env): success path returns 0 and the
caller survives the subsequent source; decrypt-failure path aborts the caller
with no /etc/thermograph.env written; both leave zero temp files.

Stop logging /healthz + internal SSR hop, truncate logged IPs (#35) 4ff113f2cf

ops/dbq.sh: read-only Postgres queries across all environments (#18) 864d38d772

Add cheap, stable content-page cache token (#47) ... a4fa72639f

Content pages keyed cache validity on the full-history hist_end, invalidating ~1-2x/day and paying a 45yr load even to compute the token. Add content_token(cell_id) = PAYLOAD_VER:CONTENT_VER:max_date, backed by an indexed MAX(date) (climate_store.history_max_date / climate.history_max_date), so it survives intra-day top-ups and needs no full-history load.

Check derived store before loading history on content routes (#48) ... 6fef57f05f

The three content handlers loaded the full ~45yr archive before checking the cache, so even hits paid the load. Restructure to resolve cell + compute content_token cheaply first, check the store, and load history only inside the miss-only build() closure. Cache hits and 304s now return without touching the archive. Keys/kinds/ETag unchanged.

Pre-warm the SEO content derived-store off-request (#49) ... bf0aaf1ec0

After a cell's archive advances, the first content request recomputes a 45yr payload cold. Add warm_cities.warm_content — idempotent, cache-only, bounded — that pre-populates the content derived-store under the same (kind,key,content_token) the routes read, wired into the leader-gated notifier loop (50 cities/30-min tick) so the ~1000-city set stays warm within hours of each daily advance.

ops/iceberg.sh: read-only Iceberg lake queries across all environments (#23) 67d1734e99

Merge remote-tracking branch 'origin/dev' into shell-lint-merged ca7bae5933

Merge pull request 'render-secrets: drop the RETURN trap, it broke deploy silently' (#50) from shell-lint into dev 5f0f5990ae

data/climate: fix geocode_nominatim's NameError on every call (P0, live) (#51) cda6e75b38

admin_emi merged commit 0935eea394 into main 2026-07-24 19:48:41 +00:00

admin_emi referenced this pull request from a commit 2026-07-24 19:48:43 +00:00

Merge pull request 'Promote dev to main: geocode P0 fix + ops tooling + privacy logging' (#52) from dev into main

admin_emi referenced this pull request 2026-07-24 19:49:41 +00:00

Promote main to release: geocode P0 fix + ops tooling + privacy logging #53

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

  

Compat/Breaking

Breaking change that won't be backward compatible

  

Kind/Bug

Something is not working

  

Kind/Documentation

Documentation changes

  

Kind/Enhancement

Improve existing functionality

  

Kind/Feature

New functionality

  

Kind/Security

This is security issue

  

Kind/Testing

Issue or pull request related to testing

  

Priority

Critical

The priority is critical

  

Priority

High

The priority is high

  

Priority

Low

The priority is low

  

Priority

Medium

The priority is medium

  

Reviewed

Confirmed

Issue has been confirmed

  

Reviewed

Duplicate

This issue or pull request already exists

  

Reviewed

Invalid

Invalid issue

  

Reviewed

Won't Fix

This issue won't be fixed

  

Status

Abandoned

Somebody has started to work on this but abandoned work

  

Status

Blocked

Something is blocking this issue or pull request

  

Status

Need More Info

Feedback is required to reproduce issue or to continue work

No labels

Compat/Breaking

Kind/Bug

Kind/Documentation

Kind/Enhancement

Kind/Feature

Kind/Security

Kind/Testing

Priority

Critical

Priority

High

Priority

Low

Priority

Medium

Reviewed

Confirmed

Reviewed

Duplicate

Reviewed

Invalid

Reviewed

Won't Fix

Status

Abandoned

Status

Blocked

Status

Need More Info

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Jinemi/thermograph#52

No description provided.