Commit graph

8 commits

Author SHA1 Message Date
Emi Griffith
9eecfc8eef frontend: rewrite the SSR content service in Go
All checks were successful
PR build (required check) / changes (pull_request) Successful in 6s
secrets-guard / encrypted (pull_request) Successful in 7s
PR build (required check) / build-backend (pull_request) Has been skipped
shell-lint / shellcheck (pull_request) Successful in 7s
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-frontend (pull_request) Successful in 1m0s
PR build (required check) / gate (pull_request) Successful in 2s
Ports frontend/ (Jinja2/FastAPI, ~1180 LOC) to Go with html/template.
No climate math, no DB, no auth here -- every route fetches from the
backend's /content/* API, so this is I/O-bound glue with no hard-porting
wall; the risk was always in reproducing the rendering exactly, not the
language.

Verified with a golden-HTML diff, not just unit tests: both the Python
original and the Go rewrite were run against the same committed fixtures
(frontend/tests/fixtures/) and every one of the 11 routes compared
byte-for-byte. The only surviving differences after that process are
insignificant inter-tag whitespace and one attribute where Go's stricter
escaper HTML-encodes an apostrophe Jinja left literal (functionally
identical in every browser) -- confirmed programmatically by normalizing
whitespace and unescaping before diffing, not by eyeballing.

That process caught defects unit tests alone would have missed, because
map[string]any has no compile-time field check:

- Render-context keys were snake_case throughout (content.py's Jinja
  convention, ported verbatim) while the templates -- written
  independently -- read PascalCase fields. A missing map key doesn't
  error in html/template, it silently renders empty, so this was invisible
  in every status code and every "it built" signal: title, meta
  description, canonical URL, OpenGraph tags, the homepage's entire ranked
  list, and the brand-tag/nav-active state were all blank across every
  page. Fixed by renaming every key to match each template's own header
  comment (the authoritative per-page field contract) and, where an
  API struct's exported fields already matched what a template needed
  (contentapi.CityInfo, Crumb, HomeRanked, HubCountry, ...), passing the
  struct straight through instead of hand-rewrapping it in a map --
  removes a whole layer of future drift risk, not just this instance of it.
- Three pages 500'd outright: `.ToolHref` needed a fully-composed href
  string, not the bare "lat,lon" fragment the handlers were building; the
  all-time-records table needed the raw contentapi.AllTimeRecords struct,
  not a re-wrapped map.
- JSON-LD was being double-encoded: `<script type="application/ld+json">`
  is JAVASCRIPT context to html/template's contextual escaper regardless
  of the script's `type` attribute, so a template.HTML-typed value placed
  there gets re-escaped as a quoted JS string instead of emitted raw --
  the entire structured-data payload shipped as a JSON string containing
  JSON, which no crawler would parse as the intended object. Needed
  template.JS instead, the type that actually means "trusted JS source."
  The glossary term page's JSON-LD was simply never built at all (the
  Jinja original assembled it inline in the template rather than through
  content.py's context dict, and that got lost in translation) -- added.
- html/template silently strips literal HTML comments AND JavaScript
  comments from the parsed output (verified in isolation, zero template
  actions involved) -- confirmed as real engine behavior, not a bug in
  either port, so both need a FuncMap function returning template.HTML /
  template.JS respectively to survive parsing rather than a literal
  `<!-- -->` or `//` in the template source.

Packaging: multi-stage Go build, final image alpine (not distroless -- the
Swarm stack's env-entrypoint.sh shim needs bash), 187MB -> 22.6MB. Two
defects caught before they reached a host:
- The Swarm stack overrides `entrypoint:` with no `command:`, which drops
  the image's own CMD entirely (Docker/Swarm semantics, not merged) --
  env-entrypoint.sh then fell through to its hardcoded `exec uvicorn
  app:app` fallback, which doesn't exist in this image. Every deploy
  would have exited 127. Fixed with an explicit `command:` on the stack's
  frontend service, and corrected the shim's stale comment claiming CMD
  passes through automatically.
- `COPY --chown=thermograph` resolves the group by NAME at copy time;
  Alpine's `adduser -S` with no `-G` doesn't create a same-named group, so
  the classic (non-BuildKit) Docker builder -- which this CI runner falls
  back to, since it installs plain `docker.io` with no buildx plugin --
  failed outright. Fixed with an explicit group and numeric --chown.

Verification: go build/vet/test -race clean across all packages; the
Docker image builds and passes its embedded go test step under both
BuildKit and the classic builder; shellcheck 0 findings on the one script
touched; rebased onto current main (the ERA5 lake stack landed on both
main and dev during this work -- confirmed additive, no overlap with
frontend/daemon).
2026-07-23 17:51:31 -07:00
Emi Griffith
30b0c70bbb Reconcile: merge main (shellcheck guard, Go daemon) into dev
All checks were successful
secrets-guard / encrypted (pull_request) Successful in 6s
PR build (required check) / changes (pull_request) Successful in 8s
shell-lint / shellcheck (pull_request) Successful in 9s
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-frontend (pull_request) Successful in 1m9s
PR build (required check) / build-backend (pull_request) Successful in 1m44s
PR build (required check) / gate (pull_request) Successful in 2s
main took two direct PRs (#19 shell-lint, #21 the thermograph-daemon Go
service) while dev accumulated the ERA5 lake stack; both sides added compose/
stack services and touched the same seams. Resolutions keep both worlds:

- compose + Swarm stack: lake AND daemon are sibling services; backend env
  carries THERMOGRAPH_LAKE_URL and THERMOGRAPH_INTERNAL_TOKEN.
- deploy.sh: backend deploys roll (backend lake daemon); all adds frontend.
- requirements.txt: websockets/apscheduler stay removed (moved to the Go
  daemon), duckdb stays (the lake role's engine).

Merged tree: backend suite green, both compose configs validate.
2026-07-23 17:06:52 -07:00
emi
8e09155aaf Run the ERA5 lake service in every environment (#20)
All checks were successful
secrets-guard / encrypted (push) Successful in 6s
2026-07-23 22:32:13 +00:00
emi
de8e847f9f shell: add shellcheck CI guard and drive the tree to zero findings (#19)
All checks were successful
Build + push backend image (Forgejo registry) / build-push (push) Successful in 1m23s
Build + push frontend image (Forgejo registry) / build-push (push) Successful in 1m22s
Sync infra to hosts / sync-beta (push) Successful in 9s
Sync infra to hosts / sync-prod (push) Successful in 8s
secrets-guard / encrypted (push) Successful in 6s
shell-lint / shellcheck (push) Successful in 9s
Deploy backend to beta VPS / deploy (push) Successful in 2m4s
Deploy frontend to beta VPS / deploy (push) Successful in 1m8s
Adds .forgejo/workflows/shell-lint.yml (pinned shellcheck v0.11.0 + sha256, -x,
default severity, fail on any finding, not path-filtered) and drives all 26
scripts to zero findings.

Two defects shellcheck cannot see:

render-secrets.sh left DECRYPTED vault contents in /tmp whenever a sops decrypt
failed -- the caller's set -e aborted the function before either cleanup ran.
Now removed on every exit path, with `|| return 1` on both sops calls so a
decrypt failure can never write a partial /etc/thermograph.env regardless of the
caller's shell options. Explicitly not a `trap ... RETURN`: such a trap set in a
sourced function persists into the caller's shell and re-fires when the caller's
next `.`/source completes, where the function-local tmp is unset -- fatal and
silent under deploy.sh's set -u. The file now records that reasoning.

autoscale.sh ran `set -eu` without pipefail while piping docker stats into awk,
so a failed left side was swallowed and the loop autoscaled on empty input.
Promoted to pipefail with a missed sample treated as a skip; verified busybox ash
in docker:27-cli supports it.

Also: capture-fixtures.sh's `jq . || cat` ran cat after jq had consumed stdin,
silently writing truncated fixtures; deploy.sh/deploy-stack.sh `# shellcheck
source=` paths corrected for the monorepo layout.
2026-07-23 22:26:05 +00:00
emi
af839c6cd0 Finish the domain-sha tag keying: three deploy workflows were missed (#6)
All checks were successful
secrets-guard / encrypted (push) Successful in 6s
PR build (required check) / changes (pull_request) Successful in 6s
secrets-guard / encrypted (pull_request) Successful in 5s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 42s
PR build (required check) / gate (pull_request) Successful in 3s
2026-07-23 13:58:57 +00:00
emi
0b46844cf5 Key image tags to the last domain-touching commit, not the branch tip (#3)
All checks were successful
secrets-guard / encrypted (push) Successful in 5s
secrets-guard / encrypted (pull_request) Successful in 6s
PR build (required check) / changes (pull_request) Successful in 9s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 54s
PR build (required check) / gate (pull_request) Successful in 3s
2026-07-23 13:42:28 +00:00
Emi Griffith
a9ceb8f03f CI: port the dev-branch in-image test step into the reusable build check
Backend runs its full hermetic suite, frontend its unit tier, inside the
just-built image -- the dev-only feature both app repos carried in their own
build.yml (deleted here in favor of the root workflow).
2026-07-22 22:24:31 -07:00
Emi Griffith
7b2db07722 CI: port the split repos' workflows to per-domain path-filtered monorepo pipelines
One root workflow set replaces the four repos' copies (deleted -- root-only
is where Forgejo reads them, and dead copies are a trap): per-domain
build-push with explicit image paths (emi/thermograph/backend|frontend; the
old github.repository-derived path collides in a monorepo), path-filtered
per-domain beta/prod/dev deploys, a domain-input reusable build check, a
single always-reporting PR gate (path-filtered required checks deadlock
auto-merge), a new infra-sync pipeline (host checkout + secrets render on
infra/** pushes), and ports of secrets-guard / ops-cron /
observability-validate to monorepo paths.
2026-07-22 22:11:33 -07:00