Three operator decisions change v1's premises, so the design is now two tiers
with separate flags:
Tier A (THERMOGRAPH_EVENTS) is v1 unchanged -- anonymous hourly aggregates, no
identifier, nothing stored on the device, no consent needed, runs for everyone.
It stays the primary signal precisely so a poor consent rate cannot take the
numbers down with it, and so Tier B's rates can be calibrated against it to
measure the consent bias rather than ignoring it.
Tier B (THERMOGRAPH_EVENT_SESSIONS) adds an ephemeral per-tab id -- 16 random
bytes in sessionStorage, rotating on a 30-minute idle and a 2-hour absolute cap,
capped at 200 events, never linked to another session, device, or to the account
(api_event does not read the auth cookie and there is no user column). Rows land
in a 30-day hypertable with minute-granularity timestamps and an in-session
sequence number instead of precise clock times. Storing an identifier engages
ePrivacy Art. 5(3) and analytics is not strictly necessary, so it is gated on
opt-in consent: an equal-weight banner that mints nothing before "Allow",
one-click withdrawal in every footer that drops the live id immediately, and
GPC/DNT treated as a refusal already given.
THERMOGRAPH_SEARCH_MISS captures zero-result search text server-side in
api_suggest only -- normalised, rejected outright on any personal-data smell,
stored as a per-day count, and pruned below a three-person floor after a week.
The privacy page is rewritten rather than deferred: "no per-visitor identifier"
becomes false the moment Tier B ships. Tests assert the load-bearing promises so
the copy and the code cannot drift apart silently.
Raw-IP truncation is NOT implemented here (the logging pipeline owns it) but
UI-EVENTS.md states what the app side must do, and records that Caddy's default
JSON log already stores full request URIs -- so every search query is in Loki
with the client IP today, which the search-miss mitigations depend on fixing.
UI-EVENTS.md carries the v1-to-v2 diff, the consent reasoning including why
legitimate interest is not available, and an explicit argument that the session
identifier is the wrong trade at this traffic volume.
Extends the existing /api/v2/event beacon into a small, typed, durable event
schema so the interactive views can answer product questions the request log
cannot: whether a visitor ever gets a location, whether search finds anything,
which controls and views earn their maintenance, and which dead ends people
actually hit.
Seven events with three enum dimension slots, stored as hourly aggregates in a
TimescaleDB hypertable plus one JSONL line per event for the 30-day Loki view.
No row per interaction and no column an identifier could go in: no cookie, no
session id, no user id, no IP, no coordinates, no free text, no URLs. The IP is
a per-minute rate-limit key and nothing else.
Abuse resistance for a public endpoint on a 60%-crawler site: closed allowlist
(unknown names collapse to one bucket), 4 KB streaming body cap, per-IP ceiling
raised to 120/min (30 was sized for four coarse events and would have silently
truncated a batched stream), soft Sec-Fetch-Site first-party check, uniform 204.
Ships inert -- THERMOGRAPH_EVENTS gates both the recorder and the flag stamp on
<html> that makes the client send anything, and is unset everywhere. See
UI-EVENTS.md for the schema, the rejected transport/storage alternatives, and
the privacy decisions that must be settled before the flag is turned on.
Steady-state usage of the Open-Meteo API is removed; it remains only as a dormant fallback. Geocoding -> local GeoNames + Nominatim; wind gusts -> Meteostat; history -> NASA POWER (curated cells seeded with keyless ERA5); recent+forecast -> NASA range + MET Norway. Plus a drift-check tool comparing NASA vs Open-Meteo. All keyless, no new infra. Backend suite green in-image (gate).
The cutover moved the host checkout's deploy tree to /opt/thermograph/infra/deploy/,
but the stack file's absolute bind sources (db init, env-entrypoint.sh, autoscale.sh)
still pointed at the old /opt/thermograph/deploy/ -- a service update kept the stale
mounts and new tasks failed with 'bind source path does not exist'.
Backend runs its full hermetic suite, frontend its unit tier, inside the
just-built image -- the dev-only feature both app repos carried in their own
build.yml (deleted here in favor of the root workflow).
/opt/thermograph becomes a monorepo checkout: deploy.sh gains INFRA_DIR (git
ops at the root, compose work cd'd into infra/), lock/tags/render paths move
under infra/deploy/, image-path defaults become emi/thermograph/backend|
frontend across compose, stack, and the tag-prune. docker-compose.yml pins
name: thermograph -- without it compose run from .../infra derives project
"infra" and recreates the whole stack beside the running one;
deploy-dev.sh pins COMPOSE_PROJECT_NAME=thermograph-dev (env wins over the
file key) to keep LAN dev's separate project.
One root workflow set replaces the four repos' copies (deleted -- root-only
is where Forgejo reads them, and dead copies are a trap): per-domain
build-push with explicit image paths (emi/thermograph/backend|frontend; the
old github.repository-derived path collides in a monorepo), path-filtered
per-domain beta/prod/dev deploys, a domain-input reusable build check, a
single always-reporting PR gate (path-filtered required checks deadlock
auto-merge), a new infra-sync pipeline (host checkout + secrets render on
infra/** pushes), and ports of secrets-guard / ops-cron /
observability-validate to monorepo paths.
main received the perf hardening line (#5, #8) and its own landing of the
Discord gateway bot (#7) while dev carried a parallel port of the same bot
(#3). The two implementations are near-identical ports of the same archived
source; main's includes one extra hardening (grading calls moved off the
gateway event loop via asyncio.to_thread) and broader tests, so bot files
resolve wholesale to main's side. dev keeps its test-runner/smoke tooling and
the run-tests-in-image CI, which main lacks.
# Conflicts:
# notifications/discord_bot.py
# tests/notifications/test_discord_bot.py
# web/app.py
