All checks were successful
PR build (required check) / changes (pull_request) Successful in 6s
secrets-guard / encrypted (pull_request) Successful in 5s
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-frontend (pull_request) Successful in 1m8s
PR build (required check) / build-backend (pull_request) Successful in 1m36s
PR build (required check) / gate (pull_request) Successful in 2s
A visitor who declines browser geolocation currently has no location at all — the copy sends them to the map picker. This adds an opt-in fallback that suggests a coarse city from the request's own IP, presented as a guess with a one-tap correction, so the dead end has a way out. backend/data/geoip.py does the lookup against a local MMDB file (DB-IP IP to City Lite or GeoLite2 City — same format, either works, attribution derived from the file's metadata). Off unless THERMOGRAPH_GEOIP is truthy AND the database exists AND maxminddb imports; every degraded case — private/reserved/ CGNAT/IPv6-ULA addresses, unparseable input, no record, a country-centroid record with no city, a record wider than the accuracy limit, a corrupt file — returns None, which the route answers as 204 and the client treats exactly as today. The IP is read in memory and dropped. GET /api/v2/geoip runs no RunAudit, records no metric dimension, and is excluded from the access log (its own "geoip" traffic category) so no stored, joinable (IP, location) pair is ever written. The response is no-store/Vary:* and takes no parameters. Client-side rather than server-rendered on purpose: the homepage's HTML and weak ETag must stay byte-identical for every visitor, or any cache added in front of it can serve one visitor's city to another. The suggestion is fetched only after a declined/unavailable prompt, and only when nothing is remembered. A guess is never persisted — no localStorage write, no URL hash — and the hero and results headings say "roughly near"/"near … approximate" rather than letting the reverse-geocoded cell name a neighbourhood the lookup never knew. The /privacy page's "never looks up your location from your IP address" paragraph now follows the same flag out of the same env file, so the published statement and the behaviour flip together. Database lifecycle is a host-side systemd timer (infra/deploy/geoip-refresh.*) that verifies a download opens and answers before swapping it in atomically, bind-mounted read-only; geoip.py re-opens on mtime change, so a refresh needs no restart. GEOIP-APPROX-LOCATION.md carries the database comparison, the SSR-vs-client argument, the privacy analysis, and the open decisions.
489 lines
28 KiB
Markdown
489 lines
28 KiB
Markdown
# Approximate-location fallback — design + prototype
|
||
|
||
Status: **prototype on `feat/geoip-approx-location`, feature-flagged OFF, not
|
||
deployed and not merged.** Nothing in this branch changes production behaviour
|
||
until `THERMOGRAPH_GEOIP` is set *and* a database file exists on the host.
|
||
|
||
This doc is the decision record. If it survives review it should move to
|
||
`thermograph-docs` (per the root `CLAUDE.md`: cross-cutting decision docs live
|
||
there); it sits here for now so the reasoning ships with the code it describes.
|
||
|
||
---
|
||
|
||
## 1. The problem
|
||
|
||
Location today comes from exactly one source: `navigator.geolocation.getCurrentPosition`
|
||
in `frontend/static/app.js`, gated on `window.isSecureContext`. If the visitor
|
||
declines the permission prompt, the prompt times out, or the browser won't offer
|
||
geolocation at all, the site has **no location** and the copy says "Pick a spot
|
||
on the map instead."
|
||
|
||
That is a dead end for the exact visitor the homepage is written for — someone
|
||
who arrived to answer "how unusual is *my* weather?" and has just been handed a
|
||
world map. This change gives that person something local, honestly labelled.
|
||
|
||
## 2. What was built
|
||
|
||
| Piece | File |
|
||
| --- | --- |
|
||
| Lookup module (flag, parsing, rejection rules, attribution) | `backend/data/geoip.py` |
|
||
| Endpoint `GET /api/v2/geoip` | `backend/web/app.py` (`api_geoip`) |
|
||
| Access-log exclusion + traffic category | `backend/web/app.py`, `backend/core/metrics.py` |
|
||
| Event names `home.geoip_shown` / `home.geoip_fixed` | `backend/core/metrics.py` |
|
||
| Client fetch + suggestion banner | `frontend/static/geoip.js` |
|
||
| Wiring, non-persistence, honest headings | `frontend/static/app.js` |
|
||
| Banner container (empty, hidden) | `frontend/templates/home.html.j2` |
|
||
| Banner styles | `frontend/static/style.css` |
|
||
| Flag-aware privacy copy | `frontend/content.py`, `frontend/templates/privacy.html.j2` |
|
||
| Host refresh job + timer | `infra/deploy/geoip-refresh.{sh,service,timer}` |
|
||
| Read-only DB mount + env plumbing | `infra/docker-compose.yml`, `infra/deploy/thermograph.env.example` |
|
||
| Tests | `backend/tests/data/test_geoip.py`, `backend/tests/web/test_geoip_api.py`, `frontend/tests/unit/test_geoip_ui.py` |
|
||
|
||
`cd backend && make test` → **449 passed, 8 skipped** (was 386/7).
|
||
`cd frontend && make test-unit` → **41 passed**.
|
||
|
||
---
|
||
|
||
## 3. Database choice — recommendation: **DB-IP IP to City Lite**
|
||
|
||
| | **DB-IP IP to City Lite** | MaxMind GeoLite2 City | IP2Location LITE DB5/DB11 |
|
||
| --- | --- | --- | --- |
|
||
| Licence | **CC BY 4.0** | GeoLite End User License Agreement (incorporates CC BY-SA 4.0 terms) | CC BY-SA 4.0 / "free with attribution" |
|
||
| Account needed | **No** — direct download URL | **Yes** — account + licence key | **Yes** — free account |
|
||
| Secret to manage | **None** | Licence key (→ SOPS vault, → CI, → rotation) | Account credentials |
|
||
| Attribution | `<a href="https://db-ip.com">IP Geolocation by DB-IP</a>` on pages showing results | "This product includes GeoLite Data created by MaxMind, available from https://www.maxmind.com" | "uses the IP2Location LITE database for IP geolocation", linked |
|
||
| Retention obligation | none | **must delete a database within 30 days of a newer release**; 30 downloads/day cap | none |
|
||
| Update cadence | monthly | weekly (Tue/Fri) | monthly |
|
||
| Size (current release) | 125 MB MMDB (685 MB CSV), ~8.05 M records | ~60–70 MB MMDB | 69 MB BIN (IPv4) + 171 MB BIN (IPv6), separate files |
|
||
| Format / reader | **MMDB → `maxminddb`** (pure-Python, mmap) | MMDB → `maxminddb` / `geoip2` | proprietary BIN → vendor `IP2Location` package (or CSV you index yourself) |
|
||
| Accuracy fields | city + lat/lon, **no `accuracy_radius`** | city + lat/lon + `accuracy_radius` | city + lat/lon |
|
||
|
||
**Recommend DB-IP IP to City Lite.** Reasoning, in priority order:
|
||
|
||
1. **No credential.** MaxMind means a licence key in the SOPS vault, in the
|
||
deploy path, and in a rotation story — new secret surface for a feature whose
|
||
entire justification is privacy hygiene. DB-IP needs none, so
|
||
`geoip-refresh.sh` reads nothing from the vault at all.
|
||
2. **No retention obligation.** GeoLite's EULA requires deleting a database
|
||
within 30 days of a newer release. That is a *compliance* clock on an ops
|
||
job, and a missed job becomes a licence breach rather than merely stale data.
|
||
CC BY 4.0 has no such clause.
|
||
3. **Same format, same reader.** Both are MMDB, so this is a swap of one file
|
||
and one URL, not a code change — `geoip.py` reads the raw record dict and
|
||
derives its attribution line from the file's own metadata. If accuracy turns
|
||
out to be materially worse in practice, moving to GeoLite2 is a config change
|
||
plus a vault entry, and the code is already written to accept it.
|
||
4. **Attribution is a link, not a legal paragraph.** One muted anchor in the
|
||
suggestion banner satisfies CC BY 4.0 and DB-IP's stated wording.
|
||
|
||
**Against IP2Location LITE**: a proprietary BIN format needing the vendor's own
|
||
Python package, IPv4 and IPv6 as *separate* databases (two files, two readers,
|
||
two refresh paths), an account to download, and CC BY-**SA** — a share-alike
|
||
term that is noise to reason about for a file we merely read.
|
||
|
||
**Cost of the DB-IP choice, stated plainly:** the Lite file carries no
|
||
`accuracy_radius`, so the "reject anything the database admits is too coarse"
|
||
guard degrades to "reject anything with no city name". That is still the guard
|
||
that matters (a country centroid is the damaging case), but it is weaker.
|
||
`suggestion_from_record()` applies the radius rule whenever the field *is*
|
||
present, so switching to GeoLite2 turns the stronger guard back on for free.
|
||
|
||
**Reader library:** `maxminddb` (pinned `2.8.2` in `backend/requirements.txt`),
|
||
not `geoip2`. `geoip2` adds a model layer over the same reader and type-checks
|
||
`database_type` against MaxMind's own names; we read four fields out of a plain
|
||
dict, and staying on `maxminddb` keeps DB-IP and MaxMind files equally usable.
|
||
It is imported *lazily inside* `geoip.py`, so a missing wheel disables the
|
||
feature rather than breaking boot.
|
||
|
||
---
|
||
|
||
## 4. Where it hooks in — recommendation: **client-side, on demand**
|
||
|
||
Two options were on the table.
|
||
|
||
**A. Server-side during SSR of `/`.** `frontend/content.py`'s `home_page()`
|
||
would look up the IP and render the guess into the first paint.
|
||
|
||
**B. Client-side after geolocation is declined/unavailable.** The browser calls
|
||
`GET /api/v2/geoip` and swaps the banner in. **This is what the prototype does.**
|
||
|
||
### Why B
|
||
|
||
**Caching is the deciding factor, and the current setup is more fragile than it
|
||
looks.** `frontend/content.py`'s `_respond_html()` returns HTML with a weak
|
||
ETag (`W/"<sha1 of the rendered body>"`) and **no `Cache-Control` and no
|
||
`Vary`**. The host Caddy (`infra/deploy/Caddyfile`) only does `encode` +
|
||
`reverse_proxy` — no cache module today. So right now nothing caches the
|
||
homepage, but nothing *declares* that it mustn't either.
|
||
|
||
Fold a location into that response and three things break at once:
|
||
|
||
- The ETag becomes a function of the visitor's city. Two visitors flap each
|
||
other's validators; conditional requests stop being useful, and a returning
|
||
visitor whose IP maps elsewhere gets a spurious 200 instead of a 304.
|
||
- Any cache added later — a Caddy `cache` directive, Cloudflare in front of the
|
||
apex, a corporate proxy on the visitor's side — will serve one visitor's city
|
||
to another, because the response carries nothing saying it varies. There is no
|
||
correct `Vary` header for this: it varies on `X-Forwarded-For`, which is not a
|
||
header a cache keys on, and `Vary: *` would make the homepage permanently
|
||
uncacheable for everyone to serve a fallback that only a minority need.
|
||
- The SSR path would also have to carry the client IP over the extra
|
||
frontend→backend hop (`api_client.py` currently forwards only `Host` and
|
||
`X-Forwarded-Proto`). That means propagating the IP into a second process's
|
||
request path — more places it can end up in a log, for no benefit.
|
||
|
||
**The first-paint argument is weaker here than it looks**, because the homepage
|
||
hero is *already* server-rendered with real content: the most unusual city
|
||
we're currently tracking, with its grade, band colour and percentile. There is
|
||
no blank space for the guess to fill. The suggestion swaps into a page that is
|
||
already complete, and it costs one small JSON request with no data dependency in
|
||
front of it.
|
||
|
||
**And SSR would do the lookup for the wrong people.** The server cannot see
|
||
`localStorage`, so it cannot know that most returning visitors already have a
|
||
saved place that is strictly better than any guess. An SSR implementation would
|
||
geolocate every homepage request, including the majority who will never use the
|
||
result — the opposite of "touch the IP as little as possible". Option B only
|
||
runs after the visitor has demonstrably ended up with nothing.
|
||
|
||
### What B costs
|
||
|
||
- **Needs JavaScript.** A no-JS visitor sees today's behaviour. Acceptable: the
|
||
entire interactive tool already needs JS.
|
||
- **One extra round trip** after the decline, ~200 bytes each way.
|
||
- **A 204 for everyone in the fallback path when the feature is off.** The
|
||
alternative — giving the frontend its own copy of the flag to decide whether
|
||
to fetch — is two flags to keep in sync and a new way for the UI and the
|
||
server to disagree. One flag, one 204, is the better trade.
|
||
|
||
### If you overrule this and want SSR anyway
|
||
|
||
Do it as a *separate* route (`/near` or `?near=1`), not on the cached `/`, and
|
||
give that route `Cache-Control: private, no-store`. Never make the canonical,
|
||
indexable homepage location-varying — that is also an SEO hazard: a crawler in
|
||
one datacentre would index a page claiming a city, and different crawler IPs
|
||
would see different "canonical" content.
|
||
|
||
---
|
||
|
||
## 5. Privacy design
|
||
|
||
### The IP is never persisted — mechanically, not by convention
|
||
|
||
- `geoip.lookup(ip)` takes a string, memory-maps a file, returns a dict. It
|
||
writes nothing, logs nothing, and never raises.
|
||
- `api_geoip` uses **no `RunAudit`** (which would record the derived lat/lon)
|
||
and adds **no metric dimension** (the counters are aggregate by category).
|
||
- The route is excluded from the access log. `metrics.classify_inbound()` now
|
||
returns a dedicated `"geoip"` category and the middleware in `web/app.py`
|
||
skips `audit.log_access` for it. Every other route still logs, and
|
||
`test_a_normal_route_still_is_logged` guards that exclusion from widening.
|
||
|
||
This is the important one. Today the access log stores a raw IP per request.
|
||
Without the exclusion, the *one* endpoint whose whole job is turning an IP
|
||
into a place would leave a stored, timestamped, joinable (IP,
|
||
approximate-location) pair on disk and in Loki. That artefact — not the
|
||
transient lookup — is what would actually constitute keeping location data
|
||
about identifiable people. The exclusion is not defence in depth; it is the
|
||
substance of the claim.
|
||
|
||
This does **not** depend on the separate access-log IP-truncation work. That
|
||
change is still worth doing, and it composes: truncation reduces what the
|
||
*other* routes retain, while this exclusion means the geoip route retains
|
||
nothing at all either way.
|
||
|
||
- `Cache-Control: no-store, private` + `Vary: *` on the response, so it cannot
|
||
be stored or reused by any intermediary.
|
||
- The endpoint takes **no parameters**. You cannot ask it about somebody else's
|
||
address; the only input is the connection you already own. `?ip=` is ignored
|
||
(tested).
|
||
|
||
### Why no consent banner is needed — and why the analytics work is different
|
||
|
||
Under GDPR the lookup is **transient processing to deliver a service the user
|
||
explicitly asked for**. The visitor pressed "Use my location" *wanting* local
|
||
weather; when the browser can't supply it, deriving a coarse city from the
|
||
connection they are already using to reach us is a proportionate substitute for
|
||
the same, requested purpose. Nothing is stored, nothing is profiled, nothing is
|
||
shared with a third party (the database is a file we host — no processor
|
||
relationship, no international transfer, no disclosure to name in the policy).
|
||
The data subject can override the result with one tap, and the result is
|
||
explicitly labelled a guess rather than presented as a finding about them.
|
||
|
||
**ePrivacy Art. 5(3) is not engaged at all**, and this is the cleanest part of
|
||
the argument. That article governs *storing information on, or gaining access to
|
||
information stored in, a user's terminal equipment* — which is why cookies,
|
||
localStorage and device fingerprinting need consent regardless of whether the
|
||
data is personal. An IP address arrives in the request headers as an unavoidable
|
||
property of routing a packet. We neither store anything on the device nor read
|
||
anything from it. There is nothing for a consent banner to be about.
|
||
|
||
**Contrast with the analytics session-id work.** That one deliberately *writes
|
||
an identifier onto the visitor's device* in order to correlate their actions
|
||
across a session. That is squarely Art. 5(3) storage/access, it is not strictly
|
||
necessary to deliver a service the user requested, and consent is therefore
|
||
required — hence `consent.js` and the consent gate on that branch. The two
|
||
features look adjacent ("both touch visitor data") and are legally opposite:
|
||
one reads a routing header and forgets it, the other plants a durable
|
||
identifier. Keep them separate in the code and in the privacy copy, and never
|
||
let the geoip fallback be folded behind the analytics consent gate — that would
|
||
imply consent is what makes it lawful, which is the wrong basis and would
|
||
silently break the fallback for everyone who declines analytics.
|
||
|
||
### The privacy page follows the flag
|
||
|
||
`frontend/templates/privacy.html.j2` currently says, in the visitor's own
|
||
words: *"Thermograph never looks up your location from your IP address."*
|
||
Turning this feature on makes that sentence **false**, and a stale privacy
|
||
statement is a worse outcome than not shipping the feature.
|
||
|
||
So the paragraph is switched on `THERMOGRAPH_GEOIP`, which both containers read
|
||
from the same `/etc/thermograph.env` (`content.py`'s `geoip_enabled()`). The
|
||
statement and the behaviour flip in the same deploy, and the frontend unit tests
|
||
assert both branches. The frontend reads the flag for *nothing else* — it does
|
||
not gate any behaviour on it.
|
||
|
||
---
|
||
|
||
## 6. Graceful degradation
|
||
|
||
Every one of these returns `None` → HTTP 204 → the client does exactly what the
|
||
site does today. There is no failure mode worse than the status quo.
|
||
|
||
| Case | Handled where | Behaviour |
|
||
| --- | --- | --- |
|
||
| Flag unset | `enabled()` | 204 |
|
||
| Flag set, no database file | `enabled()` (`os.path.isfile`) | 204 |
|
||
| `maxminddb` not installed | lazy import in `_open()` | 204 |
|
||
| Truncated / corrupt / half-written file | `_open()` try/except | 204 (tested) |
|
||
| Private ranges (10/8, 172.16/12, 192.168/16) | `parse_ip` | 204 — every LAN-dev request |
|
||
| Loopback, link-local, multicast, reserved, unspecified | `parse_ip` | 204 |
|
||
| **CGNAT 100.64/10** | `parse_ip` + explicit `_EXTRA_RESERVED` | 204 — common on mobile carriers |
|
||
| IPv6 (global) | `parse_ip` | looked up normally; MMDB covers v6 |
|
||
| IPv6 ULA `fc00::/7`, doc `2001:db8::/32` | `_EXTRA_RESERVED` | 204 |
|
||
| IPv4-mapped IPv6 `::ffff:a.b.c.d` | `parse_ip` unwraps | treated as the v4 address |
|
||
| `1.2.3.4:5678`, `[v6]:443`, `%zone` suffixes | `parse_ip` | normalised |
|
||
| Missing `X-Forwarded-For` | `_client_ip` falls back to peer | peer is loopback behind Caddy → 204 |
|
||
| A whole XFF chain passed as one string | `parse_ip` | rejected (the route already takes the left-most hop) |
|
||
| No record for the address | `lookup()` | 204 |
|
||
| Record with a country but **no city** | `suggestion_from_record` | 204 — the VPN/proxy case |
|
||
| Record whose stated radius > 100 km | `suggestion_from_record` | 204 |
|
||
| Reader raises on a bad node | `lookup()` try/except | 204 |
|
||
|
||
The explicit `_EXTRA_RESERVED` list is not redundant with `ipaddress.is_global`:
|
||
the exact membership of `is_private`/`is_global` has moved between CPython
|
||
releases (100.64/10 in particular). Which Python the container happens to run
|
||
must not decide whether we try to geolocate a carrier-NAT address.
|
||
|
||
---
|
||
|
||
## 7. The UI — presenting a guess as a guess
|
||
|
||
Copy shipped in `frontend/static/geoip.js`:
|
||
|
||
> **Showing weather near Rotterdam, South Holland.**
|
||
> That is a rough guess from your internet connection, not from your device — it
|
||
> can be tens of kilometres out. *IP Geolocation by DB-IP*
|
||
>
|
||
> **[ Not right? Choose your spot ]**
|
||
|
||
Design rules, and why each one:
|
||
|
||
1. **Named as a guess, in the same breath as the result.** Not a footnote, not a
|
||
tooltip. City-level accuracy is ~50 km against a ~2 mile grid cell — the
|
||
product's own unit of precision is 25× finer than the input.
|
||
2. **The correction is a permanent, full-width, ≥44 px control**, not a
|
||
dismissible toast. If the guess is wrong, fixing it must be the most obvious
|
||
thing on screen.
|
||
3. **A guess is never persisted.** `updateHash()` returns early while
|
||
`approx` is true: no `localStorage` write, no URL hash. A guess that became
|
||
sticky would be indistinguishable from a choice on the next visit, and a
|
||
guessed URL would propagate a rough location the sender never picked when
|
||
shared. Only an explicit pick is remembered.
|
||
4. **The headings are rewritten while approximate.** The hero says "Roughly near
|
||
Rotterdam" instead of "Today in …", and the results heading says "Near
|
||
Rotterdam" with an "approximate — from your connection, not your device"
|
||
tag. This matters more than it sounds: the guessed cell centre gets
|
||
reverse-geocoded like any other point, which would otherwise name a
|
||
*neighbourhood* — presenting street-level precision for a city-level input.
|
||
5. **Visually recessed** — `--surface-2`, dashed border, muted note text. It
|
||
reads as an aside, not as an answer.
|
||
6. **It never pre-empts the permission prompt.** It runs *after* a
|
||
declined/failed prompt, or on load only where the browser will not offer
|
||
geolocation at all (insecure context) and nothing is remembered.
|
||
7. **Never for a visitor who already has a place.** Theirs is better.
|
||
|
||
Attribution rides *with the result*, which is what CC BY 4.0 and DB-IP's own
|
||
wording ask for ("on pages displaying results") — and it is derived from the
|
||
installed file's metadata, so swapping databases changes the credit
|
||
automatically.
|
||
|
||
### Interaction with `home.locate`
|
||
|
||
`home.locate` still means exactly what it means today: "someone tapped *Use my
|
||
location*", whatever the outcome. It is unchanged, so the existing series stays
|
||
comparable across this change. Two new allowlisted events sit beside it:
|
||
|
||
- `home.geoip_shown` — a suggestion was rendered. `home.locate` minus
|
||
`home.geoip_shown` is roughly the population still hitting the dead end.
|
||
- `home.geoip_fixed` — the visitor corrected the guess. This is the accuracy
|
||
signal: a high ratio means the database is putting people in the wrong place
|
||
and the feature is doing harm, which is the number to watch after rollout.
|
||
|
||
Both are aggregate counts through the existing beacon, with **no location
|
||
dimension** — the suggestion itself is never recorded anywhere.
|
||
|
||
---
|
||
|
||
## 8. Database lifecycle
|
||
|
||
**On the host, refreshed by a systemd timer, bind-mounted read-only.** Not baked
|
||
into the image.
|
||
|
||
- `infra/deploy/geoip-refresh.sh` — resolves the current month's DB-IP release
|
||
(falling back to last month's, since the new file appears partway through the
|
||
1st), downloads, gunzips, **verifies it opens and answers for a known public
|
||
address**, and only then swaps it in atomically with `install`+`mv`. Exits 0
|
||
on "already current". A failure leaves the existing file untouched — a failed
|
||
refresh must never be worse than a stale one.
|
||
- `geoip-refresh.timer` — `OnCalendar=Mon 04:17 UTC`, `RandomizedDelaySec=2h`,
|
||
`Persistent=true`. Weekly rather than monthly even though DB-IP publishes
|
||
monthly: a no-op run costs one HEAD request, and it means a missed or failed
|
||
run self-heals within a week. (It also lands inside MaxMind's 30-day deletion
|
||
window without further thought, should you switch.)
|
||
- `infra/docker-compose.yml` mounts `${THERMOGRAPH_GEOIP_HOST_DIR:-/var/lib/thermograph/geoip}:/geoip:ro`
|
||
into the backend. Read-only on purpose — the app must never be able to write it.
|
||
- `geoip.py` re-opens when the file's `(mtime, size)` changes, so a refresh is
|
||
picked up **without a restart or a deploy**.
|
||
- `MODE_MMAP`: the 125 MB file costs page cache, not process RSS, and is shared
|
||
across the four uvicorn workers on a host.
|
||
|
||
**Why not build-time.** It is a 125 MB data file on its own release cadence, not
|
||
code. Baking it in adds 125 MB to every image pull, freezes freshness to
|
||
whenever the last deploy happened, ties a data refresh to a code deploy, and
|
||
(under GeoLite's EULA) cannot honour the 30-day deletion requirement on its own.
|
||
|
||
**Why not object storage.** We already have the bucket and could pull from it,
|
||
but that adds credentials and a network dependency to fetch a file that is
|
||
public and unauthenticated at the source. If a mirror is wanted later — to stop
|
||
depending on DB-IP's uptime, or to pin exactly which release is live across the
|
||
fleet — point `GEOIP_URL` at the bucket; the script needs no other change.
|
||
|
||
**Licence key storage.** With DB-IP: **none needed**, which is a large part of
|
||
the recommendation. If MaxMind is chosen instead, the key goes in the SOPS vault
|
||
(`infra/deploy/secrets/{prod,beta}.yaml`), renders into `/etc/thermograph.env`
|
||
like every other secret, and `geoip-refresh.service` already picks it up via its
|
||
`EnvironmentFile=` — set `GEOIP_URL` to the permalink that embeds it. Note the
|
||
key would then be in a *URL*, so keep it out of any log line that echoes the
|
||
URL: `geoip-refresh.sh` logs `$url`, and that line must be changed before using
|
||
a key-bearing URL. **Flagged as a real footgun if you pick MaxMind.**
|
||
|
||
---
|
||
|
||
## 9. Interaction with caching / CDN
|
||
|
||
- **The homepage stays byte-identical for every visitor** — the container is
|
||
server-rendered empty and hidden. `test_home_html_is_identical_regardless_of_client_address`
|
||
asserts the body *and* the ETag match across different `X-Forwarded-For`
|
||
values. That property is what keeps the homepage safe to cache, and it is the
|
||
main reason the lookup is not in SSR.
|
||
- **The endpoint is `no-store, private` + `Vary: *`** and answers 204 far more
|
||
often than 200. Nothing about it is shareable between visitors.
|
||
- **A CDN in front of the apex stays viable.** If Cloudflare (or a Caddy `cache`
|
||
directive) is added later, the rule is one line: never cache `/api/*`. That is
|
||
already the sane default and already how Caddy path-splits.
|
||
- **The existing frontend caches are untouched.** `api_client.py`'s TTL/LRU
|
||
cache never sees this request (it is browser→backend, not frontend→backend),
|
||
and `static/cache.js`'s IndexedDB layer is keyed by `/cell` URLs. A guessed
|
||
location produces ordinary `/api/v2/grade` traffic for an ordinary cell, so it
|
||
warms and reuses the same caches as any other place — no new cache keyspace.
|
||
- **One second-order effect worth naming:** guessed locations concentrate on
|
||
city centroids, so they will warm the *same* handful of cells repeatedly.
|
||
That is good for us (high cache hit rate, no extra Open-Meteo quota) but it
|
||
does mean per-cell request counts stop being a clean proxy for distinct users
|
||
near that cell.
|
||
|
||
---
|
||
|
||
## 10. Rollout
|
||
|
||
1. Merge with the flag off. Nothing changes — that is the point, and the tests
|
||
assert it.
|
||
2. On **beta only**: install the timer, run `geoip-refresh.sh` once by hand,
|
||
confirm `/var/lib/thermograph/geoip/city.mmdb` exists and
|
||
`curl -s -o /dev/null -w '%{http_code}' https://beta.thermograph.org/api/v2/geoip`
|
||
still returns 204 (flag still off).
|
||
3. Set `THERMOGRAPH_GEOIP=1` in beta's vault entry, redeploy **both** containers
|
||
(backend for the lookup, frontend for the privacy copy).
|
||
4. Verify by hand: decline the permission prompt; confirm the banner appears,
|
||
the correction button works, the guess is *not* in `localStorage` and *not*
|
||
in the URL; confirm `/privacy` now describes the fallback; confirm no
|
||
`"cat":"geoip"` line and no matching IP appears in the access log.
|
||
5. Watch `home.geoip_fixed / home.geoip_shown` for a week. If a large share of
|
||
visitors are correcting the guess, the database is putting people in the
|
||
wrong place — turn it off rather than tuning copy.
|
||
6. Only then, prod.
|
||
|
||
Rollback is `THERMOGRAPH_GEOIP=0` and a redeploy; or just delete the database
|
||
file, which disables it without touching config.
|
||
|
||
---
|
||
|
||
## 11. Decisions needed from the operator
|
||
|
||
1. **Which database?** Recommendation: **DB-IP IP to City Lite** (no account, no
|
||
key, no retention clock, CC BY 4.0). The alternative is GeoLite2 City for
|
||
`accuracy_radius` and weekly updates, at the cost of a vault-managed licence
|
||
key and a 30-day deletion obligation. Code supports either unchanged.
|
||
2. **Attribution placement.** Currently in the suggestion banner itself, beside
|
||
the result. Alternatives: also on `/about`, or only on `/about`. DB-IP's
|
||
stated wording asks for it "on pages displaying results", which the banner
|
||
satisfies — confirm you're happy with a small credit line inside the hero.
|
||
3. **SSR or client-side?** Recommendation: **client-side** (§4). Overruling this
|
||
means accepting a location-varying homepage; if you do, put it on a separate
|
||
non-canonical route, never on cached `/`.
|
||
4. **Refresh cadence.** Currently weekly-with-no-op (§8). Alternatives: monthly
|
||
to match DB-IP's release cadence exactly, or daily. Weekly is the
|
||
self-healing middle.
|
||
5. **Auto-suggest in insecure contexts?** Today the prototype suggests on load
|
||
when the browser won't offer geolocation at all (LAN dev over plain HTTP) and
|
||
nothing is remembered, since there is no prompt to decline. That is the only
|
||
path where a lookup happens without an explicit user action. Prod is HTTPS so
|
||
it never triggers there — but say if you'd rather it *never* happen without a
|
||
tap, and it becomes a one-line change.
|
||
6. **Threshold for calling it off.** Pick the `home.geoip_fixed / home.geoip_shown`
|
||
ratio at which the feature gets turned off, *before* rollout rather than
|
||
after — otherwise the number will always look explainable.
|
||
|
||
---
|
||
|
||
## 12. Things I think are bad ideas — flagged, not implemented
|
||
|
||
- **Don't put the lookup in the SSR homepage.** §4. A location-varying
|
||
canonical page is a cache-poisoning and SEO hazard, and it geolocates the
|
||
majority of visitors who already have a saved place and will never use it.
|
||
- **Don't persist the guess.** Not to `localStorage`, not to the URL, not to an
|
||
account preference. It is not a choice and must never become one silently.
|
||
The branch enforces this in `updateHash()`.
|
||
- **Don't use a third-party geolocation API**, even a "privacy-friendly" one.
|
||
Sending visitor IPs to another company creates a processor relationship and a
|
||
disclosure obligation, for a file we can just host. This was already the
|
||
brief's requirement; restating it because it is the single most tempting
|
||
shortcut here.
|
||
- **Don't extend this to country-level personalisation** (units, language,
|
||
default city) on the same lookup. Unit selection already works off the picked
|
||
location's country code, and quietly widening an IP lookup from "a fallback
|
||
the user asked for" to "we adapt the site to you" changes the legal analysis
|
||
in §5 — it stops being strictly necessary for a requested service.
|
||
- **Don't reuse the derived city anywhere it could be stored.** No metrics
|
||
dimension, no audit field, no "popular cities" table fed from guesses. The
|
||
moment a derived location is written next to anything timestamped, the
|
||
privacy claim in §5 stops being true.
|
||
- **Don't fold this behind the analytics consent gate.** §5. Different legal
|
||
basis; gating it on consent both implies the wrong basis and breaks the
|
||
fallback for everyone who declines analytics.
|
||
- **Don't log `$url` once a MaxMind key-bearing permalink is in use** (§8).
|
||
- **Be honest that this is a downgrade, not a feature.** A city-level guess is
|
||
25× coarser than the grid the whole product is built on. It exists so a
|
||
visitor who declined isn't stranded — not because IP location is good. If the
|
||
correction rate says people don't want it, delete it rather than defend it.
|