Design: approximate location from client IP (feature-flagged off, decisions needed) #34
19 changed files with 1694 additions and 4 deletions
489
GEOIP-APPROX-LOCATION.md
Normal file
489
GEOIP-APPROX-LOCATION.md
Normal file
|
|
@ -0,0 +1,489 @@
|
||||||
|
# Approximate-location fallback — design + prototype
|
||||||
|
|
||||||
|
Status: **prototype on `feat/geoip-approx-location`, feature-flagged OFF, not
|
||||||
|
deployed and not merged.** Nothing in this branch changes production behaviour
|
||||||
|
until `THERMOGRAPH_GEOIP` is set *and* a database file exists on the host.
|
||||||
|
|
||||||
|
This doc is the decision record. If it survives review it should move to
|
||||||
|
`thermograph-docs` (per the root `CLAUDE.md`: cross-cutting decision docs live
|
||||||
|
there); it sits here for now so the reasoning ships with the code it describes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The problem
|
||||||
|
|
||||||
|
Location today comes from exactly one source: `navigator.geolocation.getCurrentPosition`
|
||||||
|
in `frontend/static/app.js`, gated on `window.isSecureContext`. If the visitor
|
||||||
|
declines the permission prompt, the prompt times out, or the browser won't offer
|
||||||
|
geolocation at all, the site has **no location** and the copy says "Pick a spot
|
||||||
|
on the map instead."
|
||||||
|
|
||||||
|
That is a dead end for the exact visitor the homepage is written for — someone
|
||||||
|
who arrived to answer "how unusual is *my* weather?" and has just been handed a
|
||||||
|
world map. This change gives that person something local, honestly labelled.
|
||||||
|
|
||||||
|
## 2. What was built
|
||||||
|
|
||||||
|
| Piece | File |
|
||||||
|
| --- | --- |
|
||||||
|
| Lookup module (flag, parsing, rejection rules, attribution) | `backend/data/geoip.py` |
|
||||||
|
| Endpoint `GET /api/v2/geoip` | `backend/web/app.py` (`api_geoip`) |
|
||||||
|
| Access-log exclusion + traffic category | `backend/web/app.py`, `backend/core/metrics.py` |
|
||||||
|
| Event names `home.geoip_shown` / `home.geoip_fixed` | `backend/core/metrics.py` |
|
||||||
|
| Client fetch + suggestion banner | `frontend/static/geoip.js` |
|
||||||
|
| Wiring, non-persistence, honest headings | `frontend/static/app.js` |
|
||||||
|
| Banner container (empty, hidden) | `frontend/templates/home.html.j2` |
|
||||||
|
| Banner styles | `frontend/static/style.css` |
|
||||||
|
| Flag-aware privacy copy | `frontend/content.py`, `frontend/templates/privacy.html.j2` |
|
||||||
|
| Host refresh job + timer | `infra/deploy/geoip-refresh.{sh,service,timer}` |
|
||||||
|
| Read-only DB mount + env plumbing | `infra/docker-compose.yml`, `infra/deploy/thermograph.env.example` |
|
||||||
|
| Tests | `backend/tests/data/test_geoip.py`, `backend/tests/web/test_geoip_api.py`, `frontend/tests/unit/test_geoip_ui.py` |
|
||||||
|
|
||||||
|
`cd backend && make test` → **449 passed, 8 skipped** (was 386/7).
|
||||||
|
`cd frontend && make test-unit` → **41 passed**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Database choice — recommendation: **DB-IP IP to City Lite**
|
||||||
|
|
||||||
|
| | **DB-IP IP to City Lite** | MaxMind GeoLite2 City | IP2Location LITE DB5/DB11 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| Licence | **CC BY 4.0** | GeoLite End User License Agreement (incorporates CC BY-SA 4.0 terms) | CC BY-SA 4.0 / "free with attribution" |
|
||||||
|
| Account needed | **No** — direct download URL | **Yes** — account + licence key | **Yes** — free account |
|
||||||
|
| Secret to manage | **None** | Licence key (→ SOPS vault, → CI, → rotation) | Account credentials |
|
||||||
|
| Attribution | `<a href="https://db-ip.com">IP Geolocation by DB-IP</a>` on pages showing results | "This product includes GeoLite Data created by MaxMind, available from https://www.maxmind.com" | "uses the IP2Location LITE database for IP geolocation", linked |
|
||||||
|
| Retention obligation | none | **must delete a database within 30 days of a newer release**; 30 downloads/day cap | none |
|
||||||
|
| Update cadence | monthly | weekly (Tue/Fri) | monthly |
|
||||||
|
| Size (current release) | 125 MB MMDB (685 MB CSV), ~8.05 M records | ~60–70 MB MMDB | 69 MB BIN (IPv4) + 171 MB BIN (IPv6), separate files |
|
||||||
|
| Format / reader | **MMDB → `maxminddb`** (pure-Python, mmap) | MMDB → `maxminddb` / `geoip2` | proprietary BIN → vendor `IP2Location` package (or CSV you index yourself) |
|
||||||
|
| Accuracy fields | city + lat/lon, **no `accuracy_radius`** | city + lat/lon + `accuracy_radius` | city + lat/lon |
|
||||||
|
|
||||||
|
**Recommend DB-IP IP to City Lite.** Reasoning, in priority order:
|
||||||
|
|
||||||
|
1. **No credential.** MaxMind means a licence key in the SOPS vault, in the
|
||||||
|
deploy path, and in a rotation story — new secret surface for a feature whose
|
||||||
|
entire justification is privacy hygiene. DB-IP needs none, so
|
||||||
|
`geoip-refresh.sh` reads nothing from the vault at all.
|
||||||
|
2. **No retention obligation.** GeoLite's EULA requires deleting a database
|
||||||
|
within 30 days of a newer release. That is a *compliance* clock on an ops
|
||||||
|
job, and a missed job becomes a licence breach rather than merely stale data.
|
||||||
|
CC BY 4.0 has no such clause.
|
||||||
|
3. **Same format, same reader.** Both are MMDB, so this is a swap of one file
|
||||||
|
and one URL, not a code change — `geoip.py` reads the raw record dict and
|
||||||
|
derives its attribution line from the file's own metadata. If accuracy turns
|
||||||
|
out to be materially worse in practice, moving to GeoLite2 is a config change
|
||||||
|
plus a vault entry, and the code is already written to accept it.
|
||||||
|
4. **Attribution is a link, not a legal paragraph.** One muted anchor in the
|
||||||
|
suggestion banner satisfies CC BY 4.0 and DB-IP's stated wording.
|
||||||
|
|
||||||
|
**Against IP2Location LITE**: a proprietary BIN format needing the vendor's own
|
||||||
|
Python package, IPv4 and IPv6 as *separate* databases (two files, two readers,
|
||||||
|
two refresh paths), an account to download, and CC BY-**SA** — a share-alike
|
||||||
|
term that is noise to reason about for a file we merely read.
|
||||||
|
|
||||||
|
**Cost of the DB-IP choice, stated plainly:** the Lite file carries no
|
||||||
|
`accuracy_radius`, so the "reject anything the database admits is too coarse"
|
||||||
|
guard degrades to "reject anything with no city name". That is still the guard
|
||||||
|
that matters (a country centroid is the damaging case), but it is weaker.
|
||||||
|
`suggestion_from_record()` applies the radius rule whenever the field *is*
|
||||||
|
present, so switching to GeoLite2 turns the stronger guard back on for free.
|
||||||
|
|
||||||
|
**Reader library:** `maxminddb` (pinned `2.8.2` in `backend/requirements.txt`),
|
||||||
|
not `geoip2`. `geoip2` adds a model layer over the same reader and type-checks
|
||||||
|
`database_type` against MaxMind's own names; we read four fields out of a plain
|
||||||
|
dict, and staying on `maxminddb` keeps DB-IP and MaxMind files equally usable.
|
||||||
|
It is imported *lazily inside* `geoip.py`, so a missing wheel disables the
|
||||||
|
feature rather than breaking boot.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Where it hooks in — recommendation: **client-side, on demand**
|
||||||
|
|
||||||
|
Two options were on the table.
|
||||||
|
|
||||||
|
**A. Server-side during SSR of `/`.** `frontend/content.py`'s `home_page()`
|
||||||
|
would look up the IP and render the guess into the first paint.
|
||||||
|
|
||||||
|
**B. Client-side after geolocation is declined/unavailable.** The browser calls
|
||||||
|
`GET /api/v2/geoip` and swaps the banner in. **This is what the prototype does.**
|
||||||
|
|
||||||
|
### Why B
|
||||||
|
|
||||||
|
**Caching is the deciding factor, and the current setup is more fragile than it
|
||||||
|
looks.** `frontend/content.py`'s `_respond_html()` returns HTML with a weak
|
||||||
|
ETag (`W/"<sha1 of the rendered body>"`) and **no `Cache-Control` and no
|
||||||
|
`Vary`**. The host Caddy (`infra/deploy/Caddyfile`) only does `encode` +
|
||||||
|
`reverse_proxy` — no cache module today. So right now nothing caches the
|
||||||
|
homepage, but nothing *declares* that it mustn't either.
|
||||||
|
|
||||||
|
Fold a location into that response and three things break at once:
|
||||||
|
|
||||||
|
- The ETag becomes a function of the visitor's city. Two visitors flap each
|
||||||
|
other's validators; conditional requests stop being useful, and a returning
|
||||||
|
visitor whose IP maps elsewhere gets a spurious 200 instead of a 304.
|
||||||
|
- Any cache added later — a Caddy `cache` directive, Cloudflare in front of the
|
||||||
|
apex, a corporate proxy on the visitor's side — will serve one visitor's city
|
||||||
|
to another, because the response carries nothing saying it varies. There is no
|
||||||
|
correct `Vary` header for this: it varies on `X-Forwarded-For`, which is not a
|
||||||
|
header a cache keys on, and `Vary: *` would make the homepage permanently
|
||||||
|
uncacheable for everyone to serve a fallback that only a minority need.
|
||||||
|
- The SSR path would also have to carry the client IP over the extra
|
||||||
|
frontend→backend hop (`api_client.py` currently forwards only `Host` and
|
||||||
|
`X-Forwarded-Proto`). That means propagating the IP into a second process's
|
||||||
|
request path — more places it can end up in a log, for no benefit.
|
||||||
|
|
||||||
|
**The first-paint argument is weaker here than it looks**, because the homepage
|
||||||
|
hero is *already* server-rendered with real content: the most unusual city
|
||||||
|
we're currently tracking, with its grade, band colour and percentile. There is
|
||||||
|
no blank space for the guess to fill. The suggestion swaps into a page that is
|
||||||
|
already complete, and it costs one small JSON request with no data dependency in
|
||||||
|
front of it.
|
||||||
|
|
||||||
|
**And SSR would do the lookup for the wrong people.** The server cannot see
|
||||||
|
`localStorage`, so it cannot know that most returning visitors already have a
|
||||||
|
saved place that is strictly better than any guess. An SSR implementation would
|
||||||
|
geolocate every homepage request, including the majority who will never use the
|
||||||
|
result — the opposite of "touch the IP as little as possible". Option B only
|
||||||
|
runs after the visitor has demonstrably ended up with nothing.
|
||||||
|
|
||||||
|
### What B costs
|
||||||
|
|
||||||
|
- **Needs JavaScript.** A no-JS visitor sees today's behaviour. Acceptable: the
|
||||||
|
entire interactive tool already needs JS.
|
||||||
|
- **One extra round trip** after the decline, ~200 bytes each way.
|
||||||
|
- **A 204 for everyone in the fallback path when the feature is off.** The
|
||||||
|
alternative — giving the frontend its own copy of the flag to decide whether
|
||||||
|
to fetch — is two flags to keep in sync and a new way for the UI and the
|
||||||
|
server to disagree. One flag, one 204, is the better trade.
|
||||||
|
|
||||||
|
### If you overrule this and want SSR anyway
|
||||||
|
|
||||||
|
Do it as a *separate* route (`/near` or `?near=1`), not on the cached `/`, and
|
||||||
|
give that route `Cache-Control: private, no-store`. Never make the canonical,
|
||||||
|
indexable homepage location-varying — that is also an SEO hazard: a crawler in
|
||||||
|
one datacentre would index a page claiming a city, and different crawler IPs
|
||||||
|
would see different "canonical" content.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Privacy design
|
||||||
|
|
||||||
|
### The IP is never persisted — mechanically, not by convention
|
||||||
|
|
||||||
|
- `geoip.lookup(ip)` takes a string, memory-maps a file, returns a dict. It
|
||||||
|
writes nothing, logs nothing, and never raises.
|
||||||
|
- `api_geoip` uses **no `RunAudit`** (which would record the derived lat/lon)
|
||||||
|
and adds **no metric dimension** (the counters are aggregate by category).
|
||||||
|
- The route is excluded from the access log. `metrics.classify_inbound()` now
|
||||||
|
returns a dedicated `"geoip"` category and the middleware in `web/app.py`
|
||||||
|
skips `audit.log_access` for it. Every other route still logs, and
|
||||||
|
`test_a_normal_route_still_is_logged` guards that exclusion from widening.
|
||||||
|
|
||||||
|
This is the important one. Today the access log stores a raw IP per request.
|
||||||
|
Without the exclusion, the *one* endpoint whose whole job is turning an IP
|
||||||
|
into a place would leave a stored, timestamped, joinable (IP,
|
||||||
|
approximate-location) pair on disk and in Loki. That artefact — not the
|
||||||
|
transient lookup — is what would actually constitute keeping location data
|
||||||
|
about identifiable people. The exclusion is not defence in depth; it is the
|
||||||
|
substance of the claim.
|
||||||
|
|
||||||
|
This does **not** depend on the separate access-log IP-truncation work. That
|
||||||
|
change is still worth doing, and it composes: truncation reduces what the
|
||||||
|
*other* routes retain, while this exclusion means the geoip route retains
|
||||||
|
nothing at all either way.
|
||||||
|
|
||||||
|
- `Cache-Control: no-store, private` + `Vary: *` on the response, so it cannot
|
||||||
|
be stored or reused by any intermediary.
|
||||||
|
- The endpoint takes **no parameters**. You cannot ask it about somebody else's
|
||||||
|
address; the only input is the connection you already own. `?ip=` is ignored
|
||||||
|
(tested).
|
||||||
|
|
||||||
|
### Why no consent banner is needed — and why the analytics work is different
|
||||||
|
|
||||||
|
Under GDPR the lookup is **transient processing to deliver a service the user
|
||||||
|
explicitly asked for**. The visitor pressed "Use my location" *wanting* local
|
||||||
|
weather; when the browser can't supply it, deriving a coarse city from the
|
||||||
|
connection they are already using to reach us is a proportionate substitute for
|
||||||
|
the same, requested purpose. Nothing is stored, nothing is profiled, nothing is
|
||||||
|
shared with a third party (the database is a file we host — no processor
|
||||||
|
relationship, no international transfer, no disclosure to name in the policy).
|
||||||
|
The data subject can override the result with one tap, and the result is
|
||||||
|
explicitly labelled a guess rather than presented as a finding about them.
|
||||||
|
|
||||||
|
**ePrivacy Art. 5(3) is not engaged at all**, and this is the cleanest part of
|
||||||
|
the argument. That article governs *storing information on, or gaining access to
|
||||||
|
information stored in, a user's terminal equipment* — which is why cookies,
|
||||||
|
localStorage and device fingerprinting need consent regardless of whether the
|
||||||
|
data is personal. An IP address arrives in the request headers as an unavoidable
|
||||||
|
property of routing a packet. We neither store anything on the device nor read
|
||||||
|
anything from it. There is nothing for a consent banner to be about.
|
||||||
|
|
||||||
|
**Contrast with the analytics session-id work.** That one deliberately *writes
|
||||||
|
an identifier onto the visitor's device* in order to correlate their actions
|
||||||
|
across a session. That is squarely Art. 5(3) storage/access, it is not strictly
|
||||||
|
necessary to deliver a service the user requested, and consent is therefore
|
||||||
|
required — hence `consent.js` and the consent gate on that branch. The two
|
||||||
|
features look adjacent ("both touch visitor data") and are legally opposite:
|
||||||
|
one reads a routing header and forgets it, the other plants a durable
|
||||||
|
identifier. Keep them separate in the code and in the privacy copy, and never
|
||||||
|
let the geoip fallback be folded behind the analytics consent gate — that would
|
||||||
|
imply consent is what makes it lawful, which is the wrong basis and would
|
||||||
|
silently break the fallback for everyone who declines analytics.
|
||||||
|
|
||||||
|
### The privacy page follows the flag
|
||||||
|
|
||||||
|
`frontend/templates/privacy.html.j2` currently says, in the visitor's own
|
||||||
|
words: *"Thermograph never looks up your location from your IP address."*
|
||||||
|
Turning this feature on makes that sentence **false**, and a stale privacy
|
||||||
|
statement is a worse outcome than not shipping the feature.
|
||||||
|
|
||||||
|
So the paragraph is switched on `THERMOGRAPH_GEOIP`, which both containers read
|
||||||
|
from the same `/etc/thermograph.env` (`content.py`'s `geoip_enabled()`). The
|
||||||
|
statement and the behaviour flip in the same deploy, and the frontend unit tests
|
||||||
|
assert both branches. The frontend reads the flag for *nothing else* — it does
|
||||||
|
not gate any behaviour on it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Graceful degradation
|
||||||
|
|
||||||
|
Every one of these returns `None` → HTTP 204 → the client does exactly what the
|
||||||
|
site does today. There is no failure mode worse than the status quo.
|
||||||
|
|
||||||
|
| Case | Handled where | Behaviour |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Flag unset | `enabled()` | 204 |
|
||||||
|
| Flag set, no database file | `enabled()` (`os.path.isfile`) | 204 |
|
||||||
|
| `maxminddb` not installed | lazy import in `_open()` | 204 |
|
||||||
|
| Truncated / corrupt / half-written file | `_open()` try/except | 204 (tested) |
|
||||||
|
| Private ranges (10/8, 172.16/12, 192.168/16) | `parse_ip` | 204 — every LAN-dev request |
|
||||||
|
| Loopback, link-local, multicast, reserved, unspecified | `parse_ip` | 204 |
|
||||||
|
| **CGNAT 100.64/10** | `parse_ip` + explicit `_EXTRA_RESERVED` | 204 — common on mobile carriers |
|
||||||
|
| IPv6 (global) | `parse_ip` | looked up normally; MMDB covers v6 |
|
||||||
|
| IPv6 ULA `fc00::/7`, doc `2001:db8::/32` | `_EXTRA_RESERVED` | 204 |
|
||||||
|
| IPv4-mapped IPv6 `::ffff:a.b.c.d` | `parse_ip` unwraps | treated as the v4 address |
|
||||||
|
| `1.2.3.4:5678`, `[v6]:443`, `%zone` suffixes | `parse_ip` | normalised |
|
||||||
|
| Missing `X-Forwarded-For` | `_client_ip` falls back to peer | peer is loopback behind Caddy → 204 |
|
||||||
|
| A whole XFF chain passed as one string | `parse_ip` | rejected (the route already takes the left-most hop) |
|
||||||
|
| No record for the address | `lookup()` | 204 |
|
||||||
|
| Record with a country but **no city** | `suggestion_from_record` | 204 — the VPN/proxy case |
|
||||||
|
| Record whose stated radius > 100 km | `suggestion_from_record` | 204 |
|
||||||
|
| Reader raises on a bad node | `lookup()` try/except | 204 |
|
||||||
|
|
||||||
|
The explicit `_EXTRA_RESERVED` list is not redundant with `ipaddress.is_global`:
|
||||||
|
the exact membership of `is_private`/`is_global` has moved between CPython
|
||||||
|
releases (100.64/10 in particular). Which Python the container happens to run
|
||||||
|
must not decide whether we try to geolocate a carrier-NAT address.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. The UI — presenting a guess as a guess
|
||||||
|
|
||||||
|
Copy shipped in `frontend/static/geoip.js`:
|
||||||
|
|
||||||
|
> **Showing weather near Rotterdam, South Holland.**
|
||||||
|
> That is a rough guess from your internet connection, not from your device — it
|
||||||
|
> can be tens of kilometres out. *IP Geolocation by DB-IP*
|
||||||
|
>
|
||||||
|
> **[ Not right? Choose your spot ]**
|
||||||
|
|
||||||
|
Design rules, and why each one:
|
||||||
|
|
||||||
|
1. **Named as a guess, in the same breath as the result.** Not a footnote, not a
|
||||||
|
tooltip. City-level accuracy is ~50 km against a ~2 mile grid cell — the
|
||||||
|
product's own unit of precision is 25× finer than the input.
|
||||||
|
2. **The correction is a permanent, full-width, ≥44 px control**, not a
|
||||||
|
dismissible toast. If the guess is wrong, fixing it must be the most obvious
|
||||||
|
thing on screen.
|
||||||
|
3. **A guess is never persisted.** `updateHash()` returns early while
|
||||||
|
`approx` is true: no `localStorage` write, no URL hash. A guess that became
|
||||||
|
sticky would be indistinguishable from a choice on the next visit, and a
|
||||||
|
guessed URL would propagate a rough location the sender never picked when
|
||||||
|
shared. Only an explicit pick is remembered.
|
||||||
|
4. **The headings are rewritten while approximate.** The hero says "Roughly near
|
||||||
|
Rotterdam" instead of "Today in …", and the results heading says "Near
|
||||||
|
Rotterdam" with an "approximate — from your connection, not your device"
|
||||||
|
tag. This matters more than it sounds: the guessed cell centre gets
|
||||||
|
reverse-geocoded like any other point, which would otherwise name a
|
||||||
|
*neighbourhood* — presenting street-level precision for a city-level input.
|
||||||
|
5. **Visually recessed** — `--surface-2`, dashed border, muted note text. It
|
||||||
|
reads as an aside, not as an answer.
|
||||||
|
6. **It never pre-empts the permission prompt.** It runs *after* a
|
||||||
|
declined/failed prompt, or on load only where the browser will not offer
|
||||||
|
geolocation at all (insecure context) and nothing is remembered.
|
||||||
|
7. **Never for a visitor who already has a place.** Theirs is better.
|
||||||
|
|
||||||
|
Attribution rides *with the result*, which is what CC BY 4.0 and DB-IP's own
|
||||||
|
wording ask for ("on pages displaying results") — and it is derived from the
|
||||||
|
installed file's metadata, so swapping databases changes the credit
|
||||||
|
automatically.
|
||||||
|
|
||||||
|
### Interaction with `home.locate`
|
||||||
|
|
||||||
|
`home.locate` still means exactly what it means today: "someone tapped *Use my
|
||||||
|
location*", whatever the outcome. It is unchanged, so the existing series stays
|
||||||
|
comparable across this change. Two new allowlisted events sit beside it:
|
||||||
|
|
||||||
|
- `home.geoip_shown` — a suggestion was rendered. `home.locate` minus
|
||||||
|
`home.geoip_shown` is roughly the population still hitting the dead end.
|
||||||
|
- `home.geoip_fixed` — the visitor corrected the guess. This is the accuracy
|
||||||
|
signal: a high ratio means the database is putting people in the wrong place
|
||||||
|
and the feature is doing harm, which is the number to watch after rollout.
|
||||||
|
|
||||||
|
Both are aggregate counts through the existing beacon, with **no location
|
||||||
|
dimension** — the suggestion itself is never recorded anywhere.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Database lifecycle
|
||||||
|
|
||||||
|
**On the host, refreshed by a systemd timer, bind-mounted read-only.** Not baked
|
||||||
|
into the image.
|
||||||
|
|
||||||
|
- `infra/deploy/geoip-refresh.sh` — resolves the current month's DB-IP release
|
||||||
|
(falling back to last month's, since the new file appears partway through the
|
||||||
|
1st), downloads, gunzips, **verifies it opens and answers for a known public
|
||||||
|
address**, and only then swaps it in atomically with `install`+`mv`. Exits 0
|
||||||
|
on "already current". A failure leaves the existing file untouched — a failed
|
||||||
|
refresh must never be worse than a stale one.
|
||||||
|
- `geoip-refresh.timer` — `OnCalendar=Mon 04:17 UTC`, `RandomizedDelaySec=2h`,
|
||||||
|
`Persistent=true`. Weekly rather than monthly even though DB-IP publishes
|
||||||
|
monthly: a no-op run costs one HEAD request, and it means a missed or failed
|
||||||
|
run self-heals within a week. (It also lands inside MaxMind's 30-day deletion
|
||||||
|
window without further thought, should you switch.)
|
||||||
|
- `infra/docker-compose.yml` mounts `${THERMOGRAPH_GEOIP_HOST_DIR:-/var/lib/thermograph/geoip}:/geoip:ro`
|
||||||
|
into the backend. Read-only on purpose — the app must never be able to write it.
|
||||||
|
- `geoip.py` re-opens when the file's `(mtime, size)` changes, so a refresh is
|
||||||
|
picked up **without a restart or a deploy**.
|
||||||
|
- `MODE_MMAP`: the 125 MB file costs page cache, not process RSS, and is shared
|
||||||
|
across the four uvicorn workers on a host.
|
||||||
|
|
||||||
|
**Why not build-time.** It is a 125 MB data file on its own release cadence, not
|
||||||
|
code. Baking it in adds 125 MB to every image pull, freezes freshness to
|
||||||
|
whenever the last deploy happened, ties a data refresh to a code deploy, and
|
||||||
|
(under GeoLite's EULA) cannot honour the 30-day deletion requirement on its own.
|
||||||
|
|
||||||
|
**Why not object storage.** We already have the bucket and could pull from it,
|
||||||
|
but that adds credentials and a network dependency to fetch a file that is
|
||||||
|
public and unauthenticated at the source. If a mirror is wanted later — to stop
|
||||||
|
depending on DB-IP's uptime, or to pin exactly which release is live across the
|
||||||
|
fleet — point `GEOIP_URL` at the bucket; the script needs no other change.
|
||||||
|
|
||||||
|
**Licence key storage.** With DB-IP: **none needed**, which is a large part of
|
||||||
|
the recommendation. If MaxMind is chosen instead, the key goes in the SOPS vault
|
||||||
|
(`infra/deploy/secrets/{prod,beta}.yaml`), renders into `/etc/thermograph.env`
|
||||||
|
like every other secret, and `geoip-refresh.service` already picks it up via its
|
||||||
|
`EnvironmentFile=` — set `GEOIP_URL` to the permalink that embeds it. Note the
|
||||||
|
key would then be in a *URL*, so keep it out of any log line that echoes the
|
||||||
|
URL: `geoip-refresh.sh` logs `$url`, and that line must be changed before using
|
||||||
|
a key-bearing URL. **Flagged as a real footgun if you pick MaxMind.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Interaction with caching / CDN
|
||||||
|
|
||||||
|
- **The homepage stays byte-identical for every visitor** — the container is
|
||||||
|
server-rendered empty and hidden. `test_home_html_is_identical_regardless_of_client_address`
|
||||||
|
asserts the body *and* the ETag match across different `X-Forwarded-For`
|
||||||
|
values. That property is what keeps the homepage safe to cache, and it is the
|
||||||
|
main reason the lookup is not in SSR.
|
||||||
|
- **The endpoint is `no-store, private` + `Vary: *`** and answers 204 far more
|
||||||
|
often than 200. Nothing about it is shareable between visitors.
|
||||||
|
- **A CDN in front of the apex stays viable.** If Cloudflare (or a Caddy `cache`
|
||||||
|
directive) is added later, the rule is one line: never cache `/api/*`. That is
|
||||||
|
already the sane default and already how Caddy path-splits.
|
||||||
|
- **The existing frontend caches are untouched.** `api_client.py`'s TTL/LRU
|
||||||
|
cache never sees this request (it is browser→backend, not frontend→backend),
|
||||||
|
and `static/cache.js`'s IndexedDB layer is keyed by `/cell` URLs. A guessed
|
||||||
|
location produces ordinary `/api/v2/grade` traffic for an ordinary cell, so it
|
||||||
|
warms and reuses the same caches as any other place — no new cache keyspace.
|
||||||
|
- **One second-order effect worth naming:** guessed locations concentrate on
|
||||||
|
city centroids, so they will warm the *same* handful of cells repeatedly.
|
||||||
|
That is good for us (high cache hit rate, no extra Open-Meteo quota) but it
|
||||||
|
does mean per-cell request counts stop being a clean proxy for distinct users
|
||||||
|
near that cell.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Rollout
|
||||||
|
|
||||||
|
1. Merge with the flag off. Nothing changes — that is the point, and the tests
|
||||||
|
assert it.
|
||||||
|
2. On **beta only**: install the timer, run `geoip-refresh.sh` once by hand,
|
||||||
|
confirm `/var/lib/thermograph/geoip/city.mmdb` exists and
|
||||||
|
`curl -s -o /dev/null -w '%{http_code}' https://beta.thermograph.org/api/v2/geoip`
|
||||||
|
still returns 204 (flag still off).
|
||||||
|
3. Set `THERMOGRAPH_GEOIP=1` in beta's vault entry, redeploy **both** containers
|
||||||
|
(backend for the lookup, frontend for the privacy copy).
|
||||||
|
4. Verify by hand: decline the permission prompt; confirm the banner appears,
|
||||||
|
the correction button works, the guess is *not* in `localStorage` and *not*
|
||||||
|
in the URL; confirm `/privacy` now describes the fallback; confirm no
|
||||||
|
`"cat":"geoip"` line and no matching IP appears in the access log.
|
||||||
|
5. Watch `home.geoip_fixed / home.geoip_shown` for a week. If a large share of
|
||||||
|
visitors are correcting the guess, the database is putting people in the
|
||||||
|
wrong place — turn it off rather than tuning copy.
|
||||||
|
6. Only then, prod.
|
||||||
|
|
||||||
|
Rollback is `THERMOGRAPH_GEOIP=0` and a redeploy; or just delete the database
|
||||||
|
file, which disables it without touching config.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Decisions needed from the operator
|
||||||
|
|
||||||
|
1. **Which database?** Recommendation: **DB-IP IP to City Lite** (no account, no
|
||||||
|
key, no retention clock, CC BY 4.0). The alternative is GeoLite2 City for
|
||||||
|
`accuracy_radius` and weekly updates, at the cost of a vault-managed licence
|
||||||
|
key and a 30-day deletion obligation. Code supports either unchanged.
|
||||||
|
2. **Attribution placement.** Currently in the suggestion banner itself, beside
|
||||||
|
the result. Alternatives: also on `/about`, or only on `/about`. DB-IP's
|
||||||
|
stated wording asks for it "on pages displaying results", which the banner
|
||||||
|
satisfies — confirm you're happy with a small credit line inside the hero.
|
||||||
|
3. **SSR or client-side?** Recommendation: **client-side** (§4). Overruling this
|
||||||
|
means accepting a location-varying homepage; if you do, put it on a separate
|
||||||
|
non-canonical route, never on cached `/`.
|
||||||
|
4. **Refresh cadence.** Currently weekly-with-no-op (§8). Alternatives: monthly
|
||||||
|
to match DB-IP's release cadence exactly, or daily. Weekly is the
|
||||||
|
self-healing middle.
|
||||||
|
5. **Auto-suggest in insecure contexts?** Today the prototype suggests on load
|
||||||
|
when the browser won't offer geolocation at all (LAN dev over plain HTTP) and
|
||||||
|
nothing is remembered, since there is no prompt to decline. That is the only
|
||||||
|
path where a lookup happens without an explicit user action. Prod is HTTPS so
|
||||||
|
it never triggers there — but say if you'd rather it *never* happen without a
|
||||||
|
tap, and it becomes a one-line change.
|
||||||
|
6. **Threshold for calling it off.** Pick the `home.geoip_fixed / home.geoip_shown`
|
||||||
|
ratio at which the feature gets turned off, *before* rollout rather than
|
||||||
|
after — otherwise the number will always look explainable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Things I think are bad ideas — flagged, not implemented
|
||||||
|
|
||||||
|
- **Don't put the lookup in the SSR homepage.** §4. A location-varying
|
||||||
|
canonical page is a cache-poisoning and SEO hazard, and it geolocates the
|
||||||
|
majority of visitors who already have a saved place and will never use it.
|
||||||
|
- **Don't persist the guess.** Not to `localStorage`, not to the URL, not to an
|
||||||
|
account preference. It is not a choice and must never become one silently.
|
||||||
|
The branch enforces this in `updateHash()`.
|
||||||
|
- **Don't use a third-party geolocation API**, even a "privacy-friendly" one.
|
||||||
|
Sending visitor IPs to another company creates a processor relationship and a
|
||||||
|
disclosure obligation, for a file we can just host. This was already the
|
||||||
|
brief's requirement; restating it because it is the single most tempting
|
||||||
|
shortcut here.
|
||||||
|
- **Don't extend this to country-level personalisation** (units, language,
|
||||||
|
default city) on the same lookup. Unit selection already works off the picked
|
||||||
|
location's country code, and quietly widening an IP lookup from "a fallback
|
||||||
|
the user asked for" to "we adapt the site to you" changes the legal analysis
|
||||||
|
in §5 — it stops being strictly necessary for a requested service.
|
||||||
|
- **Don't reuse the derived city anywhere it could be stored.** No metrics
|
||||||
|
dimension, no audit field, no "popular cities" table fed from guesses. The
|
||||||
|
moment a derived location is written next to anything timestamped, the
|
||||||
|
privacy claim in §5 stops being true.
|
||||||
|
- **Don't fold this behind the analytics consent gate.** §5. Different legal
|
||||||
|
basis; gating it on consent both implies the wrong basis and breaks the
|
||||||
|
fallback for everyone who declines analytics.
|
||||||
|
- **Don't log `$url` once a MaxMind key-bearing permalink is in use** (§8).
|
||||||
|
- **Be honest that this is a downgrade, not a feature.** A city-level guess is
|
||||||
|
25× coarser than the grid the whole product is built on. It exists so a
|
||||||
|
visitor who declined isn't stranded — not because IP location is good. If the
|
||||||
|
correction rate says people don't want it, delete it rather than defend it.
|
||||||
|
|
@ -118,6 +118,11 @@ def classify_inbound(path: str, base: str = "") -> str:
|
||||||
# category keeps record_inbound from double-counting every interaction.
|
# category keeps record_inbound from double-counting every interaction.
|
||||||
if seg == "event":
|
if seg == "event":
|
||||||
return "event"
|
return "event"
|
||||||
|
# Its own category so web/app.py's access-log middleware can skip it:
|
||||||
|
# that route turns the client IP into a coarse location, so an access
|
||||||
|
# line pairing the two on disk is exactly what the feature avoids.
|
||||||
|
if seg == "geoip":
|
||||||
|
return "geoip"
|
||||||
return f"api:{seg}" if seg else "api:other"
|
return f"api:{seg}" if seg else "api:other"
|
||||||
if p.endswith((".js", ".css", ".html", ".webmanifest", ".png", ".svg", ".ico",
|
if p.endswith((".js", ".css", ".html", ".webmanifest", ".png", ".svg", ".ico",
|
||||||
".json", ".woff", ".woff2", ".map", ".txt", ".xml")):
|
".json", ".woff", ".woff2", ".map", ".txt", ".xml")):
|
||||||
|
|
@ -141,6 +146,13 @@ def classify_inbound(path: str, base: str = "") -> str:
|
||||||
|
|
||||||
EVENTS = frozenset({
|
EVENTS = frozenset({
|
||||||
"home.locate", "home.digest_signup", "home.records_click", "home.share",
|
"home.locate", "home.digest_signup", "home.records_click", "home.share",
|
||||||
|
# Approximate-location fallback (data/geoip.py). Pairs with home.locate:
|
||||||
|
# that one counts taps on "Use my location" whatever the outcome, these two
|
||||||
|
# count how often the IP fallback rescued a visitor who got no position
|
||||||
|
# (geoip_shown) and how often the guess was wrong enough to correct
|
||||||
|
# (geoip_fixed). Aggregate counts only — no location dimension, so the
|
||||||
|
# suggestion itself is never recorded anywhere.
|
||||||
|
"home.geoip_shown", "home.geoip_fixed",
|
||||||
})
|
})
|
||||||
# home.nav_{surface}: only these surfaces, so the key space stays bounded.
|
# home.nav_{surface}: only these surfaces, so the key space stays bounded.
|
||||||
NAV_SURFACES = frozenset({
|
NAV_SURFACES = frozenset({
|
||||||
|
|
|
||||||
330
backend/data/geoip.py
Normal file
330
backend/data/geoip.py
Normal file
|
|
@ -0,0 +1,330 @@
|
||||||
|
"""Coarse IP -> approximate-location lookup, for visitors who don't share
|
||||||
|
browser geolocation.
|
||||||
|
|
||||||
|
This is a *fallback suggestion*, never a position. Browser geolocation is
|
||||||
|
metres-accurate; a city-level IP database is tens of kilometres accurate against
|
||||||
|
a ~2 mile grid cell, so everything here is shaped so the caller cannot mistake
|
||||||
|
one for the other: the payload is explicitly labelled approximate, and the UI
|
||||||
|
that consumes it must offer a correction affordance (see frontend/static/geoip.js).
|
||||||
|
|
||||||
|
Design rules this module enforces, in order of importance:
|
||||||
|
|
||||||
|
1. **The IP is never persisted.** ``lookup()`` takes a string, reads a memory
|
||||||
|
-mapped database, and returns a dict. Nothing is written anywhere — no DB
|
||||||
|
row, no log line, no metric dimension. The web route deliberately keeps the
|
||||||
|
geoip category out of the access log too (see web/app.py), so there is never
|
||||||
|
a joinable (IP, derived-city) record on disk. Callers must not log the
|
||||||
|
argument.
|
||||||
|
|
||||||
|
2. **The lookup is local.** The database is a file on our own disk in MaxMind's
|
||||||
|
open MMDB format. No third-party API is called, so the visitor's IP never
|
||||||
|
leaves the server and no data-processor relationship is created.
|
||||||
|
|
||||||
|
3. **Off unless deliberately turned on.** ``THERMOGRAPH_GEOIP`` must be truthy
|
||||||
|
*and* ``THERMOGRAPH_GEOIP_DB`` must point at a readable file *and* the
|
||||||
|
``maxminddb`` reader must be importable. Any one of those missing makes
|
||||||
|
``enabled()`` False and every ``lookup()`` return None — which the route
|
||||||
|
turns into a 204 and the client treats exactly like today's no-fallback
|
||||||
|
behaviour. There is no failure mode here that is worse than the status quo.
|
||||||
|
|
||||||
|
4. **Silence beats a wrong guess.** A reserved/private/CGNAT/loopback address,
|
||||||
|
an unparseable one, a record with no city (a country centroid would put a
|
||||||
|
VPN user in the geographic middle of a country they aren't in), or a record
|
||||||
|
whose stated accuracy radius is worse than
|
||||||
|
``THERMOGRAPH_GEOIP_MAX_RADIUS_KM`` all return None rather than something
|
||||||
|
plausible-looking.
|
||||||
|
|
||||||
|
Database choice is a deployment concern, not a code one: both DB-IP's
|
||||||
|
*IP to City Lite* (CC BY 4.0, no account) and MaxMind's *GeoLite2 City*
|
||||||
|
(GeoLite EULA, account + licence key) ship the same MMDB format with the same
|
||||||
|
record shape, so either file works unchanged. ``attribution()`` reads the
|
||||||
|
file's own metadata so the credit line the UI renders follows whichever file is
|
||||||
|
installed. See GEOIP-APPROX-LOCATION.md for the comparison and the recommendation.
|
||||||
|
"""
|
||||||
|
import ipaddress
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
|
||||||
|
from data import grid
|
||||||
|
|
||||||
|
FLAG_ENV = "THERMOGRAPH_GEOIP"
|
||||||
|
DB_ENV = "THERMOGRAPH_GEOIP_DB"
|
||||||
|
RADIUS_ENV = "THERMOGRAPH_GEOIP_MAX_RADIUS_KM"
|
||||||
|
ATTRIB_ENV = "THERMOGRAPH_GEOIP_ATTRIBUTION"
|
||||||
|
|
||||||
|
# City-level databases claim ~50 km typical accuracy. Anything the file itself
|
||||||
|
# admits is worse than this is a region/country centroid dressed up as a city,
|
||||||
|
# which is exactly the "wrong part of the map" guess we'd rather not make.
|
||||||
|
DEFAULT_MAX_RADIUS_KM = 100.0
|
||||||
|
|
||||||
|
# Extra reserved ranges checked explicitly rather than leaning on
|
||||||
|
# ipaddress.is_global alone: the exact membership of is_private/is_global has
|
||||||
|
# moved between CPython releases (100.64/10 in particular), and "which Python
|
||||||
|
# is this container on" must not decide whether we try to geolocate a
|
||||||
|
# carrier-NAT address.
|
||||||
|
_EXTRA_RESERVED = (
|
||||||
|
ipaddress.ip_network("100.64.0.0/10"), # RFC 6598 carrier-grade NAT
|
||||||
|
ipaddress.ip_network("192.0.0.0/24"), # IETF protocol assignments
|
||||||
|
ipaddress.ip_network("198.18.0.0/15"), # benchmarking
|
||||||
|
ipaddress.ip_network("fc00::/7"), # IPv6 unique-local
|
||||||
|
ipaddress.ip_network("2001:db8::/32"), # documentation
|
||||||
|
)
|
||||||
|
|
||||||
|
_lock = threading.Lock()
|
||||||
|
_reader = None
|
||||||
|
_reader_key: "tuple | None" = None # (path, mtime, size) the open reader came from
|
||||||
|
|
||||||
|
|
||||||
|
def _truthy(v: "str | None") -> bool:
|
||||||
|
return (v or "").strip().lower() in ("1", "true", "yes", "on")
|
||||||
|
|
||||||
|
|
||||||
|
def db_path() -> str:
|
||||||
|
return os.environ.get(DB_ENV, "").strip()
|
||||||
|
|
||||||
|
|
||||||
|
def max_radius_km() -> float:
|
||||||
|
try:
|
||||||
|
return float(os.environ.get(RADIUS_ENV, "") or DEFAULT_MAX_RADIUS_KM)
|
||||||
|
except ValueError:
|
||||||
|
return DEFAULT_MAX_RADIUS_KM
|
||||||
|
|
||||||
|
|
||||||
|
def enabled() -> bool:
|
||||||
|
"""True only when the flag is set AND a database file is actually present.
|
||||||
|
Read on every call (not cached) so flipping the flag or dropping the file in
|
||||||
|
doesn't need a restart, and so a test can toggle it with monkeypatch.setenv."""
|
||||||
|
if not _truthy(os.environ.get(FLAG_ENV)):
|
||||||
|
return False
|
||||||
|
path = db_path()
|
||||||
|
return bool(path) and os.path.isfile(path)
|
||||||
|
|
||||||
|
|
||||||
|
def _open():
|
||||||
|
"""The memory-mapped reader for the configured database, or None.
|
||||||
|
|
||||||
|
Re-opens when the file's (mtime, size) changes so the monthly refresh job
|
||||||
|
can swap the file in atomically and be picked up without a deploy. mmap
|
||||||
|
means the 100-200 MB file costs page cache, not process RSS, and is shared
|
||||||
|
across uvicorn workers on the same host.
|
||||||
|
"""
|
||||||
|
global _reader, _reader_key
|
||||||
|
path = db_path()
|
||||||
|
if not path:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
st = os.stat(path)
|
||||||
|
except OSError:
|
||||||
|
return None
|
||||||
|
key = (path, st.st_mtime_ns, st.st_size)
|
||||||
|
with _lock:
|
||||||
|
if _reader is not None and _reader_key == key:
|
||||||
|
return _reader
|
||||||
|
try:
|
||||||
|
import maxminddb
|
||||||
|
except ImportError:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
new = maxminddb.open_database(path, maxminddb.MODE_MMAP)
|
||||||
|
except Exception: # noqa: BLE001 - a corrupt/half-written file must not 500
|
||||||
|
return None
|
||||||
|
old = _reader
|
||||||
|
_reader, _reader_key = new, key
|
||||||
|
if old is not None:
|
||||||
|
try:
|
||||||
|
old.close()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass
|
||||||
|
return new
|
||||||
|
|
||||||
|
|
||||||
|
def reset() -> None:
|
||||||
|
"""Drop the cached reader. For tests, and for anything that rewrites the
|
||||||
|
database in place rather than swapping it atomically."""
|
||||||
|
global _reader, _reader_key
|
||||||
|
with _lock:
|
||||||
|
r, _reader, _reader_key = _reader, None, None
|
||||||
|
if r is not None:
|
||||||
|
try:
|
||||||
|
r.close()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def parse_ip(raw: "str | None") -> "ipaddress._BaseAddress | None":
|
||||||
|
"""The routable address in ``raw``, or None.
|
||||||
|
|
||||||
|
Handles the shapes an X-Forwarded-For hop actually arrives in — a bare
|
||||||
|
address, ``[v6]:port``, ``v4:port``, a ``%zone`` suffix — and rejects
|
||||||
|
everything that can't identify a location on the public internet: private
|
||||||
|
LAN ranges (every LAN-dev and internal-proxy request), loopback, link-local,
|
||||||
|
multicast, carrier-grade NAT, IPv6 unique-local, and anything unparseable.
|
||||||
|
An IPv4-mapped IPv6 address is unwrapped so ``::ffff:8.8.8.8`` is treated as
|
||||||
|
the IPv4 address it is.
|
||||||
|
"""
|
||||||
|
s = (raw or "").strip()
|
||||||
|
if not s:
|
||||||
|
return None
|
||||||
|
if s.startswith("["): # [2001:db8::1]:443
|
||||||
|
s = s[1:].partition("]")[0]
|
||||||
|
elif s.count(":") == 1: # 8.8.8.8:443 (a lone colon is never v6)
|
||||||
|
s = s.partition(":")[0]
|
||||||
|
s = s.partition("%")[0] # fe80::1%eth0
|
||||||
|
try:
|
||||||
|
addr = ipaddress.ip_address(s)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
mapped = getattr(addr, "ipv4_mapped", None)
|
||||||
|
if mapped is not None:
|
||||||
|
addr = mapped
|
||||||
|
if not addr.is_global:
|
||||||
|
return None
|
||||||
|
if addr.is_private or addr.is_loopback or addr.is_link_local:
|
||||||
|
return None
|
||||||
|
if addr.is_multicast or addr.is_reserved or addr.is_unspecified:
|
||||||
|
return None
|
||||||
|
for net in _EXTRA_RESERVED:
|
||||||
|
if addr.version == net.version and addr in net:
|
||||||
|
return None
|
||||||
|
return addr
|
||||||
|
|
||||||
|
|
||||||
|
def _name(node) -> str:
|
||||||
|
"""The English display name out of an MMDB names map. Both DB-IP Lite and
|
||||||
|
GeoLite2 store ``{"names": {"en": ..., "de": ...}}``; DB-IP Lite is
|
||||||
|
English-only in some builds, so fall back to whatever single name exists
|
||||||
|
rather than dropping an otherwise-good record."""
|
||||||
|
if not isinstance(node, dict):
|
||||||
|
return ""
|
||||||
|
names = node.get("names")
|
||||||
|
if not isinstance(names, dict):
|
||||||
|
return ""
|
||||||
|
for key in ("en", "en-US"):
|
||||||
|
v = names.get(key)
|
||||||
|
if isinstance(v, str) and v.strip():
|
||||||
|
return v.strip()
|
||||||
|
for v in names.values():
|
||||||
|
if isinstance(v, str) and v.strip():
|
||||||
|
return v.strip()
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def suggestion_from_record(record, *, max_radius: "float | None" = None) -> "dict | None":
|
||||||
|
"""Turn one raw MMDB record into the wire payload, or None if it is too
|
||||||
|
coarse to be worth showing. Split out from ``lookup()`` so every rejection
|
||||||
|
rule is testable without a database file."""
|
||||||
|
if not isinstance(record, dict):
|
||||||
|
return None
|
||||||
|
loc = record.get("location")
|
||||||
|
if not isinstance(loc, dict):
|
||||||
|
return None
|
||||||
|
lat, lon = loc.get("latitude"), loc.get("longitude")
|
||||||
|
if not isinstance(lat, (int, float)) or not isinstance(lon, (int, float)):
|
||||||
|
return None
|
||||||
|
if not (-90.0 <= lat <= 90.0) or not (-180.0 <= lon <= 180.0):
|
||||||
|
return None
|
||||||
|
|
||||||
|
# No city name means a country (or continent) centroid. Showing one would
|
||||||
|
# put every VPN user in the geographic middle of a country they may not be
|
||||||
|
# in, which is worse than showing nothing: it looks like a real answer.
|
||||||
|
city = _name(record.get("city"))
|
||||||
|
if not city:
|
||||||
|
return None
|
||||||
|
|
||||||
|
radius = loc.get("accuracy_radius")
|
||||||
|
limit = max_radius_km() if max_radius is None else max_radius
|
||||||
|
if isinstance(radius, (int, float)) and radius > limit:
|
||||||
|
return None
|
||||||
|
|
||||||
|
subs = record.get("subdivisions")
|
||||||
|
region = _name(subs[0]) if isinstance(subs, list) and subs else ""
|
||||||
|
country_node = record.get("country") or record.get("registered_country") or {}
|
||||||
|
country = _name(country_node)
|
||||||
|
code = country_node.get("iso_code") if isinstance(country_node, dict) else None
|
||||||
|
|
||||||
|
# Snap to the app's own ~2 mile grid. This drops the database's fake
|
||||||
|
# precision (city centroids are published to 4+ decimal places, which is
|
||||||
|
# metres, for a value good to tens of kilometres) and returns exactly the
|
||||||
|
# cell the weather would be fetched for anyway — so the coordinate carries
|
||||||
|
# no more information than the city name already did.
|
||||||
|
cell = grid.snap(float(lat), float(lon))
|
||||||
|
|
||||||
|
# "Springfield" alone is ambiguous; "Springfield, Illinois" is not. Region
|
||||||
|
# first (more useful locally), country when there is no region.
|
||||||
|
label = f"{city}, {region}" if region else (f"{city}, {country}" if country else city)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"approximate": True, # never omit: the client keys its copy off this
|
||||||
|
"source": "ip",
|
||||||
|
"label": label,
|
||||||
|
"city": city,
|
||||||
|
"region": region or None,
|
||||||
|
"country": country or None,
|
||||||
|
"country_code": code if isinstance(code, str) else None,
|
||||||
|
"lat": cell["center_lat"],
|
||||||
|
"lon": cell["center_lon"],
|
||||||
|
"accuracy_radius_km": radius if isinstance(radius, (int, float)) else None,
|
||||||
|
"attribution": attribution(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def lookup(ip: "str | None") -> "dict | None":
|
||||||
|
"""An approximate-location suggestion for ``ip``, or None.
|
||||||
|
|
||||||
|
None covers every degraded case — feature off, database missing, private or
|
||||||
|
malformed address, no matching record, record too coarse — so the caller has
|
||||||
|
exactly one branch to write and it is always "behave as if this feature did
|
||||||
|
not exist". Never raises; never logs or stores ``ip``.
|
||||||
|
"""
|
||||||
|
if not enabled():
|
||||||
|
return None
|
||||||
|
addr = parse_ip(ip)
|
||||||
|
if addr is None:
|
||||||
|
return None
|
||||||
|
reader = _open()
|
||||||
|
if reader is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
record = reader.get(str(addr))
|
||||||
|
except Exception: # noqa: BLE001 - a bad record must degrade, not 500
|
||||||
|
return None
|
||||||
|
if record is None:
|
||||||
|
return None
|
||||||
|
return suggestion_from_record(record)
|
||||||
|
|
||||||
|
|
||||||
|
# --- attribution -----------------------------------------------------------
|
||||||
|
# Both candidate databases are free *with attribution*, and each wants its own
|
||||||
|
# wording, so the credit is derived from the installed file's own metadata
|
||||||
|
# instead of hardcoded — swapping databases stays a config change. An explicit
|
||||||
|
# THERMOGRAPH_GEOIP_ATTRIBUTION ("text|url") overrides, for a file whose
|
||||||
|
# database_type we don't recognise.
|
||||||
|
_ATTRIBUTIONS = {
|
||||||
|
"dbip": ("IP Geolocation by DB-IP", "https://db-ip.com"),
|
||||||
|
"geolite2": ("This product includes GeoLite data created by MaxMind",
|
||||||
|
"https://www.maxmind.com"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def attribution() -> "dict | None":
|
||||||
|
"""``{"text", "url"}`` for the installed database, or None if it can't be
|
||||||
|
determined. Rendered next to any result the visitor sees — both licences
|
||||||
|
require credit on pages that display their data."""
|
||||||
|
override = os.environ.get(ATTRIB_ENV, "").strip()
|
||||||
|
if override:
|
||||||
|
text, _, url = override.partition("|")
|
||||||
|
return {"text": text.strip(), "url": url.strip() or None}
|
||||||
|
reader = _open()
|
||||||
|
if reader is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
kind = (reader.metadata().database_type or "").lower()
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return None
|
||||||
|
if "dbip" in kind or "db-ip" in kind:
|
||||||
|
text, url = _ATTRIBUTIONS["dbip"]
|
||||||
|
elif "geolite" in kind:
|
||||||
|
text, url = _ATTRIBUTIONS["geolite2"]
|
||||||
|
else:
|
||||||
|
return None
|
||||||
|
return {"text": text, "url": url}
|
||||||
|
|
@ -25,6 +25,13 @@ websockets==16.0
|
||||||
# Recurring worker-tier jobs — city warming, IndexNow pings (see
|
# Recurring worker-tier jobs — city warming, IndexNow pings (see
|
||||||
# notifications/scheduler.py). In-process; no broker/queue needed at this scale.
|
# notifications/scheduler.py). In-process; no broker/queue needed at this scale.
|
||||||
apscheduler==3.11.3
|
apscheduler==3.11.3
|
||||||
|
# MMDB reader for the optional approximate-location fallback (data/geoip.py).
|
||||||
|
# The reader only, not MaxMind's geoip2 model layer — the raw dict record is all
|
||||||
|
# we read, and staying on maxminddb keeps DB-IP's and MaxMind's files equally
|
||||||
|
# usable. Imported lazily inside geoip.py so a missing wheel disables the
|
||||||
|
# feature instead of breaking boot; pinned here so the flag is flippable on a
|
||||||
|
# deployed image without a rebuild.
|
||||||
|
maxminddb==2.8.2
|
||||||
# The lake role's SQL engine (lake_app.py /query): embedded, parallel
|
# The lake role's SQL engine (lake_app.py /query): embedded, parallel
|
||||||
# partition-pruned scans over the ERA5 hive table. Only lake_app imports it.
|
# partition-pruned scans over the ERA5 hive table. Only lake_app imports it.
|
||||||
duckdb==1.4.2
|
duckdb==1.4.2
|
||||||
|
|
|
||||||
263
backend/tests/data/test_geoip.py
Normal file
263
backend/tests/data/test_geoip.py
Normal file
|
|
@ -0,0 +1,263 @@
|
||||||
|
"""Approximate-location fallback (data/geoip.py).
|
||||||
|
|
||||||
|
Every rejection rule is tested against a raw MMDB-shaped record rather than a
|
||||||
|
real database file, so the suite stays hermetic and needs no 125 MB download —
|
||||||
|
the record shape below is what both DB-IP IP-to-City Lite and MaxMind GeoLite2
|
||||||
|
City actually return.
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from data import geoip
|
||||||
|
from data import grid
|
||||||
|
|
||||||
|
|
||||||
|
def _record(**over) -> dict:
|
||||||
|
"""A well-formed city record. Rotterdam, as in the design doc's example."""
|
||||||
|
rec = {
|
||||||
|
"city": {"names": {"en": "Rotterdam"}},
|
||||||
|
"country": {"iso_code": "NL", "names": {"en": "Netherlands"}},
|
||||||
|
"subdivisions": [{"names": {"en": "South Holland"}}],
|
||||||
|
"location": {"latitude": 51.9225, "longitude": 4.47917, "accuracy_radius": 20},
|
||||||
|
}
|
||||||
|
rec.update(over)
|
||||||
|
return rec
|
||||||
|
|
||||||
|
|
||||||
|
# --- address parsing --------------------------------------------------------
|
||||||
|
@pytest.mark.parametrize("raw", [
|
||||||
|
"8.8.8.8",
|
||||||
|
"8.8.8.8:51234", # an XFF hop can carry the source port
|
||||||
|
"2001:4860:4860::8888",
|
||||||
|
"[2001:4860:4860::8888]:443",
|
||||||
|
"::ffff:8.8.8.8", # IPv4-mapped IPv6 unwraps to the v4 address
|
||||||
|
" 8.8.8.8 ",
|
||||||
|
])
|
||||||
|
def test_parse_accepts_routable_addresses(raw):
|
||||||
|
assert geoip.parse_ip(raw) is not None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("raw", [
|
||||||
|
None, "", " ", "not-an-ip", "999.1.1.1", "8.8.8.8, 9.9.9.9", # a whole XFF chain
|
||||||
|
"127.0.0.1", "::1", # loopback
|
||||||
|
"10.0.0.4", "192.168.1.20", "172.16.5.5", # RFC 1918 — every LAN-dev request
|
||||||
|
"100.64.7.9", # RFC 6598 carrier-grade NAT (mobile networks)
|
||||||
|
"169.254.10.1", "fe80::1%eth0", # link-local
|
||||||
|
"fd00::1", # IPv6 unique-local
|
||||||
|
"2001:db8::1", # documentation range
|
||||||
|
"224.0.0.1", # multicast
|
||||||
|
"0.0.0.0", "::", # unspecified
|
||||||
|
])
|
||||||
|
def test_parse_rejects_everything_unroutable(raw):
|
||||||
|
assert geoip.parse_ip(raw) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_ipv4_mapped_v6_resolves_to_the_v4_address():
|
||||||
|
assert str(geoip.parse_ip("::ffff:8.8.8.8")) == "8.8.8.8"
|
||||||
|
|
||||||
|
|
||||||
|
# --- record -> suggestion ---------------------------------------------------
|
||||||
|
def test_good_record_becomes_a_labelled_approximate_suggestion():
|
||||||
|
s = geoip.suggestion_from_record(_record())
|
||||||
|
assert s["approximate"] is True and s["source"] == "ip"
|
||||||
|
assert s["label"] == "Rotterdam, South Holland"
|
||||||
|
assert s["city"] == "Rotterdam" and s["country_code"] == "NL"
|
||||||
|
|
||||||
|
|
||||||
|
def test_coordinates_are_snapped_to_the_app_grid_not_passed_through():
|
||||||
|
"""The database publishes a city centroid to five decimal places — metre
|
||||||
|
precision for a value good to tens of kilometres. We return the grid cell
|
||||||
|
centre instead, so the payload carries no more precision than it has
|
||||||
|
information."""
|
||||||
|
s = geoip.suggestion_from_record(_record())
|
||||||
|
cell = grid.snap(51.9225, 4.47917)
|
||||||
|
assert (s["lat"], s["lon"]) == (cell["center_lat"], cell["center_lon"])
|
||||||
|
assert s["lat"] != 51.9225
|
||||||
|
|
||||||
|
|
||||||
|
def test_region_absent_falls_back_to_country_in_the_label():
|
||||||
|
s = geoip.suggestion_from_record(_record(subdivisions=[]))
|
||||||
|
assert s["label"] == "Rotterdam, Netherlands"
|
||||||
|
|
||||||
|
|
||||||
|
def test_city_only_record_still_labels():
|
||||||
|
s = geoip.suggestion_from_record(
|
||||||
|
{"city": {"names": {"en": "Rotterdam"}}, "location": {"latitude": 1.0, "longitude": 2.0}})
|
||||||
|
assert s["label"] == "Rotterdam"
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_english_only_names_are_still_usable():
|
||||||
|
s = geoip.suggestion_from_record(_record(city={"names": {"nl": "Rotterdam"}}))
|
||||||
|
assert s["city"] == "Rotterdam"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("rec", [
|
||||||
|
None, {}, "garbage",
|
||||||
|
{"location": {"latitude": 51.9, "longitude": 4.4}}, # country-only: no city
|
||||||
|
{"city": {"names": {"en": "X"}}}, # no coordinates
|
||||||
|
{"city": {"names": {"en": "X"}}, "location": {"latitude": 51.9}}, # half coordinates
|
||||||
|
{"city": {"names": {"en": "X"}},
|
||||||
|
"location": {"latitude": "51.9", "longitude": "4.4"}}, # wrong types
|
||||||
|
{"city": {"names": {"en": "X"}},
|
||||||
|
"location": {"latitude": 991.0, "longitude": 4.4}}, # out of range
|
||||||
|
])
|
||||||
|
def test_unusable_records_suggest_nothing(rec):
|
||||||
|
assert geoip.suggestion_from_record(rec) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_country_centroid_is_never_offered_as_a_city():
|
||||||
|
"""The single most damaging failure mode: a VPN/proxy user gets a record
|
||||||
|
with a country but no city, and we drop a pin in the middle of a country
|
||||||
|
they are not in. Silence is the correct answer."""
|
||||||
|
rec = _record()
|
||||||
|
del rec["city"]
|
||||||
|
assert geoip.suggestion_from_record(rec) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_record_wider_than_the_accuracy_limit_is_rejected():
|
||||||
|
assert geoip.suggestion_from_record(
|
||||||
|
_record(location={"latitude": 51.9, "longitude": 4.4, "accuracy_radius": 500})) is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_record_with_no_stated_accuracy_is_accepted():
|
||||||
|
"""DB-IP Lite omits accuracy_radius entirely; requiring it would disable the
|
||||||
|
feature for that database."""
|
||||||
|
s = geoip.suggestion_from_record(
|
||||||
|
_record(location={"latitude": 51.9, "longitude": 4.4}))
|
||||||
|
assert s is not None and s["accuracy_radius_km"] is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_accuracy_limit_is_configurable(monkeypatch):
|
||||||
|
monkeypatch.setenv(geoip.RADIUS_ENV, "1000")
|
||||||
|
assert geoip.suggestion_from_record(
|
||||||
|
_record(location={"latitude": 51.9, "longitude": 4.4, "accuracy_radius": 500})) is not None
|
||||||
|
|
||||||
|
|
||||||
|
# --- feature flag -----------------------------------------------------------
|
||||||
|
def test_disabled_by_default(monkeypatch):
|
||||||
|
monkeypatch.delenv(geoip.FLAG_ENV, raising=False)
|
||||||
|
monkeypatch.delenv(geoip.DB_ENV, raising=False)
|
||||||
|
assert geoip.enabled() is False
|
||||||
|
assert geoip.lookup("8.8.8.8") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_flag_without_a_database_stays_disabled(monkeypatch, tmp_path):
|
||||||
|
monkeypatch.setenv(geoip.FLAG_ENV, "1")
|
||||||
|
monkeypatch.setenv(geoip.DB_ENV, str(tmp_path / "missing.mmdb"))
|
||||||
|
assert geoip.enabled() is False
|
||||||
|
assert geoip.lookup("8.8.8.8") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_database_without_the_flag_stays_disabled(monkeypatch, tmp_path):
|
||||||
|
db = tmp_path / "city.mmdb"
|
||||||
|
db.write_bytes(b"not really an mmdb")
|
||||||
|
monkeypatch.delenv(geoip.FLAG_ENV, raising=False)
|
||||||
|
monkeypatch.setenv(geoip.DB_ENV, str(db))
|
||||||
|
assert geoip.enabled() is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_unreadable_database_degrades_to_no_suggestion(monkeypatch, tmp_path):
|
||||||
|
"""A truncated or half-written file (a refresh job interrupted mid-copy)
|
||||||
|
must behave like the feature is off, not raise."""
|
||||||
|
db = tmp_path / "city.mmdb"
|
||||||
|
db.write_bytes(b"not really an mmdb")
|
||||||
|
monkeypatch.setenv(geoip.FLAG_ENV, "1")
|
||||||
|
monkeypatch.setenv(geoip.DB_ENV, str(db))
|
||||||
|
geoip.reset()
|
||||||
|
assert geoip.enabled() is True
|
||||||
|
assert geoip.lookup("8.8.8.8") is None
|
||||||
|
geoip.reset()
|
||||||
|
|
||||||
|
|
||||||
|
# --- lookup, end to end over a stand-in reader ------------------------------
|
||||||
|
class _FakeReader:
|
||||||
|
"""Stands in for maxminddb's reader: same tiny surface geoip.py uses."""
|
||||||
|
|
||||||
|
def __init__(self, by_ip, database_type="DBIP-City-Lite (compat=City)"):
|
||||||
|
self.by_ip = by_ip
|
||||||
|
self.asked = []
|
||||||
|
self._type = database_type
|
||||||
|
|
||||||
|
def get(self, ip):
|
||||||
|
self.asked.append(ip)
|
||||||
|
return self.by_ip.get(ip)
|
||||||
|
|
||||||
|
def metadata(self):
|
||||||
|
return type("M", (), {"database_type": self._type})()
|
||||||
|
|
||||||
|
def close(self):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def live(monkeypatch, tmp_path):
|
||||||
|
"""Feature on, with a stand-in reader — exercises enabled() + parse_ip() +
|
||||||
|
suggestion_from_record() together without a real database file."""
|
||||||
|
db = tmp_path / "city.mmdb"
|
||||||
|
db.write_bytes(b"x")
|
||||||
|
monkeypatch.setenv(geoip.FLAG_ENV, "1")
|
||||||
|
monkeypatch.setenv(geoip.DB_ENV, str(db))
|
||||||
|
reader = _FakeReader({"8.8.8.8": _record()})
|
||||||
|
monkeypatch.setattr(geoip, "_open", lambda: reader)
|
||||||
|
return reader
|
||||||
|
|
||||||
|
|
||||||
|
def test_lookup_end_to_end(live):
|
||||||
|
s = geoip.lookup("8.8.8.8")
|
||||||
|
assert s["city"] == "Rotterdam"
|
||||||
|
assert s["attribution"] == {"text": "IP Geolocation by DB-IP", "url": "https://db-ip.com"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_lookup_never_queries_the_database_for_a_private_address(live):
|
||||||
|
assert geoip.lookup("192.168.1.20") is None
|
||||||
|
assert live.asked == [] # rejected before the file is touched at all
|
||||||
|
|
||||||
|
|
||||||
|
def test_lookup_returns_none_for_an_ip_with_no_record(live):
|
||||||
|
assert geoip.lookup("9.9.9.9") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_lookup_survives_a_reader_that_raises(live, monkeypatch):
|
||||||
|
def boom(ip):
|
||||||
|
raise ValueError("corrupt node")
|
||||||
|
monkeypatch.setattr(live, "get", boom)
|
||||||
|
assert geoip.lookup("8.8.8.8") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_attribution_follows_the_installed_database(monkeypatch, tmp_path):
|
||||||
|
db = tmp_path / "city.mmdb"
|
||||||
|
db.write_bytes(b"x")
|
||||||
|
monkeypatch.setenv(geoip.FLAG_ENV, "1")
|
||||||
|
monkeypatch.setenv(geoip.DB_ENV, str(db))
|
||||||
|
monkeypatch.setattr(geoip, "_open", lambda: _FakeReader({}, "GeoLite2-City"))
|
||||||
|
assert "MaxMind" in geoip.attribution()["text"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_attribution_can_be_overridden_by_env(monkeypatch):
|
||||||
|
monkeypatch.setenv(geoip.ATTRIB_ENV, "Data by Someone|https://example.org")
|
||||||
|
assert geoip.attribution() == {"text": "Data by Someone", "url": "https://example.org"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_attribution_is_none_for_an_unrecognised_database(monkeypatch, tmp_path):
|
||||||
|
db = tmp_path / "city.mmdb"
|
||||||
|
db.write_bytes(b"x")
|
||||||
|
monkeypatch.delenv(geoip.ATTRIB_ENV, raising=False)
|
||||||
|
monkeypatch.setenv(geoip.DB_ENV, str(db))
|
||||||
|
monkeypatch.setattr(geoip, "_open", lambda: _FakeReader({}, "Homegrown-City"))
|
||||||
|
assert geoip.attribution() is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.skipif(os.environ.get("THERMOGRAPH_GEOIP_DB", "") == "",
|
||||||
|
reason="no real MMDB installed; set THERMOGRAPH_GEOIP_DB to run")
|
||||||
|
def test_real_database_opens_and_answers():
|
||||||
|
"""Opt-in smoke test against an actual installed database — the one thing
|
||||||
|
the stand-in reader cannot prove (that our record-shape assumptions match a
|
||||||
|
real file). Skipped in CI, run by hand after the refresh job first lands a
|
||||||
|
file on a box."""
|
||||||
|
import os as _os
|
||||||
|
_os.environ[geoip.FLAG_ENV] = "1"
|
||||||
|
geoip.reset()
|
||||||
|
assert geoip.enabled()
|
||||||
|
assert geoip.lookup("8.8.8.8") is not None or geoip.lookup("1.1.1.1") is not None
|
||||||
|
geoip.reset()
|
||||||
114
backend/tests/web/test_geoip_api.py
Normal file
114
backend/tests/web/test_geoip_api.py
Normal file
|
|
@ -0,0 +1,114 @@
|
||||||
|
"""GET /api/v2/geoip — the approximate-location fallback endpoint.
|
||||||
|
|
||||||
|
The two properties worth locking down are behavioural, not cosmetic: it is
|
||||||
|
inert when the flag is off (so shipping it disabled changes nothing), and it
|
||||||
|
never writes the client IP anywhere.
|
||||||
|
"""
|
||||||
|
import glob
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from core import audit
|
||||||
|
from data import geoip
|
||||||
|
from web import app as appmod
|
||||||
|
|
||||||
|
URL = "/thermograph/api/v2/geoip"
|
||||||
|
|
||||||
|
_SUGGESTION = {
|
||||||
|
"approximate": True, "source": "ip", "label": "Rotterdam, South Holland",
|
||||||
|
"city": "Rotterdam", "region": "South Holland", "country": "Netherlands",
|
||||||
|
"country_code": "NL", "lat": 51.92028, "lon": 4.48339,
|
||||||
|
"accuracy_radius_km": 20,
|
||||||
|
"attribution": {"text": "IP Geolocation by DB-IP", "url": "https://db-ip.com"},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def client():
|
||||||
|
return TestClient(appmod.app)
|
||||||
|
|
||||||
|
|
||||||
|
def _access_lines():
|
||||||
|
lines = []
|
||||||
|
for path in glob.glob(os.path.join(audit.ACCESS_DIR, "access-*.jsonl")):
|
||||||
|
with open(path, encoding="utf-8") as fh:
|
||||||
|
lines += [json.loads(ln) for ln in fh if ln.strip()]
|
||||||
|
return lines
|
||||||
|
|
||||||
|
|
||||||
|
def test_disabled_answers_204_with_no_body(client, monkeypatch):
|
||||||
|
monkeypatch.delenv(geoip.FLAG_ENV, raising=False)
|
||||||
|
r = client.get(URL, headers={"X-Forwarded-For": "8.8.8.8"})
|
||||||
|
assert r.status_code == 204
|
||||||
|
assert r.content == b""
|
||||||
|
|
||||||
|
|
||||||
|
def test_enabled_hit_returns_the_suggestion(client, monkeypatch):
|
||||||
|
monkeypatch.setattr(geoip, "lookup", lambda ip: dict(_SUGGESTION))
|
||||||
|
r = client.get(URL, headers={"X-Forwarded-For": "8.8.8.8"})
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body["approximate"] is True and body["label"] == "Rotterdam, South Holland"
|
||||||
|
assert body["attribution"]["url"] == "https://db-ip.com"
|
||||||
|
|
||||||
|
|
||||||
|
def test_response_is_never_cacheable_by_a_shared_cache(client, monkeypatch):
|
||||||
|
"""This response is per-visitor by construction. A CDN or proxy that reused
|
||||||
|
it would show one visitor another's city — the exact hazard that keeps this
|
||||||
|
lookup out of the server-rendered homepage."""
|
||||||
|
monkeypatch.setattr(geoip, "lookup", lambda ip: dict(_SUGGESTION))
|
||||||
|
r = client.get(URL, headers={"X-Forwarded-For": "8.8.8.8"})
|
||||||
|
assert "no-store" in r.headers["cache-control"]
|
||||||
|
assert r.headers["vary"] == "*"
|
||||||
|
|
||||||
|
|
||||||
|
def test_miss_answers_204(client, monkeypatch):
|
||||||
|
monkeypatch.setattr(geoip, "lookup", lambda ip: None)
|
||||||
|
r = client.get(URL, headers={"X-Forwarded-For": "8.8.8.8"})
|
||||||
|
assert r.status_code == 204
|
||||||
|
assert "no-store" in r.headers["cache-control"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_left_most_forwarded_hop_is_what_gets_looked_up(client, monkeypatch):
|
||||||
|
seen = []
|
||||||
|
monkeypatch.setattr(geoip, "lookup", lambda ip: seen.append(ip) or None)
|
||||||
|
client.get(URL, headers={"X-Forwarded-For": "8.8.8.8, 10.0.0.1, 10.0.0.2"})
|
||||||
|
assert seen == ["8.8.8.8"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_forwarded_header_still_answers_cleanly(client, monkeypatch):
|
||||||
|
"""No XFF (direct hit, LAN dev) falls through to the peer address, which is
|
||||||
|
the loopback TestClient address — private, so no suggestion, no error."""
|
||||||
|
monkeypatch.setenv(geoip.FLAG_ENV, "1")
|
||||||
|
r = client.get(URL)
|
||||||
|
assert r.status_code == 204
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_client_ip_is_not_written_to_the_access_log(client, monkeypatch):
|
||||||
|
"""The whole point: reading an IP to derive a location must not leave a
|
||||||
|
stored, joinable (IP, location) pair behind."""
|
||||||
|
monkeypatch.setattr(geoip, "lookup", lambda ip: dict(_SUGGESTION))
|
||||||
|
before = len(_access_lines())
|
||||||
|
client.get(URL, headers={"X-Forwarded-For": "8.8.8.8"})
|
||||||
|
after = _access_lines()
|
||||||
|
assert len(after) == before
|
||||||
|
assert not any(ln.get("ip") == "8.8.8.8" for ln in after)
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_normal_route_still_is_logged(client):
|
||||||
|
"""Guards the exclusion above from silently widening to everything."""
|
||||||
|
before = len(_access_lines())
|
||||||
|
client.get("/thermograph/api/version")
|
||||||
|
assert len(_access_lines()) > before
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_route_takes_no_parameters(client, monkeypatch):
|
||||||
|
"""You cannot ask this endpoint about someone else's address: there is no
|
||||||
|
input but the connection itself, and a supplied one is ignored."""
|
||||||
|
seen = []
|
||||||
|
monkeypatch.setattr(geoip, "lookup", lambda ip: seen.append(ip) or None)
|
||||||
|
client.get(f"{URL}?ip=1.2.3.4", headers={"X-Forwarded-For": "8.8.8.8"})
|
||||||
|
assert seen == ["8.8.8.8"]
|
||||||
|
|
@ -21,6 +21,7 @@ from accounts import api_accounts
|
||||||
from api import content_routes
|
from api import content_routes
|
||||||
from core import audit
|
from core import audit
|
||||||
from data import climate
|
from data import climate
|
||||||
|
from data import geoip
|
||||||
from notifications import digest
|
from notifications import digest
|
||||||
from notifications import discord_bot
|
from notifications import discord_bot
|
||||||
from notifications import discord_interactions as discord_interactions_mod
|
from notifications import discord_interactions as discord_interactions_mod
|
||||||
|
|
@ -306,7 +307,11 @@ async def revalidate_static(request, call_next):
|
||||||
await run_in_threadpool(metrics.record_inbound, cat, response.status_code)
|
await run_in_threadpool(metrics.record_inbound, cat, response.status_code)
|
||||||
# Retain per-request client IPs for later analysis; skip static assets and the
|
# Retain per-request client IPs for later analysis; skip static assets and the
|
||||||
# dashboard's own metrics polling to keep the log to real, meaningful traffic.
|
# dashboard's own metrics polling to keep the log to real, meaningful traffic.
|
||||||
if cat not in ("static", "metrics", "event"):
|
# "geoip" is excluded for a different reason: that route derives a coarse
|
||||||
|
# location FROM the client IP, so logging the two together would create the
|
||||||
|
# one artefact this feature exists to avoid — a stored, joinable
|
||||||
|
# (IP, approximate-location) pair. See api_geoip.
|
||||||
|
if cat not in ("static", "metrics", "event", "geoip"):
|
||||||
await run_in_threadpool(audit.log_access, {
|
await run_in_threadpool(audit.log_access, {
|
||||||
"ip": _client_ip(request), "method": request.method,
|
"ip": _client_ip(request), "method": request.method,
|
||||||
"path": path, "status": response.status_code, "cat": cat})
|
"path": path, "status": response.status_code, "cat": cat})
|
||||||
|
|
@ -809,6 +814,39 @@ async def api_event(request: Request) -> Response:
|
||||||
return Response(status_code=204)
|
return Response(status_code=204)
|
||||||
|
|
||||||
|
|
||||||
|
async def api_geoip(request: Request) -> Response:
|
||||||
|
"""An approximate location suggested from this request's own IP, for a
|
||||||
|
visitor who has not shared browser geolocation. Feature-flagged OFF.
|
||||||
|
|
||||||
|
204 (no body) is the normal answer, and means "no suggestion" for every
|
||||||
|
reason there can be: the flag is off, no database is installed, the address
|
||||||
|
is private/reserved/unparseable, no record matched, or the record was too
|
||||||
|
coarse to be worth showing. The client treats 204 exactly as it behaves
|
||||||
|
today, so a disabled or broken lookup is indistinguishable from the feature
|
||||||
|
never having shipped.
|
||||||
|
|
||||||
|
Privacy posture, deliberately:
|
||||||
|
* the IP is read from this request's headers, used, and dropped — it is
|
||||||
|
not written to the access log (see the middleware's category skip), any
|
||||||
|
database, any metric dimension, or any log line here;
|
||||||
|
* ``no-store`` + ``Vary: *`` so no shared cache or CDN can ever hand one
|
||||||
|
visitor's suggestion to another — this is why the lookup is a separate,
|
||||||
|
client-called endpoint rather than folded into the server-rendered
|
||||||
|
homepage, whose HTML must stay byte-identical for everyone;
|
||||||
|
* GET with no parameters and no body: there is nothing a caller can pass
|
||||||
|
that would let them geolocate an address other than their own.
|
||||||
|
"""
|
||||||
|
# No RunAudit and no metrics dimension: both would record the derived
|
||||||
|
# location, which is the sensitive half of the pair.
|
||||||
|
hit = await run_in_threadpool(geoip.lookup, _client_ip(request))
|
||||||
|
if hit is None:
|
||||||
|
return Response(status_code=204, headers={"Cache-Control": "no-store"})
|
||||||
|
return Response(
|
||||||
|
content=json.dumps(hit), media_type="application/json",
|
||||||
|
headers={"Cache-Control": "no-store, private", "Vary": "*"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# --- API versioning --------------------------------------------------------
|
# --- API versioning --------------------------------------------------------
|
||||||
# Non-backward-compatible additions land in a new version; every version stays
|
# Non-backward-compatible additions land in a new version; every version stays
|
||||||
# mounted and served simultaneously. v1 is the original contract (grade/geocode);
|
# mounted and served simultaneously. v1 is the original contract (grade/geocode);
|
||||||
|
|
@ -828,6 +866,7 @@ v2.add_api_route("/day", api_day, methods=["GET"])
|
||||||
v2.add_api_route("/score", api_score, methods=["GET"])
|
v2.add_api_route("/score", api_score, methods=["GET"])
|
||||||
v2.add_api_route("/forecast", api_forecast, methods=["GET"])
|
v2.add_api_route("/forecast", api_forecast, methods=["GET"])
|
||||||
v2.add_api_route("/cell", api_cell, methods=["GET"])
|
v2.add_api_route("/cell", api_cell, methods=["GET"])
|
||||||
|
v2.add_api_route("/geoip", api_geoip, methods=["GET"], include_in_schema=False)
|
||||||
v2.add_api_route("/metrics", api_metrics, methods=["GET"], include_in_schema=False)
|
v2.add_api_route("/metrics", api_metrics, methods=["GET"], include_in_schema=False)
|
||||||
v2.add_api_route("/event", api_event, methods=["POST"], include_in_schema=False)
|
v2.add_api_route("/event", api_event, methods=["POST"], include_in_schema=False)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -380,12 +380,27 @@ def about_page(request: Request) -> Response:
|
||||||
return _respond_html(request, "about.html.j2", **ctx)
|
return _respond_html(request, "about.html.j2", **ctx)
|
||||||
|
|
||||||
|
|
||||||
|
def geoip_enabled() -> bool:
|
||||||
|
"""Whether the backend's approximate-location fallback is switched on
|
||||||
|
(backend/data/geoip.py). Read here for ONE reason: the privacy page's
|
||||||
|
"Your location" section makes a factual claim about IP lookups, and a
|
||||||
|
factual claim must not survive the behaviour it describes. Both containers
|
||||||
|
read the same THERMOGRAPH_GEOIP out of the same /etc/thermograph.env, so
|
||||||
|
the statement and the behaviour flip together in one deploy.
|
||||||
|
|
||||||
|
Nothing else in this repo branches on it — the interactive tool just calls
|
||||||
|
the endpoint and gets a 204 when the feature is off.
|
||||||
|
"""
|
||||||
|
return os.environ.get("THERMOGRAPH_GEOIP", "").strip().lower() in ("1", "true", "yes", "on")
|
||||||
|
|
||||||
|
|
||||||
def privacy_page(request: Request) -> Response:
|
def privacy_page(request: Request) -> Response:
|
||||||
ctx = {
|
ctx = {
|
||||||
"canonical_path": "/privacy",
|
"canonical_path": "/privacy",
|
||||||
"breadcrumb": [("Home", f"{BASE}/"), ("Privacy", None)],
|
"breadcrumb": [("Home", f"{BASE}/"), ("Privacy", None)],
|
||||||
"page_title": PAGES["privacy"]["title"],
|
"page_title": PAGES["privacy"]["title"],
|
||||||
"page_description": PAGES["privacy"]["description"],
|
"page_description": PAGES["privacy"]["description"],
|
||||||
|
"geoip": geoip_enabled(),
|
||||||
}
|
}
|
||||||
return _respond_html(request, "privacy.html.j2", **ctx)
|
return _respond_html(request, "privacy.html.j2", **ctx)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -7,10 +7,16 @@ import { initFindButton, setFindLabel } from "./mappicker.js";
|
||||||
import { W, PW, H, PL, PR, plotTop, plotBot, xLabY, setChartWidth, chartPalette,
|
import { W, PW, H, PL, PR, plotTop, plotBot, xLabY, setChartWidth, chartPalette,
|
||||||
tempChart, precipChart, dryChart, attachChartHover } from "./chart.js";
|
tempChart, precipChart, dryChart, attachChartHover } from "./chart.js";
|
||||||
import { track } from "./digest.js";
|
import { track } from "./digest.js";
|
||||||
|
import { fetchApprox, renderApprox, clearApprox } from "./geoip.js";
|
||||||
import { TIER_COLORS, SCALE_TEMP, drynessColor, pctOrd, esc, todayISO, placeLabel,
|
import { TIER_COLORS, SCALE_TEMP, drynessColor, pctOrd, esc, todayISO, placeLabel,
|
||||||
tierKeySegs, GUIDE_LINK, fmtPrecip, fmtWind, fmtHumid } from "./shared.js";
|
tierKeySegs, GUIDE_LINK, fmtPrecip, fmtWind, fmtHumid } from "./shared.js";
|
||||||
|
|
||||||
let selected = null; // {lat, lon}
|
let selected = null; // {lat, lon}
|
||||||
|
// True while `selected` came from the IP fallback rather than a real choice.
|
||||||
|
// A guess is loaded (so the page is useful) but never remembered and never
|
||||||
|
// written into the URL — see geoip.js for why that distinction matters.
|
||||||
|
let approx = false;
|
||||||
|
let approxLabel = ""; // the city the guess named, for headlines that must not overstate it
|
||||||
|
|
||||||
const dateInput = document.getElementById("date-input");
|
const dateInput = document.getElementById("date-input");
|
||||||
const todayBtn = document.getElementById("today-btn");
|
const todayBtn = document.getElementById("today-btn");
|
||||||
|
|
@ -32,8 +38,13 @@ todayBtn.addEventListener("click", () => {
|
||||||
if (selected) runGrade();
|
if (selected) runGrade();
|
||||||
});
|
});
|
||||||
|
|
||||||
function selectLocation(lat, lon) {
|
function selectLocation(lat, lon, opts) {
|
||||||
selected = { lat, lon };
|
selected = { lat, lon };
|
||||||
|
// An approximate pick keeps its banner up; any real pick retires it, so the
|
||||||
|
// "this is a guess" claim can never outlive the guess.
|
||||||
|
approx = !!(opts && opts.approx);
|
||||||
|
approxLabel = approx ? ((opts && opts.label) || "") : "";
|
||||||
|
if (!approx) clearApprox(geoBox);
|
||||||
// Show the raw coordinates immediately; render() replaces them with the resolved
|
// Show the raw coordinates immediately; render() replaces them with the resolved
|
||||||
// neighborhood + city name once the grade fetch returns.
|
// neighborhood + city name once the grade fetch returns.
|
||||||
locLabel.textContent = `${lat.toFixed(3)}, ${lon.toFixed(3)}`;
|
locLabel.textContent = `${lat.toFixed(3)}, ${lon.toFixed(3)}`;
|
||||||
|
|
@ -58,6 +69,7 @@ const hero = document.getElementById("hero");
|
||||||
const heroLocate = document.getElementById("hero-locate");
|
const heroLocate = document.getElementById("hero-locate");
|
||||||
const heroPick = document.getElementById("hero-pick");
|
const heroPick = document.getElementById("hero-pick");
|
||||||
const heroMsg = document.getElementById("hero-locate-msg");
|
const heroMsg = document.getElementById("hero-locate-msg");
|
||||||
|
const geoBox = document.getElementById("geo-approx");
|
||||||
|
|
||||||
function locateMsg(text) {
|
function locateMsg(text) {
|
||||||
if (!heroMsg) return;
|
if (!heroMsg) return;
|
||||||
|
|
@ -106,6 +118,16 @@ heroLocate?.addEventListener("click", () => {
|
||||||
? "That took too long. Try again, or pick a spot on the map."
|
? "That took too long. Try again, or pick a spot on the map."
|
||||||
: "Your location isn't available right now. Pick a spot on the map instead.",
|
: "Your location isn't available right now. Pick a spot on the map instead.",
|
||||||
);
|
);
|
||||||
|
// The dead end this fallback exists for. If the server can offer a
|
||||||
|
// coarse guess, take over the message — telling someone to "pick a spot
|
||||||
|
// on the map" while a guessed place is already loading reads as two
|
||||||
|
// contradictory instructions.
|
||||||
|
tryApprox().then((ok) => {
|
||||||
|
if (!ok) return;
|
||||||
|
locateMsg(err.code === err.PERMISSION_DENIED
|
||||||
|
? "No problem — location permission stays off."
|
||||||
|
: "Your device's location wasn't available.");
|
||||||
|
});
|
||||||
},
|
},
|
||||||
{ enableHighAccuracy: false, timeout: 10000, maximumAge: 600000 },
|
{ enableHighAccuracy: false, timeout: 10000, maximumAge: 600000 },
|
||||||
);
|
);
|
||||||
|
|
@ -113,6 +135,30 @@ heroLocate?.addEventListener("click", () => {
|
||||||
|
|
||||||
heroPick?.addEventListener("click", () => { locateMsg(""); findBtn.click(); });
|
heroPick?.addEventListener("click", () => { locateMsg(""); findBtn.click(); });
|
||||||
|
|
||||||
|
// ---- approximate-location fallback ----
|
||||||
|
// Only ever a rescue for someone who has no location at all: it runs after a
|
||||||
|
// declined/failed geolocation prompt, or on load in a context where the browser
|
||||||
|
// will not offer geolocation at all (plain HTTP). It never runs for a visitor
|
||||||
|
// who already has a place — theirs is better than any guess — and never
|
||||||
|
// pre-empts the permission prompt.
|
||||||
|
let approxTried = false;
|
||||||
|
|
||||||
|
async function tryApprox() {
|
||||||
|
if (approxTried || selected) return false;
|
||||||
|
approxTried = true;
|
||||||
|
const data = await fetchApprox();
|
||||||
|
// 204/anything unusable → exactly today's behaviour, nothing rendered.
|
||||||
|
if (!data || selected) return false;
|
||||||
|
renderApprox(geoBox, data, () => { locateMsg(""); findBtn.click(); });
|
||||||
|
selectLocation(data.lat, data.lon, { approx: true, label: data.label });
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
// No geolocation available at all (insecure context, or a browser without it)
|
||||||
|
// and nothing remembered: there is no prompt to decline, so offer the guess up
|
||||||
|
// front rather than leaving the page with no place at all.
|
||||||
|
if (!canLocate && !loadLastLocation()) tryApprox();
|
||||||
|
|
||||||
/** Re-point the hero card at the place the visitor actually chose. */
|
/** Re-point the hero card at the place the visitor actually chose. */
|
||||||
function updateHero(data) {
|
function updateHero(data) {
|
||||||
if (!hero) return;
|
if (!hero) return;
|
||||||
|
|
@ -121,7 +167,12 @@ function updateHero(data) {
|
||||||
const g = targetDay && (targetDay.tmax || targetDay.tmin);
|
const g = targetDay && (targetDay.tmax || targetDay.tmin);
|
||||||
if (!card || !g) return;
|
if (!card || !g) return;
|
||||||
|
|
||||||
document.getElementById("hero-where").textContent = `Today in ${placeLabel(data)}`;
|
// A guessed place is reverse-geocoded like any other cell, which would name a
|
||||||
|
// *neighbourhood* — far more precision than an IP lookup ever had. Say
|
||||||
|
// "roughly near <city>" instead, so the headline can't overstate what we know.
|
||||||
|
document.getElementById("hero-where").textContent =
|
||||||
|
approx ? `Roughly near ${approxLabel || placeLabel(data)}`
|
||||||
|
: `Today in ${placeLabel(data)}`;
|
||||||
document.getElementById("hero-band").textContent = g.grade;
|
document.getElementById("hero-band").textContent = g.grade;
|
||||||
document.getElementById("hero-detail").textContent =
|
document.getElementById("hero-detail").textContent =
|
||||||
`${targetDay.tmax === g ? "High temp" : "Low temp"} in the ${pctOrd(g.percentile)} percentile`;
|
`${targetDay.tmax === g ? "High temp" : "Low temp"} in the ${pctOrd(g.percentile)} percentile`;
|
||||||
|
|
@ -267,7 +318,8 @@ function render(data) {
|
||||||
results.innerHTML = `
|
results.innerHTML = `
|
||||||
<div class="loc-head">
|
<div class="loc-head">
|
||||||
<div class="loc-title">
|
<div class="loc-title">
|
||||||
<h2>${placeLabel(data)}</h2>
|
<h2>${approx ? `Near ${esc(approxLabel || placeLabel(data))}` : placeLabel(data)}</h2>
|
||||||
|
${approx ? `<p class="loc-approx-tag">approximate — from your connection, not your device</p>` : ""}
|
||||||
<ul class="loc-meta">
|
<ul class="loc-meta">
|
||||||
${data.place ? `<li><b>${coords}</b></li>` : ""}
|
${data.place ? `<li><b>${coords}</b></li>` : ""}
|
||||||
<li>Climatology from <b>${yrs}</b></li>
|
<li>Climatology from <b>${yrs}</b></li>
|
||||||
|
|
@ -605,6 +657,10 @@ function flashBtn(id, text) {
|
||||||
|
|
||||||
function updateHash() {
|
function updateHash() {
|
||||||
if (!selected) return;
|
if (!selected) return;
|
||||||
|
// A guessed location is deliberately not remembered and not written into the
|
||||||
|
// URL: next visit must ask again rather than treating a guess as a choice,
|
||||||
|
// and a shared link must never carry a rough location the sender never picked.
|
||||||
|
if (approx) return;
|
||||||
history.replaceState(null, "", locHash(selected.lat, selected.lon, dateInput.value));
|
history.replaceState(null, "", locHash(selected.lat, selected.lon, dateInput.value));
|
||||||
saveLastLocation(selected.lat, selected.lon, dateInput.value);
|
saveLastLocation(selected.lat, selected.lon, dateInput.value);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
84
frontend/static/geoip.js
Normal file
84
frontend/static/geoip.js
Normal file
|
|
@ -0,0 +1,84 @@
|
||||||
|
// Approximate-location fallback.
|
||||||
|
//
|
||||||
|
// The only reason this exists: a visitor who declines (or cannot use) browser
|
||||||
|
// geolocation currently hits a dead end — search or the map picker, or nothing.
|
||||||
|
// This asks the backend for a coarse guess derived from the request's own IP,
|
||||||
|
// and offers it as a *suggestion*.
|
||||||
|
//
|
||||||
|
// Three rules the UI must never break, because the accuracy gap is enormous
|
||||||
|
// (browser geolocation is metres; a city-level IP database is tens of
|
||||||
|
// kilometres, against a ~2 mile grid cell):
|
||||||
|
//
|
||||||
|
// 1. Say it is a guess, in the visitor's own words, right next to the result.
|
||||||
|
// Never present it as "your location".
|
||||||
|
// 2. Put the correction one obvious tap away, permanently visible — not
|
||||||
|
// hidden behind a dismissed toast.
|
||||||
|
// 3. Never persist it. An explicit choice is remembered (localStorage) and
|
||||||
|
// written into the URL hash; a guess is neither. A guess that became
|
||||||
|
// sticky would be indistinguishable from a choice on the next visit, and
|
||||||
|
// a guessed URL would propagate someone else's rough location when shared.
|
||||||
|
//
|
||||||
|
// The endpoint answers 204 for every "no" — feature off, no database, private
|
||||||
|
// or unparsable address, no match, match too coarse — so there is exactly one
|
||||||
|
// branch here and it is "behave exactly as the site does today".
|
||||||
|
import { uv } from "./account.js";
|
||||||
|
import { track } from "./digest.js";
|
||||||
|
import { esc } from "./shared.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ask the backend for an approximate location.
|
||||||
|
* Resolves to the suggestion object, or null when there is nothing to suggest.
|
||||||
|
* Never rejects: a failed fetch is just "no suggestion".
|
||||||
|
*/
|
||||||
|
export async function fetchApprox() {
|
||||||
|
try {
|
||||||
|
const res = await fetch(uv("geoip"), { headers: { Accept: "application/json" } });
|
||||||
|
if (res.status !== 200) return null; // 204 is the normal "no"
|
||||||
|
const data = await res.json();
|
||||||
|
if (!data || !data.approximate) return null; // never trust an unlabelled payload
|
||||||
|
if (typeof data.lat !== "number" || typeof data.lon !== "number") return null;
|
||||||
|
return data;
|
||||||
|
} catch (e) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function attributionHtml(a) {
|
||||||
|
// Both candidate databases are free *with attribution* and both ask for the
|
||||||
|
// credit on the page that displays a result — so it rides with the result
|
||||||
|
// rather than living only in a footer that this view doesn't have.
|
||||||
|
if (!a || !a.text) return "";
|
||||||
|
const text = esc(a.text);
|
||||||
|
return a.url
|
||||||
|
? ` <a class="geo-approx-credit" href="${esc(a.url)}" rel="noopener nofollow" target="_blank">${text}</a>`
|
||||||
|
: ` <span class="geo-approx-credit">${text}</span>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Render the suggestion banner into `host` and load the guessed place.
|
||||||
|
*
|
||||||
|
* @param host container element (hidden until there is something to show)
|
||||||
|
* @param data a suggestion from fetchApprox()
|
||||||
|
* @param onFix called when the visitor corrects the guess
|
||||||
|
*/
|
||||||
|
export function renderApprox(host, data, onFix) {
|
||||||
|
if (!host) return;
|
||||||
|
host.innerHTML =
|
||||||
|
`<p class="geo-approx-line">Showing weather near <b>${esc(data.label)}</b>.</p>` +
|
||||||
|
`<p class="geo-approx-note">That is a rough guess from your internet connection, ` +
|
||||||
|
`not from your device — it can be tens of kilometres out.` +
|
||||||
|
attributionHtml(data.attribution) + `</p>` +
|
||||||
|
`<button type="button" class="btn-ghost geo-approx-fix">Not right? Choose your spot</button>`;
|
||||||
|
host.hidden = false;
|
||||||
|
host.querySelector(".geo-approx-fix")?.addEventListener("click", () => {
|
||||||
|
track("home.geoip_fixed");
|
||||||
|
onFix?.();
|
||||||
|
});
|
||||||
|
track("home.geoip_shown");
|
||||||
|
}
|
||||||
|
|
||||||
|
export function clearApprox(host) {
|
||||||
|
if (!host) return;
|
||||||
|
host.hidden = true;
|
||||||
|
host.innerHTML = "";
|
||||||
|
}
|
||||||
|
|
@ -1962,6 +1962,24 @@ details.hub-country > summary:hover { color: var(--accent); }
|
||||||
.hero-jump a { color: var(--muted); text-decoration: none; }
|
.hero-jump a { color: var(--muted); text-decoration: none; }
|
||||||
.hero-jump a:hover { color: var(--accent); }
|
.hero-jump a:hover { color: var(--accent); }
|
||||||
|
|
||||||
|
/* Approximate-location banner (static/geoip.js). Read as an aside, not a
|
||||||
|
result: recessed surface, no accent fill, and the correction button is a
|
||||||
|
full-width ~44px target so it is never the hard thing to find on a phone. */
|
||||||
|
.geo-approx {
|
||||||
|
display: flex; flex-direction: column; gap: 8px;
|
||||||
|
margin: 0; padding: 12px 14px; max-width: 42ch;
|
||||||
|
background: var(--surface-2); border: 1px dashed var(--border); border-radius: 10px;
|
||||||
|
}
|
||||||
|
.geo-approx[hidden] { display: none; }
|
||||||
|
.geo-approx-line { margin: 0; font-size: 14px; }
|
||||||
|
.geo-approx-note { margin: 0; font-size: 12px; color: var(--muted); }
|
||||||
|
.geo-approx-credit { color: var(--muted); text-decoration: underline; }
|
||||||
|
.geo-approx-credit:hover { color: var(--accent); }
|
||||||
|
.geo-approx .geo-approx-fix { min-height: 44px; font-size: 16px; width: 100%; }
|
||||||
|
/* The same "this is a guess" claim on the results heading, so it survives a
|
||||||
|
scroll past the hero. */
|
||||||
|
.loc-approx-tag { margin: 2px 0 0; font-size: 12px; color: var(--muted); }
|
||||||
|
|
||||||
/* Desktop: copy beside the card rather than stacked above it. */
|
/* Desktop: copy beside the card rather than stacked above it. */
|
||||||
@media (min-width: 900px) {
|
@media (min-width: 900px) {
|
||||||
.hero { grid-template-columns: 1.1fr 1fr; align-items: center; gap: 40px; }
|
.hero { grid-template-columns: 1.1fr 1fr; align-items: center; gap: 40px; }
|
||||||
|
|
|
||||||
|
|
@ -68,6 +68,13 @@
|
||||||
<button type="button" id="hero-pick" class="btn-ghost">Pick on the map</button>
|
<button type="button" id="hero-pick" class="btn-ghost">Pick on the map</button>
|
||||||
</div>
|
</div>
|
||||||
<p class="hero-locate-msg" id="hero-locate-msg" role="status" aria-live="polite" hidden></p>
|
<p class="hero-locate-msg" id="hero-locate-msg" role="status" aria-live="polite" hidden></p>
|
||||||
|
{# Approximate-location fallback (static/geoip.js). Server-rendered EMPTY
|
||||||
|
and hidden on purpose: this page's HTML must stay byte-identical for
|
||||||
|
every visitor so its ETag keeps validating and no shared cache can
|
||||||
|
ever hand one person's city to another. The suggestion is fetched
|
||||||
|
client-side, and only for a visitor who ended up with no location at
|
||||||
|
all. #}
|
||||||
|
<div class="geo-approx" id="geo-approx" role="status" aria-live="polite" hidden></div>
|
||||||
<p class="hero-jump"><a href="#panel">See your week ↓</a></p>
|
<p class="hero-jump"><a href="#panel">See your week ↓</a></p>
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|
|
||||||
|
|
@ -12,9 +12,27 @@
|
||||||
there is very little about you to collect in the first place.</p>
|
there is very little about you to collect in the first place.</p>
|
||||||
|
|
||||||
<h2>Your location</h2>
|
<h2>Your location</h2>
|
||||||
|
{# This section states a fact about what the server actually does, so it follows
|
||||||
|
the THERMOGRAPH_GEOIP flag instead of being written once and left to rot.
|
||||||
|
See content.py's geoip_enabled(). #}
|
||||||
|
{% if geoip %}
|
||||||
|
<p>The precise way the site learns where you are is if you press <b>Use my location</b>,
|
||||||
|
which asks your browser for permission. You can decline, and picking a place on the map
|
||||||
|
works just as well.</p>
|
||||||
|
<p>If you decline, or your browser cannot offer it, we fall back to a rough guess at your
|
||||||
|
city from your IP address so there is something local to show instead of nothing. It is
|
||||||
|
always labelled as a guess, never as your position — it is accurate to tens of kilometres
|
||||||
|
at best, and one tap corrects it. It is also not remembered: a guess is never saved in
|
||||||
|
your browser or written into the page address, so only a place you actually pick is kept.</p>
|
||||||
|
<p>That lookup runs entirely on our own server, against a database file we host ourselves.
|
||||||
|
Your IP address is never sent to another company, and it is not stored: it is read from
|
||||||
|
the request, used to find a city, and discarded. It is deliberately kept out of our
|
||||||
|
request log for this feature too, so nothing anywhere links an address to a place.</p>
|
||||||
|
{% else %}
|
||||||
<p>Thermograph never looks up your location from your IP address. The only way the site
|
<p>Thermograph never looks up your location from your IP address. The only way the site
|
||||||
learns where you are is if you press <b>Use my location</b>, which asks your browser for
|
learns where you are is if you press <b>Use my location</b>, which asks your browser for
|
||||||
permission. You can decline, and picking a place on the map works just as well.</p>
|
permission. You can decline, and picking a place on the map works just as well.</p>
|
||||||
|
{% endif %}
|
||||||
<p>Whichever way you choose a place, the coordinates are sent to the server only as part of
|
<p>Whichever way you choose a place, the coordinates are sent to the server only as part of
|
||||||
the ordinary request for that grid square's climate data, and are not logged beyond the
|
the ordinary request for that grid square's climate data, and are not logged beyond the
|
||||||
normal web-server request handling every site does. The place you picked is remembered in
|
normal web-server request handling every site does. The place you picked is remembered in
|
||||||
|
|
|
||||||
58
frontend/tests/unit/test_geoip_ui.py
Normal file
58
frontend/tests/unit/test_geoip_ui.py
Normal file
|
|
@ -0,0 +1,58 @@
|
||||||
|
"""Approximate-location fallback, frontend side.
|
||||||
|
|
||||||
|
Two things are worth locking down here, and both are about honesty rather than
|
||||||
|
mechanics: the server-rendered HTML must not vary by visitor (so no shared
|
||||||
|
cache can leak one person's city to another), and the privacy page's factual
|
||||||
|
claim about IP lookups must track the flag that decides whether they happen.
|
||||||
|
"""
|
||||||
|
import content
|
||||||
|
|
||||||
|
B = "/thermograph"
|
||||||
|
|
||||||
|
|
||||||
|
def test_home_ships_an_empty_hidden_container(client):
|
||||||
|
"""The suggestion is fetched client-side. If it were ever server-rendered
|
||||||
|
into the homepage, that page's HTML — and its ETag — would vary per
|
||||||
|
visitor, which is exactly the cache-poisoning shape this design avoids."""
|
||||||
|
r = client.get(f"{B}/")
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert 'id="geo-approx"' in r.text
|
||||||
|
assert "hidden" in r.text.split('id="geo-approx"')[1].split(">")[0]
|
||||||
|
|
||||||
|
|
||||||
|
def test_home_html_is_identical_regardless_of_client_address(client):
|
||||||
|
"""Same bytes, same ETag, whatever the request claims about where it came
|
||||||
|
from — the property that lets the homepage stay cacheable."""
|
||||||
|
a = client.get(f"{B}/", headers={"X-Forwarded-For": "8.8.8.8"})
|
||||||
|
b = client.get(f"{B}/", headers={"X-Forwarded-For": "1.1.1.1"})
|
||||||
|
assert a.text == b.text
|
||||||
|
assert a.headers["etag"] == b.headers["etag"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_privacy_says_no_ip_lookup_while_the_flag_is_off(client, monkeypatch):
|
||||||
|
monkeypatch.delenv("THERMOGRAPH_GEOIP", raising=False)
|
||||||
|
r = client.get(f"{B}/privacy")
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert "never looks up your location from your IP address" in r.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_privacy_describes_the_fallback_when_the_flag_is_on(client, monkeypatch):
|
||||||
|
"""Turning the feature on must not leave a published claim behind that is
|
||||||
|
no longer true."""
|
||||||
|
monkeypatch.setenv("THERMOGRAPH_GEOIP", "1")
|
||||||
|
r = client.get(f"{B}/privacy")
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert "never looks up your location from your IP address" not in r.text
|
||||||
|
assert "rough guess" in r.text
|
||||||
|
assert "not stored" in r.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_geoip_enabled_reads_the_shared_flag(monkeypatch):
|
||||||
|
monkeypatch.delenv("THERMOGRAPH_GEOIP", raising=False)
|
||||||
|
assert content.geoip_enabled() is False
|
||||||
|
for on in ("1", "true", "YES", "on"):
|
||||||
|
monkeypatch.setenv("THERMOGRAPH_GEOIP", on)
|
||||||
|
assert content.geoip_enabled() is True, on
|
||||||
|
for off in ("0", "", "no", "maybe"):
|
||||||
|
monkeypatch.setenv("THERMOGRAPH_GEOIP", off)
|
||||||
|
assert content.geoip_enabled() is False, off
|
||||||
18
infra/deploy/geoip-refresh.service
Normal file
18
infra/deploy/geoip-refresh.service
Normal file
|
|
@ -0,0 +1,18 @@
|
||||||
|
[Unit]
|
||||||
|
Description=Refresh the Thermograph IP-geolocation database
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
# GEOIP_URL / GEOIP_BASE / THERMOGRAPH_GEOIP_HOST_DIR can be overridden here if
|
||||||
|
# the deployment switches databases. The leading `-` keeps a missing file
|
||||||
|
# non-fatal, matching thermograph.service.
|
||||||
|
EnvironmentFile=-/etc/thermograph.env
|
||||||
|
ExecStart=/opt/thermograph/infra/deploy/geoip-refresh.sh
|
||||||
|
# Downloading ~125 MB and verifying it needs no privilege beyond writing the
|
||||||
|
# destination directory, and nothing else on the box.
|
||||||
|
ProtectSystem=strict
|
||||||
|
ReadWritePaths=/var/lib/thermograph/geoip
|
||||||
|
PrivateTmp=yes
|
||||||
|
NoNewPrivileges=yes
|
||||||
90
infra/deploy/geoip-refresh.sh
Executable file
90
infra/deploy/geoip-refresh.sh
Executable file
|
|
@ -0,0 +1,90 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# Refresh the IP-geolocation database used by the approximate-location fallback
|
||||||
|
# (backend/data/geoip.py). Runs on the HOST (prod/beta), not in a container, on
|
||||||
|
# a systemd timer — see geoip-refresh.timer.
|
||||||
|
#
|
||||||
|
# Why on the host and not baked into the image:
|
||||||
|
# * it is a 100-200 MB data file on its own monthly release cadence, not code;
|
||||||
|
# baking it in adds that to every image pull and freezes its freshness to
|
||||||
|
# whenever the last deploy happened;
|
||||||
|
# * MaxMind's GeoLite EULA *requires* deleting a database within 30 days of a
|
||||||
|
# newer release, which a build-time copy cannot honour on its own;
|
||||||
|
# * geoip.py memory-maps the file and re-opens it when (mtime, size) changes,
|
||||||
|
# so an atomic swap here is picked up with no restart and no deploy.
|
||||||
|
#
|
||||||
|
# Default source: DB-IP's IP to City Lite, CC BY 4.0, no account and no licence
|
||||||
|
# key — which is why this script needs no secret from the vault. Point
|
||||||
|
# GEOIP_URL elsewhere (e.g. a MaxMind permalink carrying a licence key from
|
||||||
|
# /etc/thermograph.env) to use a different database; geoip.py reads the file's
|
||||||
|
# own metadata for the attribution line, so nothing else changes.
|
||||||
|
#
|
||||||
|
# sudo /opt/thermograph/infra/deploy/geoip-refresh.sh
|
||||||
|
#
|
||||||
|
# Exit status: 0 on a successful swap AND on "already current" (so the timer
|
||||||
|
# doesn't alarm); non-zero only on a real failure, leaving any existing
|
||||||
|
# database untouched — a failed refresh must never be worse than a stale one.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
DEST_DIR="${THERMOGRAPH_GEOIP_HOST_DIR:-/var/lib/thermograph/geoip}"
|
||||||
|
DEST="$DEST_DIR/city.mmdb"
|
||||||
|
# DB-IP publishes one file per month at a predictable URL. Try this month, then
|
||||||
|
# last month: the new file appears at some point on the 1st, so a timer firing
|
||||||
|
# early would otherwise fail for a day.
|
||||||
|
BASE="${GEOIP_BASE:-https://download.db-ip.com/free}"
|
||||||
|
THIS_MONTH="$(date -u +%Y-%m)"
|
||||||
|
LAST_MONTH="$(date -u -d "$(date -u +%Y-%m-01) -1 month" +%Y-%m)"
|
||||||
|
|
||||||
|
log() { echo "[geoip-refresh] $*"; }
|
||||||
|
|
||||||
|
mkdir -p "$DEST_DIR"
|
||||||
|
tmp="$(mktemp -d "${TMPDIR:-/tmp}/geoip.XXXXXX")"
|
||||||
|
trap 'rm -rf "$tmp"' EXIT
|
||||||
|
|
||||||
|
url=""
|
||||||
|
if [ -n "${GEOIP_URL:-}" ]; then
|
||||||
|
url="$GEOIP_URL"
|
||||||
|
else
|
||||||
|
for m in "$THIS_MONTH" "$LAST_MONTH"; do
|
||||||
|
candidate="$BASE/dbip-city-lite-$m.mmdb.gz"
|
||||||
|
if curl -fsSI --max-time 30 "$candidate" >/dev/null 2>&1; then
|
||||||
|
url="$candidate"; break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
[ -n "$url" ] || { log "no published database found at $BASE for $THIS_MONTH or $LAST_MONTH"; exit 1; }
|
||||||
|
log "source: $url"
|
||||||
|
|
||||||
|
curl -fsSL --max-time 900 --retry 3 --retry-delay 10 -o "$tmp/db.gz" "$url"
|
||||||
|
gunzip -c "$tmp/db.gz" > "$tmp/city.mmdb"
|
||||||
|
|
||||||
|
# Prove the file actually opens and answers before it can replace a working
|
||||||
|
# one. A truncated download that merely *exists* would otherwise silently
|
||||||
|
# disable lookups for a month.
|
||||||
|
python3 - "$tmp/city.mmdb" <<'PY'
|
||||||
|
import sys
|
||||||
|
import maxminddb
|
||||||
|
with maxminddb.open_database(sys.argv[1]) as r:
|
||||||
|
meta = r.metadata()
|
||||||
|
if "city" not in (meta.database_type or "").lower():
|
||||||
|
sys.exit(f"unexpected database_type: {meta.database_type}")
|
||||||
|
if r.get("8.8.8.8") is None and r.get("1.1.1.1") is None:
|
||||||
|
sys.exit("database opened but answered nothing for known public addresses")
|
||||||
|
print(f"[geoip-refresh] verified {meta.database_type} "
|
||||||
|
f"built {meta.build_epoch} nodes={meta.node_count}")
|
||||||
|
PY
|
||||||
|
|
||||||
|
if [ -f "$DEST" ] && cmp -s "$tmp/city.mmdb" "$DEST"; then
|
||||||
|
log "already current: $DEST unchanged"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Atomic swap on the same filesystem: readers either see the whole old file or
|
||||||
|
# the whole new one, never a half-written mmap. The app's open handle keeps the
|
||||||
|
# unlinked inode alive until it re-opens on the mtime change.
|
||||||
|
install -m 0444 "$tmp/city.mmdb" "$DEST.new"
|
||||||
|
mv -f "$DEST.new" "$DEST"
|
||||||
|
log "installed $DEST ($(stat -c %s "$DEST") bytes)"
|
||||||
|
|
||||||
|
# MaxMind's EULA requires deleting superseded copies within 30 days; keeping no
|
||||||
|
# copies at all satisfies that trivially and is the right default for DB-IP too.
|
||||||
|
find "$DEST_DIR" -maxdepth 1 -name 'city.mmdb.*' -type f -delete 2>/dev/null || true
|
||||||
18
infra/deploy/geoip-refresh.timer
Normal file
18
infra/deploy/geoip-refresh.timer
Normal file
|
|
@ -0,0 +1,18 @@
|
||||||
|
[Unit]
|
||||||
|
Description=Monthly refresh of the Thermograph IP-geolocation database
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
# Both candidate databases publish monthly. Weekly rather than monthly on
|
||||||
|
# purpose: the script exits 0 without touching anything when the file is already
|
||||||
|
# current, so the extra runs cost one HEAD request, and they mean a missed or
|
||||||
|
# failed monthly run self-heals within a week instead of leaving the data a
|
||||||
|
# month stale. (MaxMind additionally *requires* deleting a superseded copy
|
||||||
|
# within 30 days, which a strictly-monthly timer would sit right on the edge of.)
|
||||||
|
OnCalendar=Mon 04:17 UTC
|
||||||
|
# Don't have every host in the fleet hit the mirror at the same second.
|
||||||
|
RandomizedDelaySec=2h
|
||||||
|
# Catch up after a reboot that spanned the scheduled time.
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
|
|
@ -207,3 +207,37 @@ THERMOGRAPH_BASE_URL=https://thermograph.org
|
||||||
# required — Discord delivers content for mentions/DMs. Unset/0 => no gateway
|
# required — Discord delivers content for mentions/DMs. Unset/0 => no gateway
|
||||||
# connection. (Code: thermograph-backend notifications/discord_bot.py.)
|
# connection. (Code: thermograph-backend notifications/discord_bot.py.)
|
||||||
#THERMOGRAPH_DISCORD_BOT=1
|
#THERMOGRAPH_DISCORD_BOT=1
|
||||||
|
# --- approximate-location fallback (backend/data/geoip.py) ------------------
|
||||||
|
# When a visitor declines (or cannot use) browser geolocation, suggest a coarse
|
||||||
|
# city from their IP so the site has something local to show. OFF by default,
|
||||||
|
# and inert until deploy/geoip-refresh.sh has landed a database at
|
||||||
|
# /var/lib/thermograph/geoip/city.mmdb — with either missing, the site behaves
|
||||||
|
# exactly as it does today (the endpoint answers 204 and the UI does nothing).
|
||||||
|
#
|
||||||
|
# Read by BOTH containers: the backend does the lookup; the frontend reads it
|
||||||
|
# only to decide which paragraph /privacy renders about location. Turning this
|
||||||
|
# on therefore changes the published privacy statement in the same deploy, on
|
||||||
|
# purpose — do not set it in one place and not the other.
|
||||||
|
#
|
||||||
|
# The lookup is local (a file on our own disk, MaxMind MMDB format) — the
|
||||||
|
# visitor's IP never leaves the server, so no third party becomes a data
|
||||||
|
# processor. The IP is read in memory and discarded: it is written to no
|
||||||
|
# database, no metric, and not even the access log (that route is excluded).
|
||||||
|
#THERMOGRAPH_GEOIP=1
|
||||||
|
# Where the refresh job writes the database on the host; bind-mounted read-only
|
||||||
|
# into the backend container at /geoip. Only override if the default path is
|
||||||
|
# unsuitable.
|
||||||
|
#THERMOGRAPH_GEOIP_HOST_DIR=/var/lib/thermograph/geoip
|
||||||
|
# Reject a record the database itself says is worse than this many km — that is
|
||||||
|
# what stops a country-centroid guess being presented as a city. Default 100.
|
||||||
|
#THERMOGRAPH_GEOIP_MAX_RADIUS_KM=100
|
||||||
|
# Credit line shown beside any suggestion, as "text|url". Normally unset: it is
|
||||||
|
# derived from the installed database's own metadata, so swapping DB-IP for
|
||||||
|
# GeoLite2 changes the attribution automatically. Set it only for a database
|
||||||
|
# whose metadata we don't recognise.
|
||||||
|
#THERMOGRAPH_GEOIP_ATTRIBUTION=IP Geolocation by DB-IP|https://db-ip.com
|
||||||
|
# Alternate download source for deploy/geoip-refresh.sh. Unset = DB-IP IP to
|
||||||
|
# City Lite (CC BY 4.0, no account, no licence key — which is why nothing here
|
||||||
|
# is a secret). A MaxMind permalink would embed a licence key and therefore
|
||||||
|
# belongs in the SOPS vault rather than in this example file.
|
||||||
|
#GEOIP_URL=
|
||||||
|
|
|
||||||
|
|
@ -129,6 +129,13 @@ services:
|
||||||
# 503 and climate.py falls straight through to NASA — an accelerator,
|
# 503 and climate.py falls straight through to NASA — an accelerator,
|
||||||
# never a dependency.
|
# never a dependency.
|
||||||
THERMOGRAPH_LAKE_URL: http://lake:8141
|
THERMOGRAPH_LAKE_URL: http://lake:8141
|
||||||
|
# Approximate-location fallback (backend/data/geoip.py). OFF unless
|
||||||
|
# THERMOGRAPH_GEOIP is set truthy in /etc/thermograph.env, and inert even
|
||||||
|
# then until deploy/geoip-refresh.sh has landed a database at the host
|
||||||
|
# path bind-mounted below — so shipping the code changes nothing by
|
||||||
|
# itself, and the mount being empty is a supported steady state.
|
||||||
|
THERMOGRAPH_GEOIP: ${THERMOGRAPH_GEOIP:-}
|
||||||
|
THERMOGRAPH_GEOIP_DB: /geoip/city.mmdb
|
||||||
# Prod secrets live in /etc/thermograph.env: POSTGRES_PASSWORD,
|
# Prod secrets live in /etc/thermograph.env: POSTGRES_PASSWORD,
|
||||||
# THERMOGRAPH_AUTH_SECRET, THERMOGRAPH_VAPID_PRIVATE_KEY/_PUBLIC_KEY,
|
# THERMOGRAPH_AUTH_SECRET, THERMOGRAPH_VAPID_PRIVATE_KEY/_PUBLIC_KEY,
|
||||||
# THERMOGRAPH_COOKIE_SECURE=1, mail/Discord keys, ... (see
|
# THERMOGRAPH_COOKIE_SECURE=1, mail/Discord keys, ... (see
|
||||||
|
|
@ -145,6 +152,12 @@ services:
|
||||||
# (above) points runtime state here instead, clear of the code.
|
# (above) points runtime state here instead, clear of the code.
|
||||||
- appdata:/state
|
- appdata:/state
|
||||||
- applogs:/app/logs
|
- applogs:/app/logs
|
||||||
|
# The IP-geolocation database, refreshed on the HOST by
|
||||||
|
# deploy/geoip-refresh.sh (systemd timer, monthly) and mounted read-only:
|
||||||
|
# the app must never be able to write it, and a 100-200 MB data file that
|
||||||
|
# goes stale on its own schedule does not belong baked into the image.
|
||||||
|
# geoip.py re-opens on mtime change, so a refresh needs no restart.
|
||||||
|
- ${THERMOGRAPH_GEOIP_HOST_DIR:-/var/lib/thermograph/geoip}:/geoip:ro
|
||||||
# No compose-level healthcheck override -- the image's own Dockerfile
|
# No compose-level healthcheck override -- the image's own Dockerfile
|
||||||
# HEALTHCHECK (port-aware via ${PORT}) already covers this, and frontend's
|
# HEALTHCHECK (port-aware via ${PORT}) already covers this, and frontend's
|
||||||
# depends_on below reads it the same way.
|
# depends_on below reads it the same way.
|
||||||
|
|
@ -208,6 +221,13 @@ services:
|
||||||
PORT: 8080
|
PORT: 8080
|
||||||
THERMOGRAPH_SERVICE_ROLE: frontend
|
THERMOGRAPH_SERVICE_ROLE: frontend
|
||||||
THERMOGRAPH_API_BASE_INTERNAL: http://backend:8137
|
THERMOGRAPH_API_BASE_INTERNAL: http://backend:8137
|
||||||
|
# Read for ONE purpose: which paragraph the /privacy page renders about
|
||||||
|
# location. Frontend behaviour does not branch on it — the interactive
|
||||||
|
# tool simply calls the backend endpoint and gets a 204 when the feature
|
||||||
|
# is off. Same value as backend's, from the same /etc/thermograph.env, so
|
||||||
|
# the published privacy statement can never drift from what the server
|
||||||
|
# actually does.
|
||||||
|
THERMOGRAPH_GEOIP: ${THERMOGRAPH_GEOIP:-}
|
||||||
# No volumes -- stateless, holds no data of its own.
|
# No volumes -- stateless, holds no data of its own.
|
||||||
cpus: ${FRONTEND_CPUS:-2}
|
cpus: ${FRONTEND_CPUS:-2}
|
||||||
deploy:
|
deploy:
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue