083d3bd0f8
All checks were successful
PR build (required check) / changes (pull_request) Successful in 6s
secrets-guard / encrypted (pull_request) Successful in 5s
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-frontend (pull_request) Successful in 1m8s
PR build (required check) / build-backend (pull_request) Successful in 1m36s
PR build (required check) / gate (pull_request) Successful in 2s
A visitor who declines browser geolocation currently has no location at all — the copy sends them to the map picker. This adds an opt-in fallback that suggests a coarse city from the request's own IP, presented as a guess with a one-tap correction, so the dead end has a way out. backend/data/geoip.py does the lookup against a local MMDB file (DB-IP IP to City Lite or GeoLite2 City — same format, either works, attribution derived from the file's metadata). Off unless THERMOGRAPH_GEOIP is truthy AND the database exists AND maxminddb imports; every degraded case — private/reserved/ CGNAT/IPv6-ULA addresses, unparseable input, no record, a country-centroid record with no city, a record wider than the accuracy limit, a corrupt file — returns None, which the route answers as 204 and the client treats exactly as today. The IP is read in memory and dropped. GET /api/v2/geoip runs no RunAudit, records no metric dimension, and is excluded from the access log (its own "geoip" traffic category) so no stored, joinable (IP, location) pair is ever written. The response is no-store/Vary:* and takes no parameters. Client-side rather than server-rendered on purpose: the homepage's HTML and weak ETag must stay byte-identical for every visitor, or any cache added in front of it can serve one visitor's city to another. The suggestion is fetched only after a declined/unavailable prompt, and only when nothing is remembered. A guess is never persisted — no localStorage write, no URL hash — and the hero and results headings say "roughly near"/"near … approximate" rather than letting the reverse-geocoded cell name a neighbourhood the lookup never knew. The /privacy page's "never looks up your location from your IP address" paragraph now follows the same flag out of the same env file, so the published statement and the behaviour flip together. Database lifecycle is a host-side systemd timer (infra/deploy/geoip-refresh.*) that verifies a download opens and answers before swapping it in atomically, bind-mounted read-only; geoip.py re-opens on mtime change, so a refresh needs no restart. GEOIP-APPROX-LOCATION.md carries the database comparison, the SSR-vs-client argument, the privacy analysis, and the open decisions.
90 lines
3.9 KiB
Bash
Executable file
90 lines
3.9 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Refresh the IP-geolocation database used by the approximate-location fallback
|
|
# (backend/data/geoip.py). Runs on the HOST (prod/beta), not in a container, on
|
|
# a systemd timer — see geoip-refresh.timer.
|
|
#
|
|
# Why on the host and not baked into the image:
|
|
# * it is a 100-200 MB data file on its own monthly release cadence, not code;
|
|
# baking it in adds that to every image pull and freezes its freshness to
|
|
# whenever the last deploy happened;
|
|
# * MaxMind's GeoLite EULA *requires* deleting a database within 30 days of a
|
|
# newer release, which a build-time copy cannot honour on its own;
|
|
# * geoip.py memory-maps the file and re-opens it when (mtime, size) changes,
|
|
# so an atomic swap here is picked up with no restart and no deploy.
|
|
#
|
|
# Default source: DB-IP's IP to City Lite, CC BY 4.0, no account and no licence
|
|
# key — which is why this script needs no secret from the vault. Point
|
|
# GEOIP_URL elsewhere (e.g. a MaxMind permalink carrying a licence key from
|
|
# /etc/thermograph.env) to use a different database; geoip.py reads the file's
|
|
# own metadata for the attribution line, so nothing else changes.
|
|
#
|
|
# sudo /opt/thermograph/infra/deploy/geoip-refresh.sh
|
|
#
|
|
# Exit status: 0 on a successful swap AND on "already current" (so the timer
|
|
# doesn't alarm); non-zero only on a real failure, leaving any existing
|
|
# database untouched — a failed refresh must never be worse than a stale one.
|
|
set -euo pipefail
|
|
|
|
DEST_DIR="${THERMOGRAPH_GEOIP_HOST_DIR:-/var/lib/thermograph/geoip}"
|
|
DEST="$DEST_DIR/city.mmdb"
|
|
# DB-IP publishes one file per month at a predictable URL. Try this month, then
|
|
# last month: the new file appears at some point on the 1st, so a timer firing
|
|
# early would otherwise fail for a day.
|
|
BASE="${GEOIP_BASE:-https://download.db-ip.com/free}"
|
|
THIS_MONTH="$(date -u +%Y-%m)"
|
|
LAST_MONTH="$(date -u -d "$(date -u +%Y-%m-01) -1 month" +%Y-%m)"
|
|
|
|
log() { echo "[geoip-refresh] $*"; }
|
|
|
|
mkdir -p "$DEST_DIR"
|
|
tmp="$(mktemp -d "${TMPDIR:-/tmp}/geoip.XXXXXX")"
|
|
trap 'rm -rf "$tmp"' EXIT
|
|
|
|
url=""
|
|
if [ -n "${GEOIP_URL:-}" ]; then
|
|
url="$GEOIP_URL"
|
|
else
|
|
for m in "$THIS_MONTH" "$LAST_MONTH"; do
|
|
candidate="$BASE/dbip-city-lite-$m.mmdb.gz"
|
|
if curl -fsSI --max-time 30 "$candidate" >/dev/null 2>&1; then
|
|
url="$candidate"; break
|
|
fi
|
|
done
|
|
fi
|
|
[ -n "$url" ] || { log "no published database found at $BASE for $THIS_MONTH or $LAST_MONTH"; exit 1; }
|
|
log "source: $url"
|
|
|
|
curl -fsSL --max-time 900 --retry 3 --retry-delay 10 -o "$tmp/db.gz" "$url"
|
|
gunzip -c "$tmp/db.gz" > "$tmp/city.mmdb"
|
|
|
|
# Prove the file actually opens and answers before it can replace a working
|
|
# one. A truncated download that merely *exists* would otherwise silently
|
|
# disable lookups for a month.
|
|
python3 - "$tmp/city.mmdb" <<'PY'
|
|
import sys
|
|
import maxminddb
|
|
with maxminddb.open_database(sys.argv[1]) as r:
|
|
meta = r.metadata()
|
|
if "city" not in (meta.database_type or "").lower():
|
|
sys.exit(f"unexpected database_type: {meta.database_type}")
|
|
if r.get("8.8.8.8") is None and r.get("1.1.1.1") is None:
|
|
sys.exit("database opened but answered nothing for known public addresses")
|
|
print(f"[geoip-refresh] verified {meta.database_type} "
|
|
f"built {meta.build_epoch} nodes={meta.node_count}")
|
|
PY
|
|
|
|
if [ -f "$DEST" ] && cmp -s "$tmp/city.mmdb" "$DEST"; then
|
|
log "already current: $DEST unchanged"
|
|
exit 0
|
|
fi
|
|
|
|
# Atomic swap on the same filesystem: readers either see the whole old file or
|
|
# the whole new one, never a half-written mmap. The app's open handle keeps the
|
|
# unlinked inode alive until it re-opens on the mtime change.
|
|
install -m 0444 "$tmp/city.mmdb" "$DEST.new"
|
|
mv -f "$DEST.new" "$DEST"
|
|
log "installed $DEST ($(stat -c %s "$DEST") bytes)"
|
|
|
|
# MaxMind's EULA requires deleting superseded copies within 30 days; keeping no
|
|
# copies at all satisfies that trivially and is the right default for DB-IP too.
|
|
find "$DEST_DIR" -maxdepth 1 -name 'city.mmdb.*' -type f -delete 2>/dev/null || true
|