Merge pull request 'Promote main to release: NASA retirement' (#74) from main into release
All checks were successful
secrets-guard / encrypted (push) Successful in 17s
shell-lint / shellcheck (push) Successful in 24s
Build + push frontend image (Forgejo registry) / build-push (push) Successful in 1m22s
Build + push backend image (Forgejo registry) / build-push (push) Successful in 1m28s
Deploy backend to prod VPS / deploy (push) Successful in 3m24s
Deploy frontend to prod VPS / deploy (push) Successful in 3m50s

This commit is contained in:
emi 2026-07-24 23:59:26 +00:00
commit 51889d6979
21 changed files with 522 additions and 179 deletions

4
.gitignore vendored Normal file
View file

@ -0,0 +1,4 @@
# Claude Code worktrees. These are other sessions' checkouts living inside this
# repo; `git add -A` will otherwise commit them as embedded git repositories.
.claude/worktrees/

View file

@ -6,6 +6,7 @@ pattern web/app.py's own endpoints use.
""" """
import hashlib import hashlib
import os import os
from urllib.parse import urlsplit
from fastapi import APIRouter, HTTPException, Request, Response from fastapi import APIRouter, HTTPException, Request, Response
@ -84,9 +85,24 @@ def _load_history(cell):
return history return history
# The configured public origin is the source of truth for the scheme. Caddy
# terminates TLS and reverse-proxies plain HTTP, so x-forwarded-proto (and
# request.url.scheme) read "http" even though the public site is HTTPS-only --
# and this origin is folded into the jsonld "url" the payloads carry. Trusting
# the forwarded scheme makes that url (and the canonical/og the frontend builds
# from the same origin it forwards here) emit http://, which 308-redirects to
# https://. So a request that arrives on the configured public host is answered
# with the configured public scheme; localhost/LAN dev never matches that host
# and keeps the observed scheme, leaving plain-HTTP development unaffected.
_PUBLIC = urlsplit(os.environ.get("THERMOGRAPH_BASE_URL", "https://thermograph.org"))
def _origin(request: Request) -> str: def _origin(request: Request) -> str:
proto = request.headers.get("x-forwarded-proto") or request.url.scheme
host = request.headers.get("host") or request.url.netloc host = request.headers.get("host") or request.url.netloc
if _PUBLIC.scheme and _PUBLIC.netloc and host == _PUBLIC.netloc:
proto = _PUBLIC.scheme
else:
proto = request.headers.get("x-forwarded-proto") or request.url.scheme
return f"{proto}://{host}" return f"{proto}://{host}"

View file

@ -75,9 +75,10 @@ ARCHIVE_URL = os.environ.get("THERMOGRAPH_ARCHIVE_URL",
# instance serves the same 0.1° resolution. Archive requests only, not forecast. # instance serves the same 0.1° resolution. Archive requests only, not forecast.
ARCHIVE_MODEL = "era5_seamless" ARCHIVE_MODEL = "era5_seamless"
FORECAST_URL = "https://api.open-meteo.com/v1/forecast" FORECAST_URL = "https://api.open-meteo.com/v1/forecast"
# Backup archive when Open-Meteo is unavailable (e.g. its daily rate limit). NASA # NASA POWER is RETIRED from every serving path (2026-07-24): its MERRA-2
# POWER is free + keyless, global, daily from 1981. It lacks gusts + apparent temp, # record ran ~1.3°F off the ERA5 lake/archive with location-dependent sign and
# so gusts read as unavailable and "feels like" is computed from heat index/chill. # carried no gusts, so serving it beside ERA5 sources skewed percentiles. Kept
# only as drift_check.py's comparison source.
NASA_POWER_URL = "https://power.larc.nasa.gov/api/temporal/daily/point" NASA_POWER_URL = "https://power.larc.nasa.gov/api/temporal/daily/point"
NASA_START = "19810101" NASA_START = "19810101"
NASA_FILL = -900.0 # POWER's missing-value sentinel is ~-999 NASA_FILL = -900.0 # POWER's missing-value sentinel is ~-999
@ -710,14 +711,15 @@ def _read_history_cache(path):
def _fetch_history_tail(cell: dict, start_date: str, end_date: str) -> pl.DataFrame: def _fetch_history_tail(cell: dict, start_date: str, end_date: str) -> pl.DataFrame:
"""Fetch a recent history range to top up the cached tail — NASA POWER primary, """Fetch a recent history range to top up the cached tail — the Open-Meteo
Open-Meteo archive fallback (mirroring the full-history source order). The archive, which is the same ERA5 family as the lake record it extends, so
Open-Meteo fallback is skipped while it's in a rate-limit cooldown.""" topped-up days grade against the climatology without a source seam (NASA's
try: MERRA-2 days ran ~1.3°F off ERA5, sign varying by location). Raises while
return _fetch_history_nasa(cell, start=start_date, end=end_date) the archive is in a rate-limit cooldown; the top-up is best-effort and
except Exception: # noqa: BLE001 - NASA unavailable; try Open-Meteo unless it's cooling down retries next interval."""
if time.time() < _archive_cooldown_until: if time.time() < _archive_cooldown_until:
raise raise WeatherUnavailable(limit_message(_archive_limit_daily),
daily=_archive_limit_daily)
return _fetch_history_range(cell, start_date, end_date) return _fetch_history_range(cell, start_date, end_date)
@ -802,12 +804,14 @@ def _load_history(cell: dict) -> tuple[pl.DataFrame, dict]:
def _serve_stale(): def _serve_stale():
return _with_doy(stale), {"cached": True, "stale": True, "cache_age_days": None} return _with_doy(stale), {"cached": True, "stale": True, "cache_age_days": None}
# Fetch. The ERA5 lake is first when configured (true ERA5 with measured # Fetch. The ERA5 lake is first (true ERA5 with measured gusts, served
# gusts, served from our own bucket — no third-party API in the path); # from our own bucket — no third-party API in the path); unconfigured
# unconfigured or missing-point cells fall through at zero cost. NASA # or missing-point cells fall through at zero cost. The backup is the
# POWER is next (keyless, independent of Open-Meteo; its missing gusts # Open-Meteo archive — the same ERA5 family as the lake, so a
# are filled from Meteostat inside _fetch_history_nasa), then Open-Meteo # backup-sourced record grades consistently with lake-sourced
# — still guarded by its rate-limit cooldown. Every source must return a # neighbors — still guarded by its rate-limit cooldown. NASA POWER is
# retired from serving (its MERRA-2 record ran ~1.3°F off ERA5 with no
# gusts; kept only for drift_check). Every source must return a
# plausibly-full span before being accepted and cached as a complete # plausibly-full span before being accepted and cached as a complete
# record (a short/partial response is rejected rather than held # record (a short/partial response is rejected rather than held
# indefinitely). # indefinitely).
@ -818,15 +822,7 @@ def _load_history(cell: dict) -> tuple[pl.DataFrame, dict]:
if fetched.height >= MIN_ARCHIVE_DAYS: if fetched.height >= MIN_ARCHIVE_DAYS:
df = fetched df = fetched
source = "era5-lake" source = "era5-lake"
except Exception: # noqa: BLE001 - lake unconfigured/miss; fall through to NASA except Exception: # noqa: BLE001 - lake unconfigured/miss; fall through to Open-Meteo
pass
if df is None:
try:
fetched = _fetch_history_nasa(cell)
if fetched.height >= MIN_ARCHIVE_DAYS:
df = fetched
source = "nasa-power"
except Exception: # noqa: BLE001 - primary unavailable; fall through to Open-Meteo
pass pass
if df is None and time.time() >= _archive_cooldown_until: if df is None and time.time() >= _archive_cooldown_until:
try: try:
@ -920,26 +916,49 @@ def load_cached_recent_forecast(cell: dict) -> "pl.DataFrame | None":
def _fetch_recent_forecast(cell: dict) -> pl.DataFrame: def _fetch_recent_forecast(cell: dict) -> pl.DataFrame:
"""Recent observations + forward forecast WITHOUT Open-Meteo: the recent observed """Fallback recent+forecast bundle without the Open-Meteo *forecast* API:
window from a NASA POWER range (measured), the forward days from MET Norway, the recent observed window from the Open-Meteo ARCHIVE (an independent
merged by date. Gusts which neither source carries are filled from Meteostat: quota, and the same ERA5 family as the history record so the graded days
measured for the observed days, estimated from wind for the forecast days.""" sit on a consistent baseline), the forward days from MET Norway with
Meteostat-estimated gusts. The archive lags a few days more than a live
feed, so this degraded path serves a slightly shorter observed window
honest and consistent beats fresh and skewed."""
today = datetime.date.today() today = datetime.date.today()
start = (today - datetime.timedelta(days=RECENT_PAST_DAYS)).isoformat() start = (today - datetime.timedelta(days=RECENT_PAST_DAYS)).isoformat()
end = (today - datetime.timedelta(days=RECENT_END_LAG_DAYS)).isoformat() end = (today - datetime.timedelta(days=ARCHIVE_LATENCY_DAYS)).isoformat()
past = _fetch_history_nasa(cell, start=start, end=end) # measured + Meteostat gusts past = _fetch_history_range(cell, start, end) # ERA5 days, gusts included
fwd = meteostat.fill_gusts( fwd = meteostat.fill_gusts(
cell["center_lat"], cell["center_lon"], _fetch_forecast_metno(cell)) cell["center_lat"], cell["center_lon"], _fetch_forecast_metno(cell))
# Forecast wins on any overlapping day (freshest model value for today); the NASA # Forecast wins on any overlapping day (freshest model value); the archive
# observed days fill the recent past MET Norway lacks. # days fill the recent past MET Norway lacks.
return (pl.concat([past, fwd], how="diagonal_relaxed") return (pl.concat([past, fwd], how="diagonal_relaxed")
.unique(subset="date", keep="last", maintain_order=True) .unique(subset="date", keep="last", maintain_order=True)
.sort("date")) .sort("date"))
def _om_local_today(payload: dict) -> "datetime.date | None":
"""The calendar date it is *now* at the cell, from the UTC offset Open-Meteo
reports for a ``timezone=auto`` request. None when the offset is absent, which
tells the caller to skip the in-progress-day guard rather than guess a date."""
offset = payload.get("utc_offset_seconds")
if offset is None:
return None
return (datetime.datetime.now(datetime.timezone.utc)
+ datetime.timedelta(seconds=int(offset))).date()
def _fetch_recent_forecast_om(cell: dict) -> pl.DataFrame: def _fetch_recent_forecast_om(cell: dict) -> pl.DataFrame:
"""Fallback recent+forecast bundle from the Open-Meteo forecast API (the former """Fallback recent+forecast bundle from the Open-Meteo forecast API (the former
primary): recent past + forward days in one call.""" primary): recent past + forward days in one call.
The cell's in-progress *local* day is dropped from the bundle. Open-Meteo's
daily high/low for today aggregates only the hours elapsed so far, so grading it
reads a still-unfolding day as complete and produces spurious extremes (a cool
morning served as a record-low high, seen live at the 1st percentile). This is
the same failure the MET Norway path guards against with its diurnal-coverage
gate (see _metno_to_frame); the fallback needs the equivalent. Past days are
complete and future days are whole-day forecasts, so only today is excluded
the day lands in the record once it is over."""
params = { params = {
"latitude": cell["center_lat"], "latitude": cell["center_lat"],
"longitude": cell["center_lon"], "longitude": cell["center_lon"],
@ -952,16 +971,23 @@ def _fetch_recent_forecast_om(cell: dict) -> pl.DataFrame:
"forecast_days": FORECAST_DAYS, "forecast_days": FORECAST_DAYS,
} }
r = _request(FORECAST_URL, params, 60, phase="recent_forecast_fetch") r = _request(FORECAST_URL, params, 60, phase="recent_forecast_fetch")
return _to_frame(r.json()["daily"]) payload = r.json()
df = _to_frame(payload["daily"])
local_today = _om_local_today(payload)
if local_today is not None:
df = df.filter(pl.col("date") != local_today)
return df
def _load_recent_forecast(cell: dict) -> pl.DataFrame: def _load_recent_forecast(cell: dict) -> pl.DataFrame:
"""Recent observations AND the forward forecast for a cell. """Recent observations AND the forward forecast for a cell.
The primary source is keyless and Open-Meteo-free: a NASA POWER recent-past range The primary source is the Open-Meteo forecast API (recent past + forward in
plus the MET Norway forward forecast (see _fetch_recent_forecast). The Open-Meteo one call, gusts included, ERA5-consistent with the history record see
forecast API is the fallback (skipped while it's in its own rate-limit cooldown, _fetch_recent_forecast_om), skipped while it's in its own rate-limit
see _note_forecast_rate_limit), then a stale cache. Cached per cell for cooldown (_note_forecast_rate_limit). The fallback is Open-Meteo-forecast-
free: an archive recent-past range plus MET Norway forward days
(_fetch_recent_forecast). Then a stale cache. Cached per cell for
FORECAST_TTL_HOURS so it follows model updates without per-hour upstream IO. FORECAST_TTL_HOURS so it follows model updates without per-hour upstream IO.
""" """
cell_id = cell["id"] cell_id = cell["id"]
@ -972,22 +998,23 @@ def _load_recent_forecast(cell: dict) -> pl.DataFrame:
return _with_doy(cached) return _with_doy(cached)
df = None df = None
try:
df = _fetch_recent_forecast(cell) # NASA recent + MET forward (primary)
except Exception: # noqa: BLE001 - keyless primary unavailable; try Open-Meteo
pass
forecast_error = None forecast_error = None
if df is None and time.time() >= _forecast_cooldown_until: if time.time() >= _forecast_cooldown_until:
# Fall back to the Open-Meteo forecast API, skipped while it's in its own
# rate-limit cooldown so a brownout doesn't retry-storm the endpoint.
try: try:
df = _fetch_recent_forecast_om(cell) df = _fetch_recent_forecast_om(cell) # primary: one call, gusts, ERA5-consistent
except Exception as e: # noqa: BLE001 except Exception as e: # noqa: BLE001
if is_rate_limit(e): if is_rate_limit(e):
_note_forecast_rate_limit(e) _note_forecast_rate_limit(e)
forecast_error = e forecast_error = e
if df is None:
# Fallback: archive past + MET forward — no Open-Meteo forecast quota.
try:
df = _fetch_recent_forecast(cell)
forecast_error = None
except Exception: # noqa: BLE001 - fallback also down; stale cache next
pass
if df is None: if df is None:
# Last resort: serve the stale cache if we have one, WITHOUT rewriting it, so # Last resort: serve the stale cache if we have one, WITHOUT rewriting it, so
# recent_stamp stays old and the next request retries upstream first (mirrors # recent_stamp stays old and the next request retries upstream first (mirrors

View file

@ -279,8 +279,10 @@ def all_time_records(df: pl.DataFrame) -> dict:
def longest_dry_streak(df: pl.DataFrame) -> tuple[int, str | None]: def longest_dry_streak(df: pl.DataFrame) -> tuple[int, str | None]:
"""Longest run of consecutive days without measurable rain, and the ISO date the """Longest run of consecutive days without any rain, and the ISO date the streak
streak began. A dry day is precip < RAIN_THRESHOLD; null precip counts as dry.""" began. A dry day has no rain at all (precip <= 0); null precip counts as dry. The
threshold matches _grade_precip's dry/rain split — any measurable rain, however
slight, is a rain day that breaks the streak (not the 0.01" rain-frequency line)."""
if "precip" not in df.columns: if "precip" not in df.columns:
return (0, None) return (0, None)
d = df.select(["date", "precip"]).sort("date") d = df.select(["date", "precip"]).sort("date")
@ -288,7 +290,7 @@ def longest_dry_streak(df: pl.DataFrame) -> tuple[int, str | None]:
best_len, best_start = 0, None best_len, best_start = 0, None
cur_len, cur_start = 0, None cur_len, cur_start = 0, None
for dt, p in zip(dates, precips): for dt, p in zip(dates, precips):
wet = p is not None and not (isinstance(p, float) and np.isnan(p)) and p >= RAIN_THRESHOLD wet = p is not None and not (isinstance(p, float) and np.isnan(p)) and p > 0
if wet: if wet:
cur_len, cur_start = 0, None cur_len, cur_start = 0, None
else: else:
@ -305,11 +307,12 @@ def longest_dry_streak(df: pl.DataFrame) -> tuple[int, str | None]:
def _band_stats(samples: np.ndarray) -> dict | None: def _band_stats(samples: np.ndarray) -> dict | None:
if samples.size == 0: if samples.size == 0:
return None return None
# Percentiles for the chart's nested "normal" fan, matching the 9 tier bounds: # Percentiles for the chart's nested "normal" fan. p40-p60 is the Normal band;
# p40-p60 is the Normal band; p25/p75, p10/p90 and p1/p99 mark the successive # p25/p75, p10/p90 and p1/p99 mark the successive Below/Above Normal, Low/High,
# Below/Above Normal, Low/High, Very Low/High and Near-Record edges. # Very Low/High and Near-Record edges. p95 additionally splits the rain fan's top
p1, p10, p25, p40, p50, p60, p75, p90, p99 = np.percentile( # region into Very Heavy (90-95) and Severe (95-99); unused by the temperature fan.
samples, [1, 10, 25, 40, 50, 60, 75, 90, 99] p1, p10, p25, p40, p50, p60, p75, p90, p95, p99 = np.percentile(
samples, [1, 10, 25, 40, 50, 60, 75, 90, 95, 99]
) )
return { return {
"p1": round(float(p1), 1), "p1": round(float(p1), 1),
@ -320,6 +323,7 @@ def _band_stats(samples: np.ndarray) -> dict | None:
"p60": round(float(p60), 1), "p60": round(float(p60), 1),
"p75": round(float(p75), 1), "p75": round(float(p75), 1),
"p90": round(float(p90), 1), "p90": round(float(p90), 1),
"p95": round(float(p95), 1),
"p99": round(float(p99), 1), "p99": round(float(p99), 1),
} }
@ -352,13 +356,14 @@ def _grade_precip(samples: np.ndarray, value) -> dict | None:
def dry_streaks(dates, precips) -> dict[str, int]: def dry_streaks(dates, precips) -> dict[str, int]:
"""Map each date (ISO string) to days since the last measurable rain, walking a """Map each date (ISO string) to days since the last day with any rain, walking a
chronological precip series. Missing precip counts as a dry day. `dates` is an chronological precip series. Any measurable rain (precip > 0) resets the count, so
iterable of ``datetime.date`` (a polars Date column's ``.to_list()``).""" this matches _grade_precip's dry/rain split; missing precip counts as a dry day.
`dates` is an iterable of ``datetime.date`` (a polars Date column's ``.to_list()``)."""
out: dict[str, int] = {} out: dict[str, int] = {}
streak = 0 streak = 0
for d, p in zip(dates, precips): for d, p in zip(dates, precips):
wet = p is not None and not (isinstance(p, float) and np.isnan(p)) and p >= RAIN_THRESHOLD wet = p is not None and not (isinstance(p, float) and np.isnan(p)) and p > 0
streak = 0 if wet else streak + 1 streak = 0 if wet else streak + 1
out[_as_date(d).isoformat()] = streak out[_as_date(d).isoformat()] = streak
return out return out

View file

@ -37,7 +37,7 @@ log = logging.getLogger("thermograph.push")
_DATA_DIR = paths.DATA_DIR _DATA_DIR = paths.DATA_DIR
_VAPID_PATH = os.environ.get("THERMOGRAPH_VAPID_FILE") or os.path.join(_DATA_DIR, "vapid.json") _VAPID_PATH = os.environ.get("THERMOGRAPH_VAPID_FILE") or os.path.join(_DATA_DIR, "vapid.json")
# The VAPID "sub" claim — a contact the push service can reach about our traffic. # The VAPID "sub" claim — a contact the push service can reach about our traffic.
_CONTACT = os.environ.get("THERMOGRAPH_VAPID_CONTACT", "mailto:admin@thermograph.app") _CONTACT = os.environ.get("THERMOGRAPH_VAPID_CONTACT", "mailto:admin@thermograph.org")
# pywebpush 2.0.0 forwards `timeout` straight to requests.post with no default of # pywebpush 2.0.0 forwards `timeout` straight to requests.post with no default of
# its own — omit the kwarg here and the send blocks with NO timeout at all (the # its own — omit the kwarg here and the send blocks with NO timeout at all (the
@ -174,10 +174,13 @@ def send(subscription_info: dict, payload: dict) -> str:
return "ok" return "ok"
except WebPushException as e: except WebPushException as e:
status = getattr(getattr(e, "response", None), "status_code", None) status = getattr(getattr(e, "response", None), "status_code", None)
if status in (404, 410): if status in (401, 403, 404, 410):
return "gone" # endpoint retired — caller should delete it # 404/410 = endpoint retired; 401/403 = VAPID mismatch (e.g. after a key
# 401/403 = VAPID key mismatch, etc. Record it where it's visible (the errors # rotation) — the row was minted under a key we can no longer sign for and
# JSONL / dashboard), not just the system journal. # will never authenticate again. All are permanently dead: caller deletes.
return "gone"
# Anything else (rate limits, 5xx, malformed payload) may be transient — keep
# the row and record it where it's visible (the errors JSONL / dashboard).
log.warning("web push failed (status=%s): %s", status, e) log.warning("web push failed (status=%s): %s", status, e)
audit.log_event("error", {"phase": "push", "status": status, audit.log_event("error", {"phase": "push", "status": status,
"endpoint": (subscription_info.get("endpoint") or "")[:120]}) "endpoint": (subscription_info.get("endpoint") or "")[:120]})

View file

@ -118,6 +118,28 @@ def test_records_payload_shape(client):
assert body["canonical_path"] == "/climate/testville/records" assert body["canonical_path"] == "/climate/testville/records"
# --- https origin (jsonld url) -----------------------------------------------
def test_public_host_jsonld_url_is_https(client):
"""The public site is HTTPS-only behind Caddy, which proxies plain HTTP, so
x-forwarded-proto reads "http". A request on the configured public host
(THERMOGRAPH_BASE_URL defaults to https://thermograph.org) must still fold an
https:// origin into the payload's jsonld url; otherwise the frontend renders
an http:// canonical/JSON-LD that 308-redirects."""
import json as _json
hdr = {"host": "thermograph.org", "x-forwarded-proto": "http"}
for path in ("city/testville", "city/testville/records"):
s = _json.dumps(client.get(f"{BASE}/{path}", headers=hdr).json())
assert "https://thermograph.org/thermograph/climate/testville" in s
assert "http://thermograph.org" not in s
def test_non_public_host_jsonld_keeps_forwarded_scheme(client):
hdr = {"host": "192.168.1.10:8000", "x-forwarded-proto": "http"}
body = client.get(f"{BASE}/city/testville", headers=hdr).json()
assert body["jsonld"]["@graph"][0]["url"].startswith("http://192.168.1.10:8000/")
# --- 404s -------------------------------------------------------------------- # --- 404s --------------------------------------------------------------------
def test_unknown_slug_is_404(client, counts): def test_unknown_slug_is_404(client, counts):

View file

@ -165,12 +165,34 @@ def test_metno_to_frame_drops_days_without_diurnal_coverage():
assert row["tmax"] != row["tmin"] assert row["tmax"] != row["tmin"]
def test_recent_forecast_merges_nasa_past_and_metno_forward(monkeypatch, tmp_path): def test_recent_forecast_om_is_primary(monkeypatch, tmp_path):
"""The recent/forecast bundle is built from a NASA POWER recent-past range plus the """The Open-Meteo forecast bundle is the primary recent+forecast source
MET Norway forward forecast, merged by date (Open-Meteo not consulted).""" (one call, gusts included, ERA5-consistent); the archive+MET composite is
not consulted when it succeeds."""
monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path)) monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path))
monkeypatch.setattr(climate, "_forecast_cooldown_until", 0.0)
cell = {"id": "rf_primary", "center_lat": 47.6, "center_lon": -122.3} cell = {"id": "rf_primary", "center_lat": 47.6, "center_lon": -122.3}
om = climate._to_frame(_om_daily(3))
monkeypatch.setattr(climate, "_fetch_recent_forecast_om", lambda c: om)
def _fallback_should_not_run(c): raise AssertionError("composite fallback should not run")
monkeypatch.setattr(climate, "_fetch_recent_forecast", _fallback_should_not_run)
df = climate._load_recent_forecast(cell)
assert df.height == om.height
def test_recent_forecast_fallback_merges_archive_past_and_metno_forward(monkeypatch, tmp_path):
"""When the Open-Meteo forecast API is down, the bundle is built from an
Open-Meteo ARCHIVE recent-past range plus the MET Norway forward forecast
NASA POWER is never consulted."""
monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path))
monkeypatch.setattr(climate, "_forecast_cooldown_until", 0.0)
monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0)
cell = {"id": "rf_fallback", "center_lat": 47.6, "center_lon": -122.3}
def _om_forecast_down(c): raise RuntimeError("forecast API down")
monkeypatch.setattr(climate, "_fetch_recent_forecast_om", _om_forecast_down)
past = climate._finalize_approximated(pl.DataFrame({ past = climate._finalize_approximated(pl.DataFrame({
"date": [datetime.date(2026, 7, 10), datetime.date(2026, 7, 11)], "date": [datetime.date(2026, 7, 10), datetime.date(2026, 7, 11)],
"tmax": [70.0, 72.0], "tmin": [50.0, 51.0], "precip": [0.0, 0.1], "tmax": [70.0, 72.0], "tmin": [50.0, 51.0], "precip": [0.0, 0.1],
@ -181,11 +203,11 @@ def test_recent_forecast_merges_nasa_past_and_metno_forward(monkeypatch, tmp_pat
"tmax": [80.0, 82.0], "tmin": [60.0, 61.0], "precip": [0.0, 0.0], "tmax": [80.0, 82.0], "tmin": [60.0, 61.0], "precip": [0.0, 0.0],
"wind": [3.0, 4.0], "humid": [40.0, 45.0], "wind": [3.0, 4.0], "humid": [40.0, 45.0],
})) }))
monkeypatch.setattr(climate, "_fetch_history_nasa", lambda c, start, end: past) monkeypatch.setattr(climate, "_fetch_history_range", lambda c, s, e: past)
monkeypatch.setattr(climate, "_fetch_forecast_metno", lambda c: fwd) monkeypatch.setattr(climate, "_fetch_forecast_metno", lambda c: fwd)
monkeypatch.setattr(climate.meteostat, "fill_gusts", lambda lat, lon, df: df) monkeypatch.setattr(climate.meteostat, "fill_gusts", lambda lat, lon, df: df)
def _om_should_not_run(c): raise AssertionError("Open-Meteo forecast should not run") def _nasa_should_not_run(*a, **k): raise AssertionError("NASA is retired from serving")
monkeypatch.setattr(climate, "_fetch_recent_forecast_om", _om_should_not_run) monkeypatch.setattr(climate, "_fetch_history_nasa", _nasa_should_not_run)
df = climate._load_recent_forecast(cell) df = climate._load_recent_forecast(cell)
assert df["date"].to_list() == [ assert df["date"].to_list() == [
@ -193,19 +215,90 @@ def test_recent_forecast_merges_nasa_past_and_metno_forward(monkeypatch, tmp_pat
datetime.date(2026, 7, 16), datetime.date(2026, 7, 17)] datetime.date(2026, 7, 16), datetime.date(2026, 7, 17)]
def test_recent_forecast_falls_back_to_open_meteo(monkeypatch, tmp_path): def test_recent_forecast_om_drops_the_in_progress_local_day(monkeypatch):
"""When the NASA + MET Norway primary fails, the Open-Meteo forecast API serves.""" """The Open-Meteo fallback must not grade the cell's in-progress local day: its
monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path)) daily high/low is only a partial aggregate of the hours elapsed so far (a cool
monkeypatch.setattr(climate, "_forecast_cooldown_until", 0.0) morning would read as a record-low high). Past days and future forecast days
cell = {"id": "rf_fallback", "center_lat": 47.6, "center_lon": -122.3} survive; today per the UTC offset Open-Meteo reports for a timezone=auto
request is dropped, matching the MET path's diurnal-coverage gate."""
offset = 3 * 3600 # UTC+3, e.g. Europe/Vilnius (the reported Ringaudai incident)
local_today = (datetime.datetime.now(datetime.timezone.utc)
+ datetime.timedelta(seconds=offset)).date()
days = [local_today + datetime.timedelta(days=n) for n in (-2, -1, 0, 1)]
daily = {
"time": [d.isoformat() for d in days],
"temperature_2m_max": [80.0, 82.0, 61.0, 84.0], # today's 61 is the partial value
"temperature_2m_min": [60.0, 61.0, 57.0, 62.0],
"precipitation_sum": [0.0, 0.0, 0.0, 0.0],
}
payload = {"utc_offset_seconds": offset, "daily": daily}
def _primary_down(c): raise RuntimeError("NASA + MET Norway down") class Resp:
monkeypatch.setattr(climate, "_fetch_recent_forecast", _primary_down) def json(self): return payload
om = climate._to_frame(_om_daily(3)) monkeypatch.setattr(climate, "_request", lambda *a, **k: Resp())
monkeypatch.setattr(climate, "_fetch_recent_forecast_om", lambda c: om)
df = climate._load_recent_forecast(cell) got = climate._fetch_recent_forecast_om(
assert df.height == om.height {"center_lat": 54.9, "center_lon": 23.8})["date"].to_list()
assert local_today not in got # partial today dropped
assert local_today - datetime.timedelta(days=1) in got # yesterday kept
assert local_today + datetime.timedelta(days=1) in got # tomorrow's forecast kept
assert len(got) == 3
def test_recent_forecast_om_keeps_all_days_without_a_utc_offset(monkeypatch):
"""No UTC offset reported -> skip the in-progress-day guard rather than guess a
date, so the bundle passes through as before (only the usual null-day filter)."""
payload = {"daily": _om_daily(3)} # no utc_offset_seconds
class Resp:
def json(self): return payload
monkeypatch.setattr(climate, "_request", lambda *a, **k: Resp())
df = climate._fetch_recent_forecast_om({"center_lat": 1.0, "center_lon": 2.0})
assert df.height == climate._to_frame(_om_daily(3)).height
def test_recent_forecast_om_drops_the_in_progress_local_day(monkeypatch):
"""The Open-Meteo fallback must not grade the cell's in-progress local day: its
daily high/low is only a partial aggregate of the hours elapsed so far (a cool
morning would read as a record-low high). Past days and future forecast days
survive; today per the UTC offset Open-Meteo reports for a timezone=auto
request is dropped, matching the MET path's diurnal-coverage gate."""
offset = 3 * 3600 # UTC+3, e.g. Europe/Vilnius (the reported Ringaudai incident)
local_today = (datetime.datetime.now(datetime.timezone.utc)
+ datetime.timedelta(seconds=offset)).date()
days = [local_today + datetime.timedelta(days=n) for n in (-2, -1, 0, 1)]
daily = {
"time": [d.isoformat() for d in days],
"temperature_2m_max": [80.0, 82.0, 61.0, 84.0], # today's 61 is the partial value
"temperature_2m_min": [60.0, 61.0, 57.0, 62.0],
"precipitation_sum": [0.0, 0.0, 0.0, 0.0],
}
payload = {"utc_offset_seconds": offset, "daily": daily}
class Resp:
def json(self): return payload
monkeypatch.setattr(climate, "_request", lambda *a, **k: Resp())
got = climate._fetch_recent_forecast_om(
{"center_lat": 54.9, "center_lon": 23.8})["date"].to_list()
assert local_today not in got # partial today dropped
assert local_today - datetime.timedelta(days=1) in got # yesterday kept
assert local_today + datetime.timedelta(days=1) in got # tomorrow's forecast kept
assert len(got) == 3
def test_recent_forecast_om_keeps_all_days_without_a_utc_offset(monkeypatch):
"""No UTC offset reported -> skip the in-progress-day guard rather than guess a
date, so the bundle passes through as before (only the usual null-day filter)."""
payload = {"daily": _om_daily(3)} # no utc_offset_seconds
class Resp:
def json(self): return payload
monkeypatch.setattr(climate, "_request", lambda *a, **k: Resp())
df = climate._fetch_recent_forecast_om({"center_lat": 1.0, "center_lon": 2.0})
assert df.height == climate._to_frame(_om_daily(3)).height
def test_recent_forecast_serves_stale_cache_when_all_sources_fail(monkeypatch, tmp_path): def test_recent_forecast_serves_stale_cache_when_all_sources_fail(monkeypatch, tmp_path):
@ -359,9 +452,9 @@ def test_lake_is_first_history_source(monkeypatch, tmp_path):
assert df.height == full.height assert df.height == full.height
def test_lake_miss_falls_through_to_nasa(monkeypatch, tmp_path): def test_lake_miss_falls_through_to_open_meteo(monkeypatch, tmp_path):
"""An unconfigured lake (or a point outside it) costs nothing: NASA serves """An unconfigured lake (or a point outside it) costs nothing: the
exactly as before the lake existed.""" Open-Meteo archive serves, and NASA retired is never consulted."""
monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path)) monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path))
monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0) monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0)
cell = {"id": "lake_miss", "center_lat": 47.6, "center_lon": -122.3} cell = {"id": "lake_miss", "center_lat": 47.6, "center_lon": -122.3}
@ -370,89 +463,60 @@ def test_lake_miss_falls_through_to_nasa(monkeypatch, tmp_path):
raise climate.era5lake.LakeUnavailable("no lake configured") raise climate.era5lake.LakeUnavailable("no lake configured")
monkeypatch.setattr(climate.era5lake, "fetch_history_lake", _no_lake) monkeypatch.setattr(climate.era5lake, "fetch_history_lake", _no_lake)
full = _full_history_frame() full = _full_history_frame()
monkeypatch.setattr(climate, "_fetch_history_nasa", lambda c: full)
df, meta = climate._load_history(cell)
assert meta["source"] == "nasa-power"
assert df.height == full.height
def test_nasa_is_primary_history_source(monkeypatch, tmp_path):
"""NASA POWER is the primary history source: a full NASA frame is accepted and
Open-Meteo is never consulted."""
monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path))
monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0)
cell = {"id": "nasa_primary", "center_lat": 47.6, "center_lon": -122.3}
full = _full_history_frame()
monkeypatch.setattr(climate, "_fetch_history_nasa", lambda c: full)
def _om_should_not_run(c): raise AssertionError("Open-Meteo should not be called")
monkeypatch.setattr(climate, "_fetch_history", _om_should_not_run)
df, meta = climate._load_history(cell)
assert meta["source"] == "nasa-power"
assert df.height == full.height
def test_falls_back_to_open_meteo_when_nasa_unavailable(monkeypatch, tmp_path):
"""When NASA POWER fails, the load falls back to the Open-Meteo archive."""
monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path))
monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0)
cell = {"id": "om_fallback", "center_lat": 47.6, "center_lon": -122.3}
full = _full_history_frame()
def _nasa_down(c): raise RuntimeError("NASA POWER outage")
monkeypatch.setattr(climate, "_fetch_history_nasa", _nasa_down)
monkeypatch.setattr(climate, "_fetch_history", lambda c: full) monkeypatch.setattr(climate, "_fetch_history", lambda c: full)
def _nasa_should_not_run(*a, **k): raise AssertionError("NASA is retired from serving")
monkeypatch.setattr(climate, "_fetch_history_nasa", _nasa_should_not_run)
df, meta = climate._load_history(cell) df, meta = climate._load_history(cell)
assert meta["source"] == "open-meteo" assert meta["source"] == "open-meteo"
assert df.height == full.height assert df.height == full.height
def test_short_nasa_is_rejected_and_falls_back_to_open_meteo(monkeypatch, tmp_path): def test_short_lake_record_is_rejected_and_falls_back_to_open_meteo(monkeypatch, tmp_path):
"""A too-short NASA response (a partial/unavailable point) is rejected rather than """A too-short lake frame (a thin point that slipped past the client's own
cached as a complete record, and Open-Meteo serves instead.""" gate) is rejected rather than cached as a complete record; the Open-Meteo
archive serves instead."""
monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path)) monkeypatch.setattr(climate, "CACHE_DIR", str(tmp_path))
monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0) monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0)
cell = {"id": "short_nasa", "center_lat": 47.6, "center_lon": -122.3} cell = {"id": "short_lake", "center_lat": 47.6, "center_lon": -122.3}
short = climate._to_frame(_om_daily(3)) # 2 usable days « threshold short = climate._to_frame(_om_daily(3)) # 2 usable days « threshold
full = _full_history_frame() # >= MIN_ARCHIVE_DAYS full = _full_history_frame() # >= MIN_ARCHIVE_DAYS
assert short.height < climate.MIN_ARCHIVE_DAYS <= full.height assert short.height < climate.MIN_ARCHIVE_DAYS <= full.height
monkeypatch.setattr(climate, "_fetch_history_nasa", lambda c: short) monkeypatch.setattr(climate.era5lake, "fetch_history_lake",
lambda c, start=None: short)
monkeypatch.setattr(climate, "_fetch_history", lambda c: full) monkeypatch.setattr(climate, "_fetch_history", lambda c: full)
df, meta = climate._load_history(cell) df, meta = climate._load_history(cell)
assert meta["source"] == "open-meteo" # rejected the short NASA response assert meta["source"] == "open-meteo" # rejected the short lake frame
assert df.height == full.height assert df.height == full.height
def test_history_tail_prefers_nasa(monkeypatch): def test_history_tail_uses_open_meteo_archive(monkeypatch):
"""The tail top-up fetches NASA POWER first; Open-Meteo's range fetch is not """The tail top-up is an Open-Meteo archive range — ERA5-consistent with
touched when NASA succeeds.""" the record it extends; NASA is retired and never consulted."""
cell = {"center_lat": 1.0, "center_lon": 2.0}
sentinel = object()
monkeypatch.setattr(climate, "_fetch_history_nasa",
lambda c, start=None, end=None: sentinel)
def _om_should_not_run(c, s, e): raise AssertionError("Open-Meteo tail should not run")
monkeypatch.setattr(climate, "_fetch_history_range", _om_should_not_run)
assert climate._fetch_history_tail(cell, "2026-06-01", "2026-06-10") is sentinel
def test_history_tail_falls_back_to_open_meteo(monkeypatch):
"""When NASA fails and Open-Meteo isn't cooling down, the tail falls back to the
Open-Meteo archive range."""
monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0) monkeypatch.setattr(climate, "_archive_cooldown_until", 0.0)
cell = {"center_lat": 1.0, "center_lon": 2.0} cell = {"center_lat": 1.0, "center_lon": 2.0}
def _nasa_down(c, start=None, end=None): raise RuntimeError("NASA down")
monkeypatch.setattr(climate, "_fetch_history_nasa", _nasa_down)
sentinel = object() sentinel = object()
monkeypatch.setattr(climate, "_fetch_history_range", lambda c, s, e: sentinel) monkeypatch.setattr(climate, "_fetch_history_range", lambda c, s, e: sentinel)
def _nasa_should_not_run(*a, **k): raise AssertionError("NASA is retired from serving")
monkeypatch.setattr(climate, "_fetch_history_nasa", _nasa_should_not_run)
assert climate._fetch_history_tail(cell, "2026-06-01", "2026-06-10") is sentinel assert climate._fetch_history_tail(cell, "2026-06-01", "2026-06-10") is sentinel
def test_history_tail_raises_during_archive_cooldown(monkeypatch):
"""While the archive is rate-limit cooling, the top-up raises (best-effort;
retried next interval) instead of hammering the endpoint."""
import time as _time
monkeypatch.setattr(climate, "_archive_cooldown_until", _time.time() + 60)
cell = {"center_lat": 1.0, "center_lon": 2.0}
def _om_should_not_run(c, s, e): raise AssertionError("archive is cooling down")
monkeypatch.setattr(climate, "_fetch_history_range", _om_should_not_run)
with pytest.raises(climate.WeatherUnavailable):
climate._fetch_history_tail(cell, "2026-06-01", "2026-06-10")
# --- forecast cooldown (mirrors the archive path's, tracked separately) ----- # --- forecast cooldown (mirrors the archive path's, tracked separately) -----
def test_forecast_cooldown_skips_the_open_meteo_fallback(monkeypatch, tmp_path): def test_forecast_cooldown_skips_the_open_meteo_fallback(monkeypatch, tmp_path):

View file

@ -113,7 +113,9 @@ def test_dry_streaks_walk():
dates = [datetime.date(2024, 1, 1) + datetime.timedelta(days=i) for i in range(5)] dates = [datetime.date(2024, 1, 1) + datetime.timedelta(days=i) for i in range(5)]
precips = [0.5, 0.0, float("nan"), 0.02, 0.005] precips = [0.5, 0.0, float("nan"), 0.02, 0.005]
out = grading.dry_streaks(dates, precips) out = grading.dry_streaks(dates, precips)
assert list(out.values()) == [0, 1, 2, 0, 1] # NaN counts as dry # Any rain > 0 breaks the streak (matching the dry/rain grading split), so the
# 0.005" trace day resets to 0; only exact 0.0 and NaN count as dry.
assert list(out.values()) == [0, 1, 2, 0, 0]
# ---- range + day grading over a synthetic record -------------------------------- # ---- range + day grading over a synthetic record --------------------------------
@ -141,7 +143,7 @@ def test_grade_day_normals_and_departure(history):
result = grading.grade_day(history, target, {"tmax": row["tmax"], "tmin": row["tmin"], result = grading.grade_day(history, target, {"tmax": row["tmax"], "tmin": row["tmin"],
"precip": row["precip"]}) "precip": row["precip"]})
assert set(result["normals"]["tmax"]) == {"p1", "p10", "p25", "p40", "p50", "p60", assert set(result["normals"]["tmax"]) == {"p1", "p10", "p25", "p40", "p50", "p60",
"p75", "p90", "p99"} "p75", "p90", "p95", "p99"}
expected = max(abs(result["tmax"]["percentile"] - 50), abs(result["tmin"]["percentile"] - 50)) expected = max(abs(result["tmax"]["percentile"] - 50), abs(result["tmin"]["percentile"] - 50))
assert result["departure"] == round(expected, 1) assert result["departure"] == round(expected, 1)

View file

@ -1,11 +1,16 @@
"""VAPID key resolution: the atomic first-writer-wins claim of the keypair file """VAPID key resolution: the atomic first-writer-wins claim of the keypair file
(push.py's `_claim_file`), and that `_load()` caches whatever that settles on (push.py's `_claim_file`), and that `_load()` caches whatever that settles on
rather than its own local generation when it loses the race. rather than its own local generation when it loses the race. Also `send()`'s
mapping of push-service responses onto 'ok'/'gone'/'error'.
No network `send()`'s pywebpush call isn't exercised here (that's notify.py's No network where `send()` is exercised the pywebpush call is stubbed; the
`test_push_dispatched_on_new_notification` etc., which stub `push.send` itself).""" dispatch path (notify.py's `test_push_dispatched_on_new_notification` etc.) stubs
`push.send` itself."""
import json import json
import pytest
from pywebpush import WebPushException
from notifications import push from notifications import push
@ -94,3 +99,44 @@ def test_load_prefers_env_over_file(monkeypatch, tmp_path):
result = push._load() result = push._load()
assert result == {"private_key": "priv-env", "public_key": "pub-env"} assert result == {"private_key": "priv-env", "public_key": "pub-env"}
# --- send() response mapping ------------------------------------------------
_SUB = {"endpoint": "https://push.example.com/ep-1", "keys": {"p256dh": "BKEY", "auth": "YXV0aA"}}
class _Resp:
def __init__(self, status_code):
self.status_code = status_code
def _raise_status(status):
def _webpush(**kwargs):
raise WebPushException("boom", response=_Resp(status))
return _webpush
@pytest.mark.parametrize("status", [401, 403, 404, 410])
def test_send_prunes_permanently_dead_endpoints(monkeypatch, status):
# 404/410 = endpoint retired; 401/403 = VAPID key mismatch (e.g. after a key
# rotation). All are permanently dead, so send() reports 'gone' and the caller
# deletes the row — otherwise a rotated key leaves dead subscriptions forever.
monkeypatch.setattr(push, "webpush", _raise_status(status))
assert push.send(_SUB, {"hello": "world"}) == "gone"
@pytest.mark.parametrize("status", [429, 500, 502])
def test_send_keeps_row_on_transient_failure(monkeypatch, status):
# Rate limits / 5xx may recover; keep the row and report it as an error.
monkeypatch.setattr(push, "webpush", _raise_status(status))
assert push.send(_SUB, {"hello": "world"}) == "error"
def test_send_ok(monkeypatch):
monkeypatch.setattr(push, "webpush", lambda **kwargs: None)
assert push.send(_SUB, {"hello": "world"}) == "ok"
def test_default_contact_is_the_org_domain():
# The VAPID "sub" contact must be a domain we actually own; .app was a typo.
assert push._CONTACT == "mailto:admin@thermograph.org"

View file

@ -71,6 +71,18 @@ def test_api_version_reports_backend_contract(client):
assert body["backend_version"] == "2" assert body["backend_version"] == "2"
def test_malformed_date_is_rejected(client):
# A malformed or non-calendar date must be a 422, not a 500: fromisoformat
# raises ValueError on these and the handler used to leak it as a crash.
for bad in ("notadate", "2026-13-40", "2026-02-30"):
for route in ("grade", "day"):
r = client.get(f"/thermograph/api/v2/{route}", params={**Q, "date": bad})
assert r.status_code == 422, (route, bad, r.status_code)
# A valid date still works.
assert client.get("/thermograph/api/v2/grade",
params={**Q, "date": "2026-07-11"}).status_code == 200
def test_day_detail_and_ladders(client, history): def test_day_detail_and_ladders(client, history):
r = client.get("/thermograph/api/v2/day", params=Q) r = client.get("/thermograph/api/v2/day", params=Q)
assert r.status_code == 200 assert r.status_code == 200

View file

@ -79,6 +79,20 @@ def _weather_fetch_error(e) -> HTTPException:
return HTTPException(status_code=502, detail=f"weather data fetch failed: {e}") return HTTPException(status_code=502, detail=f"weather data fetch failed: {e}")
def _parse_target_date(date: str | None, default: datetime.date) -> datetime.date:
"""Parse the ``date`` query param (YYYY-MM-DD) into a date, falling back to
``default`` when it's absent. A malformed or non-calendar date (e.g. 2026-13-40)
is a client error, not a crash: surface it as a 422 rather than let
fromisoformat's ValueError become a 500."""
if not date:
return default
try:
return datetime.date.fromisoformat(date[:10])
except ValueError:
raise HTTPException(status_code=422,
detail=f"invalid date: {date!r} (expected YYYY-MM-DD)")
# --- background neighbor warming --------------------------------------------- # --- background neighbor warming ---------------------------------------------
# /cell?neighbors=1 asks the server to warm the 8 grid cells around the request # /cell?neighbors=1 asks the server to warm the 8 grid cells around the request
# (the frontend used to fire 8 staggered prefetch requests, mirroring grid.py's # (the frontend used to fire 8 staggered prefetch requests, mirroring grid.py's
@ -492,7 +506,7 @@ def api_grade(
description="days to grade after the target (observed, or forecast when future; " description="days to grade after the target (observed, or forecast when future; "
"the forecast reaches ~7 days out so future days cap there)"), "the forecast reaches ~7 days out so future days cap there)"),
): ):
target = datetime.date.fromisoformat(date[:10]) if date else datetime.date.today() target = _parse_target_date(date, datetime.date.today())
cell = grid.snap(lat, lon) cell = grid.snap(lat, lon)
with audit.RunAudit( with audit.RunAudit(
@ -580,7 +594,7 @@ def api_day(
) as run: ) as run:
history, cache_meta, _ = _fetch_history(run, cell) history, cache_meta, _ = _fetch_history(run, cell)
last = history["date"].max() last = history["date"].max()
target = datetime.date.fromisoformat(date[:10]) if date else last target = _parse_target_date(date, last)
run.set(target_date=target.isoformat()) run.set(target_date=target.isoformat())
def build(place): def build(place):

View file

@ -13,6 +13,7 @@ import hashlib
import json import json
import logging import logging
import os import os
from urllib.parse import urlsplit
import httpx import httpx
from fastapi import HTTPException, Request, Response from fastapi import HTTPException, Request, Response
@ -56,13 +57,28 @@ _log = logging.getLogger(__name__)
# --- helpers ------------------------------------------------------------- # --- helpers -------------------------------------------------------------
# The configured public origin is the source of truth for the scheme. Caddy
# terminates TLS and reverse-proxies plain HTTP, so x-forwarded-proto (and
# request.url.scheme) read "http" even though the public site is HTTPS-only.
# Trusting that scheme makes every canonical / og:url / og:image / sitemap <loc>
# emit http://, which 308-redirects to https:// -- self-conflicting canonicals
# and a sitemap full of redirecting URLs. So a request that arrives on the
# configured public host is answered with the configured public scheme (https on
# prod/beta, both of which set THERMOGRAPH_BASE_URL per host); localhost/LAN dev
# never matches it and keeps the observed scheme, so plain-HTTP dev is unchanged.
_PUBLIC = urlsplit(os.environ.get("THERMOGRAPH_BASE_URL", "https://thermograph.org"))
def _origin(request: Request) -> str: def _origin(request: Request) -> str:
# x-forwarded-host takes precedence over host: when reached via backend's # x-forwarded-host takes precedence over host: when reached via backend's
# internal proxy fallback (no Caddy in front -- see _proxy_to_frontend in # internal proxy fallback (no Caddy in front -- see _proxy_to_frontend in
# backend/web/app.py), Host is the internal hop's own address, not the # backend/web/app.py), Host is the internal hop's own address, not the
# browser-facing one. # browser-facing one.
proto = request.headers.get("x-forwarded-proto") or request.url.scheme
host = request.headers.get("x-forwarded-host") or request.headers.get("host") or request.url.netloc host = request.headers.get("x-forwarded-host") or request.headers.get("host") or request.url.netloc
if _PUBLIC.scheme and _PUBLIC.netloc and host == _PUBLIC.netloc:
proto = _PUBLIC.scheme
else:
proto = request.headers.get("x-forwarded-proto") or request.url.scheme
return f"{proto}://{host}" return f"{proto}://{host}"

View file

@ -8,7 +8,7 @@ import { W, PW, H, PL, PR, plotTop, plotBot, xLabY, setChartWidth, chartPalette,
tempChart, precipChart, dryChart, attachChartHover } from "./chart.js"; tempChart, precipChart, dryChart, attachChartHover } from "./chart.js";
import { track } from "./digest.js"; import { track } from "./digest.js";
import { TIER_COLORS, SCALE_TEMP, drynessColor, pctOrd, esc, todayISO, placeLabel, import { TIER_COLORS, SCALE_TEMP, drynessColor, pctOrd, esc, todayISO, placeLabel,
tierKeySegs, GUIDE_LINK, fmtPrecip, fmtWind, fmtHumid } from "./shared.js"; tierKeySegs, GUIDE_LINK, fmtPrecip, fmtPrecipTier, fmtWind, fmtHumid } from "./shared.js";
let selected = null; // {lat, lon} let selected = null; // {lat, lon}
@ -323,11 +323,14 @@ function precipCell(d, isFc, isToday) {
const dd = ` data-date="${d.date}"`; const dd = ` data-date="${d.date}"`;
const cls = "rd-c" + (isFc ? " rd-fc" : isToday ? " rd-today" : ""); const cls = "rd-c" + (isFc ? " rd-fc" : isToday ? " rd-today" : "");
if (!g) return `<span class="${cls}"${dd}>—</span>`; if (!g) return `<span class="${cls}"${dd}>—</span>`;
if (g.value === 0 && d.dsr != null && d.dsr > 0) { // Only a genuinely dry day (no rain at all) labels the streak; any rain day —
return `<span class="${cls}"${dd} style="--c:${drynessColor(d.dsr)}" title="${d.dsr} days since measurable rain">${d.dsr}d</span>`; // down to sub-0.01" trace that rounds to 0 — shows its depth, tinted by tier.
if (g.class === "dry" && d.dsr != null && d.dsr > 0) {
return `<span class="${cls}"${dd} style="--c:${drynessColor(d.dsr)}" title="${d.dsr} days since rain">${d.dsr}d</span>`;
} }
const col = TIER_COLORS[g.class] || ""; const col = TIER_COLORS[g.class] || "";
return `<span class="${cls}"${dd} style="--c:${col}">${cRain(g.value)}</span>`; const amt = g.class && g.class !== "dry" ? fmtPrecipTier(g.value, g.class, false) : cRain(g.value);
return `<span class="${cls}"${dd} style="--c:${col}">${amt}</span>`;
} }
// Compact "graded days" table — a ROW per metric, a COLUMN per day (newest first). // Compact "graded days" table — a ROW per metric, a COLUMN per day (newest first).

View file

@ -7,7 +7,7 @@ import { uv } from "./account.js"; // header sign-in entry + notification bell
import { TTL, chunkedFetch, prefetchViews } from "./cache.js"; import { TTL, chunkedFetch, prefetchViews } from "./cache.js";
import { initFindButton, setFindLabel } from "./mappicker.js"; import { initFindButton, setFindLabel } from "./mappicker.js";
import { TIER_COLORS, PRECIP_COLORS, SCALE_TEMP, SCALE_RAIN, drynessColor, pctOrd, import { TIER_COLORS, PRECIP_COLORS, SCALE_TEMP, SCALE_RAIN, drynessColor, pctOrd,
placeLabel, fmtPrecip, fmtWind, fmtHumid, MONTHS, isoOfDate, monthStart, monthEnd, placeLabel, fmtPrecip, fmtPrecipTier, fmtWind, fmtHumid, MONTHS, isoOfDate, monthStart, monthEnd,
CHUNK_MONTHS, buildChunks, clickOpensPicker, metricBuckets, distStrip, CHUNK_MONTHS, buildChunks, clickOpensPicker, metricBuckets, distStrip,
seasonFilterDropdown, seasonSummaryText, syncSeasonChecks, seasonFilterDropdown, seasonSummaryText, syncSeasonChecks,
applySeasonMonthChange, initSeasonExpand, allMonths, monthsToMask, maskToMonths, applySeasonMonthChange, initSeasonExpand, allMonths, monthsToMask, maskToMonths,
@ -42,7 +42,7 @@ const SKY_WORDS = [
// The shared weather summary, adapted to the calendar's compact day records // The shared weather summary, adapted to the calendar's compact day records
// (dsr included, so a long-dry no-rain day reads "dry" rather than "clear"). // (dsr included, so a long-dry no-rain day reads "dry" rather than "clear").
const weatherType = (rec) => const weatherType = (rec) =>
wxType(rec.tmax && rec.tmax.v, rec.precip && rec.precip.v, rec.dsr); wxType(rec.tmax && rec.tmax.v, rec.precip && rec.precip.v, rec.dsr, rec.precip && rec.precip.c);
let selected = null; // {lat, lon} let selected = null; // {lat, lon}
let data = null; // last /api/v2/calendar response let data = null; // last /api/v2/calendar response
@ -704,7 +704,7 @@ function attachHover(byDate) {
["humid", line("humid", "Humid", rec.humid, fmtHumid, tempColor(rec.humid))], ["humid", line("humid", "Humid", rec.humid, fmtHumid, tempColor(rec.humid))],
["wind", line("wind", "Wind", rec.wind, fmtWind, tempColor(rec.wind))], ["wind", line("wind", "Wind", rec.wind, fmtWind, tempColor(rec.wind))],
["gust", line("gust", "Gust", rec.gust, fmtWind, tempColor(rec.gust))], ["gust", line("gust", "Gust", rec.gust, fmtWind, tempColor(rec.gust))],
["precip", line("precip", "Precip", rec.precip, fmtPrecip, precipColor)], ["precip", line("precip", "Precip", rec.precip, (v) => fmtPrecipTier(v, rec.precip && rec.precip.c), precipColor)],
]; ];
if (dsrStr) { if (dsrStr) {
const rc = metric === "dsr" ? " tt-r-active" : ""; const rc = metric === "dsr" ? " tt-r-active" : "";

View file

@ -64,7 +64,7 @@ const PCT_FALLBACK = {
p1: ["p1", "p10", "min", "p50"], p10: ["p10", "p50"], p1: ["p1", "p10", "min", "p50"], p10: ["p10", "p50"],
p25: ["p25", "p30", "p50"], p40: ["p40", "p30", "p50"], p25: ["p25", "p30", "p50"], p40: ["p40", "p30", "p50"],
p60: ["p60", "p70", "p50"], p75: ["p75", "p70", "p50"], p60: ["p60", "p70", "p50"], p75: ["p75", "p70", "p50"],
p90: ["p90", "p50"], p99: ["p99", "p90", "max", "p50"], p90: ["p90", "p50"], p95: ["p95", "p90", "p50"], p99: ["p99", "p90", "max", "p50"],
}; };
const pget = (o, k) => { const pget = (o, k) => {
for (const kk of (PCT_FALLBACK[k] || [k])) if (o && o[kk] != null) return o[kk]; for (const kk of (PCT_FALLBACK[k] || [k])) if (o && o[kk] != null) return o[kk];
@ -93,8 +93,12 @@ const TEMP_FAN = [
["p90", "p99", "very-hot"], ["p75", "p90", "hot"], ["p60", "p75", "warm"], ["p90", "p99", "very-hot"], ["p75", "p90", "hot"], ["p60", "p75", "warm"],
["p40", "p60", "normal"], ["p25", "p40", "cool"], ["p10", "p25", "cold"], ["p1", "p10", "very-cold"], ["p40", "p60", "normal"], ["p25", "p40", "cool"], ["p10", "p25", "cold"], ["p1", "p10", "very-cold"],
]; ];
// Eight rain-intensity tiers (post-split): Trace / Light / Brisk / Typical / Heavy
// (60-90) / Very Heavy (90-95) / Severe (95-99) / Extreme. Each band maps a rain-day
// percentile range to its calendar tier colour; the p75 vertex inside the single
// Heavy tier keeps the fan following the p75 contour.
const RAIN_FAN = [ const RAIN_FAN = [
["p90", "p99", "wet-8"], ["p75", "p90", "wet-7"], ["p60", "p75", "wet-6"], ["p95", "p99", "wet-8"], ["p90", "p95", "wet-7"], ["p75", "p90", "wet-6"], ["p60", "p75", "wet-6"],
["p40", "p60", "wet-5"], ["p25", "p40", "wet-4"], ["p10", "p25", "wet-3"], ["p1", "p10", "wet-2"], ["p40", "p60", "wet-5"], ["p25", "p40", "wet-4"], ["p10", "p25", "wet-3"], ["p1", "p10", "wet-2"],
]; ];
@ -212,7 +216,7 @@ export function precipChart(days, n, xFor, C) {
getPct: (d) => (d.precip ? d.precip.percentile : null), getPct: (d) => (d.precip ? d.precip.percentile : null),
// Rain days take their intensity-tier color; no-rain days warm with the dry // Rain days take their intensity-tier color; no-rain days warm with the dry
// streak (tan→red) so a dry spell reads as dry, matching the Dry chart. // streak (tan→red) so a dry spell reads as dry, matching the Dry chart.
dotColor: (d) => (d.precip && d.precip.value > 0 ? (TIER_COLORS[d.precip.class] || C.precip) : (drynessColor(d.dsr) || C.precip)), dotColor: (d) => (d.precip && d.precip.class && d.precip.class !== "dry" ? (TIER_COLORS[d.precip.class] || C.precip) : (drynessColor(d.dsr) || C.precip)),
fmtAxis: (v) => fmtPrecip(v, false), fmtAxis: (v) => fmtPrecip(v, false),
fmtLabel: (v) => fmtPrecip(v, false), fmtLabel: (v) => fmtPrecip(v, false),
labelOn: (v) => v > 0, // only label rain days; dry days rest on the baseline labelOn: (v) => v > 0, // only label rain days; dry days rest on the baseline

View file

@ -6,8 +6,8 @@ import { fmtTemp, onUnitChange } from "./units.js";
import { uv } from "./account.js"; // header sign-in entry + notification bell, and the API-version resolver import { uv } from "./account.js"; // header sign-in entry + notification bell, and the API-version resolver
import { getJSON, TTL, prefetchViews } from "./cache.js"; import { getJSON, TTL, prefetchViews } from "./cache.js";
import { initFindButton, setFindLabel } from "./mappicker.js"; import { initFindButton, setFindLabel } from "./mappicker.js";
import { TIER_COLORS, PRECIP_COLORS, DRY_COLOR, pctOrd, fmtPrecip, fmtWind, fmtHumid, import { TIER_COLORS, PRECIP_COLORS, DRY_COLOR, pctOrd, fmtPrecip, fmtPrecipTier, fmtWind, fmtHumid,
todayISO, weatherType, placeLabel } from "./shared.js"; todayISO, isoOfDate, weatherType, placeLabel } from "./shared.js";
// Color a tier the same way the calendar cell would be colored for it. // Color a tier the same way the calendar cell would be colored for it.
const tierColor = (c) => (c === "dry" ? DRY_COLOR : TIER_COLORS[c] || PRECIP_COLORS[c] || ""); const tierColor = (c) => (c === "dry" ? DRY_COLOR : TIER_COLORS[c] || PRECIP_COLORS[c] || "");
@ -44,7 +44,9 @@ function stepDay(delta) {
if (!curDate) return; if (!curDate) return;
const d = new Date(curDate + "T00:00:00"); const d = new Date(curDate + "T00:00:00");
d.setDate(d.getDate() + delta); d.setDate(d.getDate() + delta);
const iso = d.toISOString().slice(0, 10); // Format in LOCAL time — curDate was parsed as local midnight, so toISOString()
// (UTC) would shift the result a day for UTC± viewers and desync the today guard.
const iso = isoOfDate(d);
if (iso > todayISO()) return; // don't step past today if (iso > todayISO()) return; // don't step past today
curDate = iso; curDate = iso;
fetchDay(); fetchDay();
@ -115,7 +117,8 @@ function render(data) {
// Weather summary from the observed high + precip (omitted for days with no obs yet). // Weather summary from the observed high + precip (omitted for days with no obs yet).
const th = d.metrics.tmax.obs ? d.metrics.tmax.obs.value : null; const th = d.metrics.tmax.obs ? d.metrics.tmax.obs.value : null;
const pr = d.metrics.precip.obs ? d.metrics.precip.obs.value : null; const pr = d.metrics.precip.obs ? d.metrics.precip.obs.value : null;
const wt = th != null || pr != null ? weatherType(th, pr) : null; const prCls = d.metrics.precip.obs ? d.metrics.precip.obs.class : null;
const wt = th != null || pr != null ? weatherType(th, pr, undefined, prCls) : null;
dayHead.innerHTML = ` dayHead.innerHTML = `
<h2>${nice}</h2> <h2>${nice}</h2>
${wt ? `<p class="day-weather">${wt.icon} ${wt.text}</p>` : ""} ${wt ? `<p class="day-weather">${wt.icon} ${wt.text}</p>` : ""}
@ -155,11 +158,14 @@ function ladderCard(title, m, fmt, kind) {
if (!m || !m.ladder) return ""; if (!m || !m.ladder) return "";
const obs = m.obs; const obs = m.obs;
const activeClass = obs ? obs.class : null; const activeClass = obs ? obs.class : null;
// The observed precip depth reads "trace" for a sub-0.01" rain day (rounds to 0
// but is graded a rain tier); ladder tier ranges keep the plain numeric formatter.
const obsFmt = kind === "precip" ? ((v) => fmtPrecipTier(v, activeClass)) : fmt;
let summary; let summary;
if (obs) { if (obs) {
const pct = obs.percentile == null ? "" : ` · ${pctOrd(obs.percentile)} pct`; const pct = obs.percentile == null ? "" : ` · ${pctOrd(obs.percentile)} pct`;
summary = `<span class="day-obs" style="--cat:${tierColor(obs.class)}">${fmt(obs.value)}</span> summary = `<span class="day-obs" style="--cat:${tierColor(obs.class)}">${obsFmt(obs.value)}</span>
<span class="day-obs-meta">${obs.grade}${pct}</span>`; <span class="day-obs-meta">${obs.grade}${pct}</span>`;
} else { } else {
summary = `<span class="day-obs-meta">No observation for this day yet</span>`; summary = `<span class="day-obs-meta">No observation for this day yet</span>`;
@ -182,7 +188,7 @@ function ladderCard(title, m, fmt, kind) {
else if (t.lo == null) val = `< ${fmt(t.hi)}`; // bottom tier: strictly below p1 else if (t.lo == null) val = `< ${fmt(t.hi)}`; // bottom tier: strictly below p1
else val = fmtRange(fmt, t.lo, t.hi); else val = fmtRange(fmt, t.lo, t.hi);
const marker = t.c === activeClass && obs const marker = t.c === activeClass && obs
? `<span class="ladder-mark">◀ ${bareVal(fmt, obs.value)}</span>` : ""; ? `<span class="ladder-mark">◀ ${bareVal(obsFmt, obs.value)}</span>` : "";
return `<div class="ladder-row${active}"> return `<div class="ladder-row${active}">
<span class="ladder-sw" style="background:${tierColor(t.c)}"></span> <span class="ladder-sw" style="background:${tierColor(t.c)}"></span>
<span class="ladder-label" style="--cat:${tierColor(t.c)}">${t.label}</span> <span class="ladder-label" style="--cat:${tierColor(t.c)}">${t.label}</span>

View file

@ -36,6 +36,21 @@ async function registration() {
return navigator.serviceWorker.ready; return navigator.serviceWorker.ready;
} }
// Does an existing subscription's baked-in applicationServerKey still match the
// server's current VAPID public key? After a key rotation it won't: the browser
// keeps signing pushes the server can no longer authenticate, so we must replace
// the subscription. `options.applicationServerKey` is an ArrayBuffer of the raw
// key bytes (or null on browsers that don't expose it — treat that as a mismatch
// and re-subscribe rather than leaving a possibly-stale subscription in place).
function applicationServerKeyMatches(sub, serverKey) {
const current = sub.options && sub.options.applicationServerKey;
if (!current) return false;
const a = new Uint8Array(current);
if (a.length !== serverKey.length) return false;
for (let i = 0; i < a.length; i++) if (a[i] !== serverKey[i]) return false;
return true;
}
// Is this device currently subscribed? (a PushSubscription exists locally) // Is this device currently subscribed? (a PushSubscription exists locally)
export async function isEnabled() { export async function isEnabled() {
if (!supported()) return false; if (!supported()) return false;
@ -60,13 +75,24 @@ export async function enable() {
const reg = await registration(); const reg = await registration();
let sub = await reg.pushManager.getSubscription(); let sub = await reg.pushManager.getSubscription();
// Always resolve the server's current VAPID key: it's needed to subscribe, and
// — when a subscription already exists — to detect a key rotation. A subscription
// minted under an old key silently stops receiving pushes, so on a mismatch we
// drop it and re-subscribe with the new key. Matching key = happy path, untouched.
const keyRes = await apiFetch(uv("push/vapid-key"));
if (!keyRes.ok) throw new Error("Couldn't fetch the server key.");
const { key } = await keyRes.json();
const serverKey = urlB64ToUint8Array(key);
if (sub && !applicationServerKeyMatches(sub, serverKey)) {
await sub.unsubscribe();
sub = null;
}
if (!sub) { if (!sub) {
const res = await apiFetch(uv("push/vapid-key"));
if (!res.ok) throw new Error("Couldn't fetch the server key.");
const { key } = await res.json();
sub = await reg.pushManager.subscribe({ sub = await reg.pushManager.subscribe({
userVisibleOnly: true, userVisibleOnly: true,
applicationServerKey: urlB64ToUint8Array(key), applicationServerKey: serverKey,
}); });
} }
const res = await apiFetch(uv("push/subscribe"), { method: "POST", json: sub.toJSON() }); const res = await apiFetch(uv("push/subscribe"), { method: "POST", json: sub.toJSON() });

View file

@ -257,7 +257,10 @@ export const pctOrd = (n) => {
if (!Number.isFinite(v)) return "—"; if (!Number.isFinite(v)) return "—";
return ord(Math.min(99, Math.max(1, Math.round(v)))); return ord(Math.min(99, Math.max(1, Math.round(v))));
}; };
export const todayISO = () => new Date().toISOString().slice(0, 10); // Today in the viewer's LOCAL zone (see isoOfDate below). A UTC date rolls a day
// early/late for UTC± viewers past local midnight, putting "today" and the
// date-picker max out of reach of the day they're actually living in.
export const todayISO = () => isoOfDate(new Date());
export const esc = (s) => String(s).replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;"); export const esc = (s) => String(s).replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
// The heading label for a graded response: the resolved place name, or the // The heading label for a graded response: the resolved place name, or the
// cell-center coordinates while (or if) no name resolves. // cell-center coordinates while (or if) no name resolves.
@ -464,13 +467,28 @@ export const WX_ICONS = {
snow: WX(`<path d="M20 17.6A5 5 0 0 0 18 8h-1.26A8 8 0 1 0 4 16.25"/><line x1="8" y1="18" x2="8" y2="18"/><line x1="12" y1="20" x2="12" y2="20"/><line x1="16" y1="18" x2="16" y2="18"/><line x1="12" y1="16" x2="12" y2="16"/>`), snow: WX(`<path d="M20 17.6A5 5 0 0 0 18 8h-1.26A8 8 0 1 0 4 16.25"/><line x1="8" y1="18" x2="8" y2="18"/><line x1="12" y1="20" x2="12" y2="20"/><line x1="16" y1="18" x2="16" y2="18"/><line x1="12" y1="16" x2="12" y2="16"/>`),
}; };
// A graded precip depth for display, given its grade class. Reanalysis "trace"
// rain — any measurable precip below 0.01" — is graded as a rain tier but rounds to
// 0.00" / 0 mm, so its depth prints as "trace" rather than a bone-dry "0.00" that
// would contradict the rain grade (and the now-reset dry streak). Genuinely dry
// days (class "dry") and real, roundable depths format as usual.
export function fmtPrecipTier(v, cls, withUnit = true) {
if (cls && cls !== "dry" && v != null && parseFloat(fmtPrecip(v, false)) === 0)
return "trace";
return fmtPrecip(v, withUnit);
}
// Plain-language "what was the day like" descriptor from the raw values: a // Plain-language "what was the day like" descriptor from the raw values: a
// temperature word (from the daily high, °F) crossed with a sky/precip word // temperature word (from the daily high, °F) crossed with a sky/precip word
// (precip in inches — falling as snow at/below freezing). `dsr` is optional: // (precip in inches — falling as snow at/below freezing). `dsr` is optional:
// when a long dry streak is known, a no-rain day reads "dry" instead of // when a long dry streak is known, a no-rain day reads "dry" instead of
// "clear" (the calendar passes it; the day page doesn't track streaks). // "clear" (the calendar passes it; the day page doesn't track streaks).
export function weatherType(t, p, dsr) { // `precipCls` is the precip grade class when known: a rain tier means it rained
const wet = p != null && p >= 0.01; // even when the rounded depth reads 0 (trace), so it decides wet/dry ahead of the
// depth — matching the dry/rain grading split (precip > 0). Without it, any
// positive depth counts as wet.
export function weatherType(t, p, dsr, precipCls) {
const wet = precipCls != null ? precipCls !== "dry" : (p != null && p > 0);
const freezing = t != null && t <= 34; const freezing = t != null && t <= 34;
const temp = t == null ? "" : const temp = t == null ? "" :
t >= 95 ? "Scorching" : t >= 85 ? "Hot" : t >= 72 ? "Warm" : t >= 95 ? "Scorching" : t >= 85 ? "Hot" : t >= 72 ? "Warm" :

View file

@ -29,6 +29,51 @@ def test_sitemap_lists_city_urls(client):
assert f"/climate/{SLUG}/records</loc>" in r.text assert f"/climate/{SLUG}/records</loc>" in r.text
# The public site is HTTPS-only behind Caddy, which terminates TLS and proxies
# plain HTTP -- so x-forwarded-proto / request.url.scheme read "http". A request
# arriving on the configured public host (THERMOGRAPH_BASE_URL defaults to
# https://thermograph.org in the test env) must still emit https:// in every
# frontend-built absolute URL, or canonicals self-conflict and the sitemap lists
# redirecting URLs.
_PUBLIC = {"host": "thermograph.org", "x-forwarded-proto": "http"}
def test_public_host_canonical_and_og_are_https(client):
b = client.get(f"{B}/climate/{SLUG}", headers=_PUBLIC).text
assert f'<link rel="canonical" href="https://thermograph.org{B}/climate/{SLUG}"' in b
assert f'<meta property="og:url" content="https://thermograph.org{B}/climate/{SLUG}"' in b
assert 'property="og:image" content="https://thermograph.org' in b
def test_public_host_sitemap_locs_are_https(client):
body = client.get(f"{B}/sitemap.xml", headers=_PUBLIC).text
assert f"<loc>https://thermograph.org{B}/climate/{SLUG}</loc>" in body
assert "<loc>http://thermograph.org" not in body
def test_public_host_robots_sitemap_is_https(client):
body = client.get(f"{B}/robots.txt", headers=_PUBLIC).text
assert f"Sitemap: https://thermograph.org{B}/sitemap.xml" in body
def test_non_public_host_keeps_forwarded_scheme(client):
# A LAN/dev host never matches the configured public host, so the observed
# (plain-http) scheme is preserved -- http development is unaffected.
b = client.get(f"{B}/climate/{SLUG}",
headers={"host": "192.168.1.10:8000", "x-forwarded-proto": "http"}).text
assert f'<link rel="canonical" href="http://192.168.1.10:8000{B}/climate/{SLUG}"' in b
def test_proxied_forwarded_host_still_https(client):
# Reached via backend's proxy fallback: Host is the internal hop, the real
# browser host arrives in X-Forwarded-Host -- which still resolves to the
# public host and must go https.
b = client.get(f"{B}/climate/{SLUG}",
headers={"host": "frontend:8080", "x-forwarded-host": "thermograph.org",
"x-forwarded-proto": "http"}).text
assert f'<link rel="canonical" href="https://thermograph.org{B}/climate/{SLUG}"' in b
def test_city_page_renders_structure(client): def test_city_page_renders_structure(client):
r = client.get(f"{B}/climate/{SLUG}") r = client.get(f"{B}/climate/{SLUG}")
assert r.status_code == 200 assert r.status_code == 200

View file

@ -34,8 +34,18 @@ Gather the key names over SSH (never the values), then audit. Hosts/keys per INF
K=~/.ssh/thermograph_agent_ed25519 K=~/.ssh/thermograph_agent_ed25519
# sudo: /etc/thermograph.env is root-owned (0640). On prod `agent` can read it # sudo: /etc/thermograph.env is root-owned (0640). On prod `agent` can read it
# directly too, but sudo works uniformly on both boxes. # directly too, but sudo works uniformly on both boxes.
ssh -i $K agent@169.58.46.181 'sudo grep -oE "^[A-Z_]+=" /etc/thermograph.env' > /tmp/prod.keys # prod #
ssh -i $K agent@75.119.132.91 'sudo grep -oE "^[A-Z_]+=" /etc/thermograph.env' > /tmp/beta.keys # beta # The character class must allow DIGITS. This was `^[A-Z_]+=` until 2026-07-24,
# which silently skipped every key whose name contains a digit — in this estate
# that is all six *_S3_* keys. Prod audited as 26 keys against 32 real ones, and
# the audit reported the S3 credentials as missing from both hosts. They were
# present the whole time, and the phantom was independently "confirmed" twice by
# people re-running the same pattern. An audit that under-reports is worse than
# no audit: the invented finding sends someone to provision a credential that
# already exists, and it was briefly recorded as a root cause of a real bug.
# The match still stops at the `=`, so no value is ever read.
ssh -i $K agent@169.58.46.181 'sudo grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph.env' > /tmp/prod.keys # prod
ssh -i $K agent@75.119.132.91 'sudo grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph.env' > /tmp/beta.keys # beta
python3 .claude/skills/key-gaps/key_gaps.py prod=/tmp/prod.keys beta=/tmp/beta.keys python3 .claude/skills/key-gaps/key_gaps.py prod=/tmp/prod.keys beta=/tmp/beta.keys
``` ```

View file

@ -17,7 +17,7 @@ It reads only KEY NAMES, never values — safe to run anywhere. Sources per envi
* a SOPS-encrypted YAML (deploy/secrets/<env>.yaml) keys are plaintext even when * a SOPS-encrypted YAML (deploy/secrets/<env>.yaml) keys are plaintext even when
encrypted, so no age key or decryption is needed; or encrypted, so no age key or decryption is needed; or
* a plain key-list file (one KEY per line, or KEY=... lines) e.g. the output of * a plain key-list file (one KEY per line, or KEY=... lines) e.g. the output of
`ssh <box> 'grep -oE "^[A-Z_]+=" /etc/thermograph.env'` for a live audit. `ssh <box> 'grep -oE "^[A-Z][A-Z0-9_]*=" /etc/thermograph.env'` for a live audit.
Usage: Usage:
key_gaps.py prod=deploy/secrets/prod.yaml beta=deploy/secrets/beta.yaml key_gaps.py prod=deploy/secrets/prod.yaml beta=deploy/secrets/beta.yaml