Move the flat backend/tests/*.py into domain subfolders so the suite
mirrors the code's concerns:
data/ climate, grading, scoring, grid, places, store
web/ api, content, homepage, views
notifications/ notify, digest, discord (+ dm/interactions/link)
accounts/ api_accounts
core/ metrics, singleton, dashboard
conftest.py stays at the tests/ root, so its sys.path setup and shared
fixtures still apply to every subfolder. The four tests that derive repo
paths from __file__ get their depth bumped one level to match their new
location. Pytest discovers the subfolders recursively; the CI command
(python -m pytest backend/tests) is unchanged.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
Score how far a location's last 6 years have drifted from its full 45-year
record. For each metric and percentile category (p10/p25/p50/p75/p90), the
recent-years value is placed on the baseline distribution and the gap from the
expected percentile is the divergence — unit-free, so metrics compare directly.
Scored per meteorological season plus annual, weighted into per-metric and
overall scores (temps, humidity and feels-like weighted heaviest).
- backend/scoring.py: divergence math, seasonal slicing, precip zero-inflation
split (wet-day frequency + amount), tier mapping onto the existing temp scale.
- climate.py: derive a wet-bulb column (Stull 2011) at the read boundary, before
the humidity column is converted to absolute — via a shared _derive_metrics
wrapper at all four read sites.
- api/v2/score endpoint + build_score payload, cached on the history token with
a scoring-version key.
- frontend score page: overall hero, per-metric cards, by-season chips, and a
button-revealed summary (sentences + metrics×season table + per-percentile
detail). Score nav link across all headers.
- Tests for the scoring math, wet-bulb formula, payload shape and route.
Complete the forecast fallback chain: Open-Meteo → MET Norway → stale cache. When
both live sources are unavailable, serve the last cached recent/forecast bundle
even if it is past its 1-hour TTL — an hours-old bundle (which still carries the
recent observed days MET Norway lacks) beats a hard 503, mirroring the archive
path's stale-serve in _load_history.
The stale bundle is returned WITHOUT rewriting it, so its mtime stays old and the
next request retries the live sources first rather than treating the stale copy as
fresh. Only when there is no cache at all does the typed WeatherUnavailable surface.
The forecast path had no backup — unlike history, which falls back to NASA POWER.
When Open-Meteo's forecast API was rate-limited or down, the recent/forecast
bundle (and every endpoint that grades future days) failed with a 503.
Add MET Norway (yr.no) Locationforecast as a keyless, global forecast backup,
mirroring the NASA POWER role for history:
- _metno_to_frame: aggregates MET Norway's sub-daily timeseries into the daily
schema, converting units (°C→°F, mm→in, m/s→mph) and deriving feels-like from
the NWS heat index / wind chill (MET has no gusts or apparent temperature).
Precip prefers the 1-hour block and falls back to the 6-hour block so the
hourly→6-hourly resolution switch never double-counts.
- _fetch_forecast_metno: the backup fetch, with the ToS-required identifying
User-Agent and coordinates rounded to 4 decimals.
- _load_recent_forecast: on any Open-Meteo forecast failure, try MET Norway
before surfacing the error; a shared rate limit still raises the typed,
daily-aware WeatherUnavailable.
MET Norway is forecast-only (no recent past days), so it's a degraded-but-working
fallback: the forecast / day-ahead views keep serving during an Open-Meteo outage.
Tests cover the daily aggregation + unit conversion (incl. the no-double-count
precip rule) and the fallback wiring.
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
climate.py repeated the same four mechanical patterns:
- the Open-Meteo daily params dict (3x) -> _om_daily_params(cell, **window)
- the doy attach line (6x) -> _with_doy
- makedirs + drop-doy + zstd to_parquet (3x) -> _write_cache
- the identical _to_frame/_nasa_to_frame tail (feels-like, valid-day
filter, doy) -> _finalize_frame
grading.py encoded the tier tables twice — TEMP_BANDS/RAIN_BANDS plus the
hand-aligned _TEMP_LADDER/_RAIN_LADDER ('kept aligned' by comment). The
ladders are now derived from the bands (_ladder_from; verified
byte-identical to the old tables before landing), so tier boundaries have
exactly one definition. grade_range's inline dry-streak walk is replaced
with the existing dry_streaks(); its per-(doy,var) sample cache now also
memoizes the window mask per doy instead of recomputing it once per
metric (9x per day-of-year).
New tests pin the refactor: _to_frame schema/day-filter/missing-series
tolerance, the combined feels-like side selection, NASA unit conversions
and fill-sentinel handling, _om_daily_params windows, and _write_cache
stripping the derived doy column.
