* Centralize filesystem paths in a single module
Add paths.py, which resolves the repo root once and derives the cache,
accounts DB, logs, templates, frontend and bundled-city-data locations
from it. Replace the 13 per-module `dirname(__file__)/..` anchors with
references to it, so a module's location no longer determines where the
app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are
unchanged; every resolved path is byte-identical to before.
Groundwork for moving modules into packages without re-pointing paths.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
* Split the backend into domain packages
Group the flat backend modules into packages that mirror their concerns:
data/ climate, grading, scoring, grid, places, cities,
city_events, store
web/ app, views, homepage, content, schemas
notifications/ notify, digest, push, mailer, discord,
discord_interactions, discord_link
accounts/ models, users, api_accounts, db
core/ metrics, singleton, audit
Intra-project imports are rewritten to the package-qualified form. The
entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor)
and paths.py stay at the backend/ root, and backend/app.py becomes a
shim re-exporting web.app:app so the launch target stays `app:app` —
run.sh, the systemd units, and CI need no change.
Verified: full suite (318) passes, `uvicorn app:app` boots and serves
the home/SEO/static/API surfaces, and every root script imports clean.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
Backend (scoring.py):
- Fold the parallel WEIGHTS/METRIC_LABELS/DIRECTION_WORDS/TEMP_DIR_METRICS maps
into one METRICS descriptor so a scored metric is defined in one place.
- Share one _build_entry between the seasonal and annual paths.
- Add _freq_for so the annual entry reads the wet-day / heat-stress-day share
directly instead of re-running the full precip_divergence just for it.
- Drop unconsumed payload: per-q v6/pct and the overall bias.
- Drop _slice's dead 'annual' branch; derive SLICES from SEASONS.
Frontend (score.js):
- One scoreCell() and signedPts() helper for the four score-cell sites and the
two signed-points chips; reuse .section-title / .table-wrap instead of cloning.
Behavior-preserving (identical scores); bumps the score cache version.
Annual scores were computed from an all-year pooled distribution, which widens
the reference spread and hides a shift confined to one season — Seattle's daily
high read 16 despite a summer high of 65. Build each metric's annual score
(and the overall) as the mean of its four seasonal divergences instead, so a
real seasonal shift shows through (that high now reads 28). Frequency read-outs
(precip wet days, wet-bulb heat-stress days) stay pooled over the year.
Also lock the by-season table to fixed, uniform columns (min-width to scroll on
a phone) so each metric lines up vertically across the seasons, and show the
per-percentile detail as the season-averaged shift.
Bumps the score cache version.
- Explain on the page what wet-bulb temperature measures (the evaporative-cooling
ceiling on shedding heat), so the metric isn't opaque.
- Report the share of heat-stress "wet-bulb" days (peak wet bulb >= 26 C) vs
normal days, recent window vs the full record — mirroring the precip wet-day
frequency.
- Present the overall total as a direction-agnostic net change (magnitude only),
not "warmer/cooler"; per-metric cards still carry direction.
Bumps the score cache version.
Score how far a location's last 6 years have drifted from its full 45-year
record. For each metric and percentile category (p10/p25/p50/p75/p90), the
recent-years value is placed on the baseline distribution and the gap from the
expected percentile is the divergence — unit-free, so metrics compare directly.
Scored per meteorological season plus annual, weighted into per-metric and
overall scores (temps, humidity and feels-like weighted heaviest).
- backend/scoring.py: divergence math, seasonal slicing, precip zero-inflation
split (wet-day frequency + amount), tier mapping onto the existing temp scale.
- climate.py: derive a wet-bulb column (Stull 2011) at the read boundary, before
the humidity column is converted to absolute — via a shared _derive_metrics
wrapper at all four read sites.
- api/v2/score endpoint + build_score payload, cached on the history token with
a scoring-version key.
- frontend score page: overall hero, per-metric cards, by-season chips, and a
button-revealed summary (sentences + metrics×season table + per-percentile
detail). Score nav link across all headers.
- Tests for the scoring math, wet-bulb formula, payload shape and route.
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
- Fix the mobile width blow-out: the distribution grid item (.cmp-dist-row) had no
min-width:0, so each row expanded to the 9–10-column connected bar chart's
min-content and widened the whole page (zoom-out, clipped filter sheet). Add
grid-template-columns: minmax(0,1fr) + min-width:0 so the strip scrolls inside its
own .ct-strip, and tighten .ct-col to 34px on phones.
- Add a Load/Refresh button inside the compare filter sheet's date-range block, wired
to the same refresh()/isDirty(); editing the range on mobile no longer needs the
sheet closed. The existing button stays in the controls (loads added places).
- Default date range on both pages is now January six years back → the present. On
the calendar this is an explicit chunked range (the months=24 path is server-capped
at ~2yr).
- Names now include country: reverse_geocode appends it to the place label; revgeo
cache key gets a v2 prefix and PAYLOAD_VER bumps to p2 so labels/payloads
repopulate.
- Location names read as a hierarchy: compact chips show just the lead segment (full
name in the title/aria), and each ranked card shows the lead segment bold over a
muted "city · region · country" line.
Frontend + a small backend name/cache change; no schema migration.
The derived store's key/token formats existed in three places — each
endpoint, api_cell's slice assembly, and migrate.py — where any drift
would silently split the cache (endpoints missing rows the bundle wrote,
migrate materializing rows nobody reads). They are now defined once in
views.py (grade_key/calendar_key/day_key/forecast_key, history_token/
recent_token/day_token) and consumed everywhere, with a pinning test so
a format change is always deliberate.
The four data endpoints shared two copy-pasted sequences, now helpers:
- _fetch_history: history (+ optional recent bundle) fetch with upstream
failures mapped to clean HTTP errors and an empty record to 404.
- _cached_response: If-None-Match 304 / token-valid store replay /
build + persist + serve, with calendar's dont-persist-placeless rule
as an explicit flag.
Each endpoint is now its audit run + identity + a build callback (~10
lines); api_day's hourly-token special case moved into day_token. New
tests: identity format pins, rate-limit 503 parametrized across all five
data routes, prefetch=1 never touching upstream (cold 204, warm
history-only slices), and calendar's placeless-payload retry behavior.
app.py had grown three co-resident strata: HTTP/caching plumbing, payload
assembly, and search policy. This moves the payload layer — the four
build_* functions, cal_span clamping, hist_end, PAYLOAD_VER, NullRun and
their helpers — into a new views.py with no web dependencies (pure moves,
public names). app.py keeps routing, ETag/derived-store plumbing, and
page serving; migrate.py imports views instead of reaching into app's
privates.
That import previously constructed the whole FastAPI app and — because
places.start_loading() ran at import time — kicked off a background
GeoNames download from an offline batch script. The index load now hangs
off the app's lifespan hook, so it fires when a server starts, not when
the module is imported (tests, migrate).
New tests: cal_span clamping (months-back default, record bounds, 2-year
cap, inversion), builder shapes (grade window ordering, day obs from the
recent bundle, forecast future-only), a regression test that build_day
degrades to climatology when the recent fetch fails, and a subprocess
layering guard that importing views/migrate pulls in neither FastAPI nor
the app module and starts no index download.
