* Centralize filesystem paths in a single module
Add paths.py, which resolves the repo root once and derives the cache,
accounts DB, logs, templates, frontend and bundled-city-data locations
from it. Replace the 13 per-module `dirname(__file__)/..` anchors with
references to it, so a module's location no longer determines where the
app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are
unchanged; every resolved path is byte-identical to before.
Groundwork for moving modules into packages without re-pointing paths.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
* Split the backend into domain packages
Group the flat backend modules into packages that mirror their concerns:
data/ climate, grading, scoring, grid, places, cities,
city_events, store
web/ app, views, homepage, content, schemas
notifications/ notify, digest, push, mailer, discord,
discord_interactions, discord_link
accounts/ models, users, api_accounts, db
core/ metrics, singleton, audit
Intra-project imports are rewritten to the package-qualified form. The
entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor)
and paths.py stay at the backend/ root, and backend/app.py becomes a
shim re-exporting web.app:app so the launch target stays `app:app` —
run.sh, the systemd units, and CI need no change.
Verified: full suite (318) passes, `uvicorn app:app` boots and serves
the home/SEO/static/API surfaces, and every root script imports clean.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
- Fix the mobile width blow-out: the distribution grid item (.cmp-dist-row) had no
min-width:0, so each row expanded to the 9–10-column connected bar chart's
min-content and widened the whole page (zoom-out, clipped filter sheet). Add
grid-template-columns: minmax(0,1fr) + min-width:0 so the strip scrolls inside its
own .ct-strip, and tighten .ct-col to 34px on phones.
- Add a Load/Refresh button inside the compare filter sheet's date-range block, wired
to the same refresh()/isDirty(); editing the range on mobile no longer needs the
sheet closed. The existing button stays in the controls (loads added places).
- Default date range on both pages is now January six years back → the present. On
the calendar this is an explicit chunked range (the months=24 path is server-capped
at ~2yr).
- Names now include country: reverse_geocode appends it to the place label; revgeo
cache key gets a v2 prefix and PAYLOAD_VER bumps to p2 so labels/payloads
repopulate.
- Location names read as a hierarchy: compact chips show just the lead segment (full
name in the title/aria), and each ranked card shows the lead segment bold over a
muted "city · region · country" line.
Frontend + a small backend name/cache change; no schema migration.
Compare was the only page loading several cells at once, so its concurrent
reverse-geocode calls burst past Nominatim's ~1 req/sec limit, got rate-limited,
and cached a null label — leaving those locations stuck on bare coordinates.
- climate.py: serialize + rate-limit reverse_geocode behind a lock (>=1.1s between
Nominatim calls, re-checking the cache under the lock), so concurrent callers
resolve reliably instead of bursting.
- store.py: shorten the reverse-geocode miss TTL 1 day -> 1 hour so a transient
null retries soon; add a `cache` flag to put_payload.
- app.py: don't persist a calendar payload whose place failed to resolve, so the
coordinates fallback can't stick for the life of the token.
Weekly showed the place name twice (top label beside the Find button AND the
results <h2>). Keep the <h2> (it carries the coords + climatology context) as the
single name display; the top label now only holds the transient coordinates
written on selection and is hidden once the named results render.
Claude-Session: https://claude.ai/code/session_01B3Q2EkHHnTUX2BfhV5q1Zc
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Compress API responses and revalidate static assets instead of re-downloading
- Add GZipMiddleware (min 1 KB): the 2-year calendar JSON shrinks ~6-8x.
- Serve pages/assets with Cache-Control: no-cache instead of no-store, so
browsers revalidate via the ETag/Last-Modified that FileResponse and
StaticFiles already emit. Unchanged assets now cost an empty 304 rather
than a full transfer on every page navigation, while deploys still show
up immediately.
* Persist derived responses in SQLite so grading is computed once per cell, not per request
New data/thermograph.sqlite (WAL) holds what's derived from the raw parquet
records — finished grade/calendar/day/forecast payloads and reverse-geocode
labels — so the expensive work (notably the 2-year calendar grade_range, ~270ms)
becomes a ~5ms database read that survives restarts and is shared across views.
Raw parquet stays the source of truth; the store is a pure accelerator (every
reader falls back to recomputing on a miss, and deleting the db is a safe reset).
Freshness is token-driven, not clock-driven: each cached payload is validated by
a token encoding what it was computed from (payload schema version, the archive
record's end date, the recent-fetch stamp). The existing freshness drivers are
untouched — get_history still tops up the tail hourly and get_recent_forecast
still refetches hourly — and tokens are derived from what they return, so cached
payloads expire exactly when their inputs change. The same tokens double as weak
ETags: If-None-Match answers with an empty 304 without touching the payload.
- backend/store.py: derived-payload + revgeo tables, thread-local WAL conns,
every helper fail-soft.
- app.py: endpoints split into pure payload builders + HTTP/caching shells; the
in-memory 10-minute _CAL_CACHE is retired (superseded by the persistent store).
- climate.py: revgeo persisted through the store; recent_stamp() and
load_cached_history() (no-network read) helpers.
- backend/migrate.py + make migrate: idempotent, resumable backfill of the store
from existing parquet caches (default calendar span + latest-day detail +
revgeo, ≤1 throttled Nominatim call per unlabeled cell). Never fetches weather.
- grid.from_id(): rebuild a cell from its cache filename (migrate tooling).
* Add /api/v2/cell: one bundle carrying every view's payload
GET /api/v2/cell?lat&lon returns the grade, forecast, calendar (last 24 months)
and day (today) payloads in one response, each the exact payload its per-view
endpoint returns — built by the same builders and cached under the same
derived-store keys/tokens — paired with the etag that endpoint would emit. The
frontend can warm all views with a single request, seed its per-view cache from
the slices, and later revalidate each view individually with If-None-Match. The
bundle's own etag combines the slices', so an unchanged bundle is an empty 304.
prefetch=1 is a warm-only mode for neighbor-cell prefetching with a hard
guarantee: it never spends weather-API quota. A cell with no cached archive
answers 204, and only the history-derived slices (calendar + latest-day detail)
are built. At most one Nominatim lookup for a never-labeled cell.
* Frontend: IndexedDB response cache with stale-while-revalidate + bundle prefetch
The per-URL response cache moves from localStorage/sessionStorage (~5 MB quota,
which multi-year calendar payloads regularly blew through) to IndexedDB, with an
in-memory map in front. Entries carry the server's ETag, so anything stale
revalidates conditionally — unchanged data costs an empty 304 and a re-stamp,
never a re-transfer. On network failure the stale copy is served over an error.
getJSON gains an optional onUpdate callback opting into stale-while-revalidate:
the three views (weekly, day, calendar) now render a cached copy immediately —
spinners are delayed 150ms so warm loads never flash-blank — and repaint only if
background revalidation finds changed data. New data shows up the moment it
exists instead of waiting out a TTL.
Cross-view prefetch collapses from one request per view to a single /api/v2/cell
bundle, whose slices (exact per-view payloads + their etags) are seeded under the
URLs each view actually requests; the bundle call itself is conditional via a
remembered etag. Afterwards the 8 surrounding grid cells are warmed server-side
with prefetch=1 (never spends weather-API quota; staggered ≥1.1s for the one
possible Nominatim lookup each), so tapping nearby lands on already-graded data.
Legacy tg:* storage entries are cleared once; cache entries untouched for two
weeks are pruned on page load.
Emi Griffith
