On prod (served at root) the dashboard first probes /thermograph/api/v2/metrics before
falling back to root; that 404 was classified as inbound 'other' and shown as an error —
the monitor polluting the traffic it displays. Two fixes: classify any */api/v2/metrics
path as 'metrics' (excluded) whatever prefix it arrives under; and cache the working
metrics URL in the dashboard so it stops re-probing the failing base every refresh.
Add a rolling short-window view alongside the since-start totals. metrics.py
keeps per-minute buckets (epoch-minute -> {key: count}) for inbound categories
and outbound sources, summed over the last 10 minutes on snapshot() and exposed
as inbound_recent / outbound_recent / window_minutes. Cheap and bounded — at most
WINDOW+1 tiny dicts, pruned as minutes roll off; self-polling still excluded.
The dashboard's TRAFFIC section now renders that count as a dim column next to
each row's total (header reads "total · last 10m"); idle sources show "-".
Dashboard ACCOUNTS section now shows only totals (users, active/total subscriptions,
notifications, push subs) — the per-user and per-subscription lists are gone, and the
dashboard no longer queries that PII at all.
Backend: add a per-request access log (logs/access/access-<date>.jsonl) recording the
client IP (real IP via the left-most X-Forwarded-For hop when proxied behind Caddy,
else the peer address), method, path, status, and category. Static assets and the
dashboard's own metrics polling are skipped. Best-effort, never raises into the request
path — retained for later traffic/IP analysis.
The ops dashboard polls the metrics endpoint every few seconds; counting those hits
just shows the monitor watching itself and inflates the inbound totals. Skip the
'metrics' category in record_inbound (so it's excluded from inbound_total too), and
hide it in the dashboard's inbound list defensively.
Adds observability for the running app, viewable over SSH (e.g. Termux):
- backend/metrics.py: thread-safe, best-effort in-process counters for inbound
requests (per category) and outbound calls (per external source), plus a
phase->source map and snapshot(). Hooked into climate._request (outbound, all
six weather/geocode sources) and the existing revalidate_static middleware
(inbound). Never raises into the request path.
- GET /api/v2/metrics: token-gated JSON of the counters + warm-queue depth +
thread names/uptime/pid. Returns 404 unless a valid THERMOGRAPH_METRICS_TOKEN
is presented or the request is direct loopback with no proxy X-Forwarded-*
header, so ops data is never exposed through Caddy on the public domain.
- scripts/dashboard.py (+ scripts/dash, make dashboard): stdlib-only terminal
dashboard. Reads cache (parquet + SQLite), accounts (users + active
subscriptions), and system resources (/proc + systemd) directly, and pulls
traffic from the metrics endpoint over loopback. Live refresh, --once, --json;
mobile-terminal width. Paths resolve relative to the script so the same tool
works on the LAN dev and prod VPS checkouts.
- Monitoring sections in DEPLOY.md / DEPLOY-DEV.md; unit tests for the counters.
Hook the production deploy (deploy/deploy.sh) to ping IndexNow after the health
check passes, so new/changed pages reach Bing/DuckDuckGo/Yandex without a manual
run. Best-effort — wrapped so it can never fail a deploy — and it sources
/etc/thermograph.env so its key matches the one the running service serves.
To avoid re-blasting ~14k URLs on every code push, indexnow.py gains a
--if-changed mode gated by a signature of the URL set (persisted to
data/indexnow_state.txt): code-only deploys skip; a deploy that adds or removes a
city submits. `make indexnow` still forces a full submit.
Faster search-engine indexing for the ~14k climate/city/record URLs:
- IndexNow (backend/indexnow.py): instantly notify Bing/DuckDuckGo/Yandex of new
or changed URLs. Per-host key resolved env → gitignored file → generated (mirrors
push.py), served at /{key}.txt, and echoed in submissions. `submit_all()` + a
`make indexnow` CLI push every indexable URL, batched under the 10k cap. Google
doesn't use IndexNow, so it stays on the sitemap.
- Sitemap: replace the per-request today() <lastmod> (which churns every fetch and
trains crawlers to ignore lastmod) with a stable content-build date; factor the
URL list into public_paths() shared with IndexNow so the two never drift.
- Search-console verification: render google-site-verification + msvalidate.01
<meta> tags from env into every page's <head> — the SEO pages via base.html.j2
and the static pages (incl. the homepage Google verifies) via _page().
- Docs: env example documents the three new vars; .gitignore ignores the key.
Tests: key-file route, stable single-date lastmod, IndexNow payload/batching, and
verification meta on both SEO and static pages.
The server-rendered SEO pages shared base.html.j2's nav but carried no JS, so
they lacked the °F/°C toggle and account/bell, and their temperatures were baked
as dual-unit strings that couldn't respond to a toggle.
- Load units.js + account.js + climate.js on base.html.j2, so every SEO page gets
the same header as the interactive pages (nav + °F/°C toggle + account + bell).
- Emit each temperature as <span class="temp" data-temp-f> (default °F text, real
for crawlers/no-JS); climate.js rewrites them to the active unit on load and on
toggle. Tint classes stay pinned to absolute °F. Drops the inline "(NN°C)".
- Make account.js base-aware: derive the app base from import.meta.url so its
api/v2 fetches and the alerts links resolve from deep /climate/{slug} URLs
instead of 404ing. Equivalent on the depth-1 interactive pages.
- Default the unit from the browser locale for first-time visitors (no stored
pref): en-US → °F, everyone else → °C. A stored choice always wins.
- Compact the mobile menu: narrower panel (208px), tighter rows, and the nav
list now stretches to the panel width and left-aligns (it was centering because
the base .view-nav align-self:center overrode the panel's stretch).
- Stylish separation between entries: an inset hairline between each nav item,
suppressed around the active pill so nothing crosses it.
- Rename the "Day" view to "Day Detail" across the app nav, and add it to the
climate/city page nav (base.html.j2), which was missing it.
On phones the header spanned two rows: the title block plus a control bar of
the °F/°C toggle, bell, account and hamburger. Wrap the nav in a .nav-panel
inside the <details> menu and inject the °F/°C toggle and account/bell into that
panel instead of into the header row. On desktop the panel is display:contents,
so the inline top bar renders exactly as before (order restored via flex order);
on phones the panel becomes a right-anchored, viewport-clamped dropdown and the
header collapses to a single row (logo + name + hamburger, tagline hidden).
The viewport clamp also fixes the SEO/city pages, whose menu dropdown previously
ran off-screen.
- Collapse the view navigation into a hamburger menu on phones. The nav is
wrapped in a <details> (pure CSS, no JS — so it works on the crawlable SEO
pages too): transparent (display:contents) on desktop so the tabs render
inline, a dropdown behind a hamburger under 640px. The °F/°C toggle, bell and
account stay in the header bar; units.js/account.js now anchor their injected
controls on the wrapper.
- Replace the header logo: the ▚ glyph was a rotated Unicode char that fonts
rendered inconsistently (warped). Use a crisp inline SVG of the two accent
squares (currentColor) across all headers.
- Month pages: turn the plain "Visiting <city> in <month>?" line into a bordered
callout with a CTA button, matching the city page's travel box.
Verified in Firefox and Chromium at desktop and 390px, light and dark: desktop
nav inline, mobile hamburger opens the dropdown, logo crisp, travel note styled.
Complete the forecast fallback chain: Open-Meteo → MET Norway → stale cache. When
both live sources are unavailable, serve the last cached recent/forecast bundle
even if it is past its 1-hour TTL — an hours-old bundle (which still carries the
recent observed days MET Norway lacks) beats a hard 503, mirroring the archive
path's stale-serve in _load_history.
The stale bundle is returned WITHOUT rewriting it, so its mtime stays old and the
next request retries the live sources first rather than treating the stale copy as
fresh. Only when there is no cache at all does the typed WeatherUnavailable surface.
The forecast path had no backup — unlike history, which falls back to NASA POWER.
When Open-Meteo's forecast API was rate-limited or down, the recent/forecast
bundle (and every endpoint that grades future days) failed with a 503.
Add MET Norway (yr.no) Locationforecast as a keyless, global forecast backup,
mirroring the NASA POWER role for history:
- _metno_to_frame: aggregates MET Norway's sub-daily timeseries into the daily
schema, converting units (°C→°F, mm→in, m/s→mph) and deriving feels-like from
the NWS heat index / wind chill (MET has no gusts or apparent temperature).
Precip prefers the 1-hour block and falls back to the 6-hour block so the
hourly→6-hourly resolution switch never double-counts.
- _fetch_forecast_metno: the backup fetch, with the ToS-required identifying
User-Agent and coordinates rounded to 4 decimals.
- _load_recent_forecast: on any Open-Meteo forecast failure, try MET Norway
before surfacing the error; a shared rate limit still raises the typed,
daily-aware WeatherUnavailable.
MET Norway is forecast-only (no recent past days), so it's a degraded-but-working
fallback: the forecast / day-ahead views keep serving during an Open-Meteo outage.
Tests cover the daily aggregation + unit conversion (incl. the no-double-count
precip rule) and the fallback wiring.
The Wikipedia "Read more →" link in the notable-weather section could break
between "Read" and "more" at narrow widths. Join it with non-breaking spaces so
it stays on one line.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
The per-month records section listed only the warmest and coldest temperature.
Extend it to a card per metric (High, Low, Feels-like, Humidity, Wind, Gust,
Precip), each showing that calendar month's record high and low with the date.
Precip is special-cased: a per-day record low is just zero, and the dry-streak
helper would bridge year boundaries on month-filtered rows, so the wettest and
driest sides use the month's largest and smallest total accumulation, dated to
the year. Partial current months (< 26 days) are excluded so an unfinished month
can't win "driest" on a fraction of its rainfall.
Reuses the records page's card grid, so it's already vertical on mobile.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
The month and season records tables showed only the record high (of the daytime
high) and the record low (of the overnight low) — one extreme each. Show both
ends of both metrics: for the daytime high, its record hottest (▲) and the
coldest a day ever stayed (▼, its record-low high); for the overnight low, its
mildest night (▲, its record-high low) and its record low (▼). All four already
came out of all_time_records; they were just being dropped.
Restructure each table to Period · Daytime high · Overnight low, with each metric
cell carrying two tinted, dated chips (▲ warmest / ▼ coldest). Fits mobile in
three columns; the all-time cards already showed both extremes, so this brings
the per-period tables in line.
The monthly temperature-range visualization now spans the 10th-percentile daily low
to the 90th-percentile daily high (from climatology's p10/p90) — the band most days
fall within — instead of just the average low to average high, giving a truer sense
of each month's spread. The normals table keeps the average high/low figures.
Add a search box to /climate that filters to a flat, alphabetical list of matching
cities (by city or country name), shown with their country; clearing it restores the
collapsible directory. The index is built from the already-crawlable links, so it's
a pure progressive enhancement (no SEO impact). Also sort cities alphabetically
within each country in the directory (were population-ordered).
Two mobile polish issues on the climate pages:
- The season records table's "(Dec–Feb)" span overflowed its narrow first
column (nowrap) and slid under the tinted value cell. Stack the span under the
season name on phones, top-align the record cells, and let the first column
wrap — so the period label sits cleanly in its own column.
- The monthly temperature-range strip stretched edge-to-edge with the values
floating far to the right. Cap the strip width so bars + values pack into a
tight group, shrink the value labels to fit (compact "41°–66°", no reserved
92px column). The overrides now sit after the base range rules so they win.
The all-time-records redesign (#106) removed the min-width floor that let the
wide records tables scroll, so the month and season records tables — still five
columns (value + separate date, twice) — compressed below their content on
phones and overlapped/clipped.
Fold each record's date under its value so those tables become three columns
(period · record high · record low), which fit any width without horizontal
scroll. Also tighten all climate tables under 560px (smaller font + padding) so
a hot city's 3-digit °F values with °C in parens never collide in the city
page's normals table.
* SEO: monthly & seasonal records on the city records page
The /climate/<city>/records page showed only all-time records per metric.
Expand it into a full records page: record high/low for each of the 12
months (each linking its month page) and for each meteorological season,
alongside the existing all-time table.
Seasons are hemisphere-aware — Dec–Feb reads as winter for northern cities
and summer for southern ones (picked from the city's latitude). Records
reuse grading.all_time_records over a month-filtered archive, so no new
data fetching or warming is needed. Adds a Dataset + breadcrumb JSON-LD
block and richer title/description for the expanded coverage.
* SEO: colour-code climate & records pages (heat-map + range strip)
The city, records and month pages were plain muted tables — generic climate-site
styling. Bring the interactive grader's diverging cold→hot palette onto them so
they read as heat maps in the site's own visual language:
- Map absolute °F to the 9-tier palette (temp_class, a Jinja global) and tint the
temperature cells across the monthly-normals, monthly/seasonal/all-time records
tables. The value stays in the ink token; the tint is a wash behind it. Non-temp
rows (humidity/wind/precip) and date columns stay untinted.
- Add a monthly temperature-range strip on the city page: one gradient bar per
month spanning the average low→high on a shared −10..115°F axis, so a city's
whole-year rhythm (and hot-vs-cold character) reads at a glance.
- Tint the month page's hero high/low and its record values inline.
Palette, light/dark, and 390–1920px layouts verified by rendering hot (Phoenix)
and cold (Anchorage) cities.
Replace the 5-column records table (which forced horizontal scrolling on mobile —
only high OR low visible) with a responsive card grid: one card per metric, each
with a red-accented 'Highest' row and a blue-accented 'Lowest' row (tier colors).
Grids on desktop, stacks to a single column on mobile with high/low stacked
vertically — no horizontal scroll. Precip reads Wettest / longest dry spell.
Record-low rainfall is meaningless (always ~0), so the precip row's low side now
shows the longest run of consecutive days without measurable rain and the date that
dry spell began (grading.longest_dry_streak), keeping the wettest day on the high
side. A footnote explains the substitution.
Drop the open-by-default on the first three countries (every <details> starts
collapsed) and order the country sections alphabetically instead of by largest
city's population.
- Climate hub (/climate) now uses native <details> per country — collapsible
expand/shrink sections with a city count and a rotating marker (top 3 countries
open by default). Links stay in the DOM, so crawling/SEO is unaffected.
- Fix nested F/C parentheses in the city lede (e.g. 'July (71°F (22°C) average
high)') by rephrasing so the already-parenthesized temperature isn't wrapped in
another paren.
- Fix the 5-column records table overlapping on mobile: give it a min-width so it
scrolls inside .table-wrap on narrow screens instead of colliding.
Add EXCLUDE_CC + KEEP_SLUGS to gen_cities.py: drop cities in countries that
criminalize LGBTQ+ people (plus China, Pakistan, Indonesia by choice), except a
hand-kept list of notable / tourist / high-English hubs — Lagos & Abuja; Nairobi &
Mombasa; Cairo, Giza, Alexandria; Dubai & Abu Dhabi; Kuala Lumpur/Malacca/Kota
Kinabalu/Kuching; Dar es Salaam/Zanzibar/Arusha; Casablanca/Fes/Rabat; Tehran;
Jeddah; Baghdad; Kabul; 7 Pakistani cities (Karachi, Lahore, Islamabad, Faisalabad,
Rawalpindi, Multan, Hyderabad); Jakarta + Surabaya + Bekasi; and China's 5 most
English-friendly cities (Shanghai, Beijing, Shenzhen, Guangzhou, Chengdu). Removed
slots backfill from the next-ranked non-excluded cities, so cities.json stays 1000.
Flavor regenerated (prunes removed, fetches backfilled).
Common-sense coverage pass on city_events.py: the list was 10/31 US and heavily
Anglosphere with no Latin America, no Middle East, and only London/Paris for all of
continental Europe. Add 9 verified events filling those gaps — Los Angeles (2025
wildfires), Moscow (2010 heat/wildfires), Madrid (2021 Storm Filomena), Athens (2018
Attica fires), Seoul (2022 floods), Dubai (2024 floods), Jeddah (2009 floods), Sao
Paulo (2014-17 drought), Rio de Janeiro (2010 floods). Every Wikipedia link
verified to resolve. 40 events total; nothing removed (no factual errors found).
Auto-sourcing a notable weather event from Wikipedia search proved unreliable
(wrong matches, low coverage), so city_events.py is a small, fact-checked list of
one iconic event per city (Katrina/New Orleans, Harvey/Houston, the 1952 Great
Smog/London, the 2021 heat dome/Seattle, ...) — 31 cities, each with a Wikipedia
link. The city page shows a 'Notable weather in {city}' callout when one exists and
otherwise just the flavor blurb. Tests check the curated slugs are all valid and the
callout renders (and is absent for uncurated cities).
Add a third tier to gen_cities.py: the top-population cities from the remaining
English-official countries plus countries where >35% speak English (Eurobarometer/
EF), that weren't already chosen. cities.json grows to 1000 (500 global + 250
core-English + 250 extended), pulling in the biggest cities of India, Nigeria,
Pakistan, the Philippines, plus Amsterdam/Stockholm/Nairobi/Tel Aviv, etc.
gen_flavor.py is now incremental (fetches only cities missing a blurb, prunes stale
ones; --full to rebuild); cities_flavor.json refreshed to cover 942/1000.
Add unique editorial content so the programmatic pages don't read as templated:
- gen_flavor.py seeds backend/cities_flavor.json with a short descriptive blurb per
city from Wikipedia's free REST summary API (no key), validating each match by
comparing article coordinates to the city's lat/lon so the wrong 'Springfield'
never attaches. Retries + modest concurrency; ~700/750 cities get a blurb, the
rest render without one. Text is CC BY-SA, attributed with a 'via Wikipedia' link.
- City pages show the blurb under the intro and a travel callout that deep-links to
the compare page with the city pre-loaded (/compare#loc=lat,lon) — the visitor
just adds their own city. Month pages get the same seasonal 'visiting in {month}?'
compare link. Both add unique per-page text and internal links.
- cities.py gains flavor(slug); tests cover the blurb + attribution + compare CTA.
The population-ranked global top-500 skewed to Asian megacities and missed
high-English-search-demand cities. gen_cities.py now tops up with the top ~250
cities from English-speaking countries (US/GB/CA/AU/NZ/IE/ZA) not already in the
global set, so US coverage goes 13->146, GB 2->42, CA 3->29, etc. (Seattle, Boston,
Manchester, Melbourne, Auckland, Dublin, ...). cities.json regenerated to 750.
Both deploy scripts now launch warm_cities.py automatically after the health check,
detached (dev: a systemd --user transient unit; prod: setsid/nohup), so the city
pages serve from cache without a manual step; idempotent, so only the first deploy
does the full warm. DEPLOY.md updated.
* SEO: generate curated city set for crawlable climate pages
gen_cities.py reuses the GeoNames index places.py already parses to select the top
~500 metros by population, assigns each a stable URL-safe slug (dropping admin1 when
it repeats the city name), and writes committed backend/cities.json. cities.py loads
it lazily with slug lookup, all_slugs(), display_name(), and by_country() grouping
for the upcoming hub + sitemap.
* SEO: rendering core, robots.txt, sitemap.xml, and metadata hygiene
- content.py: Jinja2 environment + HTML responder (ETag/304), dynamic /robots.txt
(disallows /api and /alerts, points at the sitemap) and /sitemap.xml (enumerates
the home/static pages plus every city, month, and records URL from cities.py).
Registered on the app before the StaticFiles mount so the routes win.
- templates/base.html.j2: shared layout with unique title/description, self-
referential canonical, Open Graph, favicon/manifest, header nav (adds a Climate
link) and a footer link graph.
- Give each existing page a unique <meta description> (were 5x identical) and a
self-referential <link rel=canonical>; add WebApplication JSON-LD to the home page.
- Pin jinja2.
* SEO: server-rendered per-city climate page (/climate/{slug})
The keystone crawlable page: for a city it snaps to the grid cell, loads the
archive (fetching once if missing, self-healing), and renders as real HTML — a
'how today compares' block (grade + percentile per metric from grade_day, tinted
by tier), a monthly normals table (climatology at each month's 15th, shown in °F
and °C), all-time records (new grading.all_time_records helper), a breadcrumb,
Dataset+Place+BreadcrumbList JSON-LD, self-referential canonical, and links into
the interactive tool + month/records pages. Content-page CSS added to style.css
(renamed the table class to avoid colliding with the app's .normals flex row).
* SEO: month (/climate/{slug}/{month}) and records (/climate/{slug}/records) pages
Month pages render the exact-month long-tail ('average weather in {city} in
{month}') with that month's average high/low, typical p10-p90 range, month-specific
records, and prev/next month links. Records pages show all-time record highs/lows
per metric with dates (grading.all_time_records). Shared _resolve_city helper; the
literal /records route is registered before the {month} param and month names are
validated (unknown month -> 404).
* SEO: climate hub, weather glossary, and about/methodology pages
- /climate: crawlable directory of all ~500 cities grouped by country — the
internal-link graph that lets search engines discover every city page.
- /glossary + /glossary/{term}: plain-language definitions (climate normal,
percentile, temperature anomaly, feels-like, heat index, wind chill, humidity,
reanalysis) with DefinedTerm JSON-LD and cross-links into the tool.
- /about: methodology page (ERA5 data source, 45-year baseline, +/-7-day window,
percentile grading) for E-E-A-T. All linked from the shared footer.
* SEO: archive warmer, content-page tests, and deploy docs
- warm_cities.py: paced, idempotent offline warmer that pre-fetches each city
cell's archive so /climate pages serve from cache and a crawl can't burst the
archive quota (pages self-heal if hit before warming).
- tests/test_content.py: city-set slug uniqueness/lookup, robots.txt, sitemap
enumerating city/month/records URLs, and that a rendered city page carries the
stats + canonical + Dataset JSON-LD in the HTML; plus month/records/hub/glossary/
about routing and 404s.
- DEPLOY.md: document the content pages, the warm step, and submitting the sitemap.
Make the app installable and deliver existing alert notifications as OS
push, alongside the in-app bell.
Backend:
- PushSubscription model (per-device endpoint + keys, owned by a user) and
register/unregister/test endpoints under /api/v2/push, cookie-auth scoped
to the user like the subscription routes.
- push.py: VAPID key management (env -> data/vapid.json -> generated) and a
pywebpush send helper that reports gone endpoints for pruning. No DB coupling.
- notify.py: after creating an in-app Notification, dispatch Web Push to the
user's devices (guarded — a push failure never affects the in-app write;
endpoints reported gone are pruned).
- Serve the .webmanifest with the correct media type.
Frontend:
- manifest.webmanifest + 192/maskable icons; <link rel="manifest"> on all pages.
- sw.js: push + notificationclick handlers (push-only; no fetch caching, so it
doesn't fight the existing IndexedDB cache). Registered globally in nav.js in
secure contexts.
- push-client.js + a "Notifications on this device" toggle and test-send on the
/alerts page, subscribing through the existing cookie-aware apiFetch.
Push and service workers require a secure context, so this is active over HTTPS
(or http://localhost) and cleanly no-ops on a plain-HTTP LAN origin.
A subscription to a city that had never been viewed had no cached ~45-year
archive, so the evaluator skipped it and it never fired. Now when a subscribed
cell has no cached archive, the pass fetches it once via get_history (which caches
it); every later pass reads it from cache and never re-fetches. A per-pass budget
(THERMOGRAPH_NOTIFY_ARCHIVE_FETCHES, default 4) caps how many missing archives one
pass will fetch so a burst of new subscriptions can't exhaust the archive quota,
and a rate-limited fetch just retries on the next pass. The recent/forecast bundle
keeps refreshing on its own hourly cadence.
Add integration tests covering fetch-when-missing and never-refetch-when-cached.
The notification dropdown is anchored to the bell button, which on mobile sits
left of the account button rather than at the screen edge, so the wide panel
overflowed off the left of the viewport (title/text cut off). On narrow screens,
drop .notif's positioning context so the dropdown anchors to the .acct cluster at
the header's right edge, keeping it fully on-screen.
Also add route-level tests for the accounts feature (auth flow, subscription CRUD,
duplicate/validation, ownership 404s, notifications), plus a
THERMOGRAPH_ACCOUNTS_DB override so the suite writes to a throwaway DB and stays
hermetic.
Move Thermograph to its own domain. thermograph.org now serves the app at its
root, and emigriffith.dev/thermograph* permanently redirects there (prefix
stripped, so deep links map straight across).
- app.py: BASE now supports an empty root prefix. THERMOGRAPH_BASE=/ (or empty)
yields BASE="" so routes sit at "/", "/calendar", "/api/v2/…" with no double
slashes; the bare-base redirect is skipped and the static mount falls back to
"/". Non-empty values keep the existing "/thermograph" sub-path behavior
unchanged (backward compatible).
- deploy/Caddyfile: add a thermograph.org site block reverse-proxying to the
uvicorn on 127.0.0.1:8137; turn emigriffith.dev's /thermograph handler into a
permanent redirect to thermograph.org (handle_path strips the prefix; the bare
/thermograph goes to the root).
- deploy/thermograph.env.example: default THERMOGRAPH_BASE=/ (app owns the domain).
- DEPLOY.md: document the two-domain layout and that deploy.sh does not ship the
Caddyfile or env — those are applied on the VPS by hand.
The frontend already uses base-relative URLs, so it follows the root base with no
changes. Verified both modes locally (root serves /, /calendar, /api/v2/*; the
/thermograph sub-path still redirects bare→slash and 404s at root) and the full
test suite (123) passes.
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
* Add account system foundation: email/password auth with cookie sessions
Introduce the app's first authoritative, user-owned data in a separate
data/accounts.sqlite (SQLAlchemy), kept apart from the disposable derived-cache
DB. Wire fastapi-users for email/password signup, cookie-based login/logout, and
a session-check endpoint, backed by a database session strategy so logins survive
restarts and are revocable.
- db.py: async (aiosqlite) + sync SQLAlchemy engines over accounts.sqlite, WAL +
foreign keys, create_db_and_tables().
- models.py: User, AccessToken, Subscription, Notification tables.
- users.py: pwdlib hashing, HttpOnly cookie transport (path-scoped, SameSite=Lax,
Secure via env), DatabaseStrategy sessions, current-user dependencies.
- schemas.py: user + subscription + notification Pydantic models.
- app.py: mount auth/register/users routers on v2, create tables at startup.
- Pin fastapi-users[sqlalchemy]/aiosqlite; ignore data/accounts.sqlite*.
* Add account header entry and auth modal (frontend)
account.js self-injects a header entry (following the units.js pattern) that
shows a Sign in button when logged out and an account menu when logged in, plus
an auth modal reusing the existing .mp-overlay/.mp-modal chrome for email/password
sign-in and account creation. A shared apiFetch helper sends the same-origin
cookie for authed calls; exported getUser/openAuth/onAuthChange back later phases.
Imported by every page entry module. On narrow screens the entry collapses to an
icon-only button so it doesn't crowd the title.
Enforce an 8-character minimum password in the user manager.
* Add subscription CRUD API and the alerts management page
Backend api_accounts.py adds user-scoped, cookie-authenticated endpoints to
create/list/update/delete subscriptions (and the notification reads used next):
POST snaps lat/lon to a grid cell, resolves a label, and rejects a duplicate
location+kind with 409; PATCH/DELETE are ownership-checked (404 on mismatch).
Mounted on the v2 prefix.
Frontend subscriptions.js + subscriptions.html serve the /alerts page: a sign-in
gate when logged out, an add flow that reuses the shared map picker and an editor
modal (kind, watched metrics, 95-99 percentile, two-sided), and a card list with
inline threshold/active edits and remove. Reachable from the account menu.
* Add background subscription evaluation engine
notify.py runs a daemon thread that periodically evaluates every active
subscription: it groups them by grid cell, reads history from the parquet cache
only (never spends archive quota) plus the hourly recent/forecast bundle, and
grades candidate days with the existing grading.grade_day. A watched metric that
lands at or beyond the threshold percentile fires a 'high' alert; a two-sided
subscription also fires 'low' for the symmetric cold/calm/dry tail (precip stays
one-directional). Observed subscriptions look at the last few recorded days,
forecast subscriptions at the coming week.
Two guards keep it quiet: a UNIQUE(subscription, event_date, metric, direction,
kind) constraint dedups repeat events, and a per-subscription weekly cap
(last_notified_at) limits each alert to one notification per 7 days. The loop
tolerates a bad cell or an upstream rate limit without aborting the pass. Started
and stopped from the app lifespan; gated by THERMOGRAPH_ENABLE_NOTIFIER.
* Add in-app notification center (header bell)
Extend account.js with a notification bell beside the account menu: an unread
badge, a dropdown listing recent notifications (title, body, relative time), a
per-item mark-read on click, and a Mark all read action, all through the
cookie-authed notifications API. Unread state refreshes on open and polls every
two minutes while signed in; polling stops on sign-out. Styled to match the app,
responsive down to mobile.
* Harden accounts: expired-session cleanup, engine tests, ops docs
- notify.py sweeps expired login sessions (access tokens past their lifetime)
once per evaluation pass.
- Add hermetic unit tests for the evaluation engine's trigger detection (high/low
tails, precip one-directional, normal = no trigger) and notification wording.
- Document accounts.sqlite (authoritative, back it up), the single-worker
requirement for the in-process evaluator, and the new env vars in DEPLOY.md.
- Fix the mobile width blow-out: the distribution grid item (.cmp-dist-row) had no
min-width:0, so each row expanded to the 9–10-column connected bar chart's
min-content and widened the whole page (zoom-out, clipped filter sheet). Add
grid-template-columns: minmax(0,1fr) + min-width:0 so the strip scrolls inside its
own .ct-strip, and tighten .ct-col to 34px on phones.
- Add a Load/Refresh button inside the compare filter sheet's date-range block, wired
to the same refresh()/isDirty(); editing the range on mobile no longer needs the
sheet closed. The existing button stays in the controls (loads added places).
- Default date range on both pages is now January six years back → the present. On
the calendar this is an explicit chunked range (the months=24 path is server-capped
at ~2yr).
- Names now include country: reverse_geocode appends it to the place label; revgeo
cache key gets a v2 prefix and PAYLOAD_VER bumps to p2 so labels/payloads
repopulate.
- Location names read as a hierarchy: compact chips show just the lead segment (full
name in the title/aria), and each ranked card shows the lead segment bold over a
muted "city · region · country" line.
Frontend + a small backend name/cache change; no schema migration.
cache.js contained a JavaScript clone of backend/grid.py's snapping math
(its own comment said so) to compute the 8 surrounding cells and fire 8
staggered prefetch requests — grid geometry had two homes, one per
language, plus a client-side guess at Nominatim pacing.
The server now owns it: grid.neighbors(cell) steps one cell width from
the center and re-snaps (adjacent rows have different longitude steps;
poles and the antimeridian handled by snap), and /api/v2/cell grew a
neighbors=1 flag that enqueues those cells for a single background
worker. The warm-only guarantee matches prefetch=1 — a cell with no
cached archive is skipped, so no weather-API quota is ever spent — and
reverse_geocode's own lock paces the at-most-one Nominatim call per
never-labeled cell. Re-enqueues are TTL-deduped; the worker starts from
the lifespan hook, so tests and offline importers never spawn it.
The client now sends its one conditional bundle request with
neighbors=1 (a warm spot costs an empty 304) instead of skipping the
bundle and firing 8 extra requests; the grid-math clone and the
now-unused hasFreshCache are deleted.
Tests (114): grid.neighbors mid-latitude/pole/antimeridian, the
neighbors=1 enqueue + TTL dedupe, _warm_cell materializing the
history-derived store rows, and the never-fetch-upstream guarantee.
Verified with the headless-Chromium smoke across all five pages.
climate.py repeated the same four mechanical patterns:
- the Open-Meteo daily params dict (3x) -> _om_daily_params(cell, **window)
- the doy attach line (6x) -> _with_doy
- makedirs + drop-doy + zstd to_parquet (3x) -> _write_cache
- the identical _to_frame/_nasa_to_frame tail (feels-like, valid-day
filter, doy) -> _finalize_frame
grading.py encoded the tier tables twice — TEMP_BANDS/RAIN_BANDS plus the
hand-aligned _TEMP_LADDER/_RAIN_LADDER ('kept aligned' by comment). The
ladders are now derived from the bands (_ladder_from; verified
byte-identical to the old tables before landing), so tier boundaries have
exactly one definition. grade_range's inline dry-streak walk is replaced
with the existing dry_streaks(); its per-(doy,var) sample cache now also
memoizes the window mask per doy instead of recomputing it once per
metric (9x per day-of-year).
New tests pin the refactor: _to_frame schema/day-filter/missing-series
tolerance, the combined feels-like side selection, NASA unit conversions
and fill-sentinel handling, _om_daily_params windows, and _write_cache
stripping the derived doy column.
api_suggest carried ~80 lines of ranking policy — the exactness-boosted
scoring, local/upstream blending with dedupe, and the token-respelling
correction loop — all consuming the match: prefix/fuzzy vocabulary that
places.py produces. Producer and consumer now live together:
places.suggest(q, upstream, limit) implements the whole policy with the
upstream geocoder lookup injected as a callable, so places stays free of
the fetch layer and the policy is testable without FastAPI.
The endpoint is the HTTP shim: call places.suggest, map the
nothing-at-all-to-serve failure to a 502. Behavior unchanged.
Policy tests: upstream skipped when the local answer convinces, blend +
dedupe, the exactness boost (a hamlet spelled like the typo can't beat
Seattle), single-typo queries answered by the fuzzy matcher without a
correction, the correction path verified via the upstream probe,
upstream failure degrading to local results, and the terminal
error-propagation case.
Rate-limit handling crossed the climate→app boundary as prose: climate
raised RuntimeError(cooldown text), and app._weather_fetch_error re-parsed
it by keyword ('429'/'rate-limited'/'daily'/'tomorrow') while also calling
climate's private _is_rate_limit/_rate_limit_reason — and the user-facing
daily-quota copy existed verbatim in both modules.
climate now raises WeatherUnavailable (a RuntimeError subclass carrying
the user-facing text and a daily flag): from _load_history when both
sources fail with no stale cache, and from the forecast fetch on a 429.
limit_message() is the single home of the rate-limit copy; is_rate_limit
is public for the one remaining raw-429 fallback. app maps the typed
error to 503 by isinstance — no message parsing, no private imports.
The burst-limit copy is unified on 'The weather service is rate-limited…'
(the archive path previously said 'weather archive' internally but the
API always rewrote it; user-visible text is unchanged).
Tests: daily-quota 503 carries the 'tomorrow' copy, an unclassified raw
429 still maps to a clean 503, and a genuine fault stays a 502 with the
raw error.
The derived store's key/token formats existed in three places — each
endpoint, api_cell's slice assembly, and migrate.py — where any drift
would silently split the cache (endpoints missing rows the bundle wrote,
migrate materializing rows nobody reads). They are now defined once in
views.py (grade_key/calendar_key/day_key/forecast_key, history_token/
recent_token/day_token) and consumed everywhere, with a pinning test so
a format change is always deliberate.
The four data endpoints shared two copy-pasted sequences, now helpers:
- _fetch_history: history (+ optional recent bundle) fetch with upstream
failures mapped to clean HTTP errors and an empty record to 404.
- _cached_response: If-None-Match 304 / token-valid store replay /
build + persist + serve, with calendar's dont-persist-placeless rule
as an explicit flag.
Each endpoint is now its audit run + identity + a build callback (~10
lines); api_day's hourly-token special case moved into day_token. New
tests: identity format pins, rate-limit 503 parametrized across all five
data routes, prefetch=1 never touching upstream (cold 204, warm
history-only slices), and calendar's placeless-payload retry behavior.
app.py had grown three co-resident strata: HTTP/caching plumbing, payload
assembly, and search policy. This moves the payload layer — the four
build_* functions, cal_span clamping, hist_end, PAYLOAD_VER, NullRun and
their helpers — into a new views.py with no web dependencies (pure moves,
public names). app.py keeps routing, ETag/derived-store plumbing, and
page serving; migrate.py imports views instead of reaching into app's
privates.
That import previously constructed the whole FastAPI app and — because
places.start_loading() ran at import time — kicked off a background
GeoNames download from an offline batch script. The index load now hangs
off the app's lifespan hook, so it fires when a server starts, not when
the module is imported (tests, migrate).
New tests: cal_span clamping (months-back default, record bounds, 2-year
cap, inversion), builder shapes (grade window ordering, day obs from the
recent bundle, forecast future-only), a regression test that build_day
degrades to climatology when the recent fetch fails, and a subprocess
layering guard that importing views/migrate pulls in neither FastAPI nor
the app module and starts no index download.
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
- /api/v2/place still called grid.in_north_america(), which the worldwide-
coverage change removed — every request raised AttributeError (500). The
failure was invisible because compare.js treats the call as best-effort,
so the instant chip-naming feature was silently dead. Drop the dead guard;
the lat/lon Query validators already bound the inputs.
- /api/v2/day was the only data route not wrapping get_history in
_weather_fetch_error, so a rate-limited cold cell returned a raw 500
instead of the clean 503 the other routes emit.
- calendar: the #calendar pointerleave handler was re-registered inside
attachHover on every render — and the comfort slider re-renders per input
tick, stacking dozens of copies. Register it once at module scope next to
the matching document-level dismiss handler.
- getJSON: check res.ok before parsing the body as JSON and fall back to the
status line, so a non-JSON error body (a proxy 502 HTML page) reads as
"Request failed (502 Bad Gateway)" instead of a JSON parse error.
Sharing a Thermograph URL (Discord, Slack, iMessage…) now unfurls into a
card with the site name, page title, description, accent color, and logo.
- All five pages get description/theme-color/og:*/twitter:card meta and a
favicon. og:url/og:image need absolute URLs and crawlers don't run JS, so
pages carry an __ORIGIN__ placeholder the server fills in per request from
X-Forwarded-Proto (Caddy) / scheme + the Host header + base path — correct
on both the LAN dev server and the prod domain without hardcoding either.
- New logo assets: the header's ▚ mark as an app icon (accent quadrants on
the dark surface) — logo.png (512x512, the og:image) and logo.svg (favicon).
- Page routes render the substitution with a weak ETag + 304 revalidation
(replacing plain FileResponse) and now answer HEAD, which preview crawlers
probe with (previously 404).
Add /api/v2/suggest and wire the location picker's search box to it as a
debounced type-ahead: top-5 place suggestions that tolerate a single-letter
typo (substituted, missing, or extra letter, or two adjacent letters swapped)
anywhere in the query, including the first character.
- backend/places.py: local place index built from a GeoNames cities dump
(downloaded once into data/geonames/, loaded in a background thread; the
app boots and serves without it). Exact-prefix matches rank first by
population, then names one edit away; a token vocabulary respells one
mistyped word against known place-name tokens ("pest seattle" ->
"west seattle") for retry against the upstream geocoder, which covers
neighbourhood-level places the dump lacks. THERMOGRAPH_CITIES picks the
dump (default cities1000).
- /suggest blends local and upstream results by population with an exactness
boost, so "Seatle" (a Cumbrian hamlet) can't outrank Seattle when the
query is one edit from the city, while "munchen" still surfaces Munich via
upstream (the index only knows English names). Upstream lookups are
memoized and skipped entirely when the index answers convincingly.
- mappicker.js: debounced (250ms) live suggestions with abort + sequence
guards against stale responses, arrow-key navigation, Enter-picks-highlight,
Escape dismissing the list before closing the overlay. Submit goes through
the same typo-tolerant endpoint.
- climate.geocode results now carry population (used for ranking).
Remove the US+Canada bounding box so every endpoint accepts any lat/lon.
The grading pipeline was already global-ready (ERA5 archive, timezone=auto,
day-of-year climatology), so opening it up is mostly deleting the guard —
plus the edge cases that only exist once the whole globe is in play:
- grid.py: snap() wraps longitude into [-180, 180) and clamps latitude, and
cell centers are normalized so the polar row and the cells straddling the
antimeridian always report valid coordinates to the weather/geocoding APIs.
snap() and from_id() now share one _cell() builder, making id round-trips
exact by construction (verified with a 300k-point global sweep).
- nav.js: neighbor-cell prefetch skips rows past the poles and wraps
longitudes across the dateline instead of sending out-of-range queries.
- Nominatim reverse geocoding requests accept-language=en so place labels
render in one script worldwide (matching the forward geocoder).
- mappicker: search suggestions are no longer filtered to US/CA, the
placeholder and default map view are worldwide.
- calendar: season filter labels flip for southern-hemisphere locations
(Dec-Feb shows as Summer); the underlying month groups are unchanged, so
saved filter selections keep meaning the same months.
Verified end-to-end on a scratch server: Tokyo and Sydney grade with real
labels, a Fiji cell on the antimeridian's east edge builds and serves warm
hits from the derived store, and prefetch=1 on a cold cell still answers
204 without spending weather-API quota.
