warm_cities.py and indexnow.py are one-shot scripts run manually at deploy
today - nothing keeps the curated city set topped up, or pings IndexNow,
between deploys.
Add notifications/scheduler.py: an in-process APScheduler instance driving
two recurring jobs (city warming, daily; IndexNow --if-changed, every 6h by
default), gated by the same leader election as the notifier - only the
process that already won leadership starts it, so the jobs run in exactly
one place under multi-host Swarm rather than once per replica. Neither job
runs immediately on start: warm_cities already runs at deploy time, so an
immediate duplicate on every worker boot/restart would be wasted work.
Extract indexnow.submit_if_changed() from the CLI's --if-changed branch so
the scheduler and the command line share the exact same skip-when-unchanged
logic instead of two copies drifting apart. The CLI's plain (non-flag) path
is unchanged.
APScheduler over a Postgres-native queue: exactly one process ever runs
this, no fan-out/backpressure/dead-letter need, no new infrastructure.
* Centralize filesystem paths in a single module
Add paths.py, which resolves the repo root once and derives the cache,
accounts DB, logs, templates, frontend and bundled-city-data locations
from it. Replace the 13 per-module `dirname(__file__)/..` anchors with
references to it, so a module's location no longer determines where the
app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are
unchanged; every resolved path is byte-identical to before.
Groundwork for moving modules into packages without re-pointing paths.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
* Split the backend into domain packages
Group the flat backend modules into packages that mirror their concerns:
data/ climate, grading, scoring, grid, places, cities,
city_events, store
web/ app, views, homepage, content, schemas
notifications/ notify, digest, push, mailer, discord,
discord_interactions, discord_link
accounts/ models, users, api_accounts, db
core/ metrics, singleton, audit
Intra-project imports are rewritten to the package-qualified form. The
entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor)
and paths.py stay at the backend/ root, and backend/app.py becomes a
shim re-exporting web.app:app so the launch target stays `app:app` —
run.sh, the systemd units, and CI need no change.
Verified: full suite (318) passes, `uvicorn app:app` boots and serves
the home/SEO/static/API surfaces, and every root script imports clean.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
Hook the production deploy (deploy/deploy.sh) to ping IndexNow after the health
check passes, so new/changed pages reach Bing/DuckDuckGo/Yandex without a manual
run. Best-effort — wrapped so it can never fail a deploy — and it sources
/etc/thermograph.env so its key matches the one the running service serves.
To avoid re-blasting ~14k URLs on every code push, indexnow.py gains a
--if-changed mode gated by a signature of the URL set (persisted to
data/indexnow_state.txt): code-only deploys skip; a deploy that adds or removes a
city submits. `make indexnow` still forces a full submit.
Faster search-engine indexing for the ~14k climate/city/record URLs:
- IndexNow (backend/indexnow.py): instantly notify Bing/DuckDuckGo/Yandex of new
or changed URLs. Per-host key resolved env → gitignored file → generated (mirrors
push.py), served at /{key}.txt, and echoed in submissions. `submit_all()` + a
`make indexnow` CLI push every indexable URL, batched under the 10k cap. Google
doesn't use IndexNow, so it stays on the sitemap.
- Sitemap: replace the per-request today() <lastmod> (which churns every fetch and
trains crawlers to ignore lastmod) with a stable content-build date; factor the
URL list into public_paths() shared with IndexNow so the two never drift.
- Search-console verification: render google-site-verification + msvalidate.01
<meta> tags from env into every page's <head> — the SEO pages via base.html.j2
and the static pages (incl. the homepage Google verifies) via _page().
- Docs: env example documents the three new vars; .gitignore ignores the key.
Tests: key-file route, stable single-date lastmod, IndexNow payload/batching, and
verification meta on both SEO and static pages.
