Thermograph monorepo: graded-climate API + SSR frontend + infra, domain-specific containerized deploys
956153691a
The ops dashboard marked subscription-notifier DOWN ~2/3 of the time after the notifier was gated to a single leader worker. It judged the daemon by whether its thread was present on whichever worker answered /api/v2/metrics, but only the leader runs it — so polls landing on the other two workers saw no thread and reported DOWN for a perfectly healthy notifier. Per-worker thread checks can't describe a single-leader daemon, and worse, they could no longer distinguish a real crash from a normal non-leader. Report it by liveness instead: the notifier stamps a heartbeat (timestamp + expected interval) into the shared metrics store each loop iteration, including once at startup so the dashboard shows it up immediately after a restart. Since the beat lives in the shared SQLite store, any worker's metrics response carries it, so the dashboard sees the notifier's health whichever worker it polls. - metrics.py: both stores record/expose heartbeats (SQLite reuses the meta table, so every worker's beats share one file); snapshot() converts each beat to a server-computed age so readers needn't reconcile clock skew. - notify.py: beat at loop start and every wake. - dashboard.py: judge heartbeat daemons by freshness (up to 2 missed intervals before DOWN, with beat age shown); neighbor-warmer keeps its per-worker thread check, which is correct since every worker runs it. - Tests: heartbeat store/snapshot shape + cross-worker sharing; dashboard fresh/stale/missing rendering and the unchanged warmer check. 194 passed. Co-authored-by: root <root@vmi3417050.contaboserver.net> |
||
|---|---|---|
| templates | ||
| tests | ||
| api_accounts.py | ||
| app.py | ||
| audit.py | ||
| cities.json | ||
| cities.py | ||
| cities_flavor.json | ||
| city_events.py | ||
| climate.py | ||
| content.py | ||
| db.py | ||
| gen_cities.py | ||
| gen_flavor.py | ||
| grading.py | ||
| grid.py | ||
| indexnow.py | ||
| metrics.py | ||
| migrate.py | ||
| models.py | ||
| notify.py | ||
| places.py | ||
| push.py | ||
| requirements-dev.txt | ||
| requirements.txt | ||
| schemas.py | ||
| singleton.py | ||
| store.py | ||
| users.py | ||
| views.py | ||
| warm_cities.py | ||