Thermograph monorepo: graded-climate API + SSR frontend + infra, domain-specific containerized deploys
0f978bc708
* Serve the app on thermograph.org at root; redirect emigriffith.dev/thermograph Move Thermograph to its own domain. thermograph.org serves the app at its root, and emigriffith.dev/thermograph* permanently redirects there (prefix stripped, so deep links map straight across). - app.py: BASE now supports an empty root prefix. THERMOGRAPH_BASE=/ (or empty) yields BASE="" so routes sit at "/", "/calendar", "/api/v2/…" with no double slashes; the bare-base redirect is skipped and the static mount falls back to "/". Non-empty values keep the existing "/thermograph" sub-path behavior unchanged (backward compatible). - deploy/Caddyfile: add a thermograph.org site block reverse-proxying to the uvicorn on 127.0.0.1:8137; turn emigriffith.dev's /thermograph handler into a permanent redirect to thermograph.org. - deploy/thermograph.env.example: default THERMOGRAPH_BASE=/. - DEPLOY.md: document the two-domain layout and that deploy.sh does not ship the Caddyfile or env — those are applied on the VPS by hand. The frontend already uses base-relative URLs, so it follows the root base with no changes. Verified both modes locally (root serves /, /calendar, /api/v2/*; the /thermograph sub-path still redirects bare→slash and 404s at root). * Add dashboard (#144) * Auto-submit IndexNow on deploy when the URL set changes (#130) Hook the production deploy (deploy/deploy.sh) to ping IndexNow after the health check passes, so new/changed pages reach Bing/DuckDuckGo/Yandex without a manual run. Best-effort — wrapped so it can never fail a deploy — and it sources /etc/thermograph.env so its key matches the one the running service serves. To avoid re-blasting ~14k URLs on every code push, indexnow.py gains a --if-changed mode gated by a signature of the URL set (persisted to data/indexnow_state.txt): code-only deploys skip; a deploy that adds or removes a city submits. `make indexnow` still forces a full submit. * Add ops metrics endpoint + SSH/Termux dashboard (#131) Adds observability for the running app, viewable over SSH (e.g. Termux): - backend/metrics.py: thread-safe, best-effort in-process counters for inbound requests (per category) and outbound calls (per external source), plus a phase->source map and snapshot(). Hooked into climate._request (outbound, all six weather/geocode sources) and the existing revalidate_static middleware (inbound). Never raises into the request path. - GET /api/v2/metrics: token-gated JSON of the counters + warm-queue depth + thread names/uptime/pid. Returns 404 unless a valid THERMOGRAPH_METRICS_TOKEN is presented or the request is direct loopback with no proxy X-Forwarded-* header, so ops data is never exposed through Caddy on the public domain. - scripts/dashboard.py (+ scripts/dash, make dashboard): stdlib-only terminal dashboard. Reads cache (parquet + SQLite), accounts (users + active subscriptions), and system resources (/proc + systemd) directly, and pulls traffic from the metrics endpoint over loopback. Live refresh, --once, --json; mobile-terminal width. Paths resolve relative to the script so the same tool works on the LAN dev and prod VPS checkouts. - Monitoring sections in DEPLOY.md / DEPLOY-DEV.md; unit tests for the counters. * Dashboard: run under the app venv, not the box's default python3 (#132) The box's default python3 can be a build without _sqlite3 (pyenv 3.10 here), so ./scripts/dash crashed on import. Prefer the repo's .venv interpreter (the Python the server itself runs on, which always has sqlite3); fall back to python3/python. Also make the sqlite3 import in dashboard.py optional so a fallback interpreter degrades to empty cache/accounts panels instead of crashing, and point the 'make dashboard' target at the wrapper. * Dashboard: redraw live view in place so it stops scrolling to the bottom (#133) The live loop did a full clear-and-reprint each refresh; when the output is taller than the terminal (common on a phone) that scrolls the view to the bottom every tick. Redraw from the top instead: hide the cursor, clear once, then each frame move to home and rewrite line-by-line (clear-to-EOL + clear-below), capping the body to the window height so it never prints more lines than fit and never scrolls. Restore the cursor on exit; piped (non-TTY) output prints plain frames. * Dashboard: make the live view a scrollable in-place pager (#134) The live mode is now an alt-screen pager (like top/less): it shows only the latest snapshot, you scroll it with swipe/arrows/jk/PgUp-PgDn/g/G, and each refresh replaces the page in place while keeping your scroll position — nothing gets appended below and the view is never yanked to the top or bottom. On exit it restores the cursor, leaves the alternate screen, and resets the terminal mode. --once still prints a single static snapshot (also used automatically when stdout/stdin aren't a terminal, e.g. piping); --json unchanged. * Metrics: don't count the dashboard's own /api/v2/metrics polling (#135) The ops dashboard polls the metrics endpoint every few seconds; counting those hits just shows the monitor watching itself and inflates the inbound totals. Skip the 'metrics' category in record_inbound (so it's excluded from inbound_total too), and hide it in the dashboard's inbound list defensively. * Dashboard: add a STORAGE section (archive + DB sizes) (#136) Show on-disk storage: the parquet archives (with history/recent split), the derived SQLite DB, and the accounts SQLite DB individually, plus a total. Archive bytes are summed during the existing TTL-cached cache scan; DB sizes include the -wal/-shm sidecars (WAL holds recent writes, so the base file alone understates the footprint). Drop the now-redundant 'payload db' line from CACHE. * Dashboard: show STORAGE sizes in fixed GB (0.01 GB floor) (#137) The auto-scaling formatter only switched to GB once a value passed 1 GB, so the archives read as MB and the DBs as KB. Format the STORAGE section in fixed GB with two decimals and a 0.01 GB floor (so the small databases stay visible instead of rounding to 0.00). Memory readouts keep the auto-scaling formatter. * Dashboard: show STORAGE under 50 MB in MB, GB above (#138) Values below 50 MB now render as MB with one decimal (no KB), so the databases read naturally (derived 7.7MB, accounts 0.4MB) instead of 0.01GB; 50 MB and up stay in GB with two decimals. Renames the helper _gb -> _storage_size to match. * Dashboard: move STORAGE MB->GB cutoff to 500 MB (#139) Values below 500 MB show in MB (one decimal); 500 MB and up in GB. * Dashboard: accounts totals only; log client IPs per request (#140) Dashboard ACCOUNTS section now shows only totals (users, active/total subscriptions, notifications, push subs) — the per-user and per-subscription lists are gone, and the dashboard no longer queries that PII at all. Backend: add a per-request access log (logs/access/access-<date>.jsonl) recording the client IP (real IP via the left-most X-Forwarded-For hop when proxied behind Caddy, else the peer address), method, path, status, and category. Static assets and the dashboard's own metrics polling are skipped. Best-effort, never raises into the request path — retained for later traffic/IP analysis. * Dashboard: wrap-aware pager so the header stops scrolling off (#141) The pager counted one screen row per logical line, but on a narrow phone many lines wrap to two rows — so a page printed more physical rows than the terminal had, scrolling the top (title + CACHE header) off-screen with no way to scroll back to it. Measure each line's wrapped height and fit a page to the real physical-row budget; page/End step by what's actually visible; read the true tty size (not shutil, which honors stale COLUMNS/LINES). Status line is clipped to one row. * Add log storage size to the ops dashboard STORAGE section (#142) The STORAGE section reported parquet archives and both SQLite DBs but omitted the logs/ footprint, so the "total" understated real disk use. Add a recursive _tree_size helper and a "logs" row measuring the whole logs/ tree (audit + errors + access JSONL plus stray *.log files), broken out by stream, and fold it into the STORAGE total. Flows through --json automatically via read_storage. * Dashboard: show last-10-minute traffic per category (#143) Add a rolling short-window view alongside the since-start totals. metrics.py keeps per-minute buckets (epoch-minute -> {key: count}) for inbound categories and outbound sources, summed over the last 10 minutes on snapshot() and exposed as inbound_recent / outbound_recent / window_minutes. Cheap and bounded — at most WINDOW+1 tiny dicts, pruned as minutes roll off; self-polling still excluded. The dashboard's TRAFFIC section now renders that count as a dim column next to each row's total (header reads "total · last 10m"); idle sources show "-". * Add dashboard #2 (#145) * Auto-submit IndexNow on deploy when the URL set changes (#130) Hook the production deploy (deploy/deploy.sh) to ping IndexNow after the health check passes, so new/changed pages reach Bing/DuckDuckGo/Yandex without a manual run. Best-effort — wrapped so it can never fail a deploy — and it sources /etc/thermograph.env so its key matches the one the running service serves. To avoid re-blasting ~14k URLs on every code push, indexnow.py gains a --if-changed mode gated by a signature of the URL set (persisted to data/indexnow_state.txt): code-only deploys skip; a deploy that adds or removes a city submits. `make indexnow` still forces a full submit. * Add ops metrics endpoint + SSH/Termux dashboard (#131) Adds observability for the running app, viewable over SSH (e.g. Termux): - backend/metrics.py: thread-safe, best-effort in-process counters for inbound requests (per category) and outbound calls (per external source), plus a phase->source map and snapshot(). Hooked into climate._request (outbound, all six weather/geocode sources) and the existing revalidate_static middleware (inbound). Never raises into the request path. - GET /api/v2/metrics: token-gated JSON of the counters + warm-queue depth + thread names/uptime/pid. Returns 404 unless a valid THERMOGRAPH_METRICS_TOKEN is presented or the request is direct loopback with no proxy X-Forwarded-* header, so ops data is never exposed through Caddy on the public domain. - scripts/dashboard.py (+ scripts/dash, make dashboard): stdlib-only terminal dashboard. Reads cache (parquet + SQLite), accounts (users + active subscriptions), and system resources (/proc + systemd) directly, and pulls traffic from the metrics endpoint over loopback. Live refresh, --once, --json; mobile-terminal width. Paths resolve relative to the script so the same tool works on the LAN dev and prod VPS checkouts. - Monitoring sections in DEPLOY.md / DEPLOY-DEV.md; unit tests for the counters. * Dashboard: run under the app venv, not the box's default python3 (#132) The box's default python3 can be a build without _sqlite3 (pyenv 3.10 here), so ./scripts/dash crashed on import. Prefer the repo's .venv interpreter (the Python the server itself runs on, which always has sqlite3); fall back to python3/python. Also make the sqlite3 import in dashboard.py optional so a fallback interpreter degrades to empty cache/accounts panels instead of crashing, and point the 'make dashboard' target at the wrapper. * Dashboard: redraw live view in place so it stops scrolling to the bottom (#133) The live loop did a full clear-and-reprint each refresh; when the output is taller than the terminal (common on a phone) that scrolls the view to the bottom every tick. Redraw from the top instead: hide the cursor, clear once, then each frame move to home and rewrite line-by-line (clear-to-EOL + clear-below), capping the body to the window height so it never prints more lines than fit and never scrolls. Restore the cursor on exit; piped (non-TTY) output prints plain frames. * Dashboard: make the live view a scrollable in-place pager (#134) The live mode is now an alt-screen pager (like top/less): it shows only the latest snapshot, you scroll it with swipe/arrows/jk/PgUp-PgDn/g/G, and each refresh replaces the page in place while keeping your scroll position — nothing gets appended below and the view is never yanked to the top or bottom. On exit it restores the cursor, leaves the alternate screen, and resets the terminal mode. --once still prints a single static snapshot (also used automatically when stdout/stdin aren't a terminal, e.g. piping); --json unchanged. * Metrics: don't count the dashboard's own /api/v2/metrics polling (#135) The ops dashboard polls the metrics endpoint every few seconds; counting those hits just shows the monitor watching itself and inflates the inbound totals. Skip the 'metrics' category in record_inbound (so it's excluded from inbound_total too), and hide it in the dashboard's inbound list defensively. * Dashboard: add a STORAGE section (archive + DB sizes) (#136) Show on-disk storage: the parquet archives (with history/recent split), the derived SQLite DB, and the accounts SQLite DB individually, plus a total. Archive bytes are summed during the existing TTL-cached cache scan; DB sizes include the -wal/-shm sidecars (WAL holds recent writes, so the base file alone understates the footprint). Drop the now-redundant 'payload db' line from CACHE. * Dashboard: show STORAGE sizes in fixed GB (0.01 GB floor) (#137) The auto-scaling formatter only switched to GB once a value passed 1 GB, so the archives read as MB and the DBs as KB. Format the STORAGE section in fixed GB with two decimals and a 0.01 GB floor (so the small databases stay visible instead of rounding to 0.00). Memory readouts keep the auto-scaling formatter. * Dashboard: show STORAGE under 50 MB in MB, GB above (#138) Values below 50 MB now render as MB with one decimal (no KB), so the databases read naturally (derived 7.7MB, accounts 0.4MB) instead of 0.01GB; 50 MB and up stay in GB with two decimals. Renames the helper _gb -> _storage_size to match. * Dashboard: move STORAGE MB->GB cutoff to 500 MB (#139) Values below 500 MB show in MB (one decimal); 500 MB and up in GB. * Dashboard: accounts totals only; log client IPs per request (#140) Dashboard ACCOUNTS section now shows only totals (users, active/total subscriptions, notifications, push subs) — the per-user and per-subscription lists are gone, and the dashboard no longer queries that PII at all. Backend: add a per-request access log (logs/access/access-<date>.jsonl) recording the client IP (real IP via the left-most X-Forwarded-For hop when proxied behind Caddy, else the peer address), method, path, status, and category. Static assets and the dashboard's own metrics polling are skipped. Best-effort, never raises into the request path — retained for later traffic/IP analysis. * Dashboard: wrap-aware pager so the header stops scrolling off (#141) The pager counted one screen row per logical line, but on a narrow phone many lines wrap to two rows — so a page printed more physical rows than the terminal had, scrolling the top (title + CACHE header) off-screen with no way to scroll back to it. Measure each line's wrapped height and fit a page to the real physical-row budget; page/End step by what's actually visible; read the true tty size (not shutil, which honors stale COLUMNS/LINES). Status line is clipped to one row. * Add log storage size to the ops dashboard STORAGE section (#142) The STORAGE section reported parquet archives and both SQLite DBs but omitted the logs/ footprint, so the "total" understated real disk use. Add a recursive _tree_size helper and a "logs" row measuring the whole logs/ tree (audit + errors + access JSONL plus stray *.log files), broken out by stream, and fold it into the STORAGE total. Flows through --json automatically via read_storage. * Dashboard: show last-10-minute traffic per category (#143) Add a rolling short-window view alongside the since-start totals. metrics.py keeps per-minute buckets (epoch-minute -> {key: count}) for inbound categories and outbound sources, summed over the last 10 minutes on snapshot() and exposed as inbound_recent / outbound_recent / window_minutes. Cheap and bounded — at most WINDOW+1 tiny dicts, pruned as minutes roll off; self-polling still excluded. The dashboard's TRAFFIC section now renders that count as a dim column next to each row's total (header reads "total · last 10m"); idle sources show "-". * Run prod on 3 uvicorn workers with a shared metrics store A single slow upstream weather fetch (cache-miss during an open-meteo/NASA degradation) blocked the one uvicorn worker and took the whole app down for ~2 min — real users got aborted connections. Give prod concurrency headroom so one slow request can't freeze the rest. - deploy/thermograph.service: worker count is env-driven (`--workers ${WORKERS}`, default 1); prod sets WORKERS=3 in /etc/thermograph.env. Dev's separate --user unit is untouched (stays single-worker). - metrics.py: with multiple workers, per-process counters would split the tally and the ops dashboard (polls one random worker) would see only a fraction. Add a shared SQLite store selected by THERMOGRAPH_METRICS_DB so every worker tallies into one place; unset keeps the zero-dependency in-memory store for dev/tests/single-worker. Public API and snapshot shape unchanged. - The unit defaults THERMOGRAPH_METRICS_DB and clears it on each (re)start via ExecStartPre, so bumping WORKERS can never silently fragment the dashboard and "since start" tallies keep their old meaning. - Tests: cover the SQLite store — cross-worker aggregation, window roll-off, and env-based selection. Full suite: 178 passed. Note: applying to prod also needs the unit reinstalled + WORKERS=3 in /etc/thermograph.env — deploy.sh only pulls+restarts, it doesn't reinstall the systemd unit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016mRqdHWRdUjEvEdfwig3wg --------- Co-authored-by: root <root@vmi3417050.contaboserver.net> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| deploy | ||