thermograph/models.py

187 lines
9.2 KiB
Python
Raw Normal View History

Account system with weather-notification subscriptions (#89) * Add account system foundation: email/password auth with cookie sessions Introduce the app's first authoritative, user-owned data in a separate data/accounts.sqlite (SQLAlchemy), kept apart from the disposable derived-cache DB. Wire fastapi-users for email/password signup, cookie-based login/logout, and a session-check endpoint, backed by a database session strategy so logins survive restarts and are revocable. - db.py: async (aiosqlite) + sync SQLAlchemy engines over accounts.sqlite, WAL + foreign keys, create_db_and_tables(). - models.py: User, AccessToken, Subscription, Notification tables. - users.py: pwdlib hashing, HttpOnly cookie transport (path-scoped, SameSite=Lax, Secure via env), DatabaseStrategy sessions, current-user dependencies. - schemas.py: user + subscription + notification Pydantic models. - app.py: mount auth/register/users routers on v2, create tables at startup. - Pin fastapi-users[sqlalchemy]/aiosqlite; ignore data/accounts.sqlite*. * Add account header entry and auth modal (frontend) account.js self-injects a header entry (following the units.js pattern) that shows a Sign in button when logged out and an account menu when logged in, plus an auth modal reusing the existing .mp-overlay/.mp-modal chrome for email/password sign-in and account creation. A shared apiFetch helper sends the same-origin cookie for authed calls; exported getUser/openAuth/onAuthChange back later phases. Imported by every page entry module. On narrow screens the entry collapses to an icon-only button so it doesn't crowd the title. Enforce an 8-character minimum password in the user manager. * Add subscription CRUD API and the alerts management page Backend api_accounts.py adds user-scoped, cookie-authenticated endpoints to create/list/update/delete subscriptions (and the notification reads used next): POST snaps lat/lon to a grid cell, resolves a label, and rejects a duplicate location+kind with 409; PATCH/DELETE are ownership-checked (404 on mismatch). Mounted on the v2 prefix. Frontend subscriptions.js + subscriptions.html serve the /alerts page: a sign-in gate when logged out, an add flow that reuses the shared map picker and an editor modal (kind, watched metrics, 95-99 percentile, two-sided), and a card list with inline threshold/active edits and remove. Reachable from the account menu. * Add background subscription evaluation engine notify.py runs a daemon thread that periodically evaluates every active subscription: it groups them by grid cell, reads history from the parquet cache only (never spends archive quota) plus the hourly recent/forecast bundle, and grades candidate days with the existing grading.grade_day. A watched metric that lands at or beyond the threshold percentile fires a 'high' alert; a two-sided subscription also fires 'low' for the symmetric cold/calm/dry tail (precip stays one-directional). Observed subscriptions look at the last few recorded days, forecast subscriptions at the coming week. Two guards keep it quiet: a UNIQUE(subscription, event_date, metric, direction, kind) constraint dedups repeat events, and a per-subscription weekly cap (last_notified_at) limits each alert to one notification per 7 days. The loop tolerates a bad cell or an upstream rate limit without aborting the pass. Started and stopped from the app lifespan; gated by THERMOGRAPH_ENABLE_NOTIFIER. * Add in-app notification center (header bell) Extend account.js with a notification bell beside the account menu: an unread badge, a dropdown listing recent notifications (title, body, relative time), a per-item mark-read on click, and a Mark all read action, all through the cookie-authed notifications API. Unread state refreshes on open and polls every two minutes while signed in; polling stops on sign-out. Styled to match the app, responsive down to mobile. * Harden accounts: expired-session cleanup, engine tests, ops docs - notify.py sweeps expired login sessions (access tokens past their lifetime) once per evaluation pass. - Add hermetic unit tests for the evaluation engine's trigger detection (high/low tails, precip one-directional, normal = no trigger) and notification wording. - Document accounts.sqlite (authoritative, back it up), the single-worker requirement for the in-process evaluator, and the new env vars in DEPLOY.md.
2026-07-15 18:46:46 +00:00
"""ORM tables for the account domain (data/accounts.sqlite).
``User`` / ``AccessToken`` are fastapi-users' base tables (UUID primary keys);
the access-token table backs a database session strategy, so logins survive a
process restart and are individually revocable. ``Subscription`` and
``Notification`` are our own, keyed to a user and cascading on delete.
"""
import time
import uuid
from fastapi_users_db_sqlalchemy import SQLAlchemyBaseUserTableUUID
from fastapi_users_db_sqlalchemy.access_token import SQLAlchemyBaseAccessTokenTableUUID
from fastapi_users_db_sqlalchemy.generics import GUID
from sqlalchemy import (
JSON,
Boolean,
CheckConstraint,
Float,
ForeignKey,
Index,
Integer,
String,
Text,
UniqueConstraint,
)
from sqlalchemy.orm import Mapped, mapped_column
from db import Base
class User(SQLAlchemyBaseUserTableUUID, Base):
# Inherits id (UUID), email (unique), hashed_password, is_active,
# is_superuser, is_verified. Optional extras:
Account system with weather-notification subscriptions (#89) * Add account system foundation: email/password auth with cookie sessions Introduce the app's first authoritative, user-owned data in a separate data/accounts.sqlite (SQLAlchemy), kept apart from the disposable derived-cache DB. Wire fastapi-users for email/password signup, cookie-based login/logout, and a session-check endpoint, backed by a database session strategy so logins survive restarts and are revocable. - db.py: async (aiosqlite) + sync SQLAlchemy engines over accounts.sqlite, WAL + foreign keys, create_db_and_tables(). - models.py: User, AccessToken, Subscription, Notification tables. - users.py: pwdlib hashing, HttpOnly cookie transport (path-scoped, SameSite=Lax, Secure via env), DatabaseStrategy sessions, current-user dependencies. - schemas.py: user + subscription + notification Pydantic models. - app.py: mount auth/register/users routers on v2, create tables at startup. - Pin fastapi-users[sqlalchemy]/aiosqlite; ignore data/accounts.sqlite*. * Add account header entry and auth modal (frontend) account.js self-injects a header entry (following the units.js pattern) that shows a Sign in button when logged out and an account menu when logged in, plus an auth modal reusing the existing .mp-overlay/.mp-modal chrome for email/password sign-in and account creation. A shared apiFetch helper sends the same-origin cookie for authed calls; exported getUser/openAuth/onAuthChange back later phases. Imported by every page entry module. On narrow screens the entry collapses to an icon-only button so it doesn't crowd the title. Enforce an 8-character minimum password in the user manager. * Add subscription CRUD API and the alerts management page Backend api_accounts.py adds user-scoped, cookie-authenticated endpoints to create/list/update/delete subscriptions (and the notification reads used next): POST snaps lat/lon to a grid cell, resolves a label, and rejects a duplicate location+kind with 409; PATCH/DELETE are ownership-checked (404 on mismatch). Mounted on the v2 prefix. Frontend subscriptions.js + subscriptions.html serve the /alerts page: a sign-in gate when logged out, an add flow that reuses the shared map picker and an editor modal (kind, watched metrics, 95-99 percentile, two-sided), and a card list with inline threshold/active edits and remove. Reachable from the account menu. * Add background subscription evaluation engine notify.py runs a daemon thread that periodically evaluates every active subscription: it groups them by grid cell, reads history from the parquet cache only (never spends archive quota) plus the hourly recent/forecast bundle, and grades candidate days with the existing grading.grade_day. A watched metric that lands at or beyond the threshold percentile fires a 'high' alert; a two-sided subscription also fires 'low' for the symmetric cold/calm/dry tail (precip stays one-directional). Observed subscriptions look at the last few recorded days, forecast subscriptions at the coming week. Two guards keep it quiet: a UNIQUE(subscription, event_date, metric, direction, kind) constraint dedups repeat events, and a per-subscription weekly cap (last_notified_at) limits each alert to one notification per 7 days. The loop tolerates a bad cell or an upstream rate limit without aborting the pass. Started and stopped from the app lifespan; gated by THERMOGRAPH_ENABLE_NOTIFIER. * Add in-app notification center (header bell) Extend account.js with a notification bell beside the account menu: an unread badge, a dropdown listing recent notifications (title, body, relative time), a per-item mark-read on click, and a Mark all read action, all through the cookie-authed notifications API. Unread state refreshes on open and polls every two minutes while signed in; polling stops on sign-out. Styled to match the app, responsive down to mobile. * Harden accounts: expired-session cleanup, engine tests, ops docs - notify.py sweeps expired login sessions (access tokens past their lifetime) once per evaluation pass. - Add hermetic unit tests for the evaluation engine's trigger detection (high/low tails, precip one-directional, normal = no trigger) and notification wording. - Document accounts.sqlite (authoritative, back it up), the single-worker requirement for the in-process evaluator, and the new env vars in DEPLOY.md.
2026-07-15 18:46:46 +00:00
display_name: Mapped[str | None] = mapped_column(String(120), nullable=True)
# Linked Discord account id (OAuth2 identify) — the key DM alerts reach the user
# by. NB: create_all only makes this on a fresh DB; an existing prod accounts.db
# needs the manual migration in deploy/migrations/ (no Alembic in this project).
discord_id: Mapped[str | None] = mapped_column(String(32), unique=True, nullable=True)
Account system with weather-notification subscriptions (#89) * Add account system foundation: email/password auth with cookie sessions Introduce the app's first authoritative, user-owned data in a separate data/accounts.sqlite (SQLAlchemy), kept apart from the disposable derived-cache DB. Wire fastapi-users for email/password signup, cookie-based login/logout, and a session-check endpoint, backed by a database session strategy so logins survive restarts and are revocable. - db.py: async (aiosqlite) + sync SQLAlchemy engines over accounts.sqlite, WAL + foreign keys, create_db_and_tables(). - models.py: User, AccessToken, Subscription, Notification tables. - users.py: pwdlib hashing, HttpOnly cookie transport (path-scoped, SameSite=Lax, Secure via env), DatabaseStrategy sessions, current-user dependencies. - schemas.py: user + subscription + notification Pydantic models. - app.py: mount auth/register/users routers on v2, create tables at startup. - Pin fastapi-users[sqlalchemy]/aiosqlite; ignore data/accounts.sqlite*. * Add account header entry and auth modal (frontend) account.js self-injects a header entry (following the units.js pattern) that shows a Sign in button when logged out and an account menu when logged in, plus an auth modal reusing the existing .mp-overlay/.mp-modal chrome for email/password sign-in and account creation. A shared apiFetch helper sends the same-origin cookie for authed calls; exported getUser/openAuth/onAuthChange back later phases. Imported by every page entry module. On narrow screens the entry collapses to an icon-only button so it doesn't crowd the title. Enforce an 8-character minimum password in the user manager. * Add subscription CRUD API and the alerts management page Backend api_accounts.py adds user-scoped, cookie-authenticated endpoints to create/list/update/delete subscriptions (and the notification reads used next): POST snaps lat/lon to a grid cell, resolves a label, and rejects a duplicate location+kind with 409; PATCH/DELETE are ownership-checked (404 on mismatch). Mounted on the v2 prefix. Frontend subscriptions.js + subscriptions.html serve the /alerts page: a sign-in gate when logged out, an add flow that reuses the shared map picker and an editor modal (kind, watched metrics, 95-99 percentile, two-sided), and a card list with inline threshold/active edits and remove. Reachable from the account menu. * Add background subscription evaluation engine notify.py runs a daemon thread that periodically evaluates every active subscription: it groups them by grid cell, reads history from the parquet cache only (never spends archive quota) plus the hourly recent/forecast bundle, and grades candidate days with the existing grading.grade_day. A watched metric that lands at or beyond the threshold percentile fires a 'high' alert; a two-sided subscription also fires 'low' for the symmetric cold/calm/dry tail (precip stays one-directional). Observed subscriptions look at the last few recorded days, forecast subscriptions at the coming week. Two guards keep it quiet: a UNIQUE(subscription, event_date, metric, direction, kind) constraint dedups repeat events, and a per-subscription weekly cap (last_notified_at) limits each alert to one notification per 7 days. The loop tolerates a bad cell or an upstream rate limit without aborting the pass. Started and stopped from the app lifespan; gated by THERMOGRAPH_ENABLE_NOTIFIER. * Add in-app notification center (header bell) Extend account.js with a notification bell beside the account menu: an unread badge, a dropdown listing recent notifications (title, body, relative time), a per-item mark-read on click, and a Mark all read action, all through the cookie-authed notifications API. Unread state refreshes on open and polls every two minutes while signed in; polling stops on sign-out. Styled to match the app, responsive down to mobile. * Harden accounts: expired-session cleanup, engine tests, ops docs - notify.py sweeps expired login sessions (access tokens past their lifetime) once per evaluation pass. - Add hermetic unit tests for the evaluation engine's trigger detection (high/low tails, precip one-directional, normal = no trigger) and notification wording. - Document accounts.sqlite (authoritative, back it up), the single-worker requirement for the in-process evaluator, and the new env vars in DEPLOY.md.
2026-07-15 18:46:46 +00:00
class AccessToken(SQLAlchemyBaseAccessTokenTableUUID, Base):
# Inherits token (PK), user_id (FK -> user.id), created_at. Rows here ARE the
# sessions: DatabaseStrategy looks a cookie's token up in this table.
pass
class Subscription(Base):
__tablename__ = "subscription"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
user_id: Mapped[uuid.UUID] = mapped_column(
GUID, ForeignKey("user.id", ondelete="CASCADE"), nullable=False
)
# grid.snap(lat, lon)["id"] — the stable per-location key used across the app.
cell_id: Mapped[str] = mapped_column(String(40), nullable=False)
label: Mapped[str | None] = mapped_column(String(200), nullable=True)
lat: Mapped[float] = mapped_column(Float, nullable=False)
lon: Mapped[float] = mapped_column(Float, nullable=False)
# Unusualness cutoff the user picked; the low tail mirrors it at 100-threshold.
threshold: Mapped[int] = mapped_column(Integer, nullable=False)
# Grading metric keys this subscription watches, e.g. ["tmax", "feels", "precip"].
metrics: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
# 'observed' (a recorded day crossed) or 'forecast' (an upcoming day is projected to).
kind: Mapped[str] = mapped_column(String(16), nullable=False, default="observed")
# Also alert the cold/low tail for temperature-like metrics (precip stays one-sided).
two_sided: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
# Epoch seconds of the last notification emitted — enforces the weekly cap.
last_notified_at: Mapped[float | None] = mapped_column(Float, nullable=True)
created_at: Mapped[float] = mapped_column(Float, nullable=False, default=time.time)
__table_args__ = (
CheckConstraint("threshold BETWEEN 95 AND 99", name="ck_sub_threshold"),
CheckConstraint("kind IN ('observed','forecast')", name="ck_sub_kind"),
# One observed + one forecast subscription per location per user.
UniqueConstraint("user_id", "cell_id", "kind", name="uq_sub_user_cell_kind"),
Index("idx_sub_user", "user_id"),
Index("idx_sub_active", "active"),
)
class Notification(Base):
__tablename__ = "notification"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
user_id: Mapped[uuid.UUID] = mapped_column(
GUID, ForeignKey("user.id", ondelete="CASCADE"), nullable=False
)
subscription_id: Mapped[int] = mapped_column(
Integer, ForeignKey("subscription.id", ondelete="CASCADE"), nullable=False
)
event_date: Mapped[str] = mapped_column(String(10), nullable=False) # YYYY-MM-DD
metric: Mapped[str] = mapped_column(String(16), nullable=False)
direction: Mapped[str] = mapped_column(String(4), nullable=False) # 'high' | 'low'
kind: Mapped[str] = mapped_column(String(16), nullable=False, default="observed")
percentile: Mapped[float] = mapped_column(Float, nullable=False)
value: Mapped[float | None] = mapped_column(Float, nullable=True)
grade: Mapped[str | None] = mapped_column(String(40), nullable=True)
title: Mapped[str] = mapped_column(String(200), nullable=False)
body: Mapped[str | None] = mapped_column(Text, nullable=True)
# 'inapp' today; the seam for future 'email' / 'push' delivery.
channel: Mapped[str] = mapped_column(String(16), nullable=False, default="inapp")
created_at: Mapped[float] = mapped_column(Float, nullable=False, default=time.time)
read_at: Mapped[float | None] = mapped_column(Float, nullable=True) # NULL == unread
__table_args__ = (
# The dedup key: a given event (day+metric+direction+kind) notifies a
# subscription at most once, so re-running the evaluator never repeats it.
UniqueConstraint(
"subscription_id", "event_date", "metric", "direction", "kind",
name="uq_notif_event",
),
Index("idx_notif_user_created", "user_id", "created_at"),
Index("idx_notif_user_read", "user_id", "read_at"),
)
class PushSubscription(Base):
"""A single browser/device Web Push registration, owned by a user.
One row per device (a user with a phone + a laptop has two). The `endpoint`
is the push service URL the browser handed us; it's the natural identity, so
re-subscribing from the same device updates the keys in place rather than
duplicating. Rows are pruned when the push service reports the endpoint gone
(404/410) see notify.py / api_accounts.py.
"""
__tablename__ = "push_subscription"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
user_id: Mapped[uuid.UUID] = mapped_column(
GUID, ForeignKey("user.id", ondelete="CASCADE"), nullable=False
)
# The push service URL (per-device). Unique — it identifies the device.
endpoint: Mapped[str] = mapped_column(Text, nullable=False)
# The two client keys from PushSubscription.toJSON().keys, used to encrypt the
# payload so only this device can read it.
p256dh: Mapped[str] = mapped_column(String(200), nullable=False)
auth: Mapped[str] = mapped_column(String(100), nullable=False)
# Best-effort label for a future "manage devices" view.
user_agent: Mapped[str | None] = mapped_column(String(300), nullable=True)
created_at: Mapped[float] = mapped_column(Float, nullable=False, default=time.time)
last_used_at: Mapped[float | None] = mapped_column(Float, nullable=True)
__table_args__ = (
UniqueConstraint("endpoint", name="uq_push_endpoint"),
Index("idx_push_user", "user_id"),
)
Rebuild the homepage as a distribution landing; add an SMTP seam (#178) The homepage was a bare tool: a find-bar and an empty panel reading "Find a location to begin." A visitor arriving from a search result or a shared link learned nothing about what the product does before deciding to leave. Rebuild it around the Weekly view, which is untouched: - Hero with the question as the h1, and a grade card showing a real graded example — the most unusual city we're currently tracking, either tail. The card's frame and text slots are server-rendered with reserved heights, so app.js re-pointing it at the visitor's own place shifts nothing. - "Unusual right now" strip, CSS scroll-snap, no JS carousel. A cold-tail city is force-included whenever one qualifies. - Stance line, how-it-works, explore cards, and 12 city chips linking into the ~1000-page /climate surface. - Monthly digest form, in the footer of every page. Serve / from Jinja instead of a static file with placeholder substitution, so crawlers and no-JS readers get the whole page as real HTML. home.html.j2 extends base.html.j2 and carries app.js's DOM contract over verbatim; the brand degrades to a <p> so the hero owns the sole h1. frontend/index.html is deleted rather than left behind the static mount, where it would keep serving indexable duplicate content. "Where is it most unusual right now" has no cheap answer at request time — percentiles live inside zlib-compressed payload blobs with no column to sort on. So homepage.py sweeps the warm cache and writes data/homepage.json, read by the template. The sweep is strictly cache-only (climate.load_cached_recent_forecast is new, the sibling of load_cached_history), so grading ~1000 cities costs zero upstream requests. It rides the notifier's timer behind an hourly guard rather than starting a second daemon, and also runs at the tail of warm_cities so a fresh deploy has a populated feed. Instrumentation: metrics gains a product-event counter keyed by (event, referrer domain, UTC day), behind an allowlist and a per-IP rate limit, fed by POST /api/v2/event. The referrer is taken from the request's own header, never from the client. The beacon gets its own inbound category that record_inbound ignores, so reporting an interaction doesn't also count as traffic. The dashboard grows an events block with per-referrer attribution. Email is scaffolded but sends nothing yet. mailer.py talks stdlib smtplib to a local Postfix null client on 127.0.0.1:25 (deploy/provision-mail.sh), so the choice between direct-to-MX and relaying through a provider stays a Postfix config change with no code change. The backend defaults to "console", which logs and sends nothing, so dev and tests exercise the whole signup path safely. Signups land in pending_digest unconfirmed; collecting the list shouldn't wait on delivery. Also adds /privacy, linked from the footer and kept out of the sitemap. The strip's classes are named unusual-* rather than record-*: .record-card is already the SEO records page's, and reusing it leaked layout rules onto /climate/<slug>/records.
2026-07-18 07:39:47 +00:00
class PendingDigest(Base):
"""A monthly-digest signup, collected before email delivery is wired up.
The digest form ships ahead of SMTP on purpose: building the list is the
slow part, and making people wait for the mailer would throw away every
signup in the meantime. Rows land here unconfirmed; once delivery is live, a
confirmation pass mails each address and stamps ``confirmed_at``.
Deliberately NOT tied to ``user`` signing up for the digest must not
require an account, and most subscribers won't have one.
"""
__tablename__ = "pending_digest"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# 320 = the practical maximum length of an email address (64 local + @ + 255 domain).
email: Mapped[str] = mapped_column(String(320), nullable=False)
# The place the digest should cover. Optional: an address with no place is
# still a real signup, and the place can be asked for at confirmation time.
place_label: Mapped[str | None] = mapped_column(String(200), nullable=True)
lat: Mapped[float | None] = mapped_column(Float, nullable=True)
lon: Mapped[float | None] = mapped_column(Float, nullable=True)
cell_id: Mapped[str | None] = mapped_column(String(32), nullable=True)
# Where the signup came from (bare referrer domain), for attribution only.
source: Mapped[str | None] = mapped_column(String(64), nullable=True)
created_at: Mapped[float] = mapped_column(Float, nullable=False, default=time.time)
# Set when the address is verified. The double opt-in token is stored as a
# sha256 hash, never in the clear, so a leaked database can't confirm addresses.
token_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
confirmed_at: Mapped[float | None] = mapped_column(Float, nullable=True)
last_sent_at: Mapped[float | None] = mapped_column(Float, nullable=True)
unsubscribed_at: Mapped[float | None] = mapped_column(Float, nullable=True)
__table_args__ = (
# One row per address: a re-submit updates in place rather than
# duplicating, which is what makes the form idempotent.
UniqueConstraint("email", name="uq_pending_digest_email"),
)