thermograph/backend/tests/conftest.py

190 lines
7.6 KiB
Python
Raw Normal View History

"""Shared test setup: import path, offline guards, and synthetic weather data.
Everything here keeps the suite hermetic no Open-Meteo, no Nominatim, no
GeoNames download, and no writes into the repo's data/ or logs/ folders.
"""
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
import datetime
import os
import sys
import tempfile
import threading
import numpy as np
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
import polars as pl
import pytest
BACKEND = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, BACKEND)
Split the backend into domain packages (#217) * Centralize filesystem paths in a single module Add paths.py, which resolves the repo root once and derives the cache, accounts DB, logs, templates, frontend and bundled-city-data locations from it. Replace the 13 per-module `dirname(__file__)/..` anchors with references to it, so a module's location no longer determines where the app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are unchanged; every resolved path is byte-identical to before. Groundwork for moving modules into packages without re-pointing paths. Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62 * Split the backend into domain packages Group the flat backend modules into packages that mirror their concerns: data/ climate, grading, scoring, grid, places, cities, city_events, store web/ app, views, homepage, content, schemas notifications/ notify, digest, push, mailer, discord, discord_interactions, discord_link accounts/ models, users, api_accounts, db core/ metrics, singleton, audit Intra-project imports are rewritten to the package-qualified form. The entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor) and paths.py stay at the backend/ root, and backend/app.py becomes a shim re-exporting web.app:app so the launch target stays `app:app` — run.sh, the systemd units, and CI need no change. Verified: full suite (318) passes, `uvicorn app:app` boots and serves the home/SEO/static/API surfaces, and every root script imports clean. Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
2026-07-20 05:31:03 +00:00
from data import places # noqa: E402
# app.py calls places.start_loading() at import; pretend it already ran so no
# background GeoNames download starts. Tests build their own tiny index.
places._load_started = True
_TMP = tempfile.mkdtemp(prefix="thermograph-tests-")
# Keep the authoritative accounts DB out of the repo's data/ folder, and don't
# start the background notifier thread during tests. Set before db is imported.
os.environ.setdefault("THERMOGRAPH_ACCOUNTS_DB", os.path.join(_TMP, "accounts.sqlite"))
os.environ.setdefault("THERMOGRAPH_ENABLE_NOTIFIER", "0")
os.environ.setdefault("THERMOGRAPH_ENABLE_HEARTBEAT", "0")
# web/app.py (repo-split Stage 4) fails loud at import if this is unset -- tests
# never actually hit a frontend-owned path through the live proxy (that's
# frontend_ssr/tests' job), so an unreachable placeholder is fine here.
os.environ.setdefault("THERMOGRAPH_FRONTEND_BASE_INTERNAL", "http://127.0.0.1:1")
# Pin the IndexNow key so it isn't generated + written into the repo's data/ dir.
os.environ.setdefault("THERMOGRAPH_INDEXNOW_KEY", "test0000indexnow0000key000000000")
Split the backend into domain packages (#217) * Centralize filesystem paths in a single module Add paths.py, which resolves the repo root once and derives the cache, accounts DB, logs, templates, frontend and bundled-city-data locations from it. Replace the 13 per-module `dirname(__file__)/..` anchors with references to it, so a module's location no longer determines where the app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are unchanged; every resolved path is byte-identical to before. Groundwork for moving modules into packages without re-pointing paths. Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62 * Split the backend into domain packages Group the flat backend modules into packages that mirror their concerns: data/ climate, grading, scoring, grid, places, cities, city_events, store web/ app, views, homepage, content, schemas notifications/ notify, digest, push, mailer, discord, discord_interactions, discord_link accounts/ models, users, api_accounts, db core/ metrics, singleton, audit Intra-project imports are rewritten to the package-qualified form. The entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor) and paths.py stay at the backend/ root, and backend/app.py becomes a shim re-exporting web.app:app so the launch target stays `app:app` — run.sh, the systemd units, and CI need no change. Verified: full suite (318) passes, `uvicorn app:app` boots and serves the home/SEO/static/API surfaces, and every root script imports clean. Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
2026-07-20 05:31:03 +00:00
from data import store # noqa: E402
# Keep derived-store writes out of the repo's data/ folder. Individual store
# tests re-point this per test; everything else shares one throwaway DB.
store.DB_PATH = os.path.join(_TMP, "store.sqlite")
Split the backend into domain packages (#217) * Centralize filesystem paths in a single module Add paths.py, which resolves the repo root once and derives the cache, accounts DB, logs, templates, frontend and bundled-city-data locations from it. Replace the 13 per-module `dirname(__file__)/..` anchors with references to it, so a module's location no longer determines where the app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are unchanged; every resolved path is byte-identical to before. Groundwork for moving modules into packages without re-pointing paths. Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62 * Split the backend into domain packages Group the flat backend modules into packages that mirror their concerns: data/ climate, grading, scoring, grid, places, cities, city_events, store web/ app, views, homepage, content, schemas notifications/ notify, digest, push, mailer, discord, discord_interactions, discord_link accounts/ models, users, api_accounts, db core/ metrics, singleton, audit Intra-project imports are rewritten to the package-qualified form. The entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor) and paths.py stay at the backend/ root, and backend/app.py becomes a shim re-exporting web.app:app so the launch target stays `app:app` — run.sh, the systemd units, and CI need no change. Verified: full suite (318) passes, `uvicorn app:app` boots and serves the home/SEO/static/API surfaces, and every root script imports clean. Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
2026-07-20 05:31:03 +00:00
from core import audit # noqa: E402
audit.AUDIT_DIR = os.path.join(_TMP, "logs", "audit")
audit.ERROR_DIR = os.path.join(_TMP, "logs", "errors")
# The request-logging middleware runs under TestClient too; keep its access log
# out of the repo's logs/ so self-test traffic never pollutes the live monitor.
audit.ACCESS_DIR = os.path.join(_TMP, "logs", "access")
audit.ACTIVITY_DIR = os.path.join(_TMP, "logs", "activity")
audit.HEARTBEAT_DIR = os.path.join(_TMP, "logs", "heartbeat")
tests: hard-block outbound transports, and assert the block holds A test run could message real subscribers, and would have reported green while doing it. Every send gate in notifications/ reads its config from ambient env at import time, and conftest neutralised none of it -- it set four env vars, none touching Discord, SMTP or VAPID. test_notify.py calls run_pass() seven times without patching notifications.discord, and notify.py:358 reaches discord.post_subscription_alert() for every notification it creates. The subscription channel notifies real people. The failure mode is silent: notify.py's send path and discord.py's _bot_post swallow every exception, so a real send raises nothing and no assertion goes red. An operator who had sourced /etc/thermograph.env to debug -- routine -- turned `pytest tests/notifications` into a live broadcast. CI was safe only by omission: it runs in a container with no credentials baked in, and one added -e removed that. Two layers: 1. Config blanked, so every enabled()/dm_enabled()/subscription_enabled() gate reports False. This is the state CI already runs in, now guaranteed locally instead of depending on the shell. 2. The transports themselves replaced with sentinels that raise. A test that forgets to patch one fails loudly rather than sending. Tests that need to exercise a send path patch these themselves, which is the point: the escape hatch becomes explicit in the test that takes it. The idiom is lifted from tests/test_warm_content.py, which already did this for one call. test_send_safety.py asserts the guarantee, because a block like this fails in only one direction -- weaken it and every other test still passes, the sole symptom being a message delivered to somebody. Full suite: 457 passed, 8 skipped. Nothing relied on a live transport.
2026-07-25 08:36:12 +00:00
# --- outbound transports: hard-blocked, not merely unconfigured ---------------
#
# Every send gate in notifications/ is read from ambient env AT IMPORT TIME, and
# every send site swallows its exceptions (notify.py's _flush_sends,
# discord.py's _bot_post). So before this block existed, an operator who had
# sourced /etc/thermograph.env in the shell — routine when debugging — turned
# `pytest tests/notifications` into a live broadcast to real subscribers, and it
# reported GREEN because the exceptions never surfaced. test_notify.py alone
# calls run_pass() seven times without patching notifications.discord.
#
# CI was safe only by omission: it runs the suite in a container with no
# credentials baked in. One added `-e` removed that.
#
# Two layers, deliberately:
# 1. Blank the config so every enabled()/dm_enabled()/subscription_enabled()
# gate reports False — the same state CI runs in, now guaranteed locally.
# 2. Replace the actual transports with sentinels that RAISE. A test that
# forgets to patch one now fails loudly instead of sending silently.
#
# Tests that need to exercise a send path patch these themselves, which is the
# point: the escape hatch is explicit and visible in the test that takes it.
# The idiom is borrowed from tests/test_warm_content.py.
import smtplib # noqa: E402
import indexnow # noqa: E402
from notifications import discord, mailer, push # noqa: E402
class OutboundBlocked(AssertionError):
"""Raised when a test reaches a real outbound transport.
Seeing this is not a flaky test it means the code under test tried to
contact Discord, an SMTP host, a push endpoint or IndexNow for real. Patch
the transport in the test, or assert the no-send path.
"""
def _blocked(what):
def _raise(*a, **kw):
raise OutboundBlocked(
f"test tried to reach {what} for real. Patch it in the test "
f"(see tests/conftest.py's outbound-transport block).")
return _raise
# 1. Config blanked: every gate reports False.
discord.BOT_TOKEN = ""
discord.WEBHOOK_URL = ""
discord.SUBSCRIPTION_CHANNEL_ID = ""
discord.WEATHER_CHANNEL_ID = ""
mailer.BACKEND = "console"
# 2. Transports replaced with raisers.
discord._client.post = _blocked("Discord")
discord._client.patch = _blocked("Discord")
discord._client.put = _blocked("Discord")
discord._client.delete = _blocked("Discord")
indexnow._client.post = _blocked("IndexNow")
push.webpush = _blocked("a Web Push endpoint")
smtplib.SMTP = _blocked("an SMTP host")
smtplib.SMTP_SSL = _blocked("an SMTP host")
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
def _years_before(d: datetime.date, years: int) -> datetime.date:
"""`d` shifted back `years` years, same month/day (Feb 29 → Feb 28)."""
try:
return d.replace(year=d.year - years)
except ValueError:
return d.replace(year=d.year - years, day=28)
def _daily(start: datetime.date, end: datetime.date) -> list[datetime.date]:
"""Inclusive daily date list from `start` to `end`."""
return [start + datetime.timedelta(days=i) for i in range((end - start).days + 1)]
def _with_doy(df: pl.DataFrame) -> pl.DataFrame:
return df.with_columns(pl.col("date").dt.ordinal_day().cast(pl.Int16).alias("doy"))
def make_history(years: int = 20, end: str | None = None, seed: int = 7) -> pl.DataFrame:
"""A plausible daily record (tmax/tmin/precip + doy), matching the columns
and dtypes climate.get_history returns for an older cache."""
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
end_ts = (datetime.date.fromisoformat(end) if end
else datetime.date.today() - datetime.timedelta(days=6))
dates = _daily(_years_before(end_ts, years), end_ts)
rng = np.random.default_rng(seed)
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
doy = np.array([d.timetuple().tm_yday for d in dates])
seasonal = 55 + 30 * np.sin((doy - 100) / 366.0 * 2 * np.pi)
tmax = seasonal + rng.normal(0, 8, len(dates))
tmin = tmax - 15 + rng.normal(0, 3, len(dates))
precip = np.where(rng.random(len(dates)) < 0.3, rng.gamma(1.5, 0.2, len(dates)), 0.0)
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
return _with_doy(pl.DataFrame({
"date": dates,
"tmax": np.round(tmax, 1),
"tmin": np.round(tmin, 1),
"precip": np.round(precip, 2),
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
}))
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
def make_recent(history: pl.DataFrame, future_days: int = 7, seed: int = 11) -> pl.DataFrame:
"""A recent+forecast bundle: from a couple of weeks before the archive's end
through `future_days` past today the shape climate.get_recent_forecast returns."""
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
today = datetime.date.today()
start = history["date"].max() - datetime.timedelta(days=14)
dates = _daily(start, today + datetime.timedelta(days=future_days))
rng = np.random.default_rng(seed)
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
doy = np.array([d.timetuple().tm_yday for d in dates])
seasonal = 55 + 30 * np.sin((doy - 100) / 366.0 * 2 * np.pi)
tmax = seasonal + rng.normal(0, 8, len(dates))
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
return _with_doy(pl.DataFrame({
"date": dates,
"tmax": np.round(tmax, 1),
"tmin": np.round(tmax - 15, 1),
"precip": np.where(rng.random(len(dates)) < 0.3, 0.15, 0.0),
Migrate backend dataframe layer from pandas to polars (#90) * Migrate backend dataframe layer from pandas to polars Replace pandas with polars across the backend, dropping both pandas and its pyarrow parquet engine from the dependency set. numpy stays (the grading percentile math is unchanged). - climate.py: parquet IO, source→frame mappings, cache read/topup on polars. New _normalize_read casts the cached `date` column to pl.Date (older files were written by pandas as datetime64[ns]); frames now unify missing values as null so the grading boundary drops them consistently across sources. - grading.py: keep the numpy percentile core; swap the frame→numpy bridge to .to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the per-row loop to iter_rows(named=True). - views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat; scalar dates are stdlib datetime.date; a local _months_before helper replaces DateOffset(months=) for the calendar-range default. - app.py, migrate.py: request-date parsing uses datetime.date, removing pandas from the endpoint and migrate layers entirely. - The date column is pl.Date end to end, eliminating the pandas normalize() calls and comparing cleanly against stdlib dates. Payloads are unchanged: calendar, day, grade and forecast responses are byte-for-byte identical to the pandas implementation on the same cached record. Tests ported to polars fixtures, with added coverage for the combined feels-like fallback, calendar month-offset (month-end/leap), and the concat/dedup "fresher source wins" rule. * Port notify.py to polars after merging dev's account system Merge origin/dev (accounts + notification subscriptions) and carry the pandas→ polars migration into the newly added notify.py, which the merge brought in still using pandas — with pandas removed from requirements this broke its import. - notify.py: _candidate_rows filters/sorts the recent bundle with polars expressions and returns iter_rows dicts; date scalars are datetime.date; history/recent emptiness via is_empty(). - test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
}))
@pytest.fixture(scope="session")
def history():
return make_history()
@pytest.fixture(scope="session")
def recent(history):
return make_recent(history)
@pytest.fixture
def tmp_store(tmp_path, monkeypatch):
"""store.py against a fresh database file, isolated per test."""
monkeypatch.setattr(store, "DB_PATH", str(tmp_path / "store.sqlite"))
monkeypatch.setattr(store, "_local", threading.local())
return store