Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
"""Shared test setup: import path, offline guards, and synthetic weather data.
|
|
|
|
|
|
|
|
|
|
Everything here keeps the suite hermetic — no Open-Meteo, no Nominatim, no
|
|
|
|
|
GeoNames download, and no writes into the repo's data/ or logs/ folders.
|
|
|
|
|
"""
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
import datetime
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
import os
|
|
|
|
|
import sys
|
|
|
|
|
import tempfile
|
|
|
|
|
import threading
|
|
|
|
|
|
|
|
|
|
import numpy as np
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
import polars as pl
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
import pytest
|
|
|
|
|
|
|
|
|
|
BACKEND = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
|
|
|
sys.path.insert(0, BACKEND)
|
|
|
|
|
|
Split the backend into domain packages (#217)
* Centralize filesystem paths in a single module
Add paths.py, which resolves the repo root once and derives the cache,
accounts DB, logs, templates, frontend and bundled-city-data locations
from it. Replace the 13 per-module `dirname(__file__)/..` anchors with
references to it, so a module's location no longer determines where the
app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are
unchanged; every resolved path is byte-identical to before.
Groundwork for moving modules into packages without re-pointing paths.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
* Split the backend into domain packages
Group the flat backend modules into packages that mirror their concerns:
data/ climate, grading, scoring, grid, places, cities,
city_events, store
web/ app, views, homepage, content, schemas
notifications/ notify, digest, push, mailer, discord,
discord_interactions, discord_link
accounts/ models, users, api_accounts, db
core/ metrics, singleton, audit
Intra-project imports are rewritten to the package-qualified form. The
entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor)
and paths.py stay at the backend/ root, and backend/app.py becomes a
shim re-exporting web.app:app so the launch target stays `app:app` —
run.sh, the systemd units, and CI need no change.
Verified: full suite (318) passes, `uvicorn app:app` boots and serves
the home/SEO/static/API surfaces, and every root script imports clean.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
2026-07-20 05:31:03 +00:00
|
|
|
from data import places # noqa: E402
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
|
|
|
|
|
# app.py calls places.start_loading() at import; pretend it already ran so no
|
|
|
|
|
# background GeoNames download starts. Tests build their own tiny index.
|
|
|
|
|
places._load_started = True
|
|
|
|
|
|
|
|
|
|
_TMP = tempfile.mkdtemp(prefix="thermograph-tests-")
|
|
|
|
|
|
2026-07-15 22:00:47 +00:00
|
|
|
# Keep the authoritative accounts DB out of the repo's data/ folder, and don't
|
|
|
|
|
# start the background notifier thread during tests. Set before db is imported.
|
|
|
|
|
os.environ.setdefault("THERMOGRAPH_ACCOUNTS_DB", os.path.join(_TMP, "accounts.sqlite"))
|
|
|
|
|
os.environ.setdefault("THERMOGRAPH_ENABLE_NOTIFIER", "0")
|
2026-07-25 04:13:47 +00:00
|
|
|
os.environ.setdefault("THERMOGRAPH_ENABLE_HEARTBEAT", "0")
|
2026-07-21 20:01:30 +00:00
|
|
|
# web/app.py (repo-split Stage 4) fails loud at import if this is unset -- tests
|
|
|
|
|
# never actually hit a frontend-owned path through the live proxy (that's
|
|
|
|
|
# frontend_ssr/tests' job), so an unreachable placeholder is fine here.
|
|
|
|
|
os.environ.setdefault("THERMOGRAPH_FRONTEND_BASE_INTERNAL", "http://127.0.0.1:1")
|
2026-07-16 18:08:09 +00:00
|
|
|
# Pin the IndexNow key so it isn't generated + written into the repo's data/ dir.
|
|
|
|
|
os.environ.setdefault("THERMOGRAPH_INDEXNOW_KEY", "test0000indexnow0000key000000000")
|
2026-07-15 22:00:47 +00:00
|
|
|
|
Split the backend into domain packages (#217)
* Centralize filesystem paths in a single module
Add paths.py, which resolves the repo root once and derives the cache,
accounts DB, logs, templates, frontend and bundled-city-data locations
from it. Replace the 13 per-module `dirname(__file__)/..` anchors with
references to it, so a module's location no longer determines where the
app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are
unchanged; every resolved path is byte-identical to before.
Groundwork for moving modules into packages without re-pointing paths.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
* Split the backend into domain packages
Group the flat backend modules into packages that mirror their concerns:
data/ climate, grading, scoring, grid, places, cities,
city_events, store
web/ app, views, homepage, content, schemas
notifications/ notify, digest, push, mailer, discord,
discord_interactions, discord_link
accounts/ models, users, api_accounts, db
core/ metrics, singleton, audit
Intra-project imports are rewritten to the package-qualified form. The
entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor)
and paths.py stay at the backend/ root, and backend/app.py becomes a
shim re-exporting web.app:app so the launch target stays `app:app` —
run.sh, the systemd units, and CI need no change.
Verified: full suite (318) passes, `uvicorn app:app` boots and serves
the home/SEO/static/API surfaces, and every root script imports clean.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
2026-07-20 05:31:03 +00:00
|
|
|
from data import store # noqa: E402
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
|
|
|
|
|
# Keep derived-store writes out of the repo's data/ folder. Individual store
|
|
|
|
|
# tests re-point this per test; everything else shares one throwaway DB.
|
|
|
|
|
store.DB_PATH = os.path.join(_TMP, "store.sqlite")
|
|
|
|
|
|
Split the backend into domain packages (#217)
* Centralize filesystem paths in a single module
Add paths.py, which resolves the repo root once and derives the cache,
accounts DB, logs, templates, frontend and bundled-city-data locations
from it. Replace the 13 per-module `dirname(__file__)/..` anchors with
references to it, so a module's location no longer determines where the
app reads its data. Env overrides (accounts DB, VAPID, IndexNow) are
unchanged; every resolved path is byte-identical to before.
Groundwork for moving modules into packages without re-pointing paths.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
* Split the backend into domain packages
Group the flat backend modules into packages that mirror their concerns:
data/ climate, grading, scoring, grid, places, cities,
city_events, store
web/ app, views, homepage, content, schemas
notifications/ notify, digest, push, mailer, discord,
discord_interactions, discord_link
accounts/ models, users, api_accounts, db
core/ metrics, singleton, audit
Intra-project imports are rewritten to the package-qualified form. The
entry scripts (indexnow, warm_cities, migrate, gen_cities, gen_flavor)
and paths.py stay at the backend/ root, and backend/app.py becomes a
shim re-exporting web.app:app so the launch target stays `app:app` —
run.sh, the systemd units, and CI need no change.
Verified: full suite (318) passes, `uvicorn app:app` boots and serves
the home/SEO/static/API surfaces, and every root script imports clean.
Claude-Session: https://claude.ai/code/session_01XXxmNFy9cZ6Gh8Y9thZn62
2026-07-20 05:31:03 +00:00
|
|
|
from core import audit # noqa: E402
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
|
|
|
|
|
audit.AUDIT_DIR = os.path.join(_TMP, "logs", "audit")
|
|
|
|
|
audit.ERROR_DIR = os.path.join(_TMP, "logs", "errors")
|
2026-07-17 19:44:30 +00:00
|
|
|
# The request-logging middleware runs under TestClient too; keep its access log
|
|
|
|
|
# out of the repo's logs/ so self-test traffic never pollutes the live monitor.
|
|
|
|
|
audit.ACCESS_DIR = os.path.join(_TMP, "logs", "access")
|
2026-07-22 19:07:35 +00:00
|
|
|
audit.ACTIVITY_DIR = os.path.join(_TMP, "logs", "activity")
|
|
|
|
|
audit.HEARTBEAT_DIR = os.path.join(_TMP, "logs", "heartbeat")
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
|
|
|
|
|
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
def _years_before(d: datetime.date, years: int) -> datetime.date:
|
|
|
|
|
"""`d` shifted back `years` years, same month/day (Feb 29 → Feb 28)."""
|
|
|
|
|
try:
|
|
|
|
|
return d.replace(year=d.year - years)
|
|
|
|
|
except ValueError:
|
|
|
|
|
return d.replace(year=d.year - years, day=28)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _daily(start: datetime.date, end: datetime.date) -> list[datetime.date]:
|
|
|
|
|
"""Inclusive daily date list from `start` to `end`."""
|
|
|
|
|
return [start + datetime.timedelta(days=i) for i in range((end - start).days + 1)]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def _with_doy(df: pl.DataFrame) -> pl.DataFrame:
|
|
|
|
|
return df.with_columns(pl.col("date").dt.ordinal_day().cast(pl.Int16).alias("doy"))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def make_history(years: int = 20, end: str | None = None, seed: int = 7) -> pl.DataFrame:
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
"""A plausible daily record (tmax/tmin/precip + doy), matching the columns
|
|
|
|
|
and dtypes climate.get_history returns for an older cache."""
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
end_ts = (datetime.date.fromisoformat(end) if end
|
|
|
|
|
else datetime.date.today() - datetime.timedelta(days=6))
|
|
|
|
|
dates = _daily(_years_before(end_ts, years), end_ts)
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
rng = np.random.default_rng(seed)
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
doy = np.array([d.timetuple().tm_yday for d in dates])
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
seasonal = 55 + 30 * np.sin((doy - 100) / 366.0 * 2 * np.pi)
|
|
|
|
|
tmax = seasonal + rng.normal(0, 8, len(dates))
|
|
|
|
|
tmin = tmax - 15 + rng.normal(0, 3, len(dates))
|
|
|
|
|
precip = np.where(rng.random(len(dates)) < 0.3, rng.gamma(1.5, 0.2, len(dates)), 0.0)
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
return _with_doy(pl.DataFrame({
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
"date": dates,
|
|
|
|
|
"tmax": np.round(tmax, 1),
|
|
|
|
|
"tmin": np.round(tmin, 1),
|
|
|
|
|
"precip": np.round(precip, 2),
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
}))
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
|
|
|
|
|
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
def make_recent(history: pl.DataFrame, future_days: int = 7, seed: int = 11) -> pl.DataFrame:
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
"""A recent+forecast bundle: from a couple of weeks before the archive's end
|
|
|
|
|
through `future_days` past today — the shape climate.get_recent_forecast returns."""
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
today = datetime.date.today()
|
|
|
|
|
start = history["date"].max() - datetime.timedelta(days=14)
|
|
|
|
|
dates = _daily(start, today + datetime.timedelta(days=future_days))
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
rng = np.random.default_rng(seed)
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
doy = np.array([d.timetuple().tm_yday for d in dates])
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
seasonal = 55 + 30 * np.sin((doy - 100) / 366.0 * 2 * np.pi)
|
|
|
|
|
tmax = seasonal + rng.normal(0, 8, len(dates))
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
return _with_doy(pl.DataFrame({
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
"date": dates,
|
|
|
|
|
"tmax": np.round(tmax, 1),
|
|
|
|
|
"tmin": np.round(tmax - 15, 1),
|
|
|
|
|
"precip": np.where(rng.random(len(dates)) < 0.3, 0.15, 0.0),
|
Migrate backend dataframe layer from pandas to polars (#90)
* Migrate backend dataframe layer from pandas to polars
Replace pandas with polars across the backend, dropping both pandas and its
pyarrow parquet engine from the dependency set. numpy stays (the grading
percentile math is unchanged).
- climate.py: parquet IO, source→frame mappings, cache read/topup on polars.
New _normalize_read casts the cached `date` column to pl.Date (older files
were written by pandas as datetime64[ns]); frames now unify missing values as
null so the grading boundary drops them consistently across sources.
- grading.py: keep the numpy percentile core; swap the frame→numpy bridge to
.to_numpy()/.drop_nulls(), day-of-year/year to polars dt expressions, and the
per-row loop to iter_rows(named=True).
- views.py: filter/anti-join/concat replace boolean-mask, isin and pd.concat;
scalar dates are stdlib datetime.date; a local _months_before helper replaces
DateOffset(months=) for the calendar-range default.
- app.py, migrate.py: request-date parsing uses datetime.date, removing pandas
from the endpoint and migrate layers entirely.
- The date column is pl.Date end to end, eliminating the pandas normalize() calls
and comparing cleanly against stdlib dates.
Payloads are unchanged: calendar, day, grade and forecast responses are
byte-for-byte identical to the pandas implementation on the same cached record.
Tests ported to polars fixtures, with added coverage for the combined feels-like
fallback, calendar month-offset (month-end/leap), and the concat/dedup
"fresher source wins" rule.
* Port notify.py to polars after merging dev's account system
Merge origin/dev (accounts + notification subscriptions) and carry the pandas→
polars migration into the newly added notify.py, which the merge brought in still
using pandas — with pandas removed from requirements this broke its import.
- notify.py: _candidate_rows filters/sorts the recent bundle with polars
expressions and returns iter_rows dicts; date scalars are datetime.date;
history/recent emptiness via is_empty().
- test_notify.py: synthetic history/rows built with polars + datetime.
2026-07-15 19:07:38 +00:00
|
|
|
}))
|
Add backend test suite; gate direct pushes; serialize LAN deploys (#41)
- backend/tests: 74 hermetic tests (no network, no repo data//logs/ writes)
covering grid snapping/round-trips, grading percentiles/bands/windows/
dry streaks, the places index (norm, one-edit matchers, search,
corrections), the derived store (token validity, cache=False, degraded
mode), and route-level API tests over a faked climate layer — routing,
validation, ETag/304 revalidation, store replay, the /cell bundle, and
the v1/v2 aliases. The API tests would have caught the /place
AttributeError regression.
- requirements-dev.txt + make test (venv prefers uv-pinned 3.12, matching
deploy-dev.sh — pyarrow wheels stop at 3.12 and some pyenv builds lack
sqlite).
- CI: extract the build job into a reusable build.yml, add the test run
and an API health probe (page-only curl can't catch route wiring
faults); deploy-dev.yml now runs the same build gate before deploying
direct pushes, which previously deployed with no CI at all.
- Deploys serialize under one dev-lan-deploy concurrency group across
both workflows (previously per-PR groups could interleave two deploys
to the same checkout), and are never cancelled mid-restart.
- deploy-dev.sh health check also probes /api/v2/place — best-effort
externals mean a failure there is a genuine server bug.
2026-07-11 19:37:49 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
|
|
|
def history():
|
|
|
|
|
return make_history()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
|
|
|
def recent(history):
|
|
|
|
|
return make_recent(history)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@pytest.fixture
|
|
|
|
|
def tmp_store(tmp_path, monkeypatch):
|
|
|
|
|
"""store.py against a fresh database file, isolated per test."""
|
|
|
|
|
monkeypatch.setattr(store, "DB_PATH", str(tmp_path / "store.sqlite"))
|
|
|
|
|
monkeypatch.setattr(store, "_local", threading.local())
|
|
|
|
|
return store
|