emi
2d3f37c474
All checks were successful
Sync infra to hosts / sync-beta (push) Successful in 8s
Sync infra to hosts / sync-prod (push) Successful in 7s
secrets-guard / encrypted (push) Successful in 8s
shell-lint / shellcheck (push) Successful in 10s
Build + push backend image (Forgejo registry) / build-push (push) Successful in 1m14s
Deploy backend to beta VPS / deploy (push) Successful in 2m4s
The gateway bot and APScheduler were long-lived stateful I/O loops running inside the async web app under a leader election. They move into a single Go binary that owns ONLY that I/O -- websocket, RESUME, heartbeat, backoff, timers. It owns no grading logic. Anything needing data calls back over a new internal-only surface (/internal/discord/grade, /internal/jobs/*). Grading depends on polars and the parquet cache; reimplementing it in Go would let the bot's grades drift from the API's. The grade route returns gateway-ready JSON and Go relays the bytes verbatim. The binary ships in the backend image and runs as a second compose service off the same tag, so the two ends of the /internal/* contract can never skew. deploy.sh rolls daemon alongside backend -- without that the service would never be created, since a single-service deploy uses --no-deps. It also probes the image first and skips the daemon when rolling a tag that predates the binary: infra tracks main while image tags are env-staged, so a host can legitimately be asked to roll an older backend image, and creating the service anyway would leave a container crash-looping on a missing binary. replicas: 1 with order: stop-first replaces the leader election -- Discord permits one gateway connection per bot token. THERMOGRAPH_INTERNAL_TOKEN is optional: both ends derive it from THERMOGRAPH_AUTH_SECRET via HMAC under a domain-separation label, so this needs no new vault entry. The derivation is pinned to a shared cross-language test vector asserted on both sides, so drift fails CI instead of 401ing every call. Fail closed when neither secret is set. Improvements over the Python: a close intended for RESUME uses 4000 rather than 1000 (Discord invalidates a session closed 1000, so the old default defeated its own resume); MESSAGE_CREATE runs on a bounded worker pool; and a malformed HELLO returns an error rather than a clean reconnect, which would otherwise reset backoff and hot-loop against the gateway. 365 Python tests pass; Go build/vet/test -race clean; shellcheck 0 findings.
234 lines
14 KiB
Text
234 lines
14 KiB
Text
# This is the REFERENCE for what /etc/thermograph.env contains — the full set of
|
|
# vars the app reads. On a host wired for the SOPS vault (an age key +
|
|
# /etc/thermograph/secrets-env), you do NOT hand-edit /etc/thermograph.env: it is
|
|
# rendered at deploy from the encrypted source of truth in deploy/secrets/*.yaml
|
|
# (see deploy/secrets/README.md). To change a secret there, `sops edit` + commit +
|
|
# deploy — not a host edit. (Legacy: Terraform can also render this file from tfvars,
|
|
# and a from-scratch host can start by copying this example and editing it.)
|
|
#
|
|
# Sourced by deploy/deploy.sh (and any `docker compose` invocation) so compose can
|
|
# interpolate it, AND loaded into the app container (env_file in docker-compose.yml).
|
|
# Anything secret the app needs — Postgres password, VAPID keys, auth secret —
|
|
# belongs here.
|
|
|
|
# Port uvicorn binds inside the container. The compose stack publishes it on the
|
|
# host loopback (127.0.0.1:8137) for Caddy to proxy to. Keep 8137.
|
|
PORT=8137
|
|
|
|
# --- Registry pull (deploy.sh / Terraform) ---------------------------------------
|
|
# deploy.sh logs in to git.thermograph.org's registry and `docker compose pull`s
|
|
# the image build-push.yml already pushed for the deployed commit (repo-split
|
|
# Stage 6), instead of building in place. A personal access token with at least
|
|
# read:package scope -- the same REGISTRY_TOKEN secret build-push.yml uses (that
|
|
# one needs write:package too, to push; this only needs to pull).
|
|
REGISTRY_TOKEN=
|
|
|
|
# --- PostgreSQL (docker-compose stack) ------------------------------------------
|
|
# The app and Postgres run as a docker-compose stack (see docker-compose.yml).
|
|
# POSTGRES_PASSWORD is the database password: compose uses it to initialize the
|
|
# postgres container AND to build the app's THERMOGRAPH_DATABASE_URL. It MUST be
|
|
# set here (the systemd unit sources this file so `docker compose up` can
|
|
# interpolate it). Change it from the default before the first `up`.
|
|
POSTGRES_PASSWORD=change-me
|
|
|
|
# The app's compose service already builds THERMOGRAPH_DATABASE_URL from
|
|
# POSTGRES_PASSWORD, so you normally DON'T need this. It's here for reference and
|
|
# for running the app outside compose against the same DB (keep the password in
|
|
# sync with POSTGRES_PASSWORD above).
|
|
#THERMOGRAPH_DATABASE_URL=postgresql+asyncpg://thermograph:change-me@db:5432/thermograph
|
|
|
|
# --- Historical archive (self-hosted Open-Meteo) --------------------------------
|
|
# Where the app fetches its 45-year daily history. Unset (default) → the public
|
|
# Open-Meteo archive API (rate-limited). On the self-hosting host, the
|
|
# docker-compose.openmeteo.yml overlay sets this to the internal service URL for
|
|
# you, so you normally DON'T set it here. Only pin it to run the app outside that
|
|
# overlay against a reachable Open-Meteo instance. Leave it UNSET rather than empty:
|
|
# an empty value is honored as-is and would break the fallback to the public API.
|
|
#THERMOGRAPH_ARCHIVE_URL=http://open-meteo-api:8080/v1/archive
|
|
|
|
# Mark the session cookie Secure — required behind Caddy's HTTPS. Set to 1 in prod;
|
|
# leave unset only for plain-HTTP LAN dev (a Secure cookie is never sent over HTTP).
|
|
THERMOGRAPH_COOKIE_SECURE=1
|
|
|
|
# Pin these too (see their own sections below), so container restarts don't rotate
|
|
# them: THERMOGRAPH_AUTH_SECRET (else every emailed confirm/reset link breaks on
|
|
# restart) and THERMOGRAPH_VAPID_PRIVATE_KEY / _PUBLIC_KEY (else every existing push
|
|
# subscription silently stops delivering). The data dir persists on the appdata
|
|
# volume, but pinning here is the safe default.
|
|
|
|
# Number of uvicorn worker processes. More than 1 stops a single slow upstream fetch
|
|
# (e.g. a cache-miss weather lookup) from blocking every other request — the cause of
|
|
# past brief outages. Prod runs 4 (the compose app service also defaults to WORKERS=4);
|
|
# leave unset (defaults to 1) on a small box. Workers elect one leader for the
|
|
# subscription notifier via a lockfile (THERMOGRAPH_SINGLETON_LOCK, set by the compose
|
|
# app service to /app/data/notifier.lock) so its timer-driven upstream sweep runs once,
|
|
# not once per worker. ~200 MB RAM per worker.
|
|
WORKERS=4
|
|
|
|
# THERMOGRAPH_SINGLETON_LOCK arbitrates workers on ONE host. Under multi-host Swarm,
|
|
# each host would independently elect its own leader — multiplying the Open-Meteo
|
|
# quota use N-fold again. Set THERMOGRAPH_SINGLETON_PG=1 (with THERMOGRAPH_DATABASE_URL
|
|
# pointing at Postgres) to switch to a cluster-wide Postgres advisory lock instead, so
|
|
# exactly one host — not one per host — runs the notifier. Leave unset on a single-host
|
|
# deploy (today's default); the flock above is sufficient there.
|
|
#THERMOGRAPH_SINGLETON_PG=1
|
|
|
|
# Which duties this process performs. Every replica runs the same image; ROLE just
|
|
# decides whether it's allowed to own the notifier once it wins the leader election
|
|
# above — this is what lets the web tier scale to N stateless replicas under Swarm
|
|
# without also scaling notifier instances, while a single worker replica owns it.
|
|
# all -> both (default; today's single-process behavior, unchanged)
|
|
# web -> never runs the notifier, even if it would win leader election
|
|
# worker -> runs the notifier if it wins leader election
|
|
# Read once at process start; changing it needs a restart, not a live toggle.
|
|
#THERMOGRAPH_ROLE=all
|
|
|
|
# The recurring jobs (city warming, IndexNow). Hours between runs; both are
|
|
# cheap no-op skips when there's nothing to do, so the defaults rarely need
|
|
# changing. The daemon service (below) owns these timers and fires the work
|
|
# through the backend's internal routes.
|
|
#THERMOGRAPH_WARM_CITIES_INTERVAL_HOURS=24
|
|
#THERMOGRAPH_INDEXNOW_INTERVAL_HOURS=6
|
|
|
|
# --- Internal daemon<->backend API ------------------------------------------------
|
|
# Shared secret authenticating the daemon service's calls to the backend's
|
|
# /internal/* routes (X-Thermograph-Internal-Token header). A credential:
|
|
# OPTIONAL — leave unset unless you specifically want to override the derivation.
|
|
#
|
|
# When unset, the backend and the daemon each DERIVE this token from
|
|
# THERMOGRAPH_AUTH_SECRET (HMAC-SHA256 under a fixed domain-separation label).
|
|
# That secret is already required and already in every vault, so the daemon
|
|
# stands up with NO new key to add, rotate, or forget on a new host. HMAC under a
|
|
# distinct label rather than reusing the auth secret directly, so a leak of this
|
|
# token can't be replayed as the session-signing key.
|
|
#
|
|
# Set it explicitly to rotate this surface independently of the auth secret
|
|
# (e.g. openssl rand -hex 32). On a SOPS host that means `sops edit` on
|
|
# deploy/secrets/*.yaml, never hand-editing /etc/thermograph.env — it is a
|
|
# rendered artifact. Both ends must agree, so set it on both or neither.
|
|
#
|
|
# With neither this nor THERMOGRAPH_AUTH_SECRET set, both ends fail closed: the
|
|
# backend disables the internal routes and the daemon refuses to start. Defence
|
|
# in depth: Caddy never routes /internal/* to the backend anyway.
|
|
#THERMOGRAPH_INTERNAL_TOKEN=
|
|
|
|
# Base path the app is served under.
|
|
# / -> app at the domain root (Thermograph owns the whole domain —
|
|
# this is what the Caddyfile expects: thermograph.org proxies "/")
|
|
# /thermograph -> app under a sub-path (share the host with other apps, e.g. a
|
|
# portfolio at the root)
|
|
THERMOGRAPH_BASE=/
|
|
|
|
# --- SEO: search-engine verification + IndexNow ---------------------------------
|
|
# Ownership-verification tokens, rendered as <meta> tags in every page's <head>.
|
|
# Google Search Console → add property https://thermograph.org → "HTML tag" method
|
|
# → paste just the content="…" value below. (Or verify via DNS TXT and skip this.)
|
|
#THERMOGRAPH_GOOGLE_VERIFY=
|
|
# Bing Webmaster Tools → add site → "HTML Meta Tag" (msvalidate.01) → paste the
|
|
# content value. (Bing can also import verification from Google Search Console.)
|
|
#THERMOGRAPH_BING_VERIFY=
|
|
|
|
# IndexNow key (Bing/DuckDuckGo/Yandex instant re-crawl). Auto-generated to
|
|
# data/indexnow_key.txt on first use; set here to pin a specific key.
|
|
#THERMOGRAPH_INDEXNOW_KEY=
|
|
# Public site URL for IndexNow. The deploy hook auto-pings IndexNow after a
|
|
# successful deploy, but only when the URL set changed (a new/removed city), so
|
|
# code-only deploys don't resubmit. `make indexnow` forces a full submit.
|
|
THERMOGRAPH_BASE_URL=https://thermograph.org
|
|
|
|
# --- Web Push (VAPID) -----------------------------------------------------------
|
|
# Keys that sign push notifications. If unset, the app generates a pair into
|
|
# data/vapid.json on first run — fine as long as that file PERSISTS (it lives in the
|
|
# writable data dir and survives deploys). PIN them here to be safe: if the keys ever
|
|
# change, every existing browser subscription silently stops receiving (the push
|
|
# service rejects with 401/403), and users must toggle alerts off/on to re-subscribe.
|
|
# Generate a pair: cd backend && ../.venv/bin/python -c "import push,json; k=push._generate(); print('PRIVATE=',k['private_key']); print('PUBLIC=',k['public_key'])"
|
|
#THERMOGRAPH_VAPID_PRIVATE_KEY=
|
|
#THERMOGRAPH_VAPID_PUBLIC_KEY=
|
|
# Contact (mailto: or https URL) sent to push services in the VAPID claim.
|
|
#THERMOGRAPH_VAPID_CONTACT=mailto:you@example.com
|
|
|
|
# --- Outbound email --------------------------------------------------------------
|
|
# Delivery goes through the host's Postfix null client (deploy/provision-mail.sh).
|
|
# The app runs in a container, so it can't reach the host's loopback — it speaks
|
|
# plain SMTP to the compose bridge's gateway (172.19.0.1, pinned in
|
|
# docker-compose.yml), where Postfix listens and relays out. Switching between
|
|
# "direct to MX" and "relay through a provider" is a Postfix change, no redeploy.
|
|
# Stack-mode hosts (prod) override this per-service to the docker_gwbridge
|
|
# gateway (172.18.0.1) in deploy/stack/thermograph-stack.yml -- overlay tasks
|
|
# can't reach a compose bridge gateway; leave this file's value as the compose
|
|
# default.
|
|
#
|
|
# Backends: console (log it, send nothing — the default, right for dev),
|
|
# smtp (actually send), disabled (drop silently).
|
|
# Leave unset until Postfix is provisioned: signups are still collected either way.
|
|
# THERMOGRAPH_MAIL_FROM is left unset here on purpose — the app default
|
|
# (Thermograph <no-reply@thermograph.org>) is correct, and its "<>" would need
|
|
# escaping in this shell-sourced file. Override only if the address differs.
|
|
#THERMOGRAPH_MAIL_BACKEND=smtp
|
|
#THERMOGRAPH_SMTP_HOST=172.19.0.1
|
|
#THERMOGRAPH_SMTP_PORT=25
|
|
# Only needed if talking to a remote SMTP server directly instead of local Postfix.
|
|
#THERMOGRAPH_SMTP_USER=
|
|
#THERMOGRAPH_SMTP_PASSWORD=
|
|
#THERMOGRAPH_SMTP_STARTTLS=1
|
|
#THERMOGRAPH_MAIL_FROM=Thermograph <no-reply@thermograph.org>
|
|
#THERMOGRAPH_MAIL_REPLY_TO=
|
|
|
|
# Signing secret for email confirmation / password-reset tokens. MUST be set to a
|
|
# fixed value before any such link is mailed: it defaults to a per-boot random
|
|
# value, which would invalidate every outstanding link on each restart.
|
|
# generate with: python -c "import secrets; print(secrets.token_urlsafe(48))"
|
|
#THERMOGRAPH_AUTH_SECRET=
|
|
|
|
# --- Discord ---------------------------------------------------------------------
|
|
# Incoming webhook URL for the daily "most unusual right now" post. Create it in the
|
|
# Discord server: Channel → Edit → Integrations → Webhooks → New Webhook → Copy URL.
|
|
# The URL IS the credential — anyone who has it can post as the webhook, so keep it
|
|
# here and never in the repo. Unset => the daily post is disabled (no-op).
|
|
# The post rides the notifier daemon (leader-only), once per day after the feed
|
|
# refresh, so it needs THERMOGRAPH_ENABLE_NOTIFIER on (the default).
|
|
#THERMOGRAPH_DISCORD_WEBHOOK=
|
|
# Slash commands (/grade) are served over Discord's HTTP interactions endpoint by
|
|
# the app itself (no bot process). Set the portal's "Interactions Endpoint URL" to
|
|
# https://thermograph.org/discord/interactions. These come from the Developer Portal:
|
|
# - PUBLIC_KEY: General Information -> Public Key (used to verify every request).
|
|
# - APP_ID / BOT_TOKEN: only needed to (re)register the commands, via
|
|
# scripts/register_discord_commands.py. The bot token is a credential.
|
|
#THERMOGRAPH_DISCORD_PUBLIC_KEY=
|
|
#THERMOGRAPH_DISCORD_APP_ID=
|
|
#THERMOGRAPH_DISCORD_BOT_TOKEN=
|
|
# Channel IDs (numeric snowflakes) for the bot to post into with the BOT_TOKEN above.
|
|
# Subscription test feed: this deployment mirrors every notification the notifier
|
|
# creates into this channel, so a run can be watched live. Set it per environment;
|
|
# leave unset to disable. In the Thermograph.org server the hidden channels are:
|
|
# dev = 1529274513009934336
|
|
# uat = 1529274514066636861
|
|
# prod = 1529274515127799940
|
|
#THERMOGRAPH_DISCORD_SUBSCRIPTION_CHANNEL=
|
|
# Notable-weather feed: prod broadcasts the daily "most unusual right now" digest
|
|
# into this channel via the bot (independent of the WEBHOOK above — either can run
|
|
# alone). Leave unset to disable. In the Thermograph.org server:
|
|
# weather-events = 1529274516746932307
|
|
#THERMOGRAPH_DISCORD_WEATHER_CHANNEL=
|
|
# Discord gateway bot (@mention/DM grading): runs in the daemon service (the
|
|
# Go thermograph-daemon container — no longer inside the backend leader
|
|
# process). Opt-in: any truthy value here starts it, provided BOT_TOKEN above
|
|
# is also set (flag alone is a silent no-op). Discord allows ONE gateway
|
|
# connection per bot token, so enable this in exactly one environment — prod,
|
|
# the only vault with the token (the stack pins the daemon to 1 replica for
|
|
# the same reason).
|
|
#THERMOGRAPH_DISCORD_BOT=
|
|
# Account linking (OAuth2 "identify"): lets a signed-in user connect their Discord
|
|
# account, storing their Discord user id for DM alerts. CLIENT_ID is the same App
|
|
# ID above. The CLIENT_SECRET is from OAuth2 -> Client Secret (a credential). In the
|
|
# portal, add the redirect: https://thermograph.org/api/v2/discord/link/callback
|
|
#THERMOGRAPH_DISCORD_CLIENT_SECRET=
|
|
# Gateway bot (opt-in): holds a live websocket so the bot replies to messages that
|
|
# @mention it (or DM it) with a city grade — e.g. "@Thermograph Phoenix". Reuses
|
|
# THERMOGRAPH_DISCORD_BOT_TOKEN above. Runs in the single-replica daemon service
|
|
# (which also needs THERMOGRAPH_INTERNAL_TOKEN below — it grades via the
|
|
# backend's internal API). No privileged intent required — Discord delivers
|
|
# content for mentions/DMs. Unset/0 => no gateway connection.
|
|
# (Code: backend/daemon/, calling backend's /internal/discord/grade.)
|
|
#THERMOGRAPH_DISCORD_BOT=1
|