adf824b33f
All checks were successful
shell-lint / shellcheck (pull_request) Successful in 8s
PR build (required check) / changes (pull_request) Successful in 6s
secrets-guard / encrypted (pull_request) Successful in 5s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 1m33s
PR build (required check) / gate (pull_request) Successful in 3s
web/app.py started two long-lived background jobs under a leader election: the
Discord gateway bot and an APScheduler. Both are stateful I/O loops -- reconnect,
RESUME, heartbeat, backoff, interval timers -- living inside an async web app
that also has to serve requests. This moves them into a single Go binary.
Go owns ONLY the stateful I/O. It owns no climate or grading logic: anything
needing data calls back into Python over a new internal-only HTTP surface
(/internal/discord/grade, /internal/jobs/warm-cities, /internal/jobs/indexnow).
Grading depends on polars and the parquet cache; reimplementing it in Go would
make the bot's grades drift from the API's, and the slash-command path
deliberately shares one grade builder so the two can never disagree. The grade
route returns gateway-ready JSON -- including the ephemeral-flag drop that
discord_bot.py used to do -- and Go relays those bytes verbatim without parsing
the embed.
Packaging: the binary is built by a golang:1.26 stage in the backend Dockerfile
and shipped in the SAME image, run as a second compose service off the SAME tag.
The daemon and backend share the /internal/* contract, so they must never skew
versions; one image makes that structural rather than a convention. Its
entrypoint bypasses entrypoint.sh -- the backend owns alembic, and two racing
migrators is a real hazard.
replicas: 1 in the Swarm stack is load-bearing. Discord permits exactly one
gateway connection per bot token; the pin replaces core/singleton.claim_leader
for this workload. update_config uses order: stop-first, since start-first would
briefly run two gateways. autoscale.sh targets ${STACK_NAME}_web only, so it
cannot scale this.
Security: the internal routes compare the token with hmac.compare_digest and the
whole router 404s when THERMOGRAPH_INTERNAL_TOKEN is unset -- fail closed, never
default open. Caddy only routes /api/*, /digest and /discord/interactions to the
backend, so /internal/* was never publicly reachable; the token is defence in
depth. The router mounts before the catch-all frontend proxy so /internal/*
cannot fall through to it. The daemon refuses to start without the token.
Behaviour preserved from the Python, with the reasoning carried into the Go
comments: non-privileged intents (no MESSAGE_CONTENT, so no portal review);
fatal close codes 4004/4010-4014 stop rather than loop; the bot-author and
self-author mention-loop guard; allowed_mentions locked to {"parse":[],
"replied_user":true} so a crafted query cannot turn a reply into an @everyone
ping; the first cron tick deferred one full interval rather than firing at boot,
since warm-cities already runs at deploy time; and no overlapping warm-cities
run, which would double-spend the archive-fetch quota.
Two deliberate improvements over the Python. A close intended for RESUME now
uses 4000 rather than 1000 -- Discord invalidates a session closed 1000/1001, so
the Python's default close silently defeated its own resume. And MESSAGE_CREATE
is handled on a bounded worker pool rather than an unbounded thread hand-off, so
a flood of mentions cannot spawn unbounded work against the backend.
A .dockerignore is added because a disposable backend/.venv was being swallowed
by COPY . /app/ and duplicated again by the chown layer, inflating the image to
1.8 GB; it builds at 578 MB.
Tests: 29 Go gateway tests covering every behaviour the deleted
test_discord_bot.py asserted, plus cron/config/apiclient suites; 10 new Python
tests for the internal routes (fail-closed, auth, flag drop, per-job 409 guard).
Full suite 359 passed / 7 skipped; go build, vet and test -race clean.
57 lines
2.5 KiB
Text
57 lines
2.5 KiB
Text
# Local docker-compose overrides. Copy to .env (gitignored) for `docker compose
|
|
# up` / `make up` on a dev box: cp .env.example .env
|
|
#
|
|
# Compose auto-reads a repo-root .env for ${VAR} interpolation in
|
|
# docker-compose.yml. In production these live in /etc/thermograph.env instead
|
|
# (loaded by the systemd unit), so this file is only for local runs.
|
|
|
|
# Database password. Compose uses it to initialize the postgres container AND to
|
|
# build the app's THERMOGRAPH_DATABASE_URL. Change it before first `up`.
|
|
POSTGRES_PASSWORD=change-me
|
|
|
|
# OPTIONAL. Shared secret between the daemon service (Discord gateway + job
|
|
# timers) and the backend's /internal/* routes.
|
|
#
|
|
# Leave it empty and both ends DERIVE the same token from THERMOGRAPH_AUTH_SECRET
|
|
# (HMAC-SHA256 under a fixed label), which every environment already provisions —
|
|
# so the daemon needs no new vault entry and no operator step. Set it only to
|
|
# override that, e.g. to rotate this surface independently of the auth secret:
|
|
# openssl rand -hex 32
|
|
#
|
|
# With neither this nor THERMOGRAPH_AUTH_SECRET set, both ends fail closed: the
|
|
# backend disables the internal routes and the daemon refuses to start. Never
|
|
# routed publicly — Caddy only forwards /api/*, /digest and
|
|
# /discord/interactions to the backend.
|
|
THERMOGRAPH_INTERNAL_TOKEN=
|
|
|
|
# --- Everything below is optional -- docker-compose.yml already defaults each
|
|
# --- of these, so a plain `make up` works with none of it set. Uncomment to
|
|
# --- override.
|
|
|
|
# TimescaleDB image tag. Defaults to the floating latest-pg18 tag. Pin it to an
|
|
# exact minor (e.g. 2.17.2-pg18) before any host of this stack could ever
|
|
# replicate with another -- see docker-compose.yml's db service comment.
|
|
# TIMESCALEDB_TAG=latest-pg18
|
|
|
|
# Postgres sizing. Terraform sets these per host in prod/beta (prod DB_MEMORY
|
|
# 16g); local/beta default to 8g / 2 CPUs.
|
|
# DB_MEMORY=8g
|
|
# DB_CPUS=2
|
|
|
|
# Backend uvicorn worker count and CPU cap. Terraform raises these on bigger
|
|
# hosts; defaults keep a plain `docker compose up` identical to before.
|
|
# WORKERS=4
|
|
# APP_CPUS=4
|
|
|
|
# Frontend CPU cap.
|
|
# FRONTEND_CPUS=2
|
|
|
|
# Registry + per-service image path/tag. Local dev normally builds each image
|
|
# in its own app repo (thermograph-backend / thermograph-frontend) tagged
|
|
# :local, which is the default here -- only set these to pull a specific
|
|
# published build instead of building locally.
|
|
# REGISTRY_HOST=git.thermograph.org
|
|
# BACKEND_IMAGE_PATH=emi/thermograph-backend/app
|
|
# BACKEND_IMAGE_TAG=local
|
|
# FRONTEND_IMAGE_PATH=emi/thermograph-frontend/app
|
|
# FRONTEND_IMAGE_TAG=local
|