thermograph/backend/daemon/main.go
emi 2d3f37c474
All checks were successful
Sync infra to hosts / sync-beta (push) Successful in 8s
Sync infra to hosts / sync-prod (push) Successful in 7s
secrets-guard / encrypted (push) Successful in 8s
shell-lint / shellcheck (push) Successful in 10s
Build + push backend image (Forgejo registry) / build-push (push) Successful in 1m14s
Deploy backend to beta VPS / deploy (push) Successful in 2m4s
daemon: move the Discord gateway and scheduler out of the web process into Go (#21)
The gateway bot and APScheduler were long-lived stateful I/O loops running
inside the async web app under a leader election. They move into a single Go
binary that owns ONLY that I/O -- websocket, RESUME, heartbeat, backoff, timers.

It owns no grading logic. Anything needing data calls back over a new
internal-only surface (/internal/discord/grade, /internal/jobs/*). Grading
depends on polars and the parquet cache; reimplementing it in Go would let the
bot's grades drift from the API's. The grade route returns gateway-ready JSON
and Go relays the bytes verbatim.

The binary ships in the backend image and runs as a second compose service off
the same tag, so the two ends of the /internal/* contract can never skew.
deploy.sh rolls daemon alongside backend -- without that the service would never
be created, since a single-service deploy uses --no-deps. It also probes the
image first and skips the daemon when rolling a tag that predates the binary:
infra tracks main while image tags are env-staged, so a host can legitimately be
asked to roll an older backend image, and creating the service anyway would
leave a container crash-looping on a missing binary.

replicas: 1 with order: stop-first replaces the leader election -- Discord
permits one gateway connection per bot token.

THERMOGRAPH_INTERNAL_TOKEN is optional: both ends derive it from
THERMOGRAPH_AUTH_SECRET via HMAC under a domain-separation label, so this needs
no new vault entry. The derivation is pinned to a shared cross-language test
vector asserted on both sides, so drift fails CI instead of 401ing every call.
Fail closed when neither secret is set.

Improvements over the Python: a close intended for RESUME uses 4000 rather than
1000 (Discord invalidates a session closed 1000, so the old default defeated its
own resume); MESSAGE_CREATE runs on a bounded worker pool; and a malformed HELLO
returns an error rather than a clean reconnect, which would otherwise reset
backoff and hot-loop against the gateway.

365 Python tests pass; Go build/vet/test -race clean; shellcheck 0 findings.
2026-07-23 22:49:54 +00:00

130 lines
4.4 KiB
Go

// thermograph-daemon owns the backend's long-lived stateful I/O — the Discord
// gateway websocket (IDENTIFY/RESUME/heartbeat/backoff) and the recurring-job
// timers — and nothing else. Anything that needs data calls back into the
// Python app over its internal-only HTTP routes (see internal/apiclient).
//
// This replaces the two in-process background jobs web/app.py used to start
// under leader election. One daemon process replaces the election entirely:
// there is exactly one replica of this container, so the single-connection
// invariant (Discord tolerates only one gateway session per token; duplicated
// APScheduler replicas would repeat every job) holds by deployment shape
// rather than by runtime coordination.
package main
import (
"context"
"log/slog"
"os"
"os/signal"
"sync"
"syscall"
"time"
"thermograph/daemon/internal/apiclient"
"thermograph/daemon/internal/config"
"thermograph/daemon/internal/cron"
"thermograph/daemon/internal/gateway"
)
// shutdownGrace bounds how long we wait for the gateway and cron halves to
// stop after a signal. Long enough for the gateway to close its session
// cleanly (a dirty exit leaves Discord holding a stale session), short enough
// to stay inside a container runtime's stop timeout so we exit on our own
// terms rather than being SIGKILLed mid-close.
const shutdownGrace = 10 * time.Second
func main() {
os.Exit(run())
}
// run is main minus os.Exit, so deferred cleanup actually runs.
func run() int {
// Everything to stdout: the container collects stdout, nothing else.
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
// Refuse to start on bad config rather than running half-configured: a
// daemon that cannot authenticate to the backend can only fail silently.
cfg, err := config.Load()
if err != nil {
logger.Error("invalid configuration; refusing to start", "err", err)
return 1
}
// One context for everything long-lived; SIGINT/SIGTERM cancels it so the
// gateway can close its session cleanly (Discord treats an abrupt drop
// worse than a clean close for RESUME purposes).
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
api := apiclient.New(cfg.APIBase, cfg.InternalToken)
logger.Info("thermograph-daemon starting",
"api", cfg.APIBase,
"discord", cfg.DiscordEnabled,
"warm_cities_interval", cfg.WarmCitiesInterval,
"indexnow_interval", cfg.IndexNowInterval)
var wg sync.WaitGroup
// The cron half runs unconditionally — it is the floor of what this
// daemon does, gateway or not.
wg.Add(1)
go func() {
defer wg.Done()
cron.Run(ctx, logger,
cron.Job{
Name: "warm-cities",
Interval: cfg.WarmCitiesInterval,
Run: func(ctx context.Context) error { return api.WarmCities(ctx) },
},
cron.Job{
Name: "indexnow",
Interval: cfg.IndexNowInterval,
Run: func(ctx context.Context) error { return api.IndexNow(ctx) },
},
)
}()
if cfg.DiscordEnabled {
wg.Add(1)
go func() {
defer wg.Done()
// A fatal gateway error (bad token, disallowed intents) must not
// take the cron half down: the schedule is independently useful,
// and exiting here would turn a Discord misconfiguration into a
// crash-loop that also stops warm-cities/IndexNow. Log loudly and
// keep running instead.
if err := gateway.Run(ctx, cfg, api); err != nil && ctx.Err() == nil {
logger.Error("discord gateway exited with a fatal error; cron jobs keep running without it",
"err", err)
}
}()
} else {
// An unconfigured deploy must pay nothing, same as the Python side:
// no connection, no retries — just say so once and run cron-only.
logger.Info("discord gateway disabled (THERMOGRAPH_DISCORD_BOT not truthy or no bot token); running cron-only")
}
<-ctx.Done()
// Restore default signal handling immediately: a second SIGINT/SIGTERM
// during the grace period kills us the ordinary way instead of being
// swallowed by the (already-cancelled) NotifyContext.
stop()
logger.Info("shutting down", "grace", shutdownGrace)
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
logger.Info("shutdown complete")
case <-time.After(shutdownGrace):
// This process is the sole holder of the gateway connection, so a
// clean close matters — but a wedged goroutine must not hold the
// container's stop hostage forever.
logger.Warn("shutdown grace period elapsed before all loops stopped; exiting anyway")
}
return 0
}