thermograph/backend/daemon/internal/cron/cron.go
emi 2d3f37c474
All checks were successful
Sync infra to hosts / sync-beta (push) Successful in 8s
Sync infra to hosts / sync-prod (push) Successful in 7s
secrets-guard / encrypted (push) Successful in 8s
shell-lint / shellcheck (push) Successful in 10s
Build + push backend image (Forgejo registry) / build-push (push) Successful in 1m14s
Deploy backend to beta VPS / deploy (push) Successful in 2m4s
daemon: move the Discord gateway and scheduler out of the web process into Go (#21)
The gateway bot and APScheduler were long-lived stateful I/O loops running
inside the async web app under a leader election. They move into a single Go
binary that owns ONLY that I/O -- websocket, RESUME, heartbeat, backoff, timers.

It owns no grading logic. Anything needing data calls back over a new
internal-only surface (/internal/discord/grade, /internal/jobs/*). Grading
depends on polars and the parquet cache; reimplementing it in Go would let the
bot's grades drift from the API's. The grade route returns gateway-ready JSON
and Go relays the bytes verbatim.

The binary ships in the backend image and runs as a second compose service off
the same tag, so the two ends of the /internal/* contract can never skew.
deploy.sh rolls daemon alongside backend -- without that the service would never
be created, since a single-service deploy uses --no-deps. It also probes the
image first and skips the daemon when rolling a tag that predates the binary:
infra tracks main while image tags are env-staged, so a host can legitimately be
asked to roll an older backend image, and creating the service anyway would
leave a container crash-looping on a missing binary.

replicas: 1 with order: stop-first replaces the leader election -- Discord
permits one gateway connection per bot token.

THERMOGRAPH_INTERNAL_TOKEN is optional: both ends derive it from
THERMOGRAPH_AUTH_SECRET via HMAC under a domain-separation label, so this needs
no new vault entry. The derivation is pinned to a shared cross-language test
vector asserted on both sides, so drift fails CI instead of 401ing every call.
Fail closed when neither secret is set.

Improvements over the Python: a close intended for RESUME uses 4000 rather than
1000 (Discord invalidates a session closed 1000, so the old default defeated its
own resume); MESSAGE_CREATE runs on a bounded worker pool; and a malformed HELLO
returns an error rather than a clean reconnect, which would otherwise reset
backoff and hot-loop against the gateway.

365 Python tests pass; Go build/vet/test -race clean; shellcheck 0 findings.
2026-07-23 22:49:54 +00:00

92 lines
3.4 KiB
Go

// Package cron is the Go replacement for notifications/scheduler.py's
// APScheduler pair (warm-cities + IndexNow). The Python side chose APScheduler
// over a Postgres-native queue because exactly one process ever runs these
// jobs — no fan-out, backpressure, or dead-letter need — and that shape
// carries over unchanged: this daemon is the sole scheduler, so plain
// per-job tickers are all the machinery required. procrastinate/pgqueuer
// remain the documented upgrade path if that ever changes — see
// docs/architecture/repo-topology-and-infrastructure.md §7.
//
// The jobs themselves stay in Python (called via the internal HTTP routes);
// this package owns only the timing.
package cron
import (
"context"
"log/slog"
"sync"
"time"
)
// Job is one recurring task: a name for logs, an interval, and the callback
// (in practice an apiclient method hitting a /internal/jobs/* route).
type Job struct {
Name string
Interval time.Duration
Run func(ctx context.Context) error
}
// Run drives every job on its own ticker until ctx is cancelled, then returns
// once all job goroutines have stopped. It never returns early: a failing job
// logs and keeps ticking — one bad tick (backend briefly down, timeout) must
// never kill the schedule, because nothing restarts it short of a container
// restart.
func Run(ctx context.Context, logger *slog.Logger, jobs ...Job) {
var wg sync.WaitGroup
for _, job := range jobs {
wg.Add(1)
go func(job Job) {
defer wg.Done()
runJob(ctx, logger, job)
}(job)
}
wg.Wait()
}
func runJob(ctx context.Context, logger *slog.Logger, job Job) {
// A plain ticker's first fire is one interval from now, NOT at startup —
// deliberately kept from the Python scheduler (its add_job had no
// next_run_time override for the same reason): warm-cities already runs at
// deploy time via the deploy hook, so an immediate run on every daemon
// boot/restart would just re-spend work (and, for warm-cities, archive-fetch
// quota) that the deploy already paid for.
ticker := time.NewTicker(job.Interval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
start := time.Now()
// Run synchronously in this loop: a job can never overlap with
// itself by construction. That matters most for warm-cities,
// which spends the archive-fetch quota — two overlapping runs would
// double-spend it.
if err := job.Run(ctx); err != nil {
if ctx.Err() != nil {
// Shutdown raced the job; the cancellation is the story,
// not the (expected) aborted call.
return
}
logger.Error("cron job failed; will retry next tick",
"job", job.Name, "err", err, "elapsed", time.Since(start))
} else {
logger.Info("cron job ok",
"job", job.Name, "elapsed", time.Since(start))
}
// If the job overran its interval, a tick may have fired mid-run.
// Skip it rather than running again back-to-back — a late tick is
// the same double-spend as an overlapping one, just serialized.
// (Go 1.23+ tickers already drop fires with no ready receiver;
// this drain pins the skip semantics rather than relying on the
// runtime's channel buffering.)
select {
case <-ticker.C:
logger.Warn("cron job overran its interval; skipping the tick that fired mid-run",
"job", job.Name, "interval", job.Interval, "elapsed", time.Since(start))
default:
}
}
}
}