thermograph/backend/daemon/internal/gateway/reply_test.go
Emi Griffith c3b906a9ed
All checks were successful
shell-lint / shellcheck (pull_request) Successful in 8s
PR build (required check) / build-backend (pull_request) Successful in 1m18s
PR build (required check) / changes (pull_request) Successful in 6s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / gate (pull_request) Successful in 3s
secrets-guard / encrypted (pull_request) Successful in 5s
PR build (required check) / validate-observability (pull_request) Has been skipped
daemon: move the Discord gateway and scheduler out of the web process into Go
web/app.py started two long-lived background jobs under a leader election: the
Discord gateway bot and an APScheduler. Both are stateful I/O loops -- reconnect,
RESUME, heartbeat, backoff, interval timers -- living inside an async web app
that also has to serve requests. This moves them into a single Go binary.

Go owns ONLY the stateful I/O. It owns no climate or grading logic: anything
needing data calls back into Python over a new internal-only HTTP surface
(/internal/discord/grade, /internal/jobs/warm-cities, /internal/jobs/indexnow).
Grading depends on polars and the parquet cache; reimplementing it in Go would
make the bot's grades drift from the API's, and the slash-command path
deliberately shares one grade builder so the two can never disagree. The grade
route returns gateway-ready JSON -- including the ephemeral-flag drop that
discord_bot.py used to do -- and Go relays those bytes verbatim without parsing
the embed.

Packaging: the binary is built by a golang:1.26 stage in the backend Dockerfile
and shipped in the SAME image, run as a second compose service off the SAME tag.
The daemon and backend share the /internal/* contract, so they must never skew
versions; one image makes that structural rather than a convention. Its
entrypoint bypasses entrypoint.sh -- the backend owns alembic, and two racing
migrators is a real hazard.

replicas: 1 in the Swarm stack is load-bearing. Discord permits exactly one
gateway connection per bot token; the pin replaces core/singleton.claim_leader
for this workload. update_config uses order: stop-first, since start-first would
briefly run two gateways. autoscale.sh targets ${STACK_NAME}_web only, so it
cannot scale this.

Security: the internal routes compare the token with hmac.compare_digest and the
whole router 404s when THERMOGRAPH_INTERNAL_TOKEN is unset -- fail closed, never
default open. Caddy only routes /api/*, /digest and /discord/interactions to the
backend, so /internal/* was never publicly reachable; the token is defence in
depth. The router mounts before the catch-all frontend proxy so /internal/*
cannot fall through to it. The daemon refuses to start without the token.

Behaviour preserved from the Python, with the reasoning carried into the Go
comments: non-privileged intents (no MESSAGE_CONTENT, so no portal review);
fatal close codes 4004/4010-4014 stop rather than loop; the bot-author and
self-author mention-loop guard; allowed_mentions locked to {"parse":[],
"replied_user":true} so a crafted query cannot turn a reply into an @everyone
ping; the first cron tick deferred one full interval rather than firing at boot,
since warm-cities already runs at deploy time; and no overlapping warm-cities
run, which would double-spend the archive-fetch quota.

Two deliberate improvements over the Python. A close intended for RESUME now
uses 4000 rather than 1000 -- Discord invalidates a session closed 1000/1001, so
the Python's default close silently defeated its own resume. And MESSAGE_CREATE
is handled on a bounded worker pool rather than an unbounded thread hand-off, so
a flood of mentions cannot spawn unbounded work against the backend.

A .dockerignore is added because a disposable backend/.venv was being swallowed
by COPY . /app/ and duplicated again by the chown layer, inflating the image to
1.8 GB; it builds at 578 MB.

Tests: 29 Go gateway tests covering every behaviour the deleted
test_discord_bot.py asserted, plus cron/config/apiclient suites; 10 new Python
tests for the internal routes (fail-closed, auth, flag drop, per-job 409 guard).
Full suite 359 passed / 7 skipped; go build, vet and test -race clean.
2026-07-23 15:42:44 -07:00

278 lines
8.3 KiB
Go

// Reply-path tests: a fake Grader plus an httptest server standing in for
// Discord's REST API. No live gateway is involved anywhere.
package gateway
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"sync"
"sync/atomic"
"testing"
"time"
)
type fakeGrader struct {
mu sync.Mutex
resp json.RawMessage
err error
calls []string
}
func (f *fakeGrader) Grade(_ context.Context, query string) (json.RawMessage, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, query)
return f.resp, f.err
}
func (f *fakeGrader) callCount() int {
f.mu.Lock()
defer f.mu.Unlock()
return len(f.calls)
}
type captured struct {
method, path, auth string
body []byte
}
// newTestBot wires a Bot to a Grader and an httptest stand-in for Discord.
func newTestBot(t *testing.T, g Grader, handler http.HandlerFunc) *Bot {
t.Helper()
srv := httptest.NewServer(handler)
t.Cleanup(srv.Close)
b := New("tok-123", g)
b.rest = srv.URL
return b
}
// capture records each request and replies 200 {}.
func capture(got chan<- captured) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
got <- captured{r.Method, r.URL.Path, r.Header.Get("Authorization"), body}
_, _ = w.Write([]byte(`{}`))
}
}
func TestGradeReplyIsRelayedVerbatimAsAReply(t *testing.T) {
grade := json.RawMessage(`{"embeds":[{"title":"grade:Phoenix"}]}`)
got := make(chan captured, 1)
b := newTestBot(t, &fakeGrader{resp: grade}, capture(got))
b.handleMessage(context.Background(), testMsg("<@999> Phoenix", mentioning(botID)), botID)
c := <-got
if c.method != http.MethodPost || c.path != "/channels/77/messages" {
t.Fatalf("want POST /channels/77/messages, got %s %s", c.method, c.path)
}
if c.auth != "Bot tok-123" {
t.Fatalf("want bot-token auth, got %q", c.auth)
}
var top map[string]json.RawMessage
if err := json.Unmarshal(c.body, &top); err != nil {
t.Fatalf("reply body not JSON: %v", err)
}
// The embed must be byte-for-byte what Python returned — the verbatim
// relay is the no-drift guarantee.
if string(top["embeds"]) != `[{"title":"grade:Phoenix"}]` {
t.Fatalf("embeds not relayed verbatim: %s", top["embeds"])
}
if string(top["message_reference"]) != `{"message_id":"42"}` {
t.Fatalf("bad message_reference: %s", top["message_reference"])
}
// The locked-down allowed_mentions is the anti-@everyone control; it must
// be present exactly.
if string(top["allowed_mentions"]) != `{"parse":[],"replied_user":true}` {
t.Fatalf("bad allowed_mentions: %s", top["allowed_mentions"])
}
}
func TestEmptyQueryRepliesWithHelpWithoutGrading(t *testing.T) {
got := make(chan captured, 1)
g := &fakeGrader{}
b := newTestBot(t, g, capture(got))
b.handleMessage(context.Background(), testMsg("<@999>", mentioning(botID)), botID)
c := <-got
var body struct {
Content string `json:"content"`
}
if err := json.Unmarshal(c.body, &body); err != nil {
t.Fatalf("help body not JSON: %v", err)
}
if body.Content != helpText {
t.Fatalf("want help text, got %q", body.Content)
}
if g.callCount() != 0 {
t.Fatal("help reply must not hit the grade callback")
}
}
func TestSilentMessagesMakeNoRequests(t *testing.T) {
var hits atomic.Int32
g := &fakeGrader{resp: json.RawMessage(`{}`)}
b := newTestBot(t, g, func(w http.ResponseWriter, r *http.Request) {
hits.Add(1)
})
b.handleMessage(context.Background(), testMsg("just chatting"), botID)
if g.callCount() != 0 || hits.Load() != 0 {
t.Fatalf("silent message leaked: grades=%d posts=%d", g.callCount(), hits.Load())
}
}
func TestGradeFailureStaysSilent(t *testing.T) {
var hits atomic.Int32
g := &fakeGrader{err: context.DeadlineExceeded}
b := newTestBot(t, g, func(w http.ResponseWriter, r *http.Request) {
hits.Add(1)
})
// Must log-and-return, not panic or post a broken reply.
b.handleMessage(context.Background(), testMsg("<@999> Phoenix", mentioning(botID)), botID)
if hits.Load() != 0 {
t.Fatal("failed grade must not produce a reply")
}
}
func TestFailedReplyIsSwallowed(t *testing.T) {
g := &fakeGrader{resp: json.RawMessage(`{"content":"x"}`)}
b := newTestBot(t, g, func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "boom", http.StatusInternalServerError)
})
// A failed reply must never kill the caller (in production: the read
// loop) — returning normally is the assertion.
b.handleMessage(context.Background(), testMsg("<@999> Phoenix", mentioning(botID)), botID)
}
func TestReplyRetriesOnceOn429(t *testing.T) {
var hits atomic.Int32
bodies := make(chan []byte, 2)
g := &fakeGrader{resp: json.RawMessage(`{"content":"x"}`)}
b := newTestBot(t, g, func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
bodies <- body
if hits.Add(1) == 1 {
w.WriteHeader(http.StatusTooManyRequests)
_, _ = w.Write([]byte(`{"retry_after":0.01}`))
return
}
_, _ = w.Write([]byte(`{}`))
})
b.handleMessage(context.Background(), testMsg("<@999> Phoenix", mentioning(botID)), botID)
if hits.Load() != 2 {
t.Fatalf("want 1 retry after 429, got %d requests", hits.Load())
}
first, second := <-bodies, <-bodies
if string(first) != string(second) {
t.Fatal("retry must resend the identical payload")
}
}
func TestNoChannelMeansNoPost(t *testing.T) {
var hits atomic.Int32
g := &fakeGrader{resp: json.RawMessage(`{"content":"x"}`)}
b := newTestBot(t, g, func(w http.ResponseWriter, r *http.Request) {
hits.Add(1)
})
b.handleMessage(context.Background(),
testMsg("<@999> Phoenix", mentioning(botID), func(m *gwMessage) { m.ChannelID = "" }), botID)
if hits.Load() != 0 {
t.Fatal("a message without a channel id must not be answered")
}
}
func TestInjectReplyFieldsWithoutMessageIDLeavesBodyAlone(t *testing.T) {
out, err := injectReplyFields([]byte(`{"content":"x"}`), "")
if err != nil {
t.Fatalf("inject: %v", err)
}
var top map[string]json.RawMessage
if err := json.Unmarshal(out, &top); err != nil {
t.Fatalf("out not JSON: %v", err)
}
if _, ok := top["message_reference"]; ok {
t.Fatal("no triggering id => no message_reference")
}
if _, ok := top["allowed_mentions"]; ok {
t.Fatal("no triggering id => no allowed_mentions")
}
}
// --- dispatch: same reply, off the read loop ---------------------------------
func TestDispatchDeliversTheSameReplyViaAGoroutine(t *testing.T) {
// Mirrors the Python test_dispatch_delivers_the_same_reply_via_a_thread:
// the concurrency offload (goroutine + semaphore here, asyncio.to_thread
// there) must not change what gets sent.
grade := json.RawMessage(`{"embeds":[{"title":"grade:Phoenix"}]}`)
got := make(chan captured, 1)
b := newTestBot(t, &fakeGrader{resp: grade}, capture(got))
sess := &session{userID: botID}
d, err := json.Marshal(testMsg("<@999> Phoenix", mentioning(botID)))
if err != nil {
t.Fatalf("marshal: %v", err)
}
b.dispatch(context.Background(), payload{Op: opDispatch, T: "MESSAGE_CREATE", D: d}, sess)
select {
case c := <-got:
if c.path != "/channels/77/messages" {
t.Fatalf("want /channels/77/messages, got %s", c.path)
}
var top map[string]json.RawMessage
if err := json.Unmarshal(c.body, &top); err != nil {
t.Fatalf("reply body not JSON: %v", err)
}
if string(top["embeds"]) != `[{"title":"grade:Phoenix"}]` {
t.Fatalf("embeds not relayed verbatim: %s", top["embeds"])
}
if string(top["message_reference"]) != `{"message_id":"42"}` {
t.Fatalf("bad message_reference: %s", top["message_reference"])
}
case <-time.After(2 * time.Second):
t.Fatal("dispatched message never produced a reply")
}
}
func TestMessageFloodIsBoundedWithoutBlockingTheReadLoop(t *testing.T) {
g := &fakeGrader{resp: json.RawMessage(`{}`)}
b := New("tok", g)
b.sem = make(chan struct{}, 1)
b.sem <- struct{}{} // occupy the only slot: simulate a reply in flight
sess := &session{userID: botID}
d, err := json.Marshal(testMsg("<@999> Phoenix", mentioning(botID)))
if err != nil {
t.Fatalf("marshal: %v", err)
}
done := make(chan struct{})
go func() {
b.dispatch(context.Background(), payload{Op: opDispatch, T: "MESSAGE_CREATE", D: d}, sess)
close(done)
}()
select {
case <-done:
// dispatch returned promptly — the read loop would not have stalled
case <-time.After(time.Second):
t.Fatal("dispatch blocked on a full semaphore")
}
time.Sleep(20 * time.Millisecond)
if g.callCount() != 0 {
t.Fatal("over-the-bound message must be dropped, not queued")
}
}