* Split web/worker duties with THERMOGRAPH_ROLE Background work (the subscription notifier) is welded to the same process that serves requests, so scaling the web tier to N replicas would also scale notifier instances unless something restricts it further than leader election alone. Add THERMOGRAPH_ROLE (web|worker|all, default all - unchanged single-process behavior). Every replica runs the same image; ROLE only gates whether a process is allowed to own the notifier at all, layered on top of the existing leader election: web replicas never start it even if they'd win leader election, worker replicas start it if they win. The decision is pulled into _should_run_notifier() so it's unit-testable without booting the full app (DB init, places index, neighbor warmer). Add a minimal /healthz liveness route (no DB/upstream I/O, not under BASE) so a worker replica - which serves no real traffic - still has something Swarm can health-check. * Add the Swarm interim stack file, a pinnable TimescaleDB tag, and a Caddy health-gate Three changes toward the hop-1 interim cutover, all inert until Track B stands up the platform: docker-stack.yml: the Swarm stack file for the interim cutover, distinct from docker-compose.yml (today's plain-compose deploy, unaffected). Pulls a pre-built image (IMAGE_TAG) instead of building in place; app/worker publish no host port (127.0.0.1:8137:8137 has no Swarm equivalent - Swarm's routing mesh publishes on 0.0.0.0, which would expose the plaintext app un-fronted), reaching Caddy only over an MTU-lowered overlay network (VXLAN-over-WireGuard needs a smaller MTU or large payloads silently stall); db is placement- pinned to a labelled node; app/worker skip inline migrations (RUN_MIGRATIONS=0) so the runbook's one-shot migrate task is the only thing that ever runs Alembic; secrets are real Swarm secrets mounted at /run/secrets, read by the entrypoint shim rather than plain env vars. TIMESCALEDB_TAG: docker-compose.yml's db image now reads this (default latest-pg18, today's behavior unchanged), wired through Terraform (timescaledb_tag, default "latest-pg18") so it can actually be pinned to an exact minor without hand-editing the host - required before any host of the stack could replicate with another (a floating tag risks mismatched extension minors, which blocks a physical replica and risks compressed- chunk corruption on restore). Caddy active health-gate: both the Terraform-rendered Caddyfile and the live deploy/Caddyfile now health-check the app on the same cheap /healthz route its own Docker HEALTHCHECK uses (now /healthz instead of the SSR homepage, so it's cheap enough for a tight interval and works identically for a worker replica, which serves no public traffic at all) - Caddy won't forward into a container that's still booting or unhealthy. Verified live: built and booted the real image via docker compose - both containers report healthy via the new /healthz-based HEALTHCHECK, and GET / still renders the full SSR homepage unchanged. Both Caddyfiles validated with the real caddy binary. docker-stack.yml validated with docker compose config (required-var guards fire with clear messages; secrets correctly mount at /run/secrets/<name>, matching the entrypoint shim's mapping). docker-compose.yml validated with and without TIMESCALEDB_TAG set, alongside the existing openmeteo overlay. terraform validate + fmt clean.
220 lines
8.5 KiB
HCL
220 lines
8.5 KiB
HCL
# ---------------------------------------------------------------------------------
|
|
# Hosts
|
|
# ---------------------------------------------------------------------------------
|
|
# One entry per VPS. The same module is instantiated for each (see main.tf's
|
|
# for_each). This config manages TWO hosts: prod (new 48 GB / 12-core VPS, branch
|
|
# `release`, thermograph.org) and beta (the old VPS 75.119.132.91, branch `main`,
|
|
# no public domain by default). The `dev` branch deploys to the LAN dev server via
|
|
# deploy/deploy-dev.sh (a self-hosted runner) and is NOT managed here.
|
|
#
|
|
# A host with `domain = ""` gets no Caddy/TLS: the app port is opened on the firewall
|
|
# and the app is reached directly (beta does this by default; set a domain to front
|
|
# it with Caddy TLS).
|
|
#
|
|
# Resource sizing (workers / app_cpus / db_cpus / db_memory) defaults to the historical
|
|
# 4 / 4 / 2 / 8g budget (right for beta). The prod box is 48 GB / 12 cores — raise these
|
|
# there (the example uses app_cpus 8, db_cpus 4, db_memory "16g"). The Postgres internal
|
|
# budget scales from db_memory automatically (deploy/db/init/20-tuning.sh), so no
|
|
# separate tuning edit is needed to exploit the RAM.
|
|
variable "hosts" {
|
|
description = "Map of hosts to manage, keyed by a short name (e.g. \"prod\", \"dev\")."
|
|
type = map(object({
|
|
host = string # IP or hostname to SSH to
|
|
ssh_user = optional(string, "deploy") # SSH login user
|
|
ssh_private_key_path = string # path to the private key for that user
|
|
role = string # "prod" | "dev" (informational + outputs)
|
|
git_branch = string # branch the checkout is reset to
|
|
domain = optional(string, "") # public domain; "" => no Caddy/TLS
|
|
compose_files = optional(list(string), ["docker-compose.yml"])
|
|
app_dir = optional(string, "/opt/thermograph") # checkout path on the host
|
|
workers = optional(number, 4) # uvicorn workers (WORKERS)
|
|
app_cpus = optional(number, 4) # app container CPU cap (APP_CPUS)
|
|
db_cpus = optional(number, 2) # db container CPU cap (DB_CPUS)
|
|
db_memory = optional(string, "8g") # db container memory cap (DB_MEMORY)
|
|
# The floating tag matches today's behavior everywhere until you pin it. Pin to
|
|
# an exact minor (SELECT extversion FROM pg_extension WHERE extname='timescaledb'
|
|
# on the live DB) before any host of this stack could ever replicate with
|
|
# another — a floating tag risks mismatched extension minors, which blocks a
|
|
# physical replica (hop-1 cutover runbook hazard #7). Use the SAME tag everywhere.
|
|
timescaledb_tag = optional(string, "latest-pg18")
|
|
# Self-host the ERA5 archive (docker-compose.openmeteo.yml + a host rclone mount
|
|
# of the object-storage bucket). Only the self-hosting host (prod) sets true.
|
|
openmeteo = optional(bool, false)
|
|
om_data_dir = optional(string, "/mnt/om-archive") # host rclone mount point (OM_DATA_DIR)
|
|
}))
|
|
}
|
|
|
|
variable "repo_url" {
|
|
description = "Git remote to clone from when a host has no checkout yet (a fresh prod box)."
|
|
type = string
|
|
default = "https://github.com/griffemi/thermograph.git"
|
|
}
|
|
|
|
variable "app_port" {
|
|
description = "Port the app binds inside the container / is health-checked on."
|
|
type = number
|
|
default = 8137
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------------
|
|
# Self-hosted Open-Meteo (object storage) — consumed only by hosts with openmeteo=true
|
|
# ---------------------------------------------------------------------------------
|
|
# The ERA5 .om archive lives in an object-storage bucket, surfaced on the host by an
|
|
# rclone FUSE mount at each host's om_data_dir. These describe that bucket + mount.
|
|
# om_rclone_conf holds credentials, so it's sensitive and lands in state — keep the
|
|
# real value in terraform.tfvars (gitignored), never committed.
|
|
variable "om_bucket_remote" {
|
|
description = "rclone remote:path for the archive bucket, e.g. \"om-archive:thermograph-era5\" (matches a [remote] in om_rclone_conf)."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "om_rclone_conf" {
|
|
description = "Full rclone.conf contents defining the archive remote (installed to /etc/rclone/rclone.conf, 0600). Sensitive."
|
|
type = string
|
|
default = ""
|
|
sensitive = true
|
|
}
|
|
|
|
variable "om_vfs_cache_max" {
|
|
description = "rclone --vfs-cache-max-size: bounds the on-disk hot cache for the mount (keep within the disk budget)."
|
|
type = string
|
|
default = "80G"
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------------
|
|
# Shared secrets (rendered into /etc/thermograph.env on every host)
|
|
# ---------------------------------------------------------------------------------
|
|
# These live in terraform.tfvars (gitignored) and land in local state — never commit
|
|
# real values. Every host in this config shares the same secret set; split the config
|
|
# if prod and dev must diverge on a credential.
|
|
variable "postgres_password" {
|
|
description = "PostgreSQL password (initializes the db container AND builds THERMOGRAPH_DATABASE_URL)."
|
|
type = string
|
|
sensitive = true
|
|
}
|
|
|
|
variable "auth_secret" {
|
|
description = "THERMOGRAPH_AUTH_SECRET — signs email-confirm / password-reset tokens. Pin it."
|
|
type = string
|
|
sensitive = true
|
|
}
|
|
|
|
variable "vapid_private_key" {
|
|
description = "THERMOGRAPH_VAPID_PRIVATE_KEY — signs Web Push. Pin it or existing subscriptions break."
|
|
type = string
|
|
sensitive = true
|
|
}
|
|
|
|
variable "vapid_public_key" {
|
|
description = "THERMOGRAPH_VAPID_PUBLIC_KEY — public half of the VAPID pair."
|
|
type = string
|
|
sensitive = true
|
|
}
|
|
|
|
variable "vapid_contact" {
|
|
description = "THERMOGRAPH_VAPID_CONTACT — mailto: or https URL sent to push services."
|
|
type = string
|
|
sensitive = true
|
|
default = ""
|
|
}
|
|
|
|
# ---- Optional: search-engine verification -------------------------------------
|
|
variable "google_verify" {
|
|
description = "THERMOGRAPH_GOOGLE_VERIFY — Search Console HTML-tag content value."
|
|
type = string
|
|
sensitive = true
|
|
default = ""
|
|
}
|
|
|
|
variable "bing_verify" {
|
|
description = "THERMOGRAPH_BING_VERIFY — Bing Webmaster msvalidate.01 content value."
|
|
type = string
|
|
sensitive = true
|
|
default = ""
|
|
}
|
|
|
|
# ---- Optional: outbound email (SMTP) ------------------------------------------
|
|
variable "mail_backend" {
|
|
description = "THERMOGRAPH_MAIL_BACKEND — console | smtp | disabled. Empty => unset (console default)."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "smtp_host" {
|
|
description = "THERMOGRAPH_SMTP_HOST (usually 127.0.0.1 for the local Postfix null client)."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "smtp_port" {
|
|
description = "THERMOGRAPH_SMTP_PORT."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "smtp_user" {
|
|
description = "THERMOGRAPH_SMTP_USER (only for a remote relay)."
|
|
type = string
|
|
sensitive = true
|
|
default = ""
|
|
}
|
|
|
|
variable "smtp_password" {
|
|
description = "THERMOGRAPH_SMTP_PASSWORD (only for a remote relay)."
|
|
type = string
|
|
sensitive = true
|
|
default = ""
|
|
}
|
|
|
|
variable "smtp_starttls" {
|
|
description = "THERMOGRAPH_SMTP_STARTTLS — \"1\" to enable. Empty => unset."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "mail_from" {
|
|
description = "THERMOGRAPH_MAIL_FROM — From: header, e.g. \"Thermograph <no-reply@thermograph.org>\"."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "mail_reply_to" {
|
|
description = "THERMOGRAPH_MAIL_REPLY_TO."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
# ---- Optional: Discord --------------------------------------------------------
|
|
variable "discord_webhook" {
|
|
description = "THERMOGRAPH_DISCORD_WEBHOOK — the URL IS the credential."
|
|
type = string
|
|
sensitive = true
|
|
default = ""
|
|
}
|
|
|
|
variable "discord_public_key" {
|
|
description = "THERMOGRAPH_DISCORD_PUBLIC_KEY — verifies interaction requests."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "discord_app_id" {
|
|
description = "THERMOGRAPH_DISCORD_APP_ID."
|
|
type = string
|
|
default = ""
|
|
}
|
|
|
|
variable "discord_bot_token" {
|
|
description = "THERMOGRAPH_DISCORD_BOT_TOKEN — a credential (command registration only)."
|
|
type = string
|
|
sensitive = true
|
|
default = ""
|
|
}
|
|
|
|
variable "discord_client_secret" {
|
|
description = "THERMOGRAPH_DISCORD_CLIENT_SECRET — OAuth2 account linking (a credential)."
|
|
type = string
|
|
sensitive = true
|
|
default = ""
|
|
}
|