fed307e75c
All checks were successful
PR build (required check) / changes (pull_request) Successful in 7s
secrets-guard / encrypted (pull_request) Successful in 5s
PR build (required check) / build-backend (pull_request) Has been skipped
shell-lint / shellcheck (pull_request) Successful in 6s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / gate (pull_request) Successful in 2s
Dev was mesh-only: bound to 10.10.0.2, no DNS record, no Caddy site. A public
record now exists for it, so it gets a real front door instead of a name that
resolves to a timeout.
Three things keep that acceptable on a box that also runs Forgejo and its CI
runner, and all three are load-bearing together:
- the stack binds LOOPBACK, so the site block is the only route in — not even
the mesh reaches the app directly, which is what makes the guard total
rather than decorative,
- HTTP basic auth, so a stumbled-upon URL is not a running build,
- X-Robots-Tag: noindex, so dev never competes with thermograph.org for the
same content in a search index.
The credential in Caddyfile.vps1 is a bcrypt hash, not a password.
241 lines
12 KiB
Bash
241 lines
12 KiB
Bash
#!/usr/bin/env bash
|
|
# WHERE EACH ENVIRONMENT LIVES — the single source of truth, sourced by every
|
|
# deploy path. Not executable on its own; `. env-topology.sh` then call
|
|
# `thermograph_topology <env>`.
|
|
#
|
|
# Why this file exists at all. Until the vps1/vps2 split, "which environment is
|
|
# this?" was answerable from the machine you were standing on: one host ran one
|
|
# environment, so a host-wide marker (/etc/thermograph/secrets-env) and a
|
|
# host-wide deploy mode (/etc/thermograph/deploy-mode) were enough, and every
|
|
# path could hardcode /opt/thermograph, /etc/thermograph.env, ports 8137/8080.
|
|
# vps2 now runs BOTH beta and prod. Every one of those assumptions becomes a
|
|
# collision: two checkouts, two rendered env files, two stacks, two loopback
|
|
# port pairs, two image-tag files, two deploy locks — on one box. So the
|
|
# environment is now an INPUT (THERMOGRAPH_ENV, passed by the deploy caller),
|
|
# and everything else is derived here rather than re-guessed per script.
|
|
#
|
|
# The host markers survive as the fallback for a by-hand run with no explicit
|
|
# env (`deploy.sh` on prod still means prod), but they are no longer the thing
|
|
# that decides where files go.
|
|
#
|
|
# THE ESTATE
|
|
#
|
|
# vps1 75.119.132.91 10.10.0.2 "operational programs"
|
|
# Forgejo (git + CI + registry), Grafana/Loki/Alloy, emigriffith.dev,
|
|
# and the DEV environment with its own Postgres. Nothing here is
|
|
# reachable by an external user except git/dashboard/the portfolio site;
|
|
# dev itself is mesh-only, deliberately (see DEV_BIND_ADDR below).
|
|
#
|
|
# vps2 169.58.46.181 10.10.0.1 "the deployed environment"
|
|
# Everything an external user can touch: prod (thermograph.org) AND beta
|
|
# (beta.thermograph.org), Centralis, Postfix, the backups, and the ONE
|
|
# TimescaleDB instance that serves both environments on separate
|
|
# databases with separate roles.
|
|
#
|
|
# desktop 10.10.0.3 operator's box
|
|
# AI model hosting (voice-to-text, the LLM behind upcoming features) and
|
|
# flex capacity as a Swarm worker. It hosts NO Thermograph environment —
|
|
# that is the whole point of the vps1/vps2 split. A laptop-style
|
|
# `make dev-up` still works locally; it is just not the dev server.
|
|
#
|
|
# WHY BETA SITS NEXT TO PROD RATHER THAN NEXT TO DEV
|
|
#
|
|
# Beta is a pre-production rehearsal, so what it needs to rehearse is prod's
|
|
# environment: the same orchestrator (Swarm, not compose), the same Postgres
|
|
# major and extension build, the same Caddy in front, the same mail path, the
|
|
# same mesh position. Co-locating beta with dev bought resemblance to the thing
|
|
# it is NOT trying to predict. Co-locating it with prod means a beta green light
|
|
# is evidence about prod. The cost — a bad beta deploy shares a machine with
|
|
# prod — is bounded by the per-environment isolation this file defines: separate
|
|
# stacks, separate volumes, separate DB roles, separate CPU limits.
|
|
|
|
# thermograph_topology <env>
|
|
#
|
|
# Exports the TG_* variables describing that environment. Every value is a
|
|
# derived fact about the estate, not a preference: change one here and the
|
|
# deploy scripts, the stack files and the runbooks all follow.
|
|
thermograph_topology() {
|
|
local env_name="${1:?thermograph_topology needs an environment: dev|beta|prod}"
|
|
|
|
# Reset first: this function is sourced into long-lived shells (deploy.sh
|
|
# sources it before and after its self re-exec), and a stale TG_STACK_NAME
|
|
# from a previous call would silently target the wrong stack.
|
|
TG_ENV=""; TG_HOST=""; TG_APP_DIR=""; TG_BRANCH=""; TG_DEPLOY_MODE=""
|
|
TG_STACK_NAME=""; TG_STACK_FILE=""; TG_LB_CONFIG=""
|
|
TG_COMPOSE_PROJECT=""; TG_COMPOSE_FILE=""
|
|
TG_ENV_FILE=""; TG_STACK_ENV_FILE=""; TG_LB_NAME=""
|
|
TG_LB_HTTP_PORT=""; TG_LB_FE_PORT=""; TG_DB_NAME=""; TG_DB_USER=""
|
|
TG_TAGS_FILE=""; TG_LOCK_FILE=""; TG_BIND_ADDR=""; TG_SKIP_COMMON=0
|
|
TG_SVC_PREFIX=""; TG_DATA_NETWORK=""; TG_DB_SERVICE=""; TG_POST_DEPLOY=0
|
|
TG_SSH_HOST=""; TG_SSH_TARGET=""
|
|
|
|
case "$env_name" in
|
|
prod)
|
|
TG_ENV=prod
|
|
TG_HOST=vps2
|
|
# Unchanged from before the split — prod keeps every path it already has,
|
|
# so nothing about the live prod host moves during the cutover.
|
|
TG_SSH_HOST=169.58.46.181
|
|
TG_SSH_TARGET=agent@169.58.46.181
|
|
TG_APP_DIR=/opt/thermograph
|
|
TG_BRANCH=main
|
|
TG_DEPLOY_MODE=stack
|
|
TG_STACK_NAME=thermograph
|
|
TG_STACK_FILE=deploy/stack/thermograph-stack.yml
|
|
TG_ENV_FILE=/etc/thermograph.env
|
|
TG_STACK_ENV_FILE=/etc/thermograph/stack.env
|
|
TG_LB_NAME=thermograph-lb
|
|
TG_LB_CONFIG=deploy/stack/lb/Caddyfile
|
|
TG_LB_HTTP_PORT=8137
|
|
TG_LB_FE_PORT=8080
|
|
TG_DB_NAME=thermograph
|
|
TG_DB_USER=thermograph
|
|
# Prod's services keep their bare names (web, worker, frontend, ...): the
|
|
# cutover must not touch prod's stack file, its env or its LB config.
|
|
TG_SVC_PREFIX=""
|
|
# Where the database lives. Prod's own overlay, which is `attachable` and
|
|
# which beta joins as an external network.
|
|
TG_DATA_NETWORK=thermograph_internal
|
|
TG_DB_SERVICE=thermograph_db
|
|
# Post-deploy city warm + IndexNow ping. Prod only — see the beta note.
|
|
TG_POST_DEPLOY=1
|
|
;;
|
|
beta)
|
|
TG_ENV=beta
|
|
TG_HOST=vps2
|
|
# Same box, same public IP as prod — beta and prod are two stacks on one
|
|
# Swarm manager now, not two separate hosts. A beta SSH target is
|
|
# therefore prod's blast radius too (see .claude/hooks/prod-guard.sh).
|
|
TG_SSH_HOST=169.58.46.181
|
|
TG_SSH_TARGET=agent@169.58.46.181
|
|
# A SECOND checkout on the same box. Separate from prod's so the two can
|
|
# sit on different commits of this repo, and so `git reset --hard` in one
|
|
# deploy can never yank the tree out from under the other's running
|
|
# deploy. Everything below is likewise a distinct name/port/path from
|
|
# prod's, on purpose: co-location is only safe if nothing is shared by
|
|
# accident. The one deliberate exception is the database SERVER (see
|
|
# TG_DB_* — separate role and database on a shared instance).
|
|
TG_APP_DIR=/opt/thermograph-beta
|
|
TG_BRANCH=main
|
|
TG_DEPLOY_MODE=stack
|
|
TG_STACK_NAME=thermograph-beta
|
|
TG_STACK_FILE=deploy/stack/thermograph-beta-stack.yml
|
|
TG_ENV_FILE=/etc/thermograph-beta.env
|
|
TG_STACK_ENV_FILE=/etc/thermograph/beta-stack.env
|
|
TG_LB_NAME=thermograph-beta-lb
|
|
TG_LB_CONFIG=deploy/stack/lb/Caddyfile.beta
|
|
# NOT 8137/8080: prod's loopback LB already owns those on this host.
|
|
TG_LB_HTTP_PORT=8237
|
|
TG_LB_FE_PORT=8180
|
|
TG_DB_NAME=thermograph_beta
|
|
# Its own role, not prod's `thermograph` superuser-of-its-own-database.
|
|
# One instance is a capacity decision; it is not a decision to let a beta
|
|
# deploy running an unmerged branch read or write the production database.
|
|
TG_DB_USER=thermograph_beta
|
|
# Beta's Swarm services are named beta-web, beta-worker, ... Swarm
|
|
# registers a service's SHORT name as a DNS alias on every network it
|
|
# joins, so two stacks that both call a service `web` on one shared
|
|
# network make `web` ambiguous — prod's frontend could resolve a beta
|
|
# task. Prefixing beta's names removes the collision without editing a
|
|
# single line of prod's stack.
|
|
TG_SVC_PREFIX="beta-"
|
|
# Beta has no db service of its own: it joins prod's overlay (declared
|
|
# `external` in its stack file) purely to reach `db`, and keeps its own
|
|
# `internal` overlay for beta-to-beta traffic.
|
|
TG_DATA_NETWORK=thermograph_internal
|
|
TG_DB_SERVICE=thermograph_db
|
|
# No warm, no IndexNow on beta. IndexNow announces URLs to Bing/DDG/
|
|
# Yandex — from beta that means asking search engines to index
|
|
# beta.thermograph.org, which is the opposite of what beta is for. The
|
|
# city warm is worse than useless here: it spends the shared upstream
|
|
# archive quota to fill a cache that only a rehearsal environment reads.
|
|
# (Both ran on beta under compose; that was a side effect of beta and
|
|
# prod sharing one script, not a decision.)
|
|
TG_POST_DEPLOY=0
|
|
;;
|
|
dev)
|
|
TG_ENV=dev
|
|
TG_HOST=vps1
|
|
# vps1 — Forgejo/Grafana/dev, mesh-only for dev itself (see TG_BIND_ADDR).
|
|
TG_SSH_HOST=75.119.132.91
|
|
TG_SSH_TARGET=agent@75.119.132.91
|
|
TG_APP_DIR=/opt/thermograph-dev
|
|
# The only environment that tracks `dev`; beta and prod both run this
|
|
# repo's `main` (infra is not environment-staged — app code is, via image
|
|
# tags). See the branch model in infra/README.md.
|
|
TG_BRANCH=dev
|
|
# Compose, not Swarm: dev's value is a fast, legible, single-box stack
|
|
# you can `docker compose logs` at. It is the one environment not
|
|
# pretending to be prod.
|
|
TG_DEPLOY_MODE=compose
|
|
TG_COMPOSE_PROJECT=thermograph-dev
|
|
TG_COMPOSE_FILE="docker-compose.yml:docker-compose.dev.yml"
|
|
TG_ENV_FILE=/etc/thermograph.env
|
|
# Dev keeps its OWN Postgres container (a `db` service in its compose
|
|
# project). It is not on the shared instance and must not be: the shared
|
|
# instance lives on vps2 and holds real user data.
|
|
TG_DB_NAME=thermograph
|
|
TG_DB_USER=thermograph
|
|
# LOOPBACK ONLY. vps1 is a public VPS and dev runs whatever branch is in
|
|
# flight, including unreviewed ones, so the stack must never publish on
|
|
# 0.0.0.0 the way it did on the operator's LAN box.
|
|
#
|
|
# It was briefly mesh-only (10.10.0.2). It is now fronted by Caddy at
|
|
# dev.thermograph.org — TLS, HTTP basic auth, and X-Robots-Tag: noindex
|
|
# (see deploy/Caddyfile.vps1) — so the only reachable path is through that
|
|
# site block, and binding loopback is what guarantees it: nothing can
|
|
# reach the app without passing the password, not even from the mesh.
|
|
TG_BIND_ADDR=127.0.0.1
|
|
# Dev NEVER layers common.yaml — that file is the internet-facing fleet's
|
|
# shared production credential set (both S3 keypairs, the VAPID signing
|
|
# key, REGISTRY_TOKEN, the metrics token), and dev runs whatever branch is
|
|
# in flight on the same box as Forgejo and its CI runner. This lives here
|
|
# rather than only in deploy-dev.sh so the protection is a property of the
|
|
# ENVIRONMENT, not of which entry point happened to be used. See the long
|
|
# note in render-secrets.sh.
|
|
TG_SKIP_COMMON=1
|
|
TG_DB_SERVICE=db
|
|
# Dev warms nothing and pings nothing: it must never spend the upstream
|
|
# archive quota, and it must never tell a search engine it exists.
|
|
TG_POST_DEPLOY=0
|
|
;;
|
|
*)
|
|
echo "!! unknown environment '$env_name' (expected dev|beta|prod)" >&2
|
|
return 2
|
|
;;
|
|
esac
|
|
|
|
# Derived, never hand-set: keeping these next to the definitions above is what
|
|
# stops a second environment on one host from sharing a lock or a tag file.
|
|
TG_LOCK_FILE="$TG_APP_DIR/infra/deploy/.deploy.lock"
|
|
if [ "$TG_DEPLOY_MODE" = stack ]; then
|
|
TG_TAGS_FILE="$TG_APP_DIR/infra/deploy/.stack-image-tags.env"
|
|
else
|
|
TG_TAGS_FILE="$TG_APP_DIR/infra/deploy/.image-tags.env"
|
|
fi
|
|
|
|
export TG_ENV TG_HOST TG_APP_DIR TG_BRANCH TG_DEPLOY_MODE
|
|
export TG_STACK_NAME TG_STACK_FILE TG_LB_CONFIG TG_COMPOSE_PROJECT TG_COMPOSE_FILE
|
|
export TG_ENV_FILE TG_STACK_ENV_FILE TG_LB_NAME
|
|
export TG_LB_HTTP_PORT TG_LB_FE_PORT TG_DB_NAME TG_DB_USER
|
|
export TG_TAGS_FILE TG_LOCK_FILE TG_BIND_ADDR TG_SKIP_COMMON
|
|
export TG_SVC_PREFIX TG_DATA_NETWORK TG_DB_SERVICE TG_POST_DEPLOY
|
|
export TG_SSH_HOST TG_SSH_TARGET
|
|
}
|
|
|
|
# thermograph_env_name
|
|
#
|
|
# Which environment is this run for? Explicit input first (the deploy workflows
|
|
# pass THERMOGRAPH_ENV, and on vps2 it is the ONLY thing distinguishing a beta
|
|
# deploy from a prod one), then the host marker for a by-hand run on a
|
|
# single-environment box. Empty output means "cannot tell" and the caller must
|
|
# fail rather than guess — guessing wrong on vps2 means deploying beta's image
|
|
# tag onto prod.
|
|
thermograph_env_name() {
|
|
local marker="${THERMOGRAPH_SECRETS_ENV_FILE:-/etc/thermograph/secrets-env}"
|
|
if [ -n "${THERMOGRAPH_ENV:-}" ]; then
|
|
printf '%s\n' "$THERMOGRAPH_ENV"
|
|
return 0
|
|
fi
|
|
cat "$marker" 2>/dev/null || true
|
|
}
|