Jinemi/thermograph
4
0
Fork 0
Code Issues 2 Pull requests 9 Projects Releases Packages Wiki Activity Actions

deploy.sh + compose: two independent app images, per-service rolls

Browse source 
backend and frontend now reference their own images (BACKEND_IMAGE_PATH/TAG,
FRONTEND_IMAGE_PATH/TAG) instead of the shared emi/thermograph/app. deploy.sh
takes SERVICE=backend|frontend|all, rolls just that service with --no-deps,
persists each service's live tag in deploy/.image-tags.env so a single-service
deploy never disturbs the sibling, and fixes the old IMAGE_TAG:? guard that
made every workflow deploy hard-fail. Health-checks per service (8137/8080).

Claude-Session: https://claude.ai/code/session_01RdARHDJaYC1wSQRTYM6t3Z
This commit is contained in:
Emi Griffith 2026-07-22 11:39:16 -07:00
parent 0e3196fa05
commit ce454e0be6
2 changed files with 168 additions and 72 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

192
deploy/deploy.sh
View file

@ -1,22 +1,45 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Pull this INFRA repo's checkout up to date, then roll the docker-compose stack # Pull this INFRA repo's checkout up to date, then roll ONE (or all) of the
# (backend + frontend + PostgreSQL) onto the app image tagged IMAGE_TAG. Run on # docker-compose app services onto its separately-published image. Run on the
# the VPS — the app repo's Forgejo Actions workflow invokes this over SSH (see # VPS — each app repo's Forgejo Actions workflow invokes this over SSH (see
# .forgejo/workflows/deploy.yml there), and you can run it by hand too. # .forgejo/workflows/deploy.yml in thermograph-backend / thermograph-frontend),
# and you can run it by hand too.
# #
# ssh deploy@vps 'IMAGE_TAG=sha-<12hex> /opt/thermograph/deploy/deploy.sh' # # roll just the backend onto a specific image:
# ssh deploy@vps 'SERVICE=backend BACKEND_IMAGE_TAG=sha-<12hex> /opt/thermograph/deploy/deploy.sh'
# # roll just the frontend:
# ssh deploy@vps 'SERVICE=frontend FRONTEND_IMAGE_TAG=sha-<12hex> /opt/thermograph/deploy/deploy.sh'
# # bring the whole stack up (both tags required):
# ssh deploy@vps 'SERVICE=all BACKEND_IMAGE_TAG=sha-<a> FRONTEND_IMAGE_TAG=sha-<b> /opt/thermograph/deploy/deploy.sh'
# #
# This checkout is thermograph-infra, not the app repo: BRANCH is this repo's # FE/BE CI-CD split: backend and frontend are published from separate repos as
# separate images (emi/thermograph-backend/app, emi/thermograph-frontend/app),
# so a deploy targets ONE service and leaves the other's running container +
# tag untouched. Each service's live tag is persisted host-side in
# deploy/.image-tags.env (untracked -- survives the git reset below) so a
# single-service roll re-renders compose with BOTH services' real tags and
# never accidentally recreates or downgrades the sibling.
#
# This checkout is thermograph-infra, not an app repo: BRANCH is this repo's
# branch (compose files, db init, the secrets vault, this script itself); # branch (compose files, db init, the secrets vault, this script itself);
# IMAGE_TAG is the separately-published app image to run (see the IMAGE_TAG # the *_IMAGE_TAG values are the separately-published app images to run -- the
# guard below) -- the two are independent and rarely change together. # two axes are independent and rarely change together.
set -euo pipefail set -euo pipefail
APP_DIR="${APP_DIR:-/opt/thermograph}" APP_DIR="${APP_DIR:-/opt/thermograph}"
BRANCH="${BRANCH:-main}" BRANCH="${BRANCH:-main}"
# Which service this deploy rolls: backend | frontend | all. Defaults to `all`
# (a full-stack bring-up) so a by-hand run with both tags still works; the
# per-repo deploy.yml workflows always pass an explicit single service.
SERVICE="${SERVICE:-all}"
HEALTH_PORT="${HEALTH_PORT:-8137}" HEALTH_PORT="${HEALTH_PORT:-8137}"
cd "$APP_DIR" cd "$APP_DIR"
case "$SERVICE" in
backend|frontend|all) ;;
*) echo "!! SERVICE must be backend|frontend|all, got '$SERVICE'" >&2; exit 2 ;;
esac
# Secrets (POSTGRES_PASSWORD, VAPID keys, AUTH_SECRET, ...) drive compose # Secrets (POSTGRES_PASSWORD, VAPID keys, AUTH_SECRET, ...) drive compose
# interpolation and are also loaded into the backend container via env_file. # interpolation and are also loaded into the backend container via env_file.
# #
@ -76,31 +99,61 @@ if [ -z "${DEPLOY_SH_REEXECED:-}" ]; then
exec "$0" "$@" exec "$0" "$@"
fi fi
# Registry-pull cutover (repo-split Stage 6): pull the app image build-push.yml # Registry-pull cutover: pull the image each app repo's build-push.yml already
# (in the APP repo) already built and pushed, instead of building in place. This # built and pushed, instead of building in place. This checkout is
# checkout is thermograph-infra, not the app repo, so there is no "current commit" # thermograph-infra, not an app repo, so there's no "current commit" to derive
# to derive the tag from the way the old same-repo deploy.sh once did -- IMAGE_TAG # a tag from -- the caller (the deploying repo's deploy.yml) exports its own
# must be supplied explicitly (e.g. by the caller exporting # service's tag (BACKEND_IMAGE_TAG or FRONTEND_IMAGE_TAG = sha-<12 hex> of the
# IMAGE_TAG="sha-<12 hex>" matching the app-repo commit to run, or a semver tag). # app commit, or a semver tag).
REGISTRY_HOST="${REGISTRY_HOST:-git.thermograph.org}" REGISTRY_HOST="${REGISTRY_HOST:-git.thermograph.org}"
IMAGE_PATH="${IMAGE_PATH:-emi/thermograph/app}" export REGISTRY_HOST BACKEND_IMAGE_PATH FRONTEND_IMAGE_PATH
IMAGE_TAG="${IMAGE_TAG:?set IMAGE_TAG=sha-<12-hex-app-commit> (or a semver tag) -- this checkout can't derive it, it isn't the app repo}"
export IMAGE_TAG REGISTRY_HOST IMAGE_PATH # Load the last-deployed tag for BOTH services first, so a single-service roll
# still renders compose with the sibling's real, currently-running tag (never a
# bare `local` that would recreate/downgrade it). The incoming env for the
# service being deployed then overrides its line below. This file is untracked
# (see .gitignore), so `git reset --hard` above leaves it in place.
TAGS_FILE="$APP_DIR/deploy/.image-tags.env"
if [ -f "$TAGS_FILE" ]; then
set -a; . "$TAGS_FILE"; set +a
fi
# Guard: the service(s) being rolled MUST have a concrete tag supplied now (the
# sibling's may come from the persisted file). `all` needs both.
case "$SERVICE" in
backend) : "${BACKEND_IMAGE_TAG:?set BACKEND_IMAGE_TAG=sha-<12-hex> for a backend deploy}" ;;
frontend) : "${FRONTEND_IMAGE_TAG:?set FRONTEND_IMAGE_TAG=sha-<12-hex> for a frontend deploy}" ;;
all)
: "${BACKEND_IMAGE_TAG:?set BACKEND_IMAGE_TAG=sha-<12-hex> (SERVICE=all needs both)}"
: "${FRONTEND_IMAGE_TAG:?set FRONTEND_IMAGE_TAG=sha-<12-hex> (SERVICE=all needs both)}"
;;
esac
# Compose interpolates both vars for the whole file even when we act on one
# service; default the not-yet-known sibling (first-ever deploy) to `local` so
# interpolation doesn't warn -- harmless since --no-deps never touches it.
export BACKEND_IMAGE_TAG="${BACKEND_IMAGE_TAG:-local}"
export FRONTEND_IMAGE_TAG="${FRONTEND_IMAGE_TAG:-local}"
# Which compose services this run pulls/rolls.
case "$SERVICE" in
backend) TARGETS=(backend) ;;
frontend) TARGETS=(frontend) ;;
all) TARGETS=(backend frontend) ;;
esac
echo "==> Logging in to the registry ($REGISTRY_HOST)" echo "==> Logging in to the registry ($REGISTRY_HOST)"
echo "$REGISTRY_TOKEN" | docker login "$REGISTRY_HOST" --username emi --password-stdin echo "$REGISTRY_TOKEN" | docker login "$REGISTRY_HOST" --username emi --password-stdin
echo "==> Pulling images ($IMAGE_TAG)" echo "==> Pulling images (backend=$BACKEND_IMAGE_TAG frontend=$FRONTEND_IMAGE_TAG; rolling: ${TARGETS[*]})"
# Retry: build-push.yml (triggered by the same push) has no ordering # Retry: build-push.yml (triggered by the same push) has no ordering guarantee
# guarantee against this deploy -- Forgejo/GitHub Actions `needs:` only # against this deploy -- Forgejo Actions `needs:` only works between jobs in ONE
# works between jobs in ONE workflow file, not across two workflows # workflow file, not across the separate build-push.yml triggered by the same
# triggered by the same event. Confirmed live: a real deploy raced ahead of # event. Confirmed live: a deploy raced ahead of the push and failed with "not
# the push and failed with "not found". A bounded retry (~5 min) comfortably # found". A bounded retry (~5 min) covers a normal build; a genuine problem
# covers a normal build; a genuine problem (bad tag, registry down) still # (bad tag, registry down) still fails loudly after that.
# fails loudly after that, just a bit slower to surface.
pull_ok=0 pull_ok=0
for i in $(seq 1 30); do for i in $(seq 1 30); do
if docker compose pull backend frontend; then if docker compose pull "${TARGETS[@]}"; then
pull_ok=1 pull_ok=1
break break
fi fi
@ -112,32 +165,67 @@ if [ "$pull_ok" != 1 ]; then
exit 1 exit 1
fi fi
# Schema migrations run inside the backend container's entrypoint (alembic # Roll only the target service(s). Backend schema migrations run inside its own
# upgrade head) before uvicorn starts, so there's no separate migrate step or # entrypoint (alembic upgrade head) before uvicorn, so there's no separate
# service restart here — compose owns the process model. frontend has no # migrate step; frontend is stateless.
# migrations (stateless). --remove-orphans matters whenever the compose #
# service topology itself changes (e.g. the old single `app` service -> # Single-service rolls use --no-deps so recreating backend doesn't also bounce
# `backend`+`frontend`, repo-split Stage 4): compose only manages containers # db, and recreating frontend doesn't touch backend -- that independence is the
# for services CURRENTLY defined in the file, so a plain `up -d` leaves a # whole point of the split. A full `all` deploy instead uses --remove-orphans,
# renamed-away service's old container running and squatting its port # which matters when the service topology itself changes (a renamed-away
# forever -- confirmed live, this is exactly what blocked beta's first # service's old container would otherwise keep running and squat its port --
# dual-service deploy ("port is already allocated"). # confirmed live, this is what blocked beta's first dual-service deploy with
echo "==> Starting stack" # "port is already allocated").
docker compose up -d --remove-orphans echo "==> Rolling ${TARGETS[*]}"
if [ "$SERVICE" = all ]; then
docker compose up -d --remove-orphans
else
docker compose up -d --no-deps "${TARGETS[@]}"
fi
echo "==> Health check" # Persist the now-live tags so the next single-service deploy knows the
url="http://127.0.0.1:${HEALTH_PORT}/" # sibling's real tag. Written after `up` so a failed pull never records a tag
for i in $(seq 1 30); do # that isn't actually running.
if curl -fsS -o /dev/null "$url"; then mkdir -p "$(dirname "$TAGS_FILE")"
echo "==> OK: $url is serving" cat > "$TAGS_FILE" <<EOF
warm_city_archives # Written by deploy.sh -- the image tag each service is currently running.
ping_indexnow # Untracked (see .gitignore); lets a single-service deploy leave the other alone.
exit 0 BACKEND_IMAGE_TAG=$BACKEND_IMAGE_TAG
FRONTEND_IMAGE_TAG=$FRONTEND_IMAGE_TAG
EOF
# Health check the service(s) we rolled: backend on 8137, frontend on 8080.
# (For `all`, backend's `/` serving is the readiness signal the old script used
# and the frontend depends_on backend anyway.)
declare -A HEALTH_PORTS=( [backend]=8137 [frontend]=8080 )
health_ok=1
for svc in "${TARGETS[@]}"; do
port="${HEALTH_PORTS[$svc]}"
url="http://127.0.0.1:${port}/healthz"
echo "==> Health check: $svc ($url)"
ok=0
for i in $(seq 1 30); do
if curl -fsS -o /dev/null "$url"; then ok=1; break; fi
sleep 1
done
if [ "$ok" = 1 ]; then
echo "==> OK: $svc is serving"
else
echo "!! Health check failed for $svc ($url)" >&2
health_ok=0
fi fi
sleep 1
done done
echo "!! Health check failed for $url" >&2
docker compose ps || true if [ "$health_ok" != 1 ]; then
docker compose logs --tail=50 backend || true docker compose ps || true
docker compose logs --tail=50 frontend || true for svc in "${TARGETS[@]}"; do docker compose logs --tail=50 "$svc" || true; done
exit 1 exit 1
fi
# Post-deploy warm/IndexNow only make sense once the backend is (re)deployed --
# they exec inside the backend container. Skip them on a frontend-only roll.
if [ "$SERVICE" = backend ] || [ "$SERVICE" = all ]; then
warm_city_archives
ping_indexnow
fi
exit 0

48
docker-compose.yml
View file

@ -1,14 +1,21 @@
# Thermograph production stack: backend (FastAPI/API + TestClient(app.py)), # Thermograph production stack: backend (FastAPI/API + TestClient(app.py)),
# frontend (the SSR content service), and TimescaleDB (PostgreSQL 18). # frontend (the SSR content service), and TimescaleDB (PostgreSQL 18).
# #
# Repo-split Stage 4: backend and frontend are two containers built from the # FE/BE CI-CD split (finishes repo-split Stage 6/7): backend and frontend are
# SAME image (THERMOGRAPH_SERVICE_ROLE picks which process runs -- see # two containers, each running its OWN image published by its OWN repo's
# Dockerfile/deploy/entrypoint.sh), not one "app" service anymore. Both get # build-push.yml -- backend = ${BACKEND_IMAGE_PATH}, frontend =
# their own loopback-published port since prod/beta's host Caddy path-splits # ${FRONTEND_IMAGE_PATH}, tagged independently by BACKEND_IMAGE_TAG /
# directly to each; backend also gets a reverse-proxy fallback to frontend # FRONTEND_IMAGE_TAG. This replaces the earlier Stage-4 model where both
# (THERMOGRAPH_FRONTEND_BASE_INTERNAL) for any environment with no Caddy in # containers shared the single emi/thermograph/app image and
# front (LAN dev, bare-metal run.sh) -- see backend/web/app.py's # THERMOGRAPH_SERVICE_ROLE picked the process; the split Dockerfiles now start
# _proxy_to_frontend and the repo-split plan's Stage 4 section. # the right process directly (frontend `uvicorn app:app`, backend
# entrypoint.sh), so a backend deploy and a frontend deploy are fully
# independent -- deploy.sh rolls one service without touching the other's tag.
# Both get their own loopback-published port since prod/beta's host Caddy
# path-splits directly to each; backend also gets a reverse-proxy fallback to
# frontend (THERMOGRAPH_FRONTEND_BASE_INTERNAL) for any environment with no
# Caddy in front (LAN dev, bare-metal run.sh) -- see backend/web/app.py's
# _proxy_to_frontend.
# #
# docker compose up -d --build # or: make up # docker compose up -d --build # or: make up
# #
@ -79,14 +86,14 @@ services:
# compose network. Nothing outside the stack should touch the database. # compose network. Nothing outside the stack should touch the database.
backend: backend:
build: . # Backend's OWN image, published by thermograph-backend's build-push.yml.
# Named so `docker compose pull` can fetch a prebuilt tag instead of # No `build:` here -- infra holds no Dockerfile; each service's Dockerfile
# `--build`ing in place (repo-split Stage 6's registry-pull cutover -- # lives in its own app repo. deploy.sh sets BACKEND_IMAGE_TAG to the SHA
# deploy.sh/Terraform set IMAGE_TAG to the SHA build-push.yml just pushed # build-push.yml pushed for the backend commit being deployed; local dev
# for the branch being deployed). Harmless default for local/CI # builds `emi/thermograph-backend/app:local` from the backend repo (see
# `--build` usage, which never needs to resolve this against the # infra Makefile) and this pulls/uses it. BACKEND_IMAGE_PATH/TAG are
# registry -- `build:` + `image:` together just tag the local build. # independent of the frontend's, so the two services deploy separately.
image: ${REGISTRY_HOST:-git.thermograph.org}/${IMAGE_PATH:-emi/thermograph/app}:${IMAGE_TAG:-local} image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-emi/thermograph-backend/app}:${BACKEND_IMAGE_TAG:-local}
depends_on: depends_on:
db: db:
condition: service_healthy condition: service_healthy
@ -136,10 +143,11 @@ services:
restart: unless-stopped restart: unless-stopped
frontend: frontend:
build: . # Frontend's OWN image, published by thermograph-frontend's build-push.yml
# Same image as backend (see backend's `image:` comment above) -- # from its own Dockerfile (which starts `uvicorn app:app` directly -- no
# THERMOGRAPH_SERVICE_ROLE below is what picks the process. # THERMOGRAPH_SERVICE_ROLE process-picking anymore). Independent
image: ${REGISTRY_HOST:-git.thermograph.org}/${IMAGE_PATH:-emi/thermograph/app}:${IMAGE_TAG:-local} # FRONTEND_IMAGE_TAG so a frontend deploy never disturbs the backend's tag.
image: ${REGISTRY_HOST:-git.thermograph.org}/${FRONTEND_IMAGE_PATH:-emi/thermograph-frontend/app}:${FRONTEND_IMAGE_TAG:-local}
# Its own register() fetches the IndexNow key from backend at boot -- must # Its own register() fetches the IndexNow key from backend at boot -- must
# wait for a real, healthy backend, not just a started container. # wait for a real, healthy backend, not just a started container.
depends_on: depends_on: