Compare commits

...

3 commits

Author SHA1 Message Date
0eb4f63895 Merge pull request 'promote dev -> main: registry host migration to dev.jinemi.com' (#162) from dev into main
Some checks failed
Sync infra to hosts / sync-dev (push) Has been skipped
Sync infra to hosts / sync-centralis (push) Failing after 6s
Sync infra to hosts / sync-beta (push) Successful in 10s
secrets-guard / encrypted (push) Successful in 12s
Validate observability stack / validate (push) Successful in 15s
Sync infra to hosts / sync-prod (push) Successful in 10s
Build + push images (Forgejo registry) / build-push (frontend) (push) Successful in 39s
Build + push images (Forgejo registry) / build-push (backend) (push) Successful in 42s
shell-lint / shellcheck (push) Successful in 12s
Deploy / deploy (frontend) (push) Successful in 40s
Deploy / deploy (backend) (push) Successful in 1m56s
secrets-guard / encrypted (pull_request) Successful in 8s
shell-lint / shellcheck (pull_request) Successful in 10s
2026-08-01 23:25:00 +00:00
Emi Griffith
53e2eb7c84 registry: move the registry host to dev.jinemi.com, MCP to mcp.jinemi.com
All checks were successful
Sync infra to hosts / sync-beta (push) Has been skipped
Sync infra to hosts / sync-prod (push) Has been skipped
Sync infra to hosts / sync-centralis (push) Has been skipped
Sync infra to hosts / sync-dev (push) Successful in 9s
secrets-guard / encrypted (push) Successful in 5s
Validate observability stack / validate (push) Successful in 12s
shell-lint / shellcheck (push) Successful in 8s
Build + push images (Forgejo registry) / build-push (frontend) (push) Successful in 26s
Build + push images (Forgejo registry) / build-push (backend) (push) Successful in 1m10s
Deploy / deploy (backend) (push) Successful in 1m41s
Deploy / deploy (frontend) (push) Successful in 1m48s
secrets-guard / encrypted (pull_request) Successful in 8s
shell-lint / shellcheck (pull_request) Successful in 9s
PR build (required check) / changes (pull_request) Successful in 20s
PR build (required check) / validate-observability (pull_request) Successful in 15s
PR build (required check) / build-frontend (pull_request) Successful in 17s
PR build (required check) / build-backend (pull_request) Successful in 2m26s
PR build (required check) / gate (pull_request) Successful in 1s
Completes the Forgejo domain migration. ROOT_URL moved to dev.jinemi.com
earlier; the registry half was deliberately deferred. Every image name,
REGISTRY_HOST default, runner label and --add-host pin now names
dev.jinemi.com, so the registry host and the bearer-token realm agree again.

Both names address the same Forgejo, so no image needs re-pushing and a
rollback to a tag pushed under the old prefix still resolves.
git.thermograph.org therefore stays served off the same Caddy site block --
one block, so the /v2/* mesh-only matcher keeps covering both names -- for
pre-migration tags and for runners holding it as their registered instance
URL.

Mesh clients now pin both names in /etc/hosts: the new one as registry host
and token realm, the old one for pre-migration tags. runner-vps2/config.yaml
carries both --add-host entries for the same reason.

Also renames Centralis' endpoint to mcp.jinemi.com in the two places this
repo names it; Centralis itself is provisioned outside this repo.

Host-side steps this cannot do (documented in deploy/forgejo/README.md,
"Host-side steps"): the Forgejo Actions variable REGISTRY_HOST, docker login
against the new host, and the /etc/hosts pins.
2026-08-01 16:06:22 -07:00
Emi Griffith
4dcd6775c4 make: add root install/up/down for the local stack
All checks were successful
secrets-guard / encrypted (pull_request) Successful in 5s
shell-lint / shellcheck (pull_request) Successful in 7s
PR build (required check) / changes (pull_request) Successful in 19s
PR build (required check) / build-backend (pull_request) Has been skipped
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / gate (pull_request) Successful in 1s
Sync infra to hosts / sync-beta (push) Has been skipped
Sync infra to hosts / sync-prod (push) Has been skipped
Sync infra to hosts / sync-centralis (push) Has been skipped
secrets-guard / encrypted (push) Successful in 6s
Sync infra to hosts / sync-dev (push) Successful in 12s
shell-lint / shellcheck (push) Successful in 8s
A fresh checkout had no working path to the stack locally. Neither compose
file carries a `build:` for backend or frontend -- each ships as its own
registry image -- so with no image present `make dev-up` fell through to
pulling the `:local` tag, which nothing publishes, and failed with 403
against a registry a laptop has no login for.

`install` builds both images from backend/Dockerfile and frontend/Dockerfile
under exactly the tag the compose files default to; `up` verifies they exist
and says what to run instead of retrying the pull. Image names are read back
from `docker compose config --images` rather than restated here, so an org
rename cannot leave this file building a tag nothing runs.

infra's dev-up/dev-down now set COMPOSE_PROJECT_NAME=thermograph-dev, which
matches deploy/deploy-dev.sh and what the root CLAUDE.md already described;
a local run previously landed in the prod-shaped `thermograph` project. Drop
dev-up's `--build`, a no-op since the Dockerfiles moved out of infra, and
correct the comment claiming a 0.0.0.0 default -- the overlay has published
on loopback since dev moved to a public VPS.
2026-08-01 13:14:32 -07:00
39 changed files with 343 additions and 157 deletions

View file

@ -16,9 +16,14 @@ name: Build + push images (Forgejo registry)
# docker.sock automounted (Docker-outside-of-Docker, no privileged mode); the # docker.sock automounted (Docker-outside-of-Docker, no privileged mode); the
# job image (node:20-bookworm) ships no docker CLI, hence the install step. # job image (node:20-bookworm) ships no docker CLI, hence the install step.
# #
# The registry is deliberately mesh-only: git.thermograph.org's public DNS # The registry is deliberately mesh-only: dev.jinemi.com's public DNS resolves
# resolves to beta's public IP, whose Caddy ACL rejects /v2/ paths, so any # to vps1's public IP, whose Caddy ACL rejects /v2/ paths, so any runner host
# runner host that pushes or pulls needs an /etc/hosts entry. # that pushes or pulls needs an /etc/hosts entry pinning it to 10.10.0.2.
#
# The host itself comes from the Forgejo Actions variable REGISTRY_HOST, not
# from this file — a repo-side rename (git.thermograph.org -> dev.jinemi.com)
# does nothing until that variable is updated too. See
# infra/deploy/forgejo/README.md, "Migrating the domain".
on: on:
push: push:

View file

@ -34,8 +34,10 @@ own Postgres container, reachable only from vps1 itself (`127.0.0.1:8137` —
reach it either; use an SSH tunnel): no public DNS record, no Caddy site, no reach it either; use an SSH tunnel): no public DNS record, no Caddy site, no
TLS. It is deployed TLS. It is deployed
by CI like beta and prod. The desktop hosts no Thermograph environment at all; by CI like beta and prod. The desktop hosts no Thermograph environment at all;
`make dev-up` there is a laptop convenience for running the stack locally, not the root `Makefile` (`make install` once, then `make up` / `make down`, wrapping
a deployment target. `infra`'s `dev-up`/`dev-down`) is a laptop convenience for running the stack
locally, not a deployment target. `install` is what builds the `:local` image
tags those compose files default to — nothing publishes them.
**One workflow deploys everything.** `deploy.yml` handles both services and all **One workflow deploys everything.** `deploy.yml` handles both services and all
three environments: the branch selects the environment (`dev` → vps1, `main` three environments: the branch selects the environment (`dev` → vps1, `main`

View file

@ -121,7 +121,7 @@ Checked every split-repo branch by dry-run merge; migrated everything real:
1. Create `Jinemi/thermograph` on Forgejo (a **new** repo — do not resurrect the 1. Create `Jinemi/thermograph` on Forgejo (a **new** repo — do not resurrect the
archived monorepo) and push `main`. archived monorepo) and push `main`.
2. Actions config, once: secrets `REGISTRY_TOKEN`, `SSH_HOST/USER/KEY/PORT` 2. Actions config, once: secrets `REGISTRY_TOKEN`, `SSH_HOST/USER/KEY/PORT`
(beta), `PROD_SSH_*` (prod); variable `REGISTRY_HOST=git.thermograph.org`. (beta), `PROD_SSH_*` (prod); variable `REGISTRY_HOST=dev.jinemi.com`.
3. Branch protection on `dev` + `main`: required check `gate`, squash merges, 3. Branch protection on `dev` + `main`: required check `gate`, squash merges,
auto-merge on green. auto-merge on green.
4. Hosts (beta, then prod): clone the monorepo to `/opt/thermograph.new`; 4. Hosts (beta, then prod): clone the monorepo to `/opt/thermograph.new`;

81
Makefile Normal file
View file

@ -0,0 +1,81 @@
# Laptop-local entry point: build the two service images, run the dev stack.
#
# No fleet host uses this file. dev, beta and prod all deploy through
# infra/deploy/deploy.sh, which pulls registry images by their sha-<12hex> tag.
# A laptop has no registry login, so `install` BUILDS both images under the
# `:local` tag that infra/docker-compose.yml already falls back to
# (BACKEND_IMAGE_TAG / FRONTEND_IMAGE_TAG default to `local`). Without that
# build step `up` would try to PULL `:local`, which is published nowhere and
# fails with a 403 -- so `up` checks for the images first and says what to run.
#
# This wraps infra's own compose targets; `make -C infra dev-up` remains
# equivalent to `make up` here, same project name, same overlay, same stack.
SHELL := /bin/bash
# Keeps this stack's volumes off the prod-shaped `thermograph` project that
# infra/docker-compose.yml pins by default (the env var always wins over the
# file's `name:` key).
COMPOSE_PROJECT_NAME ?= thermograph-dev
POSTGRES_PASSWORD ?= thermograph-dev
# The daemon refuses to start without this, and the backend answers 404 on the
# whole /internal/* surface while it is unset -- both fail closed. A throwaway
# value is all a laptop needs; nothing outside this stack accepts it.
THERMOGRAPH_INTERNAL_TOKEN ?= dev-token
export COMPOSE_PROJECT_NAME POSTGRES_PASSWORD THERMOGRAPH_INTERNAL_TOKEN
# The dev overlay publishes backend on $DEV_BIND_ADDR, default loopback. Set
# DEV_BIND_ADDR=0.0.0.0 to reach it from phones on your own Wi-Fi -- safe on a
# laptop LAN, never on a fleet host (see infra/docker-compose.dev.yml).
COMPOSE := docker compose -f infra/docker-compose.yml -f infra/docker-compose.dev.yml
# Ask compose what it will run rather than restating the image names here --
# REGISTRY_HOST / *_IMAGE_PATH / *_IMAGE_TAG all have defaults in
# infra/docker-compose.yml, and a second copy of them in this file is a rename
# away from silently building a tag nothing runs. `--images <service>` also
# lists that service's dependencies, hence the per-service filter. Override by
# passing BACKEND_IMAGE=... / FRONTEND_IMAGE=... if you need a one-off tag.
# POSTGRES_PASSWORD is inlined because `config` refuses to interpolate without
# it, and $(shell) does not see make's `export` directives.
CONFIG_ENV := POSTGRES_PASSWORD=$(POSTGRES_PASSWORD)
ifndef BACKEND_IMAGE
BACKEND_IMAGE := $(shell $(CONFIG_ENV) $(COMPOSE) config --images backend 2>/dev/null | grep -m1 '/backend:')
endif
ifndef FRONTEND_IMAGE
FRONTEND_IMAGE := $(shell $(CONFIG_ENV) $(COMPOSE) config --images frontend 2>/dev/null | grep -m1 '/frontend:')
endif
.DEFAULT_GOAL := help
.PHONY: help install up down logs ps check-images-resolved
help: ## List targets
@grep -hE '^[a-z-]+:.*?## ' $(MAKEFILE_LIST) \
| awk -F':.*?## ' '{printf " \033[36m%-8s\033[0m %s\n", $$1, $$2}'
# Empty means `docker compose config` itself failed -- a broken compose file or
# no docker at all. Fail here rather than running `docker build -t ""`.
check-images-resolved:
@[[ -n "$(BACKEND_IMAGE)" && -n "$(FRONTEND_IMAGE)" ]] || { \
echo "could not resolve image names from infra/docker-compose.yml;" >&2; \
echo "check 'docker compose -f infra/docker-compose.yml config'" >&2; exit 1; }
install: check-images-resolved ## Build backend + frontend images as :local (run once before `up`)
docker build -t $(BACKEND_IMAGE) backend
docker build -t $(FRONTEND_IMAGE) frontend
up: check-images-resolved ## Start the local dev stack, uncapped, backend on 127.0.0.1:8137
@for img in $(BACKEND_IMAGE) $(FRONTEND_IMAGE); do \
docker image inspect "$$img" >/dev/null 2>&1 || { \
echo "missing image $$img -- run 'make install' first" >&2; exit 1; }; \
done
$(COMPOSE) up -d
@echo "backend: http://$${DEV_BIND_ADDR:-127.0.0.1}:8137"
down: ## Stop the stack (named volumes persist)
$(COMPOSE) down
logs: ## Follow the stack's logs (SERVICE=backend to narrow)
$(COMPOSE) logs -f $(SERVICE)
ps: ## Show the stack's containers
$(COMPOSE) ps

View file

@ -26,7 +26,7 @@ services:
# It read admin_emi/thermograph-backend/app (the retired split-era path) until this # It read admin_emi/thermograph-backend/app (the retired split-era path) until this
# was corrected; harmless for the default `:local` build, wrong the moment # was corrected; harmless for the default `:local` build, wrong the moment
# BACKEND_IMAGE_TAG names a real published tag. # BACKEND_IMAGE_TAG names a real published tag.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
depends_on: depends_on:
db: db:
condition: service_healthy condition: service_healthy

View file

@ -14,7 +14,7 @@ export POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-smoke}"
# The image to smoke, pulled from the registry (no local build). CI passes the just- # The image to smoke, pulled from the registry (no local build). CI passes the just-
# pushed sha (BACKEND_IMAGE_TAG=sha-<12hex>); locally it defaults to a published tag. # pushed sha (BACKEND_IMAGE_TAG=sha-<12hex>); locally it defaults to a published tag.
export BACKEND_IMAGE_TAG="${BACKEND_IMAGE_TAG:-v0.0.2-split-ci}" export BACKEND_IMAGE_TAG="${BACKEND_IMAGE_TAG:-v0.0.2-split-ci}"
IMG="${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-admin_emi/thermograph-backend/app}:${BACKEND_IMAGE_TAG}" IMG="${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-admin_emi/thermograph-backend/app}:${BACKEND_IMAGE_TAG}"
export SMOKE_HOST_PORT="${SMOKE_HOST_PORT:-18137}" export SMOKE_HOST_PORT="${SMOKE_HOST_PORT:-18137}"
COMPOSE=(docker compose -f docker-compose.test.yml) COMPOSE=(docker compose -f docker-compose.test.yml)
PORT_URL="http://127.0.0.1:${SMOKE_HOST_PORT}" PORT_URL="http://127.0.0.1:${SMOKE_HOST_PORT}"

View file

@ -60,7 +60,7 @@ all on a WireGuard mesh (`10.10.0.0/24`):
| Host | Mesh IP | Public | Runs | | Host | Mesh IP | Public | Runs |
|---|---|---|---| |---|---|---|---|
| **vps1** | `10.10.0.2` | `75.119.132.91`, git.thermograph.org, dashboard.thermograph.org | Forgejo (git + CI + registry), Grafana + Loki + Alloy, the `emigriffith.dev` portfolio, and **dev** — its own Postgres, plain compose, **mesh-only** (no public DNS, no Caddy site, no TLS) | | **vps1** | `10.10.0.2` | `75.119.132.91`, dev.jinemi.com, dashboard.thermograph.org | Forgejo (git + CI + registry), Grafana + Loki + Alloy, the `emigriffith.dev` portfolio, and **dev** — its own Postgres, plain compose, **mesh-only** (no public DNS, no Caddy site, no TLS) |
| **vps2** | `10.10.0.1` | `169.58.46.181`, thermograph.org, beta.thermograph.org | **prod** and **beta** as two separate Docker **Swarm** stacks, Centralis, Postfix, backups | | **vps2** | `10.10.0.1` | `169.58.46.181`, thermograph.org, beta.thermograph.org | **prod** and **beta** as two separate Docker **Swarm** stacks, Centralis, Postfix, backups |
| **desktop** | `10.10.0.3` | — | AI-model hosting (voice-to-text, an upcoming-feature LLM) + flex Swarm capacity. Hosts **no** Thermograph environment — `make dev-up` there is a laptop convenience only | | **desktop** | `10.10.0.3` | — | AI-model hosting (voice-to-text, an upcoming-feature LLM) + flex Swarm capacity. Hosts **no** Thermograph environment — `make dev-up` there is a laptop convenience only |
| phone | — | — | alerts | | phone | — | — | alerts |
@ -84,10 +84,12 @@ tell beta and prod apart by which host it's running on.
Two consequences you will hit within the first week: Two consequences you will hit within the first week:
- **Forgejo is mesh-only.** `git.thermograph.org` resolves publicly to vps1's - **Forgejo is mesh-only.** `dev.jinemi.com` resolves publicly to vps1's
IP, but vps1's Caddy rejects `/v2/*` (the registry API) from outside the IP, but vps1's Caddy rejects `/v2/*` (the registry API) from outside the
mesh. Any host that pulls images needs `10.10.0.2 git.thermograph.org` in mesh. Any host that pulls images needs
`/etc/hosts`. `10.10.0.2 dev.jinemi.com git.thermograph.org` in `/etc/hosts` — both names;
`git.thermograph.org` is the old registry host, still served because
pre-migration image tags carry it.
- **Beta and prod run the same orchestrator now — Swarm, as two stacks on one - **Beta and prod run the same orchestrator now — Swarm, as two stacks on one
box.** Dev is the only environment on compose (`infra/docker-compose.yml`), box.** Dev is the only environment on compose (`infra/docker-compose.yml`),
and it lives alone on vps1. Most of the time you don't care which stack and it lives alone on vps1. Most of the time you don't care which stack

View file

@ -169,21 +169,37 @@ ends fail closed. With Discord unconfigured it logs once and runs cron-only.
## The full stack, locally ## The full stack, locally
From the repo root:
```bash ```bash
cd infra make install # build both images as :local -- required once, see below
make dev-up # docker-compose.yml + docker-compose.dev.yml overlay: make up # docker-compose.yml + docker-compose.dev.yml overlay:
# uncapped CPU, backend published on 0.0.0.0:8137 for your LAN # uncapped CPU, backend on 127.0.0.1:8137
make dev-down make down
``` ```
`make install` is not optional on a fresh checkout. Neither compose file
carries a `build:` for backend or frontend — each ships as its own registry
image — so with no image present compose tries to **pull**
`dev.jinemi.com/emi/thermograph/backend:local`, a tag published nowhere,
and fails with `403 Forbidden` against a registry you have no login for.
`install` builds that tag locally from `backend/Dockerfile` and
`frontend/Dockerfile`. `make up` checks both images exist and points you here
if they don't.
`make -C infra dev-up` / `dev-down` remain equivalent to `up` / `down` — same
overlay, same project name, same stack.
Backend publishes on `${DEV_BIND_ADDR}`, **loopback by default**. Pass
`DEV_BIND_ADDR=0.0.0.0 make up` to reach it from phones on your own Wi-Fi —
fine on a laptop LAN, never on a fleet host.
This is a **laptop convenience**, not the hosted dev environment: the actual This is a **laptop convenience**, not the hosted dev environment: the actual
`dev` environment now runs on vps1 with its own Postgres, deployed by CI, and `dev` environment now runs on vps1 with its own Postgres, deployed by CI, and
bound only to the WireGuard mesh (`10.10.0.2:8137`) — never `0.0.0.0`, since bound only to the WireGuard mesh (`10.10.0.2:8137`) — never `0.0.0.0`, since
vps1 is a public box. See [Infra and secrets](08-infra-secrets.md). A local vps1 is a public box. See [Infra and secrets](08-infra-secrets.md).
`make dev-up` is fine on `0.0.0.0` because it's your own machine's LAN, not the
public internet.
The dev overlay exports `COMPOSE_PROJECT_NAME=thermograph-dev` so this stack Both entry points set `COMPOSE_PROJECT_NAME=thermograph-dev` so this stack
keeps volumes separate from anything else. **Do not remove either half of the keeps volumes separate from anything else. **Do not remove either half of the
project-name pinning** — `infra/docker-compose.yml` pins `name: thermograph`, project-name pinning** — `infra/docker-compose.yml` pins `name: thermograph`,
and without it running compose from `infra/` derives the project name `infra`, and without it running compose from `infra/` derives the project name `infra`,
@ -202,7 +218,7 @@ database, the ERA5 lake, fleet logs, Grafana, Forgejo, docs and notes, and
Discord. Discord.
```bash ```bash
claude mcp add --transport http centralis https://mcp.thermograph.org/mcp \ claude mcp add --transport http centralis https://mcp.jinemi.com/mcp \
--header "Authorization: Bearer $CENTRALIS_TOKEN" --header "Authorization: Bearer $CENTRALIS_TOKEN"
``` ```

View file

@ -102,7 +102,7 @@ alembic + `/healthz` 200 + a real `/api/v2/place` 200.
Triggers on pushes to `dev`/`main`/`release` touching `backend/**` or Triggers on pushes to `dev`/`main`/`release` touching `backend/**` or
`frontend/**`, and on `v*.*.*` tags. Publishes `frontend/**`, and on `v*.*.*` tags. Publishes
`git.thermograph.org/Jinemi/thermograph/{backend,frontend}:sha-<12hex>`. `dev.jinemi.com/Jinemi/thermograph/{backend,frontend}:sha-<12hex>`.
- **Serialised** (`max-parallel: 1`): one physical runner, and two whole image - **Serialised** (`max-parallel: 1`): one physical runner, and two whole image
builds racing each other buys nothing. builds racing each other buys nothing.
@ -110,7 +110,9 @@ Triggers on pushes to `dev`/`main`/`release` touching `backend/**` or
release tag builds **both** images. That's intended — a release names the release tag builds **both** images. That's intended — a release names the
whole set, not whichever domain moved last. whole set, not whichever domain moved last.
- The registry is mesh-only, so the runner host needs - The registry is mesh-only, so the runner host needs
`10.10.0.2 git.thermograph.org` in `/etc/hosts`. `10.10.0.2 dev.jinemi.com git.thermograph.org` in `/etc/hosts`.
- The registry host itself comes from the Forgejo Actions variable
`REGISTRY_HOST`, not from the workflow file.
### `deploy.yml` — one file, three environments, two services ### `deploy.yml` — one file, three environments, two services

View file

@ -12,7 +12,7 @@ desktop:
| Host | Public | Mesh | Runs | Login | | Host | Public | Mesh | Runs | Login |
|---|---|---|---|---| |---|---|---|---|---|
| **vps1** | `75.119.132.91` / git.thermograph.org, dashboard.thermograph.org | `10.10.0.2` | Forgejo, Grafana + Loki + Alloy, `emigriffith.dev`, and **dev** (own Postgres, compose, mesh-only) | `agent`, passwordless sudo | | **vps1** | `75.119.132.91` / dev.jinemi.com, dashboard.thermograph.org | `10.10.0.2` | Forgejo, Grafana + Loki + Alloy, `emigriffith.dev`, and **dev** (own Postgres, compose, mesh-only) | `agent`, passwordless sudo |
| **vps2** | `169.58.46.181` / thermograph.org, beta.thermograph.org | `10.10.0.1` | **prod** and **beta**, as two co-resident Docker **Swarm** stacks; Centralis; Postfix; backups | `agent`, passwordless sudo | | **vps2** | `169.58.46.181` / thermograph.org, beta.thermograph.org | `10.10.0.1` | **prod** and **beta**, as two co-resident Docker **Swarm** stacks; Centralis; Postfix; backups | `agent`, passwordless sudo |
| **desktop** | — | `10.10.0.3` | AI-model hosting + flex Swarm capacity. No Thermograph environment; `make dev-up` is a laptop convenience there | it's your box | | **desktop** | — | `10.10.0.3` | AI-model hosting + flex Swarm capacity. No Thermograph environment; `make dev-up` is a laptop convenience there | it's your box |

View file

@ -227,10 +227,12 @@ It's executable documentation until state is bootstrapped.
### The registry is mesh-only ### The registry is mesh-only
`git.thermograph.org` resolves publicly to vps1's IP, but vps1's Caddy rejects `dev.jinemi.com` resolves publicly to vps1's IP, but vps1's Caddy rejects
`/v2/*` from outside the WireGuard mesh. Any host that pulls needs `/v2/*` from outside the WireGuard mesh. Any host that pulls needs
`10.10.0.2 git.thermograph.org` in `/etc/hosts`. vps1 itself doesn't — it *is* `10.10.0.2 dev.jinemi.com git.thermograph.org` in `/etc/hosts`**both** names:
the box. the new one is the registry host and the bearer-token realm, the old one still
prefixes every image tag pushed before the migration. vps1 itself doesn't need
the pin — it *is* the box.
### Forgejo runners only have the labels they registered with ### Forgejo runners only have the labels they registered with

View file

@ -29,7 +29,7 @@ services:
# to the split-era v0.0.2-split-ci, under the retired # to the split-era v0.0.2-split-ci, under the retired
# admin_emi/thermograph-backend/app path, long after both were superseded.) # admin_emi/thermograph-backend/app path, long after both were superseded.)
# Override THERMOGRAPH_BACKEND_TEST_TAG to pin a specific build. # Override THERMOGRAPH_BACKEND_TEST_TAG to pin a specific build.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${THERMOGRAPH_BACKEND_TEST_TAG:?set it, or run via scripts/backend-for-tests.sh which derives it} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${THERMOGRAPH_BACKEND_TEST_TAG:?set it, or run via scripts/backend-for-tests.sh which derives it}
depends_on: depends_on:
db: db:
condition: service_healthy condition: service_healthy

View file

@ -57,7 +57,7 @@ THERMOGRAPH_INTERNAL_TOKEN=
# in its own app repo (thermograph-backend / thermograph-frontend) tagged # in its own app repo (thermograph-backend / thermograph-frontend) tagged
# :local, which is the default here -- only set these to pull a specific # :local, which is the default here -- only set these to pull a specific
# published build instead of building locally. # published build instead of building locally.
# REGISTRY_HOST=git.thermograph.org # REGISTRY_HOST=dev.jinemi.com
# BACKEND_IMAGE_PATH=admin_emi/thermograph-backend/app # BACKEND_IMAGE_PATH=admin_emi/thermograph-backend/app
# BACKEND_IMAGE_TAG=local # BACKEND_IMAGE_TAG=local
# FRONTEND_IMAGE_PATH=admin_emi/thermograph-frontend/app # FRONTEND_IMAGE_PATH=admin_emi/thermograph-frontend/app

View file

@ -26,7 +26,7 @@ rather than which environment happens to live there:
| Host | Public IP | Mesh IP | Role | | Host | Public IP | Mesh IP | Role |
|------|-----------|---------|------| |------|-----------|---------|------|
| **vps1** | `75.119.132.91` | `10.10.0.2` | "Operational programs": Forgejo (git + CI + registry; `dev.jinemi.com` canonical, `git.thermograph.org` still served and still used by the registry/mesh pins/runners), Grafana/Loki/Alloy (`dashboard.thermograph.org`), the `emigriffith.dev` portfolio, and the **dev** environment (mesh-only, `10.10.0.2:8137`, no public DNS/Caddy/TLS) | | **vps1** | `75.119.132.91` | `10.10.0.2` | "Operational programs": Forgejo (git + CI + registry, both at `dev.jinemi.com`; `git.thermograph.org` still served for pre-migration image tags and already-registered runners), Grafana/Loki/Alloy (`dashboard.thermograph.org`), the `emigriffith.dev` portfolio, and the **dev** environment (mesh-only, `10.10.0.2:8137`, no public DNS/Caddy/TLS) |
| **vps2** | `169.58.46.181` | `10.10.0.1` | "The deployed environment": **prod** and **beta**, as two separate Docker Swarm stacks, plus Centralis, Postfix, the backups, and the one shared TimescaleDB instance both stacks use | | **vps2** | `169.58.46.181` | `10.10.0.1` | "The deployed environment": **prod** and **beta**, as two separate Docker Swarm stacks, plus Centralis, Postfix, the backups, and the one shared TimescaleDB instance both stacks use |
| **desktop** | — | `10.10.0.3` | AI-model hosting (voice-to-text, an upcoming-feature LLM) plus flex Swarm-worker capacity. Hosts **no** Thermograph environment — `make dev-up` still works there, but only as a laptop-local rehearsal | | **desktop** | — | `10.10.0.3` | AI-model hosting (voice-to-text, an upcoming-feature LLM) plus flex Swarm-worker capacity. Hosts **no** Thermograph environment — `make dev-up` still works there, but only as a laptop-local rehearsal |
@ -70,7 +70,7 @@ Both VPS boxes are provisioned and reachable as of this writing:
| Host | Role | Public IP | Login | | Host | Role | Public IP | Login |
|------|------|-----------|-------| |------|------|-----------|-------|
| vps2 | Swarm manager; prod AND beta live here (`thermograph.org`, `beta.thermograph.org`) | `169.58.46.181` | `agent` | | vps2 | Swarm manager; prod AND beta live here (`thermograph.org`, `beta.thermograph.org`) | `169.58.46.181` | `agent` |
| vps1 | Swarm worker; Forgejo + Grafana/Loki + dev (`git.thermograph.org`, `dashboard.thermograph.org`) | `75.119.132.91` | `agent` | | vps1 | Swarm worker; Forgejo + Grafana/Loki + dev (`dev.jinemi.com`, `dashboard.thermograph.org`) | `75.119.132.91` | `agent` |
| desktop | Swarm worker, AI-model hosting | (local) | (already have access) | | desktop | Swarm worker, AI-model hosting | (local) | (already have access) |
``` ```
@ -179,8 +179,9 @@ The GitHub → Forgejo cutover is complete; this records what was done:
failed login attempt, not just config inspection) failed login attempt, not just config inspection)
- [x] `docker node ls` (from vps2, the manager) shows all three nodes `Ready` - [x] `docker node ls` (from vps2, the manager) shows all three nodes `Ready`
- [x] Swarm ports (2377/7946/4789) unreachable from outside the WireGuard tunnel - [x] Swarm ports (2377/7946/4789) unreachable from outside the WireGuard tunnel
- [x] `https://git.thermograph.org` serves Forgejo over TLS; `/v2/` registry - [x] `https://dev.jinemi.com` serves Forgejo over TLS (and
API is 403 from off-mesh (firewalled to the WireGuard CIDR) `https://git.thermograph.org` still does, off the same site block);
`/v2/` registry API is 403 from off-mesh (firewalled to the WireGuard CIDR)
- [x] A test PR flow verified: required `build` check runs → explicit squash - [x] A test PR flow verified: required `build` check runs → explicit squash
merge → deploy lands on the branch's mapped environment merge → deploy lands on the branch's mapped environment
- [x] GitHub retired as git host + CI; kept only as a read-only history mirror - [x] GitHub retired as git host + CI; kept only as a read-only history mirror

View file

@ -12,10 +12,11 @@ Two VPS boxes plus the operator's desktop, on one WireGuard mesh, named by
ROLE rather than by environment: ROLE rather than by environment:
- **vps1**`75.119.132.91`, mesh `10.10.0.2`. "Operational programs": Forgejo - **vps1**`75.119.132.91`, mesh `10.10.0.2`. "Operational programs": Forgejo
(git + CI + registry; `dev.jinemi.com` is canonical — Forgejo's `ROOT_URL` (git + CI + registry; `dev.jinemi.com` is canonical on both axes — Forgejo's
and `git.thermograph.org` is still served off the same Caddy site block and `ROOT_URL` *and* the registry host in every image name — while
still load-bearing for the registry, mesh pins and runner URLs, so don't `git.thermograph.org` is still served off the same Caddy site block for
retire it; see `deploy/forgejo/README.md`), Grafana/Loki/Alloy pre-migration image tags and already-registered runners, so don't retire it;
see `deploy/forgejo/README.md`), Grafana/Loki/Alloy
(`dashboard.thermograph.org`), the `emigriffith.dev` portfolio site, and the (`dashboard.thermograph.org`), the `emigriffith.dev` portfolio site, and the
**dev** environment (`/opt/thermograph-dev`, tracks `dev`), with its own **dev** environment (`/opt/thermograph-dev`, tracks `dev`), with its own
Postgres container. Dev is mesh-only here — published on `10.10.0.2:8137`, Postgres container. Dev is mesh-only here — published on `10.10.0.2:8137`,

View file

@ -5,7 +5,7 @@ the **`dev`** branch continuously deploys to the **dev environment on vps1**
(`75.119.132.91`, mesh `10.10.0.2`) — `/opt/thermograph-dev`, a normal fleet (`75.119.132.91`, mesh `10.10.0.2`) — `/opt/thermograph-dev`, a normal fleet
checkout reached over SSH exactly like beta and prod, **not** a stack on the checkout reached over SSH exactly like beta and prod, **not** a stack on the
operator's desktop or LAN any more. Git hosting and CI are self-hosted operator's desktop or LAN any more. Git hosting and CI are self-hosted
**Forgejo** (`git.thermograph.org`, also on vps1, reachable at **Forgejo** (`dev.jinemi.com`, also on vps1, reachable at
`http://10.10.0.2:3080` over the WireGuard mesh) — GitHub is retired. `http://10.10.0.2:3080` over the WireGuard mesh) — GitHub is retired.
``` ```
@ -125,13 +125,14 @@ bash deploy/forgejo/register-lan-runner.sh <forgejo_url> <registration_token>
Registry pushes (`build-push.yml`) run against the runner host's own Registry pushes (`build-push.yml`) run against the runner host's own
automounted Docker daemon, so it's that **host's** own resolver that needs to automounted Docker daemon, so it's that **host's** own resolver that needs to
route `git.thermograph.org` over the mesh, not anything settable from a job route `dev.jinemi.com` over the mesh, not anything settable from a job
container. If registry pushes ever start failing with an HTTP/TLS mismatch or container. If registry pushes ever start failing with an HTTP/TLS mismatch or
a 403 on `/v2/`, check `getent hosts git.thermograph.org` on the runner host — a 403 on `/v2/`, check `getent hosts dev.jinemi.com` on the runner host —
it needs to resolve to vps1's WireGuard IP (`10.10.0.2`), typically via a it needs to resolve to vps1's WireGuard IP (`10.10.0.2`), typically via a
`/etc/hosts` line, not the public one (a non-issue if the runner is co-located `/etc/hosts` line, not the public one (a non-issue if the runner is co-located
with Forgejo on vps1 itself, since `git.thermograph.org` then resolves to with Forgejo on vps1 itself, since the name then resolves to itself either
itself either way). way). Check `git.thermograph.org` the same way: it is still served, and a
rollback to an image tag pushed before the domain migration dials it.
## The dev environment (Docker Compose, on vps1) ## The dev environment (Docker Compose, on vps1)

View file

@ -24,7 +24,7 @@ own loopback Caddy LB fronts it with TLS.** Credentials are keyed by **host**
(`VPS2_SSH_*`), not by environment — vps2 answers to both prod and beta, so the (`VPS2_SSH_*`), not by environment — vps2 answers to both prod and beta, so the
environment is passed as data (`THERMOGRAPH_ENV`), not baked into which secret environment is passed as data (`THERMOGRAPH_ENV`), not baked into which secret
is used. (GitHub is retired/archived — Forgejo, self-hosted at is used. (GitHub is retired/archived — Forgejo, self-hosted at
`git.thermograph.org` on **vps1** over the WireGuard mesh, is now the sole git `dev.jinemi.com` on **vps1** over the WireGuard mesh, is now the sole git
host and CI.) host and CI.)
Files that make this work: Files that make this work:
@ -49,14 +49,17 @@ gitignored cache under `data/cache/`, so a `git pull` in one never touches the
other's. other's.
**Any deploy host needs a `/etc/hosts` entry for the registry.** **Any deploy host needs a `/etc/hosts` entry for the registry.**
`git.thermograph.org`'s public DNS resolves to **vps1's** public IP, and `dev.jinemi.com`'s public DNS resolves to **vps1's** public IP, and
vps1's own Caddy rejects `/v2/*` (the registry API) from anything outside the vps1's own Caddy rejects `/v2/*` (the registry API) from anything outside the
WireGuard mesh (10.10.0.0/24) — confirmed live: `docker login`/`pull` from WireGuard mesh (10.10.0.0/24) — confirmed live: `docker login`/`pull` from
vps2 failed with a 403 until adding `10.10.0.2 git.thermograph.org` to vps2 failed with a 403 until adding `10.10.0.2 dev.jinemi.com git.thermograph.org`
`/etc/hosts` (10.10.0.2 is vps1's mesh IP; every deploy host is already on to `/etc/hosts` (10.10.0.2 is vps1's mesh IP; every deploy host is already on
the mesh, this is purely a DNS-routing gap, not a connectivity one). vps1 the mesh, this is purely a DNS-routing gap, not a connectivity one). Pin
itself doesn't need this — it's the box Forgejo runs on, so `git.thermograph.org` **both** names: `dev.jinemi.com` is the registry host and the bearer-token
just resolves to itself either way. `build-push.yml`'s own header comment realm, and `git.thermograph.org` still prefixes every image tag pushed before
the domain migration, so a rollback to one of those dials it directly. vps1
itself doesn't need this — it's the box Forgejo runs on, so both names
just resolve to it either way. `build-push.yml`'s own header comment
documents the same requirement for the CI runner host. documents the same requirement for the CI runner host.
**Current production layout**: prod's Caddy config owns **`thermograph.org`** **Current production layout**: prod's Caddy config owns **`thermograph.org`**

View file

@ -30,20 +30,30 @@ db-up:
db-down: db-down:
docker compose stop db docker compose stop db
# The dev overlay: uncapped (no CPU limits). Default publish is 0.0.0.0:8137, # The dev overlay: uncapped (no CPU limits), backend published on
# right for a laptop-local rehearsal (e.g. phones on the same Wi-Fi) -- this is # ${DEV_BIND_ADDR}, which docker-compose.dev.yml defaults to loopback. Set
# also the *fleet* dev environment's own overlay, but that environment now runs # DEV_BIND_ADDR=0.0.0.0 for a laptop-local rehearsal reachable from phones on
# on vps1 (a public VPS, not a LAN box) and deploys via deploy/deploy-dev.sh, # the same Wi-Fi. This is also the *fleet* dev environment's own overlay, but
# which overrides the bind address to the WireGuard mesh IP only # that environment runs on vps1 (a public VPS, not a LAN box) and deploys via
# (DEV_BIND_ADDR, see deploy/env-topology.sh) -- never invoke `make dev-up` # deploy/deploy-dev.sh, which pins the bind address to the WireGuard mesh IP
# directly on a fleet host, or it publishes on every interface. (The base file # (see deploy/env-topology.sh) -- never invoke `make dev-up` directly on a
# is prod/beta's shape: capped + loopback, fronted by their own Caddy LB.) # fleet host. (The base file is prod/beta's shape: capped + loopback, fronted
# by their own Caddy LB.)
#
# COMPOSE_PROJECT_NAME matches deploy/deploy-dev.sh, so a laptop's dev stack
# keeps its volumes off the `thermograph` project that docker-compose.yml pins
# by default. No `--build`: neither compose file carries a `build:` for backend
# or frontend, so the images must exist first -- the root Makefile's `install`
# builds them under the `:local` tag these files default to.
DEV_COMPOSE = docker compose -f docker-compose.yml -f docker-compose.dev.yml DEV_COMPOSE = docker compose -f docker-compose.yml -f docker-compose.dev.yml
dev-up: dev-up:
POSTGRES_PASSWORD=$${POSTGRES_PASSWORD:-thermograph-dev} $(DEV_COMPOSE) up -d --build COMPOSE_PROJECT_NAME=$${COMPOSE_PROJECT_NAME:-thermograph-dev} \
POSTGRES_PASSWORD=$${POSTGRES_PASSWORD:-thermograph-dev} \
THERMOGRAPH_INTERNAL_TOKEN=$${THERMOGRAPH_INTERNAL_TOKEN:-dev-token} \
$(DEV_COMPOSE) up -d
dev-down: dev-down:
$(DEV_COMPOSE) down COMPOSE_PROJECT_NAME=$${COMPOSE_PROJECT_NAME:-thermograph-dev} $(DEV_COMPOSE) down
# Self-hosted Open-Meteo (prod-only overlay). Serves the ERA5 archive from object # Self-hosted Open-Meteo (prod-only overlay). Serves the ERA5 archive from object
# storage so the app is off the public archive API. See deploy/openmeteo/README.md # storage so the app is off the public archive API. See deploy/openmeteo/README.md

View file

@ -2,11 +2,16 @@
# #
# vps1 serves: # vps1 serves:
# emigriffith.dev -> static portfolio site (from disk) # emigriffith.dev -> static portfolio site (from disk)
# dev.jinemi.com -> Forgejo, CANONICAL (Forgejo's ROOT_URL) # dev.jinemi.com -> Forgejo, CANONICAL: both its ROOT_URL and the
# git.thermograph.org -> Forgejo, same site block — still served and # registry host in every image name
# still needed: the registry host in image names, # git.thermograph.org -> Forgejo, same site block — still served, now
# the mesh /etc/hosts pins and the runners' URLs # only for already-pushed image tags carrying the
# all use it (see deploy/forgejo/caddy-git.conf) # old prefix and for runners registered against
# it (see deploy/forgejo/caddy-git.conf)
#
# Do not confuse `dev.jinemi.com` (Forgejo — git, CI, registry) with
# `dev.thermograph.org` (the dev *application* environment, further down this
# file). Same box, same word, unrelated services.
# dashboard.thermograph.org -> Grafana (see observability/caddy-grafana.conf) # dashboard.thermograph.org -> Grafana (see observability/caddy-grafana.conf)
# #
# and hosts the **dev** environment at dev.thermograph.org (below). # and hosts the **dev** environment at dev.thermograph.org (below).

View file

@ -4,7 +4,7 @@
# thermograph.org -> prod (loopback LB on 127.0.0.1:8137 / :8080) # thermograph.org -> prod (loopback LB on 127.0.0.1:8137 / :8080)
# beta.thermograph.org -> beta (loopback LB on 127.0.0.1:8237 / :8180) # beta.thermograph.org -> beta (loopback LB on 127.0.0.1:8237 / :8180)
# #
# and Centralis at mcp.thermograph.org, which is provisioned separately. # and Centralis at mcp.jinemi.com, which is provisioned separately.
# #
# Each domain's A/AAAA record must already point here — Caddy provisions a # Each domain's A/AAAA record must already point here — Caddy provisions a
# Let's Encrypt cert on first request and auto-renews. Nothing else to do for # Let's Encrypt cert on first request and auto-renews. Nothing else to do for

View file

@ -215,7 +215,7 @@ fi
# a tag from -- the caller (the deploying repo's deploy.yml) exports its own # a tag from -- the caller (the deploying repo's deploy.yml) exports its own
# service's tag (BACKEND_IMAGE_TAG or FRONTEND_IMAGE_TAG = sha-<12 hex> of the # service's tag (BACKEND_IMAGE_TAG or FRONTEND_IMAGE_TAG = sha-<12 hex> of the
# app commit, or a semver tag). # app commit, or a semver tag).
REGISTRY_HOST="${REGISTRY_HOST:-git.thermograph.org}" REGISTRY_HOST="${REGISTRY_HOST:-dev.jinemi.com}"
export REGISTRY_HOST BACKEND_IMAGE_PATH FRONTEND_IMAGE_PATH export REGISTRY_HOST BACKEND_IMAGE_PATH FRONTEND_IMAGE_PATH
# Load the last-deployed tag for BOTH services first, so a single-service roll # Load the last-deployed tag for BOTH services first, so a single-service roll
@ -328,7 +328,7 @@ case " ${TARGETS[*]} " in
# then always found no daemon binary in a Postgres image and dropped # then always found no daemon binary in a Postgres image and dropped
# daemon from EVERY backend deploy, regardless of what the real backend # daemon from EVERY backend deploy, regardless of what the real backend
# image contained -- reproduced and confirmed against beta directly. # image contained -- reproduced and confirmed against beta directly.
daemon_img="${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG}" daemon_img="${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG}"
if ! docker run --rm --entrypoint sh "$daemon_img" -c 'test -x /usr/local/bin/thermograph-daemon' 2>/dev/null; then if ! docker run --rm --entrypoint sh "$daemon_img" -c 'test -x /usr/local/bin/thermograph-daemon' 2>/dev/null; then
echo "==> $daemon_img predates the daemon binary; rolling without the daemon service this run" echo "==> $daemon_img predates the daemon binary; rolling without the daemon service this run"
kept=() kept=()

View file

@ -70,7 +70,7 @@ web port publishes to `127.0.0.1:3080` only (host-local), and vps1's
*existing* Caddy gets one more site block reverse-proxying to it — same *existing* Caddy gets one more site block reverse-proxying to it — same
pattern as its other site blocks, same automatic-HTTPS. pattern as its other site blocks, same automatic-HTTPS.
1. Point the Forgejo domain (default `git.thermograph.org`; override with 1. Point the Forgejo domain (default `dev.jinemi.com`; override with
`FORGEJO_DOMAIN=...` before `docker stack deploy`) at **vps1's** public IP `FORGEJO_DOMAIN=...` before `docker stack deploy`) at **vps1's** public IP
— that's where the task actually runs, not vps2's or the desktop's. — that's where the task actually runs, not vps2's or the desktop's.
2. Append `deploy/forgejo/caddy-git.conf` to vps1's `/etc/caddy/Caddyfile`, 2. Append `deploy/forgejo/caddy-git.conf` to vps1's `/etc/caddy/Caddyfile`,
@ -89,26 +89,54 @@ pattern as its other site blocks, same automatic-HTTPS.
(`FORGEJO_DOMAIN` in `docker-stack.yml`), so it is the name Forgejo generates in (`FORGEJO_DOMAIN` in `docker-stack.yml`), so it is the name Forgejo generates in
clone URLs, the Google OAuth callback, webhook payload URLs and mail links. clone URLs, the Google OAuth callback, webhook payload URLs and mail links.
**Registry: deliberately not done.** Renaming the registry host is a separate **Registry: done in this repo, and it needs host-side steps to match.** Every
migration from renaming the web UI, and the two need not happen together. image name, `REGISTRY_HOST` default, runner label and `--add-host` pin under
version control now says `dev.jinemi.com`. None of that is self-applying — see
"Host-side steps" below before expecting a build to push.
Both names address the *same* Forgejo and therefore the same packages. A tag
pushed as `git.thermograph.org/jinemi/thermograph/backend:sha-abc` is pullable
as `dev.jinemi.com/jinemi/thermograph/backend:sha-abc` — the host part is only
how you reach the registry, not part of the package's identity. That is what
makes this migration safe to do in one step instead of a dual-push period: no
image needs re-pushing, and a rollback to an old tag still resolves.
`caddy-git.conf` lists both names on a *single* site block. That is deliberate, `caddy-git.conf` lists both names on a *single* site block. That is deliberate,
not cosmetic: the `/v2/*` mesh-only matcher is per-block, so a second block for not cosmetic: the `/v2/*` mesh-only matcher is per-block, so a second block for
either name would re-expose the registry API publicly and undo hazard #15. either name would re-expose the registry API publicly and undo hazard #15.
Whatever else changes, keep the two names in one block. Whatever else changes, keep the two names in one block.
**`git.thermograph.org` must stay served.** It is not a courtesy redirect for **`git.thermograph.org` must stay served.** The list of reasons is shorter than
old bookmarks — CI resolves it: it was, but it is not empty, and neither remaining item is a bookmark:
- images are named by registry host, and the `git.thermograph.org/` prefix is - image tags already pushed under the old prefix name that host, so a rollback
baked into the runner labels (`register-lan-runner.sh`, to one asks for `git.thermograph.org/...` even though nothing builds that
`runner-vps2/README.md`), the CI-runner image (`ci-runner/Dockerfile`), name any more;
`REGISTRY_HOST` in `infra/.env.example`, and every already-pushed tag;
- every mesh client in "Registry access from mesh clients" below pins
`git.thermograph.org` to `10.10.0.2` in `/etc/hosts`, which is what makes
Caddy's `/v2/*` matcher see a mesh source IP instead of returning 403;
- registered runners hold the instance URL they registered with - registered runners hold the instance URL they registered with
(`https://git.thermograph.org`); they keep working while that name resolves. (`https://git.thermograph.org`); they keep working while that name resolves,
and stop the moment it doesn't. Re-registering them is the only way to retire
the name, and it is not required for this migration.
### Host-side steps
None of these live in the repo, and CI will not do them for you. In order:
1. **`/etc/hosts` on every mesh client** — pin `dev.jinemi.com` as well as
`git.thermograph.org` (both, for the reasons under "Registry access from
mesh clients" below). A client that resolves the new name publicly gets a
403 from the `/v2/*` matcher, which reads exactly like a bad credential.
2. **`docker login dev.jinemi.com`** on each host that pushes or pulls — the
CI runners, vps2 (prod and beta), the desktop. A login against the old name
does **not** carry over: docker keys stored credentials by registry host.
3. **The Forgejo Actions variable `REGISTRY_HOST`**`dev.jinemi.com`
(Site Administration → Actions → Variables, or the repo's own). This is what
`build-push.yml` reads; the defaults in this repo are fallbacks for by-hand
runs, so leaving the variable stale silently keeps CI on the old name.
4. **Re-register the runners** against `https://dev.jinemi.com`, *optionally*
see above. Until you do, leave `git.thermograph.org` served.
Verify with a push before assuming: the failure mode for a missed step 1 or 2
is `unauthorized: reqPackageAccess`, which names neither.
### Before changing ROOT_URL again ### Before changing ROOT_URL again
@ -139,37 +167,39 @@ effect when someone re-runs `docker stack deploy` by hand on the manager (vps2)
Any node that needs `docker login`/push/pull against the registry (the CI Any node that needs `docker login`/push/pull against the registry (the CI
runner building/pushing images, any Swarm node pulling them, prod or beta on runner building/pushing images, any Swarm node pulling them, prod or beta on
vps2 pulling app images) must reach `git.thermograph.org` **over the vps2 pulling app images) must reach Forgejo **over the WireGuard tunnel**, not
WireGuard tunnel**, not vps1's public IP — otherwise Caddy's `/v2/*` block vps1's public IP — otherwise Caddy's `/v2/*` block above refuses the
above refuses the connection. Public DNS resolves the domain to vps1's public connection. Public DNS resolves both names to vps1's public IP, so add an
IP, so add a `/etc/hosts` override on each such node pinning it to vps1's `/etc/hosts` override on each such node pinning them to vps1's WireGuard
WireGuard address instead: address instead:
``` ```
echo "10.10.0.2 git.thermograph.org" | sudo tee -a /etc/hosts echo "10.10.0.2 dev.jinemi.com git.thermograph.org" | sudo tee -a /etc/hosts
``` ```
(`10.10.0.2` is vps1's WG address per `deploy/swarm/README.md`'s peer (`10.10.0.2` is vps1's WG address per `deploy/swarm/README.md`'s peer
numbering — adjust if you assigned it differently.) The git/web UI keeps numbering — adjust if you assigned it differently.) The git/web UI keeps
working normally for everyone else since only `/v2/*` is restricted. working normally for everyone else since only `/v2/*` is restricted.
**Pin the canonical name too, not just the image host.** Forgejo derives the **Both names, not just the one in the image.** `dev.jinemi.com` now carries two
registry's bearer-token realm from `ROOT_URL`, so a client that already independent jobs and needs the pin for each: it is the host in every image
resolves `git.thermograph.org` over the mesh is still sent to name, *and* it is `ROOT_URL`, from which Forgejo derives the registry's
`https://<ROOT_URL host>/v2/token` to collect a token. When `ROOT_URL` became bearer-token realm — so a client is sent to `https://<ROOT_URL host>/v2/token`
`dev.jinemi.com`, that second request took the public route, the `/v2/*` to collect a token no matter which name it dialled. `git.thermograph.org` stays
matcher above answered 403, and docker fell back to anonymous — every push then pinned for a third reason: image tags pushed before the migration name it, and
failed with `unauthorized: reqPackageAccess`, which reads exactly like a a rollback to one of those dials it directly.
revoked token or a missing scope. It is neither. Pin both names:
``` This is not hypothetical. When `ROOT_URL` first became `dev.jinemi.com` while
echo "10.10.0.2 git.thermograph.org dev.jinemi.com" | sudo tee -a /etc/hosts images were still named `git.thermograph.org`, the token request took the
``` public route, the `/v2/*` matcher answered 403, and docker fell back to
anonymous — every push failed with `unauthorized: reqPackageAccess`, which
reads exactly like a revoked token or a missing scope. It was neither. Pinning
one name and not the other reproduces it in either direction.
The realm moves whenever `ROOT_URL` does, so read it rather than assuming: The realm moves whenever `ROOT_URL` does, so read it rather than assuming:
```sh ```sh
curl -sI https://git.thermograph.org/v2/ | grep -i www-authenticate curl -sI https://dev.jinemi.com/v2/ | grep -i www-authenticate
``` ```
These `/etc/hosts` pins are host state. Nothing in this repo writes them, so a These `/etc/hosts` pins are host state. Nothing in this repo writes them, so a
@ -259,14 +289,14 @@ bug hit during the frontend Go rewrite). `ci-runner` adds `docker-ce-cli` +
`git`/`python3`/`python3-yaml` for the other jobs that need them `git`/`python3`/`python3-yaml` for the other jobs that need them
(`shell-lint`, `observability-validate`). (`shell-lint`, `observability-validate`).
Current tag: `git.thermograph.org/jinemi/thermograph/ci-runner:v2` (`v1` is Current tag: `dev.jinemi.com/jinemi/thermograph/ci-runner:v2` (`v1` is
broken — do not register any runner against it). Rebuild/push (requires a PAT broken — do not register any runner against it). Rebuild/push (requires a PAT
with `write:package` scope — the embedded git-remote token lacks it, same with `write:package` scope — the embedded git-remote token lacks it, same
requirement documented in `build-push.yml`): requirement documented in `build-push.yml`):
```bash ```bash
docker build -t git.thermograph.org/jinemi/thermograph/ci-runner:vN deploy/forgejo/ci-runner docker build -t dev.jinemi.com/jinemi/thermograph/ci-runner:vN deploy/forgejo/ci-runner
docker push git.thermograph.org/jinemi/thermograph/ci-runner:vN docker push dev.jinemi.com/jinemi/thermograph/ci-runner:vN
``` ```
`register-lan-runner.sh`'s `LABELS` default points at the current tag, so `register-lan-runner.sh`'s `LABELS` default points at the current tag, so
@ -291,8 +321,9 @@ independently.
```bash ```bash
docker service ls # forgejo_db, forgejo_forgejo both Running, 1/1 docker service ls # forgejo_db, forgejo_forgejo both Running, 1/1
curl -I https://git.thermograph.org/ # 200, valid cert (Caddy's, not a new one) curl -I https://dev.jinemi.com/ # 200, valid cert (Caddy's, not a new one)
curl -I https://git.thermograph.org/v2/ # 403 from anywhere off the WireGuard mesh curl -I https://dev.jinemi.com/v2/ # 403 from anywhere off the WireGuard mesh
curl -I https://git.thermograph.org/ # 200 too — the alias must stay served
# On the runner host, after registering the runner: # On the runner host, after registering the runner:
systemctl --user status forgejo-runner # active, both labels registered systemctl --user status forgejo-runner # active, both labels registered
``` ```

View file

@ -1,4 +1,4 @@
# Only the block below (from "git.thermograph.org, dev.jinemi.com {") goes into # Only the block below (from "dev.jinemi.com, git.thermograph.org {") goes into
# the live Caddyfile — append just that part, not this instructional header, or # the live Caddyfile — append just that part, not this instructional header, or
# it ends up as a confusing orphaned comment in production config with no # it ends up as a confusing orphaned comment in production config with no
# surrounding context (docker-stack.yml isn't visible from there). # surrounding context (docker-stack.yml isn't visible from there).
@ -14,14 +14,17 @@
# both. Splitting dev.jinemi.com into its own block serves the same Forgejo # both. Splitting dev.jinemi.com into its own block serves the same Forgejo
# with the registry API open to the public internet. # with the registry API open to the public internet.
# #
# dev.jinemi.com is CANONICAL — it is Forgejo's ROOT_URL (docker-stack.yml, # dev.jinemi.com is CANONICAL, now on both axes — it is Forgejo's ROOT_URL
# FORGEJO_DOMAIN), so it is the name Forgejo puts in clone URLs, the OAuth # (docker-stack.yml, FORGEJO_DOMAIN), so it is the name Forgejo puts in clone
# callback, webhook payload URLs and mail links. # URLs, the OAuth callback, webhook payload URLs and mail links; and it is the
# registry host in every image name this repo builds or pulls.
# #
# git.thermograph.org is kept as a served alias, NOT as dead weight: the # git.thermograph.org is kept as a served alias, NOT as dead weight, though the
# registry host baked into image names, the mesh /etc/hosts pins and the # reason narrowed when the registry moved. What still needs it: image tags
# runners' registered instance URL all still say git.thermograph.org. Removing # already pushed under the old prefix (a rollback to one names that host), and
# this name from the block breaks CI, not just old bookmarks. Both names get # runners registered against `https://git.thermograph.org`, which hold the
# instance URL they registered with. Removing this name from the block breaks a
# rollback and de-registers runners — not just old bookmarks. Both names get
# their own Let's Encrypt cert automatically (HTTP-01); both A records point # their own Let's Encrypt cert automatically (HTTP-01); both A records point
# at vps1. See README.md, "Migrating the domain". # at vps1. See README.md, "Migrating the domain".
# #
@ -33,7 +36,7 @@
# --- copy from here down into /etc/caddy/Caddyfile --- # --- copy from here down into /etc/caddy/Caddyfile ---
git.thermograph.org, dev.jinemi.com { dev.jinemi.com, git.thermograph.org {
encode zstd gzip encode zstd gzip
@registry_external { @registry_external {

View file

@ -19,9 +19,9 @@
# class, and skips the per-job install entirely. # class, and skips the per-job install entirely.
# #
# Build/push (manual — see ../README.md for when to rebuild): # Build/push (manual — see ../README.md for when to rebuild):
# docker build -t git.thermograph.org/jinemi/thermograph/ci-runner:vN \ # docker build -t dev.jinemi.com/jinemi/thermograph/ci-runner:vN \
# infra/deploy/forgejo/ci-runner # infra/deploy/forgejo/ci-runner
# docker push git.thermograph.org/jinemi/thermograph/ci-runner:vN # docker push dev.jinemi.com/jinemi/thermograph/ci-runner:vN
# #
# Cutting over the live runner to a new tag means editing the `labels` array # Cutting over the live runner to a new tag means editing the `labels` array
# in ~/forgejo-runner/.runner on the runner host directly (same runner # in ~/forgejo-runner/.runner on the runner host directly (same runner

View file

@ -20,7 +20,7 @@
# box). A second reverse proxy binding those same ports would either fail to # box). A second reverse proxy binding those same ports would either fail to
# start or fight Caddy. Instead: forgejo's web port publishes to # start or fight Caddy. Instead: forgejo's web port publishes to
# 127.0.0.1:3080 only (host-local, mode: host), and vps1's existing Caddy gets # 127.0.0.1:3080 only (host-local, mode: host), and vps1's existing Caddy gets
# a new site block reverse-proxying git.thermograph.org -> 127.0.0.1:3080, # a new site block reverse-proxying dev.jinemi.com -> 127.0.0.1:3080,
# same pattern as its other blocks. TLS is Caddy's existing automatic-HTTPS # same pattern as its other blocks. TLS is Caddy's existing automatic-HTTPS
# (HTTP-01), not a second ACME flow. # (HTTP-01), not a second ACME flow.
# #
@ -82,7 +82,8 @@ services:
FORGEJO__database__USER: forgejo FORGEJO__database__USER: forgejo
# Canonical domain. Forgejo has exactly ONE ROOT_URL and derives every # Canonical domain. Forgejo has exactly ONE ROOT_URL and derives every
# absolute URL from it — clone URLs, the Google OAuth callback, webhook # absolute URL from it — clone URLs, the Google OAuth callback, webhook
# payload URLs, mail links. git.thermograph.org is still served (Caddy # payload URLs, mail links, and the registry's bearer-token realm.
# git.thermograph.org is still served (Caddy
# answers for both names off one site block, deploy/forgejo/caddy-git.conf) # answers for both names off one site block, deploy/forgejo/caddy-git.conf)
# and still works for browsing, git-over-HTTP and the API; it is simply no # and still works for browsing, git-over-HTTP and the API; it is simply no
# longer the name Forgejo generates. # longer the name Forgejo generates.
@ -94,9 +95,12 @@ services:
# git.thermograph.org one are registered; verify with a probe before # git.thermograph.org one are registered; verify with a probe before
# changing this again (deploy/forgejo/README.md, "Migrating the domain"). # changing this again (deploy/forgejo/README.md, "Migrating the domain").
# #
# NOT changed by this: the registry host baked into image names, the mesh # The registry host in image names and the mesh /etc/hosts pins now say
# /etc/hosts pins, and the runners' registered instance URL all still say # dev.jinemi.com too, so ROOT_URL and the registry name agree again — which
# git.thermograph.org. That is a separate migration. # matters, because Forgejo derives the registry's bearer-token realm from
# ROOT_URL. Still on the old name: runners registered against
# https://git.thermograph.org (they hold the URL they registered with) and
# image tags already pushed under the old prefix.
FORGEJO__server__DOMAIN: "${FORGEJO_DOMAIN:-dev.jinemi.com}" FORGEJO__server__DOMAIN: "${FORGEJO_DOMAIN:-dev.jinemi.com}"
FORGEJO__server__ROOT_URL: "https://${FORGEJO_DOMAIN:-dev.jinemi.com}/" FORGEJO__server__ROOT_URL: "https://${FORGEJO_DOMAIN:-dev.jinemi.com}/"
FORGEJO__server__SSH_PORT: "2222" FORGEJO__server__SSH_PORT: "2222"

View file

@ -40,7 +40,7 @@ set -euo pipefail
FORGEJO_URL="${1:?usage: $0 <forgejo_url> <registration_token>}" FORGEJO_URL="${1:?usage: $0 <forgejo_url> <registration_token>}"
TOKEN="${2:?}" TOKEN="${2:?}"
RUNNER_DIR="${RUNNER_DIR:-$HOME/forgejo-runner}" RUNNER_DIR="${RUNNER_DIR:-$HOME/forgejo-runner}"
LABELS="${LABELS:-docker:docker://git.thermograph.org/jinemi/thermograph/ci-runner:v2,thermograph-lan}" LABELS="${LABELS:-docker:docker://dev.jinemi.com/jinemi/thermograph/ci-runner:v2,thermograph-lan}"
echo "==> Stopping and disabling the old GitHub Actions runner service, if present" echo "==> Stopping and disabling the old GitHub Actions runner service, if present"
systemctl --user stop github-actions-runner 2>/dev/null || true systemctl --user stop github-actions-runner 2>/dev/null || true

View file

@ -26,12 +26,14 @@ documentation one.
## Prerequisites ## Prerequisites
- vps2's docker daemon is logged in to `git.thermograph.org` (it is — - vps2's docker daemon is logged in to `dev.jinemi.com`
`/root/.docker/config.json`), so the daemon can pull the job image. (`/root/.docker/config.json`), so the daemon can pull the job image. A login
- `git.thermograph.org` resolves to the **mesh** address from vps2 for registry against the old `git.thermograph.org` does not carry over — docker keys
credentials by registry host, so this needs its own `docker login`.
- `dev.jinemi.com` resolves to the **mesh** address from vps2 for registry
traffic. Job containers get this via `--add-host` in `config.yaml`; the host traffic. Job containers get this via `--add-host` in `config.yaml`; the host
daemon already has it. daemon needs the same pin in `/etc/hosts`.
- The job image exists: `git.thermograph.org/jinemi/thermograph/ci-runner:v2`. - The job image exists: `dev.jinemi.com/jinemi/thermograph/ci-runner:v2`.
Built from `../ci-runner/Dockerfile`; see that file for when to rebuild. Built from `../ci-runner/Dockerfile`; see that file for when to rebuild.
## Where it is deployed, and why not from the checkout ## Where it is deployed, and why not from the checkout
@ -89,14 +91,20 @@ docker run --rm -it \
-v "$PWD/data:/data" -w /data --user root \ -v "$PWD/data:/data" -w /data --user root \
code.forgejo.org/forgejo/runner:6.3.1 \ code.forgejo.org/forgejo/runner:6.3.1 \
forgejo-runner register --no-interactive \ forgejo-runner register --no-interactive \
--instance https://git.thermograph.org \ --instance https://dev.jinemi.com \
--token "$REG_TOKEN" \ --token "$REG_TOKEN" \
--name thermograph-vps2 \ --name thermograph-vps2 \
--labels 'docker:docker://git.thermograph.org/jinemi/thermograph/ci-runner:v2' --labels 'docker:docker://dev.jinemi.com/jinemi/thermograph/ci-runner:v2'
unset REG_TOKEN unset REG_TOKEN
``` ```
The runner registered before the domain migration holds
`https://git.thermograph.org` as its instance URL and keeps working while Caddy
serves that name (it does — see `../caddy-git.conf`). Re-registering with the
command above is how it moves to the canonical name; it is not required, and it
costs a fresh registration token and a runner restart.
That writes `data/.runner`, which holds the **long-lived** runner token. It is That writes `data/.runner`, which holds the **long-lived** runner token. It is
`0600` and must stay out of git — `data/` is gitignored for exactly this reason. `0600` and must stay out of git — `data/` is gitignored for exactly this reason.
Re-running `register` with a fresh registration token replaces it; the old Re-running `register` with a fresh registration token replaces it; the old
@ -147,5 +155,6 @@ only, and the job container has no other way to learn the mesh address.
from the compose file. from the compose file.
**The runner is up but `last_online` is stale.** It cannot reach **The runner is up but `last_online` is stale.** It cannot reach
`https://git.thermograph.org` — check egress from vps2 and that Forgejo on vps1 the instance URL it registered with (`https://dev.jinemi.com`, or
is serving. `https://git.thermograph.org` for a runner registered before the migration) —
check egress from vps2 and that Forgejo on vps1 is serving that name.

View file

@ -25,18 +25,26 @@ runner:
shutdown_timeout: 3h shutdown_timeout: 3h
container: container:
# 1) --add-host: git.thermograph.org resolves publicly to vps1, but vps1's # 1) --add-host: both Forgejo names resolve publicly to vps1, but vps1's
# Caddy rejects the /v2/* registry API from off-mesh. Job containers that # Caddy rejects the /v2/* registry API from off-mesh. Job containers that
# docker-pull or docker-push therefore need the MESH address, and DNS will # docker-pull or docker-push therefore need the MESH address, and DNS will
# not give it to them. Same line the desktop runner carries; the value is # not give it to them. Same line the desktop runner carries; the value is
# the same from vps2 because both boxes are on 10.10.0.0/24. # the same from vps2 because both boxes are on 10.10.0.0/24.
# #
# BOTH names are pinned, and that is not belt-and-braces. dev.jinemi.com is
# the registry host in image names AND Forgejo's ROOT_URL, from which the
# registry's bearer-token realm is derived -- so a pull needs it twice over.
# git.thermograph.org stays pinned because already-pushed tags carry it as
# their prefix; unpinned, a rollback to such a tag takes the public route
# and gets a 403 that reads like a credential failure. See
# ../README.md, "Migrating the domain".
#
# 2) --cpus/--memory: vps2 runs prod. Without a ceiling a runaway job (a # 2) --cpus/--memory: vps2 runs prod. Without a ceiling a runaway job (a
# `docker build` that fans out, a test that leaks) competes with the # `docker build` that fans out, a test that leaks) competes with the
# production app for the same 6 cores. These bound the JOB container, which # production app for the same 6 cores. These bound the JOB container, which
# is a sibling of the runner rather than its child, so the runner's own # is a sibling of the runner rather than its child, so the runner's own
# compose limits do not cover it -- this is the only lever that does. # compose limits do not cover it -- this is the only lever that does.
options: --add-host=git.thermograph.org:10.10.0.2 --cpus=2 --memory=4g options: --add-host=dev.jinemi.com:10.10.0.2 --add-host=git.thermograph.org:10.10.0.2 --cpus=2 --memory=4g
# Jobs may mount NOTHING from the host. The workflows here do not need to: # Jobs may mount NOTHING from the host. The workflows here do not need to:
# they check out into the job container's own workspace and reach the estate # they check out into the job container's own workspace and reach the estate

View file

@ -106,7 +106,7 @@ export STACK_ENV_FILE
# --- image tags ----------------------------------------------------------------- # --- image tags -----------------------------------------------------------------
# Same persisted-tags contract as deploy.sh: incoming env wins, the file # Same persisted-tags contract as deploy.sh: incoming env wins, the file
# supplies the sibling. Stack mode keeps its own file so test/real never mix. # supplies the sibling. Stack mode keeps its own file so test/real never mix.
REGISTRY_HOST="${REGISTRY_HOST:-git.thermograph.org}" REGISTRY_HOST="${REGISTRY_HOST:-dev.jinemi.com}"
export REGISTRY_HOST export REGISTRY_HOST
TAGS_FILE="$APP_DIR/infra/deploy/.stack-image-tags.env" TAGS_FILE="$APP_DIR/infra/deploy/.stack-image-tags.env"
_incoming_backend="${BACKEND_IMAGE_TAG:-}" _incoming_backend="${BACKEND_IMAGE_TAG:-}"

View file

@ -62,7 +62,7 @@
services: services:
beta-web: beta-web:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"] entrypoint: ["/host/env-entrypoint.sh"]
environment: environment:
# Beta's OWN role and OWN database on the shared instance. `db` resolves # Beta's OWN role and OWN database on the shared instance. `db` resolves
@ -108,7 +108,7 @@ services:
failure_action: rollback failure_action: rollback
beta-worker: beta-worker:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"] entrypoint: ["/host/env-entrypoint.sh"]
environment: environment:
THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph_beta:${POSTGRES_PASSWORD}@db:5432/thermograph_beta THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph_beta:${POSTGRES_PASSWORD}@db:5432/thermograph_beta
@ -144,7 +144,7 @@ services:
condition: on-failure condition: on-failure
beta-lake: beta-lake:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"] entrypoint: ["/host/env-entrypoint.sh"]
environment: environment:
THERMOGRAPH_ROLE: lake THERMOGRAPH_ROLE: lake
@ -176,7 +176,7 @@ services:
failure_action: rollback failure_action: rollback
beta-daemon: beta-daemon:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
# Same reasoning as prod's daemon: NOT env-entrypoint.sh, because that shim # Same reasoning as prod's daemon: NOT env-entrypoint.sh, because that shim
# execs the image's own entrypoint (Alembic + uvicorn) and migrations belong # execs the image's own entrypoint (Alembic + uvicorn) and migrations belong
# to the one-shot task. Source the host-rendered env and exec the binary. # to the one-shot task. Source the host-rendered env and exec the binary.
@ -218,7 +218,7 @@ services:
failure_action: rollback failure_action: rollback
beta-frontend: beta-frontend:
image: ${REGISTRY_HOST:-git.thermograph.org}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"] entrypoint: ["/host/env-entrypoint.sh"]
# REQUIRED: overriding `entrypoint:` with no `command:` drops the image's # REQUIRED: overriding `entrypoint:` with no `command:` drops the image's
# CMD entirely, and env-entrypoint.sh's fallback (`exec uvicorn app:app`) # CMD entirely, and env-entrypoint.sh's fallback (`exec uvicorn app:app`)

View file

@ -64,7 +64,7 @@ services:
condition: on-failure condition: on-failure
web: web:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"] entrypoint: ["/host/env-entrypoint.sh"]
environment: environment:
THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph:${POSTGRES_PASSWORD}@db:5432/thermograph THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph:${POSTGRES_PASSWORD}@db:5432/thermograph
@ -109,7 +109,7 @@ services:
failure_action: rollback failure_action: rollback
worker: worker:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"] entrypoint: ["/host/env-entrypoint.sh"]
environment: environment:
THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph:${POSTGRES_PASSWORD}@db:5432/thermograph THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph:${POSTGRES_PASSWORD}@db:5432/thermograph
@ -153,7 +153,7 @@ services:
# healthy and answers 503/404, and web falls through to NASA — the lake is # healthy and answers 503/404, and web falls through to NASA — the lake is
# an accelerator, never a point of failure. # an accelerator, never a point of failure.
lake: lake:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"] entrypoint: ["/host/env-entrypoint.sh"]
environment: environment:
THERMOGRAPH_ROLE: lake THERMOGRAPH_ROLE: lake
@ -186,7 +186,7 @@ services:
# (/usr/local/bin/thermograph-daemon, built into the backend image) and the # (/usr/local/bin/thermograph-daemon, built into the backend image) and the
# app share the /internal/* API contract, so one BACKEND_IMAGE_TAG rolls # app share the /internal/* API contract, so one BACKEND_IMAGE_TAG rolls
# them together and they can never skew. # them together and they can never skew.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
# Not env-entrypoint.sh: that shim's handoff execs the image's # Not env-entrypoint.sh: that shim's handoff execs the image's
# deploy/entrypoint.sh (Alembic + uvicorn) whenever it exists, which is # deploy/entrypoint.sh (Alembic + uvicorn) whenever it exists, which is
# exactly what this service must never run — migrations belong to the # exactly what this service must never run — migrations belong to the
@ -243,7 +243,7 @@ services:
failure_action: rollback failure_action: rollback
frontend: frontend:
image: ${REGISTRY_HOST:-git.thermograph.org}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:?required} image: ${REGISTRY_HOST:-dev.jinemi.com}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"] entrypoint: ["/host/env-entrypoint.sh"]
# REQUIRED, not cosmetic: overriding `entrypoint:` with no `command:` drops # REQUIRED, not cosmetic: overriding `entrypoint:` with no `command:` drops
# the image's own CMD entirely (Docker/Swarm semantics, not merged) -- # the image's own CMD entirely (Docker/Swarm semantics, not merged) --

View file

@ -36,7 +36,7 @@ if ! ufw status | grep -q "^Status: active"; then
echo "ports above. Before enabling, explicitly allow every port this node" echo "ports above. Before enabling, explicitly allow every port this node"
echo "already serves publicly (SSH at minimum; on vps2 specifically, also" echo "already serves publicly (SSH at minimum; on vps2 specifically, also"
echo "80/tcp and 443/tcp for the live thermograph.org/beta.thermograph.org" echo "80/tcp and 443/tcp for the live thermograph.org/beta.thermograph.org"
echo "Caddy; on vps1, 80/tcp and 443/tcp for git.thermograph.org," echo "Caddy; on vps1, 80/tcp and 443/tcp for dev.jinemi.com,"
echo "dashboard.thermograph.org and emigriffith.dev) — check 'ss -tlnp' for" echo "dashboard.thermograph.org and emigriffith.dev) — check 'ss -tlnp' for"
echo "what's actually listening first. Enabling ufw without doing this WILL" echo "what's actually listening first. Enabling ufw without doing this WILL"
echo "drop live traffic the moment it activates." echo "drop live traffic the moment it activates."

View file

@ -16,7 +16,7 @@
PORT=8137 PORT=8137
# --- Registry pull (deploy.sh / Terraform) --------------------------------------- # --- Registry pull (deploy.sh / Terraform) ---------------------------------------
# deploy.sh logs in to git.thermograph.org's registry and `docker compose pull`s # deploy.sh logs in to dev.jinemi.com's registry and `docker compose pull`s
# the image build-push.yml already pushed for the deployed commit (repo-split # the image build-push.yml already pushed for the deployed commit (repo-split
# Stage 6), instead of building in place. A personal access token with at least # Stage 6), instead of building in place. A personal access token with at least
# read:package scope -- the same REGISTRY_TOKEN secret build-push.yml uses (that # read:package scope -- the same REGISTRY_TOKEN secret build-push.yml uses (that

View file

@ -102,7 +102,7 @@ services:
# builds `jinemi/thermograph/backend:local` from the backend repo (see # builds `jinemi/thermograph/backend:local` from the backend repo (see
# infra Makefile) and this pulls/uses it. BACKEND_IMAGE_PATH/TAG are # infra Makefile) and this pulls/uses it. BACKEND_IMAGE_PATH/TAG are
# independent of the frontend's, so the two services deploy separately. # independent of the frontend's, so the two services deploy separately.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
depends_on: depends_on:
db: db:
condition: service_healthy condition: service_healthy
@ -180,7 +180,7 @@ services:
# without them the service stays healthy and every read falls through to # without them the service stays healthy and every read falls through to
# NASA. Never published on a host port: only the backend talks to it. # NASA. Never published on a host port: only the backend talks to it.
lake: lake:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
environment: environment:
THERMOGRAPH_ROLE: lake THERMOGRAPH_ROLE: lake
PORT: 8141 PORT: 8141
@ -207,7 +207,7 @@ services:
# share the /internal/* API contract, so they must roll together — a # share the /internal/* API contract, so they must roll together — a
# separate tag could skew them. It owns the Discord gateway websocket and # separate tag could skew them. It owns the Discord gateway websocket and
# the recurring-job timers; anything needing data calls back into backend. # the recurring-job timers; anything needing data calls back into backend.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local} image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
# Bypass deploy/entrypoint.sh entirely: that script runs Alembic, and the # Bypass deploy/entrypoint.sh entirely: that script runs Alembic, and the
# backend service's boot already owns migrations — two containers racing # backend service's boot already owns migrations — two containers racing
# `alembic upgrade head` against one database is a real hazard, not a # `alembic upgrade head` against one database is a real hazard, not a
@ -262,7 +262,7 @@ services:
# frontend/Dockerfile (which starts the thermograph-frontend Go binary # frontend/Dockerfile (which starts the thermograph-frontend Go binary
# directly -- no THERMOGRAPH_SERVICE_ROLE process-picking). Independent # directly -- no THERMOGRAPH_SERVICE_ROLE process-picking). Independent
# FRONTEND_IMAGE_TAG so a frontend deploy never disturbs the backend's tag. # FRONTEND_IMAGE_TAG so a frontend deploy never disturbs the backend's tag.
image: ${REGISTRY_HOST:-git.thermograph.org}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:-local} image: ${REGISTRY_HOST:-dev.jinemi.com}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:-local}
# Its own register() fetches the IndexNow key from backend at boot -- must # Its own register() fetches the IndexNow key from backend at boot -- must
# wait for a real, healthy backend, not just a started container. # wait for a real, healthy backend, not just a started container.
depends_on: depends_on:

View file

@ -33,7 +33,7 @@ would silently overwrite the first's site instead of adding to it (there is no
merge). Prod claims that slot on vps2 today, so beta's public reverse-proxy (if merge). Prod claims that slot on vps2 today, so beta's public reverse-proxy (if
and when `beta.thermograph.org` is exposed) has to be a site block in vps2's and when `beta.thermograph.org` is exposed) has to be a site block in vps2's
shared, hand-maintained Caddy config instead — the same pattern shared, hand-maintained Caddy config instead — the same pattern
`deploy/forgejo/docker-stack.yml` already uses to put `git.thermograph.org` in `deploy/forgejo/docker-stack.yml` already uses to put `dev.jinemi.com` in
front of Forgejo's loopback port on vps1, alongside that box's own Caddy-fronted front of Forgejo's loopback port on vps1, alongside that box's own Caddy-fronted
`emigriffith.dev`. `emigriffith.dev`.
@ -180,7 +180,7 @@ Terraform brings the stack up — don't let Terraform recreate containers mid-mi
`/etc/caddy/Caddyfile` prod's apply just installed (see the table above). Reach `/etc/caddy/Caddyfile` prod's apply just installed (see the table above). Reach
beta via an SSH tunnel, or front it with a site block in vps2's own beta via an SSH tunnel, or front it with a site block in vps2's own
hand-maintained Caddy config (outside Terraform), the same way Forgejo's hand-maintained Caddy config (outside Terraform), the same way Forgejo's
`git.thermograph.org` reaches Forgejo's loopback port on vps1. `dev.jinemi.com` reaches Forgejo's loopback port on vps1.
`COOKIE_SECURE` is auto-set to `0` when there's no domain (a Secure `COOKIE_SECURE` is auto-set to `0` when there's no domain (a Secure
cookie is never sent over plain HTTP) and `1` behind Caddy TLS. cookie is never sent over plain HTTP) and `1` behind Caddy TLS.
- The rendered Caddyfile only reverse-proxies the app. The repo's - The rendered Caddyfile only reverse-proxies the app. The repo's

View file

@ -256,7 +256,7 @@ resource "null_resource" "host" {
. deploy/render-secrets.sh . deploy/render-secrets.sh
render_thermograph_secrets ${var.app_dir} render_thermograph_secrets ${var.app_dir}
set -a; . /etc/thermograph-topology.env; . /etc/thermograph.env; set +a set -a; . /etc/thermograph-topology.env; . /etc/thermograph.env; set +a
export REGISTRY_HOST="git.thermograph.org" BACKEND_IMAGE_TAG="${var.backend_image_tag}" FRONTEND_IMAGE_TAG="${var.frontend_image_tag}" export REGISTRY_HOST="dev.jinemi.com" BACKEND_IMAGE_TAG="${var.backend_image_tag}" FRONTEND_IMAGE_TAG="${var.frontend_image_tag}"
echo "$REGISTRY_TOKEN" | docker login "$REGISTRY_HOST" --username admin_emi --password-stdin echo "$REGISTRY_TOKEN" | docker login "$REGISTRY_HOST" --username admin_emi --password-stdin
docker compose ${local.compose_flags} pull backend frontend docker compose ${local.compose_flags} pull backend frontend
docker compose ${local.compose_flags} up -d --remove-orphans docker compose ${local.compose_flags} up -d --remove-orphans

View file

@ -87,7 +87,7 @@ hosts = {
# ONE environment on a box can own that file. Prod already claims it on # ONE environment on a box can own that file. Prod already claims it on
# this host, so beta.thermograph.org's public reverse-proxy has to be a # this host, so beta.thermograph.org's public reverse-proxy has to be a
# site block in vps2's shared, hand-maintained Caddy instead (the same # site block in vps2's shared, hand-maintained Caddy instead (the same
# pattern deploy/forgejo/docker-stack.yml uses for git.thermograph.org # pattern deploy/forgejo/docker-stack.yml uses for dev.jinemi.com
# on vps1) -- not something this module can render for a second # on vps1) -- not something this module can render for a second
# environment on the same host. # environment on the same host.
domain = "" domain = ""
@ -151,13 +151,13 @@ hosts = {
# } # }
# Optional overrides (shown with their defaults): # Optional overrides (shown with their defaults):
# repo_url = "https://git.thermograph.org/admin_emi/thermograph-infra.git" # repo_url = "https://dev.jinemi.com/admin_emi/thermograph-infra.git"
# app_port = 8137 # app_port = 8137
# frontend_port = 8080 # frontend_port = 8080
# #
# thermograph-infra is a PRIVATE repo, so a host cloning it for the first time # thermograph-infra is a PRIVATE repo, so a host cloning it for the first time
# needs read credentials embedded in repo_url, e.g. a Forgejo deploy token: # needs read credentials embedded in repo_url, e.g. a Forgejo deploy token:
# repo_url = "https://deploy:REPLACE_WITH_TOKEN@git.thermograph.org/admin_emi/thermograph-infra.git" # repo_url = "https://deploy:REPLACE_WITH_TOKEN@dev.jinemi.com/admin_emi/thermograph-infra.git"
# --------------------------------------------------------------------------------- # ---------------------------------------------------------------------------------
# Self-hosted Open-Meteo archive (only used by environments with openmeteo = true) # Self-hosted Open-Meteo archive (only used by environments with openmeteo = true)

View file

@ -127,9 +127,9 @@ variable "gcp_hosts" {
} }
variable "repo_url" { variable "repo_url" {
description = "Git remote to clone from when a host has no checkout yet. Points at THIS repo (thermograph-infra) now, not the app repo -- the app's own source is never checked out on a host; only its published registry images are pulled (see var.hosts[*].backend_image_tag / frontend_image_tag). thermograph-infra is a private repo, so this typically needs embedded read credentials, e.g. a Forgejo deploy token: \"https://<token-name>:<token>@git.thermograph.org/admin_emi/thermograph-infra.git\"." description = "Git remote to clone from when a host has no checkout yet. Points at THIS repo (thermograph-infra) now, not the app repo -- the app's own source is never checked out on a host; only its published registry images are pulled (see var.hosts[*].backend_image_tag / frontend_image_tag). thermograph-infra is a private repo, so this typically needs embedded read credentials, e.g. a Forgejo deploy token: \"https://<token-name>:<token>@dev.jinemi.com/admin_emi/thermograph-infra.git\"."
type = string type = string
default = "https://git.thermograph.org/admin_emi/thermograph-infra.git" default = "https://dev.jinemi.com/admin_emi/thermograph-infra.git"
sensitive = true sensitive = true
} }

View file

@ -94,7 +94,7 @@ Both the dashboard and Forgejo log in with Google. Create **one** Google Cloud
OAuth 2.0 Client (type: *Web application*) and register both redirect URIs: OAuth 2.0 Client (type: *Web application*) and register both redirect URIs:
- Grafana: `https://dashboard.thermograph.org/login/google` - Grafana: `https://dashboard.thermograph.org/login/google`
- Forgejo: `https://git.thermograph.org/user/oauth2/google/callback` - Forgejo: `https://dev.jinemi.com/user/oauth2/google/callback`
**Grafana:** put the client id/secret in `.env`, set `OAUTH_ENABLED=true`, and **Grafana:** put the client id/secret in `.env`, set `OAUTH_ENABLED=true`, and
`docker compose up -d`. Login is locked to pre-provisioned accounts `docker compose up -d`. Login is locked to pre-provisioned accounts