Merge pull request 'promote dev -> main: registry host migration to dev.jinemi.com' (#162) from dev into main
Some checks failed
Sync infra to hosts / sync-dev (push) Has been skipped
Sync infra to hosts / sync-centralis (push) Failing after 6s
Sync infra to hosts / sync-beta (push) Successful in 10s
secrets-guard / encrypted (push) Successful in 12s
Validate observability stack / validate (push) Successful in 15s
Sync infra to hosts / sync-prod (push) Successful in 10s
Build + push images (Forgejo registry) / build-push (frontend) (push) Successful in 39s
Build + push images (Forgejo registry) / build-push (backend) (push) Successful in 42s
shell-lint / shellcheck (push) Successful in 12s
Deploy / deploy (frontend) (push) Successful in 40s
Deploy / deploy (backend) (push) Successful in 1m56s
secrets-guard / encrypted (pull_request) Successful in 8s
shell-lint / shellcheck (pull_request) Successful in 10s

This commit is contained in:
Emi Griffith 2026-08-01 23:25:00 +00:00
commit 0eb4f63895
39 changed files with 343 additions and 157 deletions

View file

@ -16,9 +16,14 @@ name: Build + push images (Forgejo registry)
# docker.sock automounted (Docker-outside-of-Docker, no privileged mode); the
# job image (node:20-bookworm) ships no docker CLI, hence the install step.
#
# The registry is deliberately mesh-only: git.thermograph.org's public DNS
# resolves to beta's public IP, whose Caddy ACL rejects /v2/ paths, so any
# runner host that pushes or pulls needs an /etc/hosts entry.
# The registry is deliberately mesh-only: dev.jinemi.com's public DNS resolves
# to vps1's public IP, whose Caddy ACL rejects /v2/ paths, so any runner host
# that pushes or pulls needs an /etc/hosts entry pinning it to 10.10.0.2.
#
# The host itself comes from the Forgejo Actions variable REGISTRY_HOST, not
# from this file — a repo-side rename (git.thermograph.org -> dev.jinemi.com)
# does nothing until that variable is updated too. See
# infra/deploy/forgejo/README.md, "Migrating the domain".
on:
push:

View file

@ -34,8 +34,10 @@ own Postgres container, reachable only from vps1 itself (`127.0.0.1:8137` —
reach it either; use an SSH tunnel): no public DNS record, no Caddy site, no
TLS. It is deployed
by CI like beta and prod. The desktop hosts no Thermograph environment at all;
`make dev-up` there is a laptop convenience for running the stack locally, not
a deployment target.
the root `Makefile` (`make install` once, then `make up` / `make down`, wrapping
`infra`'s `dev-up`/`dev-down`) is a laptop convenience for running the stack
locally, not a deployment target. `install` is what builds the `:local` image
tags those compose files default to — nothing publishes them.
**One workflow deploys everything.** `deploy.yml` handles both services and all
three environments: the branch selects the environment (`dev` → vps1, `main`

View file

@ -121,7 +121,7 @@ Checked every split-repo branch by dry-run merge; migrated everything real:
1. Create `Jinemi/thermograph` on Forgejo (a **new** repo — do not resurrect the
archived monorepo) and push `main`.
2. Actions config, once: secrets `REGISTRY_TOKEN`, `SSH_HOST/USER/KEY/PORT`
(beta), `PROD_SSH_*` (prod); variable `REGISTRY_HOST=git.thermograph.org`.
(beta), `PROD_SSH_*` (prod); variable `REGISTRY_HOST=dev.jinemi.com`.
3. Branch protection on `dev` + `main`: required check `gate`, squash merges,
auto-merge on green.
4. Hosts (beta, then prod): clone the monorepo to `/opt/thermograph.new`;

81
Makefile Normal file
View file

@ -0,0 +1,81 @@
# Laptop-local entry point: build the two service images, run the dev stack.
#
# No fleet host uses this file. dev, beta and prod all deploy through
# infra/deploy/deploy.sh, which pulls registry images by their sha-<12hex> tag.
# A laptop has no registry login, so `install` BUILDS both images under the
# `:local` tag that infra/docker-compose.yml already falls back to
# (BACKEND_IMAGE_TAG / FRONTEND_IMAGE_TAG default to `local`). Without that
# build step `up` would try to PULL `:local`, which is published nowhere and
# fails with a 403 -- so `up` checks for the images first and says what to run.
#
# This wraps infra's own compose targets; `make -C infra dev-up` remains
# equivalent to `make up` here, same project name, same overlay, same stack.
SHELL := /bin/bash
# Keeps this stack's volumes off the prod-shaped `thermograph` project that
# infra/docker-compose.yml pins by default (the env var always wins over the
# file's `name:` key).
COMPOSE_PROJECT_NAME ?= thermograph-dev
POSTGRES_PASSWORD ?= thermograph-dev
# The daemon refuses to start without this, and the backend answers 404 on the
# whole /internal/* surface while it is unset -- both fail closed. A throwaway
# value is all a laptop needs; nothing outside this stack accepts it.
THERMOGRAPH_INTERNAL_TOKEN ?= dev-token
export COMPOSE_PROJECT_NAME POSTGRES_PASSWORD THERMOGRAPH_INTERNAL_TOKEN
# The dev overlay publishes backend on $DEV_BIND_ADDR, default loopback. Set
# DEV_BIND_ADDR=0.0.0.0 to reach it from phones on your own Wi-Fi -- safe on a
# laptop LAN, never on a fleet host (see infra/docker-compose.dev.yml).
COMPOSE := docker compose -f infra/docker-compose.yml -f infra/docker-compose.dev.yml
# Ask compose what it will run rather than restating the image names here --
# REGISTRY_HOST / *_IMAGE_PATH / *_IMAGE_TAG all have defaults in
# infra/docker-compose.yml, and a second copy of them in this file is a rename
# away from silently building a tag nothing runs. `--images <service>` also
# lists that service's dependencies, hence the per-service filter. Override by
# passing BACKEND_IMAGE=... / FRONTEND_IMAGE=... if you need a one-off tag.
# POSTGRES_PASSWORD is inlined because `config` refuses to interpolate without
# it, and $(shell) does not see make's `export` directives.
CONFIG_ENV := POSTGRES_PASSWORD=$(POSTGRES_PASSWORD)
ifndef BACKEND_IMAGE
BACKEND_IMAGE := $(shell $(CONFIG_ENV) $(COMPOSE) config --images backend 2>/dev/null | grep -m1 '/backend:')
endif
ifndef FRONTEND_IMAGE
FRONTEND_IMAGE := $(shell $(CONFIG_ENV) $(COMPOSE) config --images frontend 2>/dev/null | grep -m1 '/frontend:')
endif
.DEFAULT_GOAL := help
.PHONY: help install up down logs ps check-images-resolved
help: ## List targets
@grep -hE '^[a-z-]+:.*?## ' $(MAKEFILE_LIST) \
| awk -F':.*?## ' '{printf " \033[36m%-8s\033[0m %s\n", $$1, $$2}'
# Empty means `docker compose config` itself failed -- a broken compose file or
# no docker at all. Fail here rather than running `docker build -t ""`.
check-images-resolved:
@[[ -n "$(BACKEND_IMAGE)" && -n "$(FRONTEND_IMAGE)" ]] || { \
echo "could not resolve image names from infra/docker-compose.yml;" >&2; \
echo "check 'docker compose -f infra/docker-compose.yml config'" >&2; exit 1; }
install: check-images-resolved ## Build backend + frontend images as :local (run once before `up`)
docker build -t $(BACKEND_IMAGE) backend
docker build -t $(FRONTEND_IMAGE) frontend
up: check-images-resolved ## Start the local dev stack, uncapped, backend on 127.0.0.1:8137
@for img in $(BACKEND_IMAGE) $(FRONTEND_IMAGE); do \
docker image inspect "$$img" >/dev/null 2>&1 || { \
echo "missing image $$img -- run 'make install' first" >&2; exit 1; }; \
done
$(COMPOSE) up -d
@echo "backend: http://$${DEV_BIND_ADDR:-127.0.0.1}:8137"
down: ## Stop the stack (named volumes persist)
$(COMPOSE) down
logs: ## Follow the stack's logs (SERVICE=backend to narrow)
$(COMPOSE) logs -f $(SERVICE)
ps: ## Show the stack's containers
$(COMPOSE) ps

View file

@ -26,7 +26,7 @@ services:
# It read admin_emi/thermograph-backend/app (the retired split-era path) until this
# was corrected; harmless for the default `:local` build, wrong the moment
# BACKEND_IMAGE_TAG names a real published tag.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
depends_on:
db:
condition: service_healthy

View file

@ -14,7 +14,7 @@ export POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-smoke}"
# The image to smoke, pulled from the registry (no local build). CI passes the just-
# pushed sha (BACKEND_IMAGE_TAG=sha-<12hex>); locally it defaults to a published tag.
export BACKEND_IMAGE_TAG="${BACKEND_IMAGE_TAG:-v0.0.2-split-ci}"
IMG="${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-admin_emi/thermograph-backend/app}:${BACKEND_IMAGE_TAG}"
IMG="${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-admin_emi/thermograph-backend/app}:${BACKEND_IMAGE_TAG}"
export SMOKE_HOST_PORT="${SMOKE_HOST_PORT:-18137}"
COMPOSE=(docker compose -f docker-compose.test.yml)
PORT_URL="http://127.0.0.1:${SMOKE_HOST_PORT}"

View file

@ -60,7 +60,7 @@ all on a WireGuard mesh (`10.10.0.0/24`):
| Host | Mesh IP | Public | Runs |
|---|---|---|---|
| **vps1** | `10.10.0.2` | `75.119.132.91`, git.thermograph.org, dashboard.thermograph.org | Forgejo (git + CI + registry), Grafana + Loki + Alloy, the `emigriffith.dev` portfolio, and **dev** — its own Postgres, plain compose, **mesh-only** (no public DNS, no Caddy site, no TLS) |
| **vps1** | `10.10.0.2` | `75.119.132.91`, dev.jinemi.com, dashboard.thermograph.org | Forgejo (git + CI + registry), Grafana + Loki + Alloy, the `emigriffith.dev` portfolio, and **dev** — its own Postgres, plain compose, **mesh-only** (no public DNS, no Caddy site, no TLS) |
| **vps2** | `10.10.0.1` | `169.58.46.181`, thermograph.org, beta.thermograph.org | **prod** and **beta** as two separate Docker **Swarm** stacks, Centralis, Postfix, backups |
| **desktop** | `10.10.0.3` | — | AI-model hosting (voice-to-text, an upcoming-feature LLM) + flex Swarm capacity. Hosts **no** Thermograph environment — `make dev-up` there is a laptop convenience only |
| phone | — | — | alerts |
@ -84,10 +84,12 @@ tell beta and prod apart by which host it's running on.
Two consequences you will hit within the first week:
- **Forgejo is mesh-only.** `git.thermograph.org` resolves publicly to vps1's
- **Forgejo is mesh-only.** `dev.jinemi.com` resolves publicly to vps1's
IP, but vps1's Caddy rejects `/v2/*` (the registry API) from outside the
mesh. Any host that pulls images needs `10.10.0.2 git.thermograph.org` in
`/etc/hosts`.
mesh. Any host that pulls images needs
`10.10.0.2 dev.jinemi.com git.thermograph.org` in `/etc/hosts` — both names;
`git.thermograph.org` is the old registry host, still served because
pre-migration image tags carry it.
- **Beta and prod run the same orchestrator now — Swarm, as two stacks on one
box.** Dev is the only environment on compose (`infra/docker-compose.yml`),
and it lives alone on vps1. Most of the time you don't care which stack

View file

@ -169,21 +169,37 @@ ends fail closed. With Discord unconfigured it logs once and runs cron-only.
## The full stack, locally
From the repo root:
```bash
cd infra
make dev-up # docker-compose.yml + docker-compose.dev.yml overlay:
# uncapped CPU, backend published on 0.0.0.0:8137 for your LAN
make dev-down
make install # build both images as :local -- required once, see below
make up # docker-compose.yml + docker-compose.dev.yml overlay:
# uncapped CPU, backend on 127.0.0.1:8137
make down
```
`make install` is not optional on a fresh checkout. Neither compose file
carries a `build:` for backend or frontend — each ships as its own registry
image — so with no image present compose tries to **pull**
`dev.jinemi.com/emi/thermograph/backend:local`, a tag published nowhere,
and fails with `403 Forbidden` against a registry you have no login for.
`install` builds that tag locally from `backend/Dockerfile` and
`frontend/Dockerfile`. `make up` checks both images exist and points you here
if they don't.
`make -C infra dev-up` / `dev-down` remain equivalent to `up` / `down` — same
overlay, same project name, same stack.
Backend publishes on `${DEV_BIND_ADDR}`, **loopback by default**. Pass
`DEV_BIND_ADDR=0.0.0.0 make up` to reach it from phones on your own Wi-Fi —
fine on a laptop LAN, never on a fleet host.
This is a **laptop convenience**, not the hosted dev environment: the actual
`dev` environment now runs on vps1 with its own Postgres, deployed by CI, and
bound only to the WireGuard mesh (`10.10.0.2:8137`) — never `0.0.0.0`, since
vps1 is a public box. See [Infra and secrets](08-infra-secrets.md). A local
`make dev-up` is fine on `0.0.0.0` because it's your own machine's LAN, not the
public internet.
vps1 is a public box. See [Infra and secrets](08-infra-secrets.md).
The dev overlay exports `COMPOSE_PROJECT_NAME=thermograph-dev` so this stack
Both entry points set `COMPOSE_PROJECT_NAME=thermograph-dev` so this stack
keeps volumes separate from anything else. **Do not remove either half of the
project-name pinning** — `infra/docker-compose.yml` pins `name: thermograph`,
and without it running compose from `infra/` derives the project name `infra`,
@ -202,7 +218,7 @@ database, the ERA5 lake, fleet logs, Grafana, Forgejo, docs and notes, and
Discord.
```bash
claude mcp add --transport http centralis https://mcp.thermograph.org/mcp \
claude mcp add --transport http centralis https://mcp.jinemi.com/mcp \
--header "Authorization: Bearer $CENTRALIS_TOKEN"
```

View file

@ -102,7 +102,7 @@ alembic + `/healthz` 200 + a real `/api/v2/place` 200.
Triggers on pushes to `dev`/`main`/`release` touching `backend/**` or
`frontend/**`, and on `v*.*.*` tags. Publishes
`git.thermograph.org/Jinemi/thermograph/{backend,frontend}:sha-<12hex>`.
`dev.jinemi.com/Jinemi/thermograph/{backend,frontend}:sha-<12hex>`.
- **Serialised** (`max-parallel: 1`): one physical runner, and two whole image
builds racing each other buys nothing.
@ -110,7 +110,9 @@ Triggers on pushes to `dev`/`main`/`release` touching `backend/**` or
release tag builds **both** images. That's intended — a release names the
whole set, not whichever domain moved last.
- The registry is mesh-only, so the runner host needs
`10.10.0.2 git.thermograph.org` in `/etc/hosts`.
`10.10.0.2 dev.jinemi.com git.thermograph.org` in `/etc/hosts`.
- The registry host itself comes from the Forgejo Actions variable
`REGISTRY_HOST`, not from the workflow file.
### `deploy.yml` — one file, three environments, two services

View file

@ -12,7 +12,7 @@ desktop:
| Host | Public | Mesh | Runs | Login |
|---|---|---|---|---|
| **vps1** | `75.119.132.91` / git.thermograph.org, dashboard.thermograph.org | `10.10.0.2` | Forgejo, Grafana + Loki + Alloy, `emigriffith.dev`, and **dev** (own Postgres, compose, mesh-only) | `agent`, passwordless sudo |
| **vps1** | `75.119.132.91` / dev.jinemi.com, dashboard.thermograph.org | `10.10.0.2` | Forgejo, Grafana + Loki + Alloy, `emigriffith.dev`, and **dev** (own Postgres, compose, mesh-only) | `agent`, passwordless sudo |
| **vps2** | `169.58.46.181` / thermograph.org, beta.thermograph.org | `10.10.0.1` | **prod** and **beta**, as two co-resident Docker **Swarm** stacks; Centralis; Postfix; backups | `agent`, passwordless sudo |
| **desktop** | — | `10.10.0.3` | AI-model hosting + flex Swarm capacity. No Thermograph environment; `make dev-up` is a laptop convenience there | it's your box |

View file

@ -227,10 +227,12 @@ It's executable documentation until state is bootstrapped.
### The registry is mesh-only
`git.thermograph.org` resolves publicly to vps1's IP, but vps1's Caddy rejects
`dev.jinemi.com` resolves publicly to vps1's IP, but vps1's Caddy rejects
`/v2/*` from outside the WireGuard mesh. Any host that pulls needs
`10.10.0.2 git.thermograph.org` in `/etc/hosts`. vps1 itself doesn't — it *is*
the box.
`10.10.0.2 dev.jinemi.com git.thermograph.org` in `/etc/hosts`**both** names:
the new one is the registry host and the bearer-token realm, the old one still
prefixes every image tag pushed before the migration. vps1 itself doesn't need
the pin — it *is* the box.
### Forgejo runners only have the labels they registered with

View file

@ -29,7 +29,7 @@ services:
# to the split-era v0.0.2-split-ci, under the retired
# admin_emi/thermograph-backend/app path, long after both were superseded.)
# Override THERMOGRAPH_BACKEND_TEST_TAG to pin a specific build.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${THERMOGRAPH_BACKEND_TEST_TAG:?set it, or run via scripts/backend-for-tests.sh which derives it}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${THERMOGRAPH_BACKEND_TEST_TAG:?set it, or run via scripts/backend-for-tests.sh which derives it}
depends_on:
db:
condition: service_healthy

View file

@ -57,7 +57,7 @@ THERMOGRAPH_INTERNAL_TOKEN=
# in its own app repo (thermograph-backend / thermograph-frontend) tagged
# :local, which is the default here -- only set these to pull a specific
# published build instead of building locally.
# REGISTRY_HOST=git.thermograph.org
# REGISTRY_HOST=dev.jinemi.com
# BACKEND_IMAGE_PATH=admin_emi/thermograph-backend/app
# BACKEND_IMAGE_TAG=local
# FRONTEND_IMAGE_PATH=admin_emi/thermograph-frontend/app

View file

@ -26,7 +26,7 @@ rather than which environment happens to live there:
| Host | Public IP | Mesh IP | Role |
|------|-----------|---------|------|
| **vps1** | `75.119.132.91` | `10.10.0.2` | "Operational programs": Forgejo (git + CI + registry; `dev.jinemi.com` canonical, `git.thermograph.org` still served and still used by the registry/mesh pins/runners), Grafana/Loki/Alloy (`dashboard.thermograph.org`), the `emigriffith.dev` portfolio, and the **dev** environment (mesh-only, `10.10.0.2:8137`, no public DNS/Caddy/TLS) |
| **vps1** | `75.119.132.91` | `10.10.0.2` | "Operational programs": Forgejo (git + CI + registry, both at `dev.jinemi.com`; `git.thermograph.org` still served for pre-migration image tags and already-registered runners), Grafana/Loki/Alloy (`dashboard.thermograph.org`), the `emigriffith.dev` portfolio, and the **dev** environment (mesh-only, `10.10.0.2:8137`, no public DNS/Caddy/TLS) |
| **vps2** | `169.58.46.181` | `10.10.0.1` | "The deployed environment": **prod** and **beta**, as two separate Docker Swarm stacks, plus Centralis, Postfix, the backups, and the one shared TimescaleDB instance both stacks use |
| **desktop** | — | `10.10.0.3` | AI-model hosting (voice-to-text, an upcoming-feature LLM) plus flex Swarm-worker capacity. Hosts **no** Thermograph environment — `make dev-up` still works there, but only as a laptop-local rehearsal |
@ -70,7 +70,7 @@ Both VPS boxes are provisioned and reachable as of this writing:
| Host | Role | Public IP | Login |
|------|------|-----------|-------|
| vps2 | Swarm manager; prod AND beta live here (`thermograph.org`, `beta.thermograph.org`) | `169.58.46.181` | `agent` |
| vps1 | Swarm worker; Forgejo + Grafana/Loki + dev (`git.thermograph.org`, `dashboard.thermograph.org`) | `75.119.132.91` | `agent` |
| vps1 | Swarm worker; Forgejo + Grafana/Loki + dev (`dev.jinemi.com`, `dashboard.thermograph.org`) | `75.119.132.91` | `agent` |
| desktop | Swarm worker, AI-model hosting | (local) | (already have access) |
```
@ -179,8 +179,9 @@ The GitHub → Forgejo cutover is complete; this records what was done:
failed login attempt, not just config inspection)
- [x] `docker node ls` (from vps2, the manager) shows all three nodes `Ready`
- [x] Swarm ports (2377/7946/4789) unreachable from outside the WireGuard tunnel
- [x] `https://git.thermograph.org` serves Forgejo over TLS; `/v2/` registry
API is 403 from off-mesh (firewalled to the WireGuard CIDR)
- [x] `https://dev.jinemi.com` serves Forgejo over TLS (and
`https://git.thermograph.org` still does, off the same site block);
`/v2/` registry API is 403 from off-mesh (firewalled to the WireGuard CIDR)
- [x] A test PR flow verified: required `build` check runs → explicit squash
merge → deploy lands on the branch's mapped environment
- [x] GitHub retired as git host + CI; kept only as a read-only history mirror

View file

@ -12,10 +12,11 @@ Two VPS boxes plus the operator's desktop, on one WireGuard mesh, named by
ROLE rather than by environment:
- **vps1**`75.119.132.91`, mesh `10.10.0.2`. "Operational programs": Forgejo
(git + CI + registry; `dev.jinemi.com` is canonical — Forgejo's `ROOT_URL`
and `git.thermograph.org` is still served off the same Caddy site block and
still load-bearing for the registry, mesh pins and runner URLs, so don't
retire it; see `deploy/forgejo/README.md`), Grafana/Loki/Alloy
(git + CI + registry; `dev.jinemi.com` is canonical on both axes — Forgejo's
`ROOT_URL` *and* the registry host in every image name — while
`git.thermograph.org` is still served off the same Caddy site block for
pre-migration image tags and already-registered runners, so don't retire it;
see `deploy/forgejo/README.md`), Grafana/Loki/Alloy
(`dashboard.thermograph.org`), the `emigriffith.dev` portfolio site, and the
**dev** environment (`/opt/thermograph-dev`, tracks `dev`), with its own
Postgres container. Dev is mesh-only here — published on `10.10.0.2:8137`,

View file

@ -5,7 +5,7 @@ the **`dev`** branch continuously deploys to the **dev environment on vps1**
(`75.119.132.91`, mesh `10.10.0.2`) — `/opt/thermograph-dev`, a normal fleet
checkout reached over SSH exactly like beta and prod, **not** a stack on the
operator's desktop or LAN any more. Git hosting and CI are self-hosted
**Forgejo** (`git.thermograph.org`, also on vps1, reachable at
**Forgejo** (`dev.jinemi.com`, also on vps1, reachable at
`http://10.10.0.2:3080` over the WireGuard mesh) — GitHub is retired.
```
@ -125,13 +125,14 @@ bash deploy/forgejo/register-lan-runner.sh <forgejo_url> <registration_token>
Registry pushes (`build-push.yml`) run against the runner host's own
automounted Docker daemon, so it's that **host's** own resolver that needs to
route `git.thermograph.org` over the mesh, not anything settable from a job
route `dev.jinemi.com` over the mesh, not anything settable from a job
container. If registry pushes ever start failing with an HTTP/TLS mismatch or
a 403 on `/v2/`, check `getent hosts git.thermograph.org` on the runner host —
a 403 on `/v2/`, check `getent hosts dev.jinemi.com` on the runner host —
it needs to resolve to vps1's WireGuard IP (`10.10.0.2`), typically via a
`/etc/hosts` line, not the public one (a non-issue if the runner is co-located
with Forgejo on vps1 itself, since `git.thermograph.org` then resolves to
itself either way).
with Forgejo on vps1 itself, since the name then resolves to itself either
way). Check `git.thermograph.org` the same way: it is still served, and a
rollback to an image tag pushed before the domain migration dials it.
## The dev environment (Docker Compose, on vps1)

View file

@ -24,7 +24,7 @@ own loopback Caddy LB fronts it with TLS.** Credentials are keyed by **host**
(`VPS2_SSH_*`), not by environment — vps2 answers to both prod and beta, so the
environment is passed as data (`THERMOGRAPH_ENV`), not baked into which secret
is used. (GitHub is retired/archived — Forgejo, self-hosted at
`git.thermograph.org` on **vps1** over the WireGuard mesh, is now the sole git
`dev.jinemi.com` on **vps1** over the WireGuard mesh, is now the sole git
host and CI.)
Files that make this work:
@ -49,14 +49,17 @@ gitignored cache under `data/cache/`, so a `git pull` in one never touches the
other's.
**Any deploy host needs a `/etc/hosts` entry for the registry.**
`git.thermograph.org`'s public DNS resolves to **vps1's** public IP, and
`dev.jinemi.com`'s public DNS resolves to **vps1's** public IP, and
vps1's own Caddy rejects `/v2/*` (the registry API) from anything outside the
WireGuard mesh (10.10.0.0/24) — confirmed live: `docker login`/`pull` from
vps2 failed with a 403 until adding `10.10.0.2 git.thermograph.org` to
`/etc/hosts` (10.10.0.2 is vps1's mesh IP; every deploy host is already on
the mesh, this is purely a DNS-routing gap, not a connectivity one). vps1
itself doesn't need this — it's the box Forgejo runs on, so `git.thermograph.org`
just resolves to itself either way. `build-push.yml`'s own header comment
vps2 failed with a 403 until adding `10.10.0.2 dev.jinemi.com git.thermograph.org`
to `/etc/hosts` (10.10.0.2 is vps1's mesh IP; every deploy host is already on
the mesh, this is purely a DNS-routing gap, not a connectivity one). Pin
**both** names: `dev.jinemi.com` is the registry host and the bearer-token
realm, and `git.thermograph.org` still prefixes every image tag pushed before
the domain migration, so a rollback to one of those dials it directly. vps1
itself doesn't need this — it's the box Forgejo runs on, so both names
just resolve to it either way. `build-push.yml`'s own header comment
documents the same requirement for the CI runner host.
**Current production layout**: prod's Caddy config owns **`thermograph.org`**

View file

@ -30,20 +30,30 @@ db-up:
db-down:
docker compose stop db
# The dev overlay: uncapped (no CPU limits). Default publish is 0.0.0.0:8137,
# right for a laptop-local rehearsal (e.g. phones on the same Wi-Fi) -- this is
# also the *fleet* dev environment's own overlay, but that environment now runs
# on vps1 (a public VPS, not a LAN box) and deploys via deploy/deploy-dev.sh,
# which overrides the bind address to the WireGuard mesh IP only
# (DEV_BIND_ADDR, see deploy/env-topology.sh) -- never invoke `make dev-up`
# directly on a fleet host, or it publishes on every interface. (The base file
# is prod/beta's shape: capped + loopback, fronted by their own Caddy LB.)
# The dev overlay: uncapped (no CPU limits), backend published on
# ${DEV_BIND_ADDR}, which docker-compose.dev.yml defaults to loopback. Set
# DEV_BIND_ADDR=0.0.0.0 for a laptop-local rehearsal reachable from phones on
# the same Wi-Fi. This is also the *fleet* dev environment's own overlay, but
# that environment runs on vps1 (a public VPS, not a LAN box) and deploys via
# deploy/deploy-dev.sh, which pins the bind address to the WireGuard mesh IP
# (see deploy/env-topology.sh) -- never invoke `make dev-up` directly on a
# fleet host. (The base file is prod/beta's shape: capped + loopback, fronted
# by their own Caddy LB.)
#
# COMPOSE_PROJECT_NAME matches deploy/deploy-dev.sh, so a laptop's dev stack
# keeps its volumes off the `thermograph` project that docker-compose.yml pins
# by default. No `--build`: neither compose file carries a `build:` for backend
# or frontend, so the images must exist first -- the root Makefile's `install`
# builds them under the `:local` tag these files default to.
DEV_COMPOSE = docker compose -f docker-compose.yml -f docker-compose.dev.yml
dev-up:
POSTGRES_PASSWORD=$${POSTGRES_PASSWORD:-thermograph-dev} $(DEV_COMPOSE) up -d --build
COMPOSE_PROJECT_NAME=$${COMPOSE_PROJECT_NAME:-thermograph-dev} \
POSTGRES_PASSWORD=$${POSTGRES_PASSWORD:-thermograph-dev} \
THERMOGRAPH_INTERNAL_TOKEN=$${THERMOGRAPH_INTERNAL_TOKEN:-dev-token} \
$(DEV_COMPOSE) up -d
dev-down:
$(DEV_COMPOSE) down
COMPOSE_PROJECT_NAME=$${COMPOSE_PROJECT_NAME:-thermograph-dev} $(DEV_COMPOSE) down
# Self-hosted Open-Meteo (prod-only overlay). Serves the ERA5 archive from object
# storage so the app is off the public archive API. See deploy/openmeteo/README.md

View file

@ -2,11 +2,16 @@
#
# vps1 serves:
# emigriffith.dev -> static portfolio site (from disk)
# dev.jinemi.com -> Forgejo, CANONICAL (Forgejo's ROOT_URL)
# git.thermograph.org -> Forgejo, same site block — still served and
# still needed: the registry host in image names,
# the mesh /etc/hosts pins and the runners' URLs
# all use it (see deploy/forgejo/caddy-git.conf)
# dev.jinemi.com -> Forgejo, CANONICAL: both its ROOT_URL and the
# registry host in every image name
# git.thermograph.org -> Forgejo, same site block — still served, now
# only for already-pushed image tags carrying the
# old prefix and for runners registered against
# it (see deploy/forgejo/caddy-git.conf)
#
# Do not confuse `dev.jinemi.com` (Forgejo — git, CI, registry) with
# `dev.thermograph.org` (the dev *application* environment, further down this
# file). Same box, same word, unrelated services.
# dashboard.thermograph.org -> Grafana (see observability/caddy-grafana.conf)
#
# and hosts the **dev** environment at dev.thermograph.org (below).

View file

@ -4,7 +4,7 @@
# thermograph.org -> prod (loopback LB on 127.0.0.1:8137 / :8080)
# beta.thermograph.org -> beta (loopback LB on 127.0.0.1:8237 / :8180)
#
# and Centralis at mcp.thermograph.org, which is provisioned separately.
# and Centralis at mcp.jinemi.com, which is provisioned separately.
#
# Each domain's A/AAAA record must already point here — Caddy provisions a
# Let's Encrypt cert on first request and auto-renews. Nothing else to do for

View file

@ -215,7 +215,7 @@ fi
# a tag from -- the caller (the deploying repo's deploy.yml) exports its own
# service's tag (BACKEND_IMAGE_TAG or FRONTEND_IMAGE_TAG = sha-<12 hex> of the
# app commit, or a semver tag).
REGISTRY_HOST="${REGISTRY_HOST:-git.thermograph.org}"
REGISTRY_HOST="${REGISTRY_HOST:-dev.jinemi.com}"
export REGISTRY_HOST BACKEND_IMAGE_PATH FRONTEND_IMAGE_PATH
# Load the last-deployed tag for BOTH services first, so a single-service roll
@ -328,7 +328,7 @@ case " ${TARGETS[*]} " in
# then always found no daemon binary in a Postgres image and dropped
# daemon from EVERY backend deploy, regardless of what the real backend
# image contained -- reproduced and confirmed against beta directly.
daemon_img="${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG}"
daemon_img="${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG}"
if ! docker run --rm --entrypoint sh "$daemon_img" -c 'test -x /usr/local/bin/thermograph-daemon' 2>/dev/null; then
echo "==> $daemon_img predates the daemon binary; rolling without the daemon service this run"
kept=()

View file

@ -70,7 +70,7 @@ web port publishes to `127.0.0.1:3080` only (host-local), and vps1's
*existing* Caddy gets one more site block reverse-proxying to it — same
pattern as its other site blocks, same automatic-HTTPS.
1. Point the Forgejo domain (default `git.thermograph.org`; override with
1. Point the Forgejo domain (default `dev.jinemi.com`; override with
`FORGEJO_DOMAIN=...` before `docker stack deploy`) at **vps1's** public IP
— that's where the task actually runs, not vps2's or the desktop's.
2. Append `deploy/forgejo/caddy-git.conf` to vps1's `/etc/caddy/Caddyfile`,
@ -89,26 +89,54 @@ pattern as its other site blocks, same automatic-HTTPS.
(`FORGEJO_DOMAIN` in `docker-stack.yml`), so it is the name Forgejo generates in
clone URLs, the Google OAuth callback, webhook payload URLs and mail links.
**Registry: deliberately not done.** Renaming the registry host is a separate
migration from renaming the web UI, and the two need not happen together.
**Registry: done in this repo, and it needs host-side steps to match.** Every
image name, `REGISTRY_HOST` default, runner label and `--add-host` pin under
version control now says `dev.jinemi.com`. None of that is self-applying — see
"Host-side steps" below before expecting a build to push.
Both names address the *same* Forgejo and therefore the same packages. A tag
pushed as `git.thermograph.org/jinemi/thermograph/backend:sha-abc` is pullable
as `dev.jinemi.com/jinemi/thermograph/backend:sha-abc` — the host part is only
how you reach the registry, not part of the package's identity. That is what
makes this migration safe to do in one step instead of a dual-push period: no
image needs re-pushing, and a rollback to an old tag still resolves.
`caddy-git.conf` lists both names on a *single* site block. That is deliberate,
not cosmetic: the `/v2/*` mesh-only matcher is per-block, so a second block for
either name would re-expose the registry API publicly and undo hazard #15.
Whatever else changes, keep the two names in one block.
**`git.thermograph.org` must stay served.** It is not a courtesy redirect for
old bookmarks — CI resolves it:
**`git.thermograph.org` must stay served.** The list of reasons is shorter than
it was, but it is not empty, and neither remaining item is a bookmark:
- images are named by registry host, and the `git.thermograph.org/` prefix is
baked into the runner labels (`register-lan-runner.sh`,
`runner-vps2/README.md`), the CI-runner image (`ci-runner/Dockerfile`),
`REGISTRY_HOST` in `infra/.env.example`, and every already-pushed tag;
- every mesh client in "Registry access from mesh clients" below pins
`git.thermograph.org` to `10.10.0.2` in `/etc/hosts`, which is what makes
Caddy's `/v2/*` matcher see a mesh source IP instead of returning 403;
- image tags already pushed under the old prefix name that host, so a rollback
to one asks for `git.thermograph.org/...` even though nothing builds that
name any more;
- registered runners hold the instance URL they registered with
(`https://git.thermograph.org`); they keep working while that name resolves.
(`https://git.thermograph.org`); they keep working while that name resolves,
and stop the moment it doesn't. Re-registering them is the only way to retire
the name, and it is not required for this migration.
### Host-side steps
None of these live in the repo, and CI will not do them for you. In order:
1. **`/etc/hosts` on every mesh client** — pin `dev.jinemi.com` as well as
`git.thermograph.org` (both, for the reasons under "Registry access from
mesh clients" below). A client that resolves the new name publicly gets a
403 from the `/v2/*` matcher, which reads exactly like a bad credential.
2. **`docker login dev.jinemi.com`** on each host that pushes or pulls — the
CI runners, vps2 (prod and beta), the desktop. A login against the old name
does **not** carry over: docker keys stored credentials by registry host.
3. **The Forgejo Actions variable `REGISTRY_HOST`**`dev.jinemi.com`
(Site Administration → Actions → Variables, or the repo's own). This is what
`build-push.yml` reads; the defaults in this repo are fallbacks for by-hand
runs, so leaving the variable stale silently keeps CI on the old name.
4. **Re-register the runners** against `https://dev.jinemi.com`, *optionally*
see above. Until you do, leave `git.thermograph.org` served.
Verify with a push before assuming: the failure mode for a missed step 1 or 2
is `unauthorized: reqPackageAccess`, which names neither.
### Before changing ROOT_URL again
@ -139,37 +167,39 @@ effect when someone re-runs `docker stack deploy` by hand on the manager (vps2)
Any node that needs `docker login`/push/pull against the registry (the CI
runner building/pushing images, any Swarm node pulling them, prod or beta on
vps2 pulling app images) must reach `git.thermograph.org` **over the
WireGuard tunnel**, not vps1's public IP — otherwise Caddy's `/v2/*` block
above refuses the connection. Public DNS resolves the domain to vps1's public
IP, so add a `/etc/hosts` override on each such node pinning it to vps1's
WireGuard address instead:
vps2 pulling app images) must reach Forgejo **over the WireGuard tunnel**, not
vps1's public IP — otherwise Caddy's `/v2/*` block above refuses the
connection. Public DNS resolves both names to vps1's public IP, so add an
`/etc/hosts` override on each such node pinning them to vps1's WireGuard
address instead:
```
echo "10.10.0.2 git.thermograph.org" | sudo tee -a /etc/hosts
echo "10.10.0.2 dev.jinemi.com git.thermograph.org" | sudo tee -a /etc/hosts
```
(`10.10.0.2` is vps1's WG address per `deploy/swarm/README.md`'s peer
numbering — adjust if you assigned it differently.) The git/web UI keeps
working normally for everyone else since only `/v2/*` is restricted.
**Pin the canonical name too, not just the image host.** Forgejo derives the
registry's bearer-token realm from `ROOT_URL`, so a client that already
resolves `git.thermograph.org` over the mesh is still sent to
`https://<ROOT_URL host>/v2/token` to collect a token. When `ROOT_URL` became
`dev.jinemi.com`, that second request took the public route, the `/v2/*`
matcher above answered 403, and docker fell back to anonymous — every push then
failed with `unauthorized: reqPackageAccess`, which reads exactly like a
revoked token or a missing scope. It is neither. Pin both names:
**Both names, not just the one in the image.** `dev.jinemi.com` now carries two
independent jobs and needs the pin for each: it is the host in every image
name, *and* it is `ROOT_URL`, from which Forgejo derives the registry's
bearer-token realm — so a client is sent to `https://<ROOT_URL host>/v2/token`
to collect a token no matter which name it dialled. `git.thermograph.org` stays
pinned for a third reason: image tags pushed before the migration name it, and
a rollback to one of those dials it directly.
```
echo "10.10.0.2 git.thermograph.org dev.jinemi.com" | sudo tee -a /etc/hosts
```
This is not hypothetical. When `ROOT_URL` first became `dev.jinemi.com` while
images were still named `git.thermograph.org`, the token request took the
public route, the `/v2/*` matcher answered 403, and docker fell back to
anonymous — every push failed with `unauthorized: reqPackageAccess`, which
reads exactly like a revoked token or a missing scope. It was neither. Pinning
one name and not the other reproduces it in either direction.
The realm moves whenever `ROOT_URL` does, so read it rather than assuming:
```sh
curl -sI https://git.thermograph.org/v2/ | grep -i www-authenticate
curl -sI https://dev.jinemi.com/v2/ | grep -i www-authenticate
```
These `/etc/hosts` pins are host state. Nothing in this repo writes them, so a
@ -259,14 +289,14 @@ bug hit during the frontend Go rewrite). `ci-runner` adds `docker-ce-cli` +
`git`/`python3`/`python3-yaml` for the other jobs that need them
(`shell-lint`, `observability-validate`).
Current tag: `git.thermograph.org/jinemi/thermograph/ci-runner:v2` (`v1` is
Current tag: `dev.jinemi.com/jinemi/thermograph/ci-runner:v2` (`v1` is
broken — do not register any runner against it). Rebuild/push (requires a PAT
with `write:package` scope — the embedded git-remote token lacks it, same
requirement documented in `build-push.yml`):
```bash
docker build -t git.thermograph.org/jinemi/thermograph/ci-runner:vN deploy/forgejo/ci-runner
docker push git.thermograph.org/jinemi/thermograph/ci-runner:vN
docker build -t dev.jinemi.com/jinemi/thermograph/ci-runner:vN deploy/forgejo/ci-runner
docker push dev.jinemi.com/jinemi/thermograph/ci-runner:vN
```
`register-lan-runner.sh`'s `LABELS` default points at the current tag, so
@ -291,8 +321,9 @@ independently.
```bash
docker service ls # forgejo_db, forgejo_forgejo both Running, 1/1
curl -I https://git.thermograph.org/ # 200, valid cert (Caddy's, not a new one)
curl -I https://git.thermograph.org/v2/ # 403 from anywhere off the WireGuard mesh
curl -I https://dev.jinemi.com/ # 200, valid cert (Caddy's, not a new one)
curl -I https://dev.jinemi.com/v2/ # 403 from anywhere off the WireGuard mesh
curl -I https://git.thermograph.org/ # 200 too — the alias must stay served
# On the runner host, after registering the runner:
systemctl --user status forgejo-runner # active, both labels registered
```

View file

@ -1,4 +1,4 @@
# Only the block below (from "git.thermograph.org, dev.jinemi.com {") goes into
# Only the block below (from "dev.jinemi.com, git.thermograph.org {") goes into
# the live Caddyfile — append just that part, not this instructional header, or
# it ends up as a confusing orphaned comment in production config with no
# surrounding context (docker-stack.yml isn't visible from there).
@ -14,14 +14,17 @@
# both. Splitting dev.jinemi.com into its own block serves the same Forgejo
# with the registry API open to the public internet.
#
# dev.jinemi.com is CANONICAL — it is Forgejo's ROOT_URL (docker-stack.yml,
# FORGEJO_DOMAIN), so it is the name Forgejo puts in clone URLs, the OAuth
# callback, webhook payload URLs and mail links.
# dev.jinemi.com is CANONICAL, now on both axes — it is Forgejo's ROOT_URL
# (docker-stack.yml, FORGEJO_DOMAIN), so it is the name Forgejo puts in clone
# URLs, the OAuth callback, webhook payload URLs and mail links; and it is the
# registry host in every image name this repo builds or pulls.
#
# git.thermograph.org is kept as a served alias, NOT as dead weight: the
# registry host baked into image names, the mesh /etc/hosts pins and the
# runners' registered instance URL all still say git.thermograph.org. Removing
# this name from the block breaks CI, not just old bookmarks. Both names get
# git.thermograph.org is kept as a served alias, NOT as dead weight, though the
# reason narrowed when the registry moved. What still needs it: image tags
# already pushed under the old prefix (a rollback to one names that host), and
# runners registered against `https://git.thermograph.org`, which hold the
# instance URL they registered with. Removing this name from the block breaks a
# rollback and de-registers runners — not just old bookmarks. Both names get
# their own Let's Encrypt cert automatically (HTTP-01); both A records point
# at vps1. See README.md, "Migrating the domain".
#
@ -33,7 +36,7 @@
# --- copy from here down into /etc/caddy/Caddyfile ---
git.thermograph.org, dev.jinemi.com {
dev.jinemi.com, git.thermograph.org {
encode zstd gzip
@registry_external {

View file

@ -19,9 +19,9 @@
# class, and skips the per-job install entirely.
#
# Build/push (manual — see ../README.md for when to rebuild):
# docker build -t git.thermograph.org/jinemi/thermograph/ci-runner:vN \
# docker build -t dev.jinemi.com/jinemi/thermograph/ci-runner:vN \
# infra/deploy/forgejo/ci-runner
# docker push git.thermograph.org/jinemi/thermograph/ci-runner:vN
# docker push dev.jinemi.com/jinemi/thermograph/ci-runner:vN
#
# Cutting over the live runner to a new tag means editing the `labels` array
# in ~/forgejo-runner/.runner on the runner host directly (same runner

View file

@ -20,7 +20,7 @@
# box). A second reverse proxy binding those same ports would either fail to
# start or fight Caddy. Instead: forgejo's web port publishes to
# 127.0.0.1:3080 only (host-local, mode: host), and vps1's existing Caddy gets
# a new site block reverse-proxying git.thermograph.org -> 127.0.0.1:3080,
# a new site block reverse-proxying dev.jinemi.com -> 127.0.0.1:3080,
# same pattern as its other blocks. TLS is Caddy's existing automatic-HTTPS
# (HTTP-01), not a second ACME flow.
#
@ -82,7 +82,8 @@ services:
FORGEJO__database__USER: forgejo
# Canonical domain. Forgejo has exactly ONE ROOT_URL and derives every
# absolute URL from it — clone URLs, the Google OAuth callback, webhook
# payload URLs, mail links. git.thermograph.org is still served (Caddy
# payload URLs, mail links, and the registry's bearer-token realm.
# git.thermograph.org is still served (Caddy
# answers for both names off one site block, deploy/forgejo/caddy-git.conf)
# and still works for browsing, git-over-HTTP and the API; it is simply no
# longer the name Forgejo generates.
@ -94,9 +95,12 @@ services:
# git.thermograph.org one are registered; verify with a probe before
# changing this again (deploy/forgejo/README.md, "Migrating the domain").
#
# NOT changed by this: the registry host baked into image names, the mesh
# /etc/hosts pins, and the runners' registered instance URL all still say
# git.thermograph.org. That is a separate migration.
# The registry host in image names and the mesh /etc/hosts pins now say
# dev.jinemi.com too, so ROOT_URL and the registry name agree again — which
# matters, because Forgejo derives the registry's bearer-token realm from
# ROOT_URL. Still on the old name: runners registered against
# https://git.thermograph.org (they hold the URL they registered with) and
# image tags already pushed under the old prefix.
FORGEJO__server__DOMAIN: "${FORGEJO_DOMAIN:-dev.jinemi.com}"
FORGEJO__server__ROOT_URL: "https://${FORGEJO_DOMAIN:-dev.jinemi.com}/"
FORGEJO__server__SSH_PORT: "2222"

View file

@ -40,7 +40,7 @@ set -euo pipefail
FORGEJO_URL="${1:?usage: $0 <forgejo_url> <registration_token>}"
TOKEN="${2:?}"
RUNNER_DIR="${RUNNER_DIR:-$HOME/forgejo-runner}"
LABELS="${LABELS:-docker:docker://git.thermograph.org/jinemi/thermograph/ci-runner:v2,thermograph-lan}"
LABELS="${LABELS:-docker:docker://dev.jinemi.com/jinemi/thermograph/ci-runner:v2,thermograph-lan}"
echo "==> Stopping and disabling the old GitHub Actions runner service, if present"
systemctl --user stop github-actions-runner 2>/dev/null || true

View file

@ -26,12 +26,14 @@ documentation one.
## Prerequisites
- vps2's docker daemon is logged in to `git.thermograph.org` (it is —
`/root/.docker/config.json`), so the daemon can pull the job image.
- `git.thermograph.org` resolves to the **mesh** address from vps2 for registry
- vps2's docker daemon is logged in to `dev.jinemi.com`
(`/root/.docker/config.json`), so the daemon can pull the job image. A login
against the old `git.thermograph.org` does not carry over — docker keys
credentials by registry host, so this needs its own `docker login`.
- `dev.jinemi.com` resolves to the **mesh** address from vps2 for registry
traffic. Job containers get this via `--add-host` in `config.yaml`; the host
daemon already has it.
- The job image exists: `git.thermograph.org/jinemi/thermograph/ci-runner:v2`.
daemon needs the same pin in `/etc/hosts`.
- The job image exists: `dev.jinemi.com/jinemi/thermograph/ci-runner:v2`.
Built from `../ci-runner/Dockerfile`; see that file for when to rebuild.
## Where it is deployed, and why not from the checkout
@ -89,14 +91,20 @@ docker run --rm -it \
-v "$PWD/data:/data" -w /data --user root \
code.forgejo.org/forgejo/runner:6.3.1 \
forgejo-runner register --no-interactive \
--instance https://git.thermograph.org \
--instance https://dev.jinemi.com \
--token "$REG_TOKEN" \
--name thermograph-vps2 \
--labels 'docker:docker://git.thermograph.org/jinemi/thermograph/ci-runner:v2'
--labels 'docker:docker://dev.jinemi.com/jinemi/thermograph/ci-runner:v2'
unset REG_TOKEN
```
The runner registered before the domain migration holds
`https://git.thermograph.org` as its instance URL and keeps working while Caddy
serves that name (it does — see `../caddy-git.conf`). Re-registering with the
command above is how it moves to the canonical name; it is not required, and it
costs a fresh registration token and a runner restart.
That writes `data/.runner`, which holds the **long-lived** runner token. It is
`0600` and must stay out of git — `data/` is gitignored for exactly this reason.
Re-running `register` with a fresh registration token replaces it; the old
@ -147,5 +155,6 @@ only, and the job container has no other way to learn the mesh address.
from the compose file.
**The runner is up but `last_online` is stale.** It cannot reach
`https://git.thermograph.org` — check egress from vps2 and that Forgejo on vps1
is serving.
the instance URL it registered with (`https://dev.jinemi.com`, or
`https://git.thermograph.org` for a runner registered before the migration) —
check egress from vps2 and that Forgejo on vps1 is serving that name.

View file

@ -25,18 +25,26 @@ runner:
shutdown_timeout: 3h
container:
# 1) --add-host: git.thermograph.org resolves publicly to vps1, but vps1's
# 1) --add-host: both Forgejo names resolve publicly to vps1, but vps1's
# Caddy rejects the /v2/* registry API from off-mesh. Job containers that
# docker-pull or docker-push therefore need the MESH address, and DNS will
# not give it to them. Same line the desktop runner carries; the value is
# the same from vps2 because both boxes are on 10.10.0.0/24.
#
# BOTH names are pinned, and that is not belt-and-braces. dev.jinemi.com is
# the registry host in image names AND Forgejo's ROOT_URL, from which the
# registry's bearer-token realm is derived -- so a pull needs it twice over.
# git.thermograph.org stays pinned because already-pushed tags carry it as
# their prefix; unpinned, a rollback to such a tag takes the public route
# and gets a 403 that reads like a credential failure. See
# ../README.md, "Migrating the domain".
#
# 2) --cpus/--memory: vps2 runs prod. Without a ceiling a runaway job (a
# `docker build` that fans out, a test that leaks) competes with the
# production app for the same 6 cores. These bound the JOB container, which
# is a sibling of the runner rather than its child, so the runner's own
# compose limits do not cover it -- this is the only lever that does.
options: --add-host=git.thermograph.org:10.10.0.2 --cpus=2 --memory=4g
options: --add-host=dev.jinemi.com:10.10.0.2 --add-host=git.thermograph.org:10.10.0.2 --cpus=2 --memory=4g
# Jobs may mount NOTHING from the host. The workflows here do not need to:
# they check out into the job container's own workspace and reach the estate

View file

@ -106,7 +106,7 @@ export STACK_ENV_FILE
# --- image tags -----------------------------------------------------------------
# Same persisted-tags contract as deploy.sh: incoming env wins, the file
# supplies the sibling. Stack mode keeps its own file so test/real never mix.
REGISTRY_HOST="${REGISTRY_HOST:-git.thermograph.org}"
REGISTRY_HOST="${REGISTRY_HOST:-dev.jinemi.com}"
export REGISTRY_HOST
TAGS_FILE="$APP_DIR/infra/deploy/.stack-image-tags.env"
_incoming_backend="${BACKEND_IMAGE_TAG:-}"

View file

@ -62,7 +62,7 @@
services:
beta-web:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"]
environment:
# Beta's OWN role and OWN database on the shared instance. `db` resolves
@ -108,7 +108,7 @@ services:
failure_action: rollback
beta-worker:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"]
environment:
THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph_beta:${POSTGRES_PASSWORD}@db:5432/thermograph_beta
@ -144,7 +144,7 @@ services:
condition: on-failure
beta-lake:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"]
environment:
THERMOGRAPH_ROLE: lake
@ -176,7 +176,7 @@ services:
failure_action: rollback
beta-daemon:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
# Same reasoning as prod's daemon: NOT env-entrypoint.sh, because that shim
# execs the image's own entrypoint (Alembic + uvicorn) and migrations belong
# to the one-shot task. Source the host-rendered env and exec the binary.
@ -218,7 +218,7 @@ services:
failure_action: rollback
beta-frontend:
image: ${REGISTRY_HOST:-git.thermograph.org}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"]
# REQUIRED: overriding `entrypoint:` with no `command:` drops the image's
# CMD entirely, and env-entrypoint.sh's fallback (`exec uvicorn app:app`)

View file

@ -64,7 +64,7 @@ services:
condition: on-failure
web:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"]
environment:
THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph:${POSTGRES_PASSWORD}@db:5432/thermograph
@ -109,7 +109,7 @@ services:
failure_action: rollback
worker:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"]
environment:
THERMOGRAPH_DATABASE_URL: postgresql+asyncpg://thermograph:${POSTGRES_PASSWORD}@db:5432/thermograph
@ -153,7 +153,7 @@ services:
# healthy and answers 503/404, and web falls through to NASA — the lake is
# an accelerator, never a point of failure.
lake:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"]
environment:
THERMOGRAPH_ROLE: lake
@ -186,7 +186,7 @@ services:
# (/usr/local/bin/thermograph-daemon, built into the backend image) and the
# app share the /internal/* API contract, so one BACKEND_IMAGE_TAG rolls
# them together and they can never skew.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:?required}
# Not env-entrypoint.sh: that shim's handoff execs the image's
# deploy/entrypoint.sh (Alembic + uvicorn) whenever it exists, which is
# exactly what this service must never run — migrations belong to the
@ -243,7 +243,7 @@ services:
failure_action: rollback
frontend:
image: ${REGISTRY_HOST:-git.thermograph.org}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:?required}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:?required}
entrypoint: ["/host/env-entrypoint.sh"]
# REQUIRED, not cosmetic: overriding `entrypoint:` with no `command:` drops
# the image's own CMD entirely (Docker/Swarm semantics, not merged) --

View file

@ -36,7 +36,7 @@ if ! ufw status | grep -q "^Status: active"; then
echo "ports above. Before enabling, explicitly allow every port this node"
echo "already serves publicly (SSH at minimum; on vps2 specifically, also"
echo "80/tcp and 443/tcp for the live thermograph.org/beta.thermograph.org"
echo "Caddy; on vps1, 80/tcp and 443/tcp for git.thermograph.org,"
echo "Caddy; on vps1, 80/tcp and 443/tcp for dev.jinemi.com,"
echo "dashboard.thermograph.org and emigriffith.dev) — check 'ss -tlnp' for"
echo "what's actually listening first. Enabling ufw without doing this WILL"
echo "drop live traffic the moment it activates."

View file

@ -16,7 +16,7 @@
PORT=8137
# --- Registry pull (deploy.sh / Terraform) ---------------------------------------
# deploy.sh logs in to git.thermograph.org's registry and `docker compose pull`s
# deploy.sh logs in to dev.jinemi.com's registry and `docker compose pull`s
# the image build-push.yml already pushed for the deployed commit (repo-split
# Stage 6), instead of building in place. A personal access token with at least
# read:package scope -- the same REGISTRY_TOKEN secret build-push.yml uses (that

View file

@ -102,7 +102,7 @@ services:
# builds `jinemi/thermograph/backend:local` from the backend repo (see
# infra Makefile) and this pulls/uses it. BACKEND_IMAGE_PATH/TAG are
# independent of the frontend's, so the two services deploy separately.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
depends_on:
db:
condition: service_healthy
@ -180,7 +180,7 @@ services:
# without them the service stays healthy and every read falls through to
# NASA. Never published on a host port: only the backend talks to it.
lake:
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
environment:
THERMOGRAPH_ROLE: lake
PORT: 8141
@ -207,7 +207,7 @@ services:
# share the /internal/* API contract, so they must roll together — a
# separate tag could skew them. It owns the Discord gateway websocket and
# the recurring-job timers; anything needing data calls back into backend.
image: ${REGISTRY_HOST:-git.thermograph.org}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${BACKEND_IMAGE_PATH:-jinemi/thermograph/backend}:${BACKEND_IMAGE_TAG:-local}
# Bypass deploy/entrypoint.sh entirely: that script runs Alembic, and the
# backend service's boot already owns migrations — two containers racing
# `alembic upgrade head` against one database is a real hazard, not a
@ -262,7 +262,7 @@ services:
# frontend/Dockerfile (which starts the thermograph-frontend Go binary
# directly -- no THERMOGRAPH_SERVICE_ROLE process-picking). Independent
# FRONTEND_IMAGE_TAG so a frontend deploy never disturbs the backend's tag.
image: ${REGISTRY_HOST:-git.thermograph.org}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:-local}
image: ${REGISTRY_HOST:-dev.jinemi.com}/${FRONTEND_IMAGE_PATH:-jinemi/thermograph/frontend}:${FRONTEND_IMAGE_TAG:-local}
# Its own register() fetches the IndexNow key from backend at boot -- must
# wait for a real, healthy backend, not just a started container.
depends_on:

View file

@ -33,7 +33,7 @@ would silently overwrite the first's site instead of adding to it (there is no
merge). Prod claims that slot on vps2 today, so beta's public reverse-proxy (if
and when `beta.thermograph.org` is exposed) has to be a site block in vps2's
shared, hand-maintained Caddy config instead — the same pattern
`deploy/forgejo/docker-stack.yml` already uses to put `git.thermograph.org` in
`deploy/forgejo/docker-stack.yml` already uses to put `dev.jinemi.com` in
front of Forgejo's loopback port on vps1, alongside that box's own Caddy-fronted
`emigriffith.dev`.
@ -180,7 +180,7 @@ Terraform brings the stack up — don't let Terraform recreate containers mid-mi
`/etc/caddy/Caddyfile` prod's apply just installed (see the table above). Reach
beta via an SSH tunnel, or front it with a site block in vps2's own
hand-maintained Caddy config (outside Terraform), the same way Forgejo's
`git.thermograph.org` reaches Forgejo's loopback port on vps1.
`dev.jinemi.com` reaches Forgejo's loopback port on vps1.
`COOKIE_SECURE` is auto-set to `0` when there's no domain (a Secure
cookie is never sent over plain HTTP) and `1` behind Caddy TLS.
- The rendered Caddyfile only reverse-proxies the app. The repo's

View file

@ -256,7 +256,7 @@ resource "null_resource" "host" {
. deploy/render-secrets.sh
render_thermograph_secrets ${var.app_dir}
set -a; . /etc/thermograph-topology.env; . /etc/thermograph.env; set +a
export REGISTRY_HOST="git.thermograph.org" BACKEND_IMAGE_TAG="${var.backend_image_tag}" FRONTEND_IMAGE_TAG="${var.frontend_image_tag}"
export REGISTRY_HOST="dev.jinemi.com" BACKEND_IMAGE_TAG="${var.backend_image_tag}" FRONTEND_IMAGE_TAG="${var.frontend_image_tag}"
echo "$REGISTRY_TOKEN" | docker login "$REGISTRY_HOST" --username admin_emi --password-stdin
docker compose ${local.compose_flags} pull backend frontend
docker compose ${local.compose_flags} up -d --remove-orphans

View file

@ -87,7 +87,7 @@ hosts = {
# ONE environment on a box can own that file. Prod already claims it on
# this host, so beta.thermograph.org's public reverse-proxy has to be a
# site block in vps2's shared, hand-maintained Caddy instead (the same
# pattern deploy/forgejo/docker-stack.yml uses for git.thermograph.org
# pattern deploy/forgejo/docker-stack.yml uses for dev.jinemi.com
# on vps1) -- not something this module can render for a second
# environment on the same host.
domain = ""
@ -151,13 +151,13 @@ hosts = {
# }
# Optional overrides (shown with their defaults):
# repo_url = "https://git.thermograph.org/admin_emi/thermograph-infra.git"
# repo_url = "https://dev.jinemi.com/admin_emi/thermograph-infra.git"
# app_port = 8137
# frontend_port = 8080
#
# thermograph-infra is a PRIVATE repo, so a host cloning it for the first time
# needs read credentials embedded in repo_url, e.g. a Forgejo deploy token:
# repo_url = "https://deploy:REPLACE_WITH_TOKEN@git.thermograph.org/admin_emi/thermograph-infra.git"
# repo_url = "https://deploy:REPLACE_WITH_TOKEN@dev.jinemi.com/admin_emi/thermograph-infra.git"
# ---------------------------------------------------------------------------------
# Self-hosted Open-Meteo archive (only used by environments with openmeteo = true)

View file

@ -127,9 +127,9 @@ variable "gcp_hosts" {
}
variable "repo_url" {
description = "Git remote to clone from when a host has no checkout yet. Points at THIS repo (thermograph-infra) now, not the app repo -- the app's own source is never checked out on a host; only its published registry images are pulled (see var.hosts[*].backend_image_tag / frontend_image_tag). thermograph-infra is a private repo, so this typically needs embedded read credentials, e.g. a Forgejo deploy token: \"https://<token-name>:<token>@git.thermograph.org/admin_emi/thermograph-infra.git\"."
description = "Git remote to clone from when a host has no checkout yet. Points at THIS repo (thermograph-infra) now, not the app repo -- the app's own source is never checked out on a host; only its published registry images are pulled (see var.hosts[*].backend_image_tag / frontend_image_tag). thermograph-infra is a private repo, so this typically needs embedded read credentials, e.g. a Forgejo deploy token: \"https://<token-name>:<token>@dev.jinemi.com/admin_emi/thermograph-infra.git\"."
type = string
default = "https://git.thermograph.org/admin_emi/thermograph-infra.git"
default = "https://dev.jinemi.com/admin_emi/thermograph-infra.git"
sensitive = true
}

View file

@ -94,7 +94,7 @@ Both the dashboard and Forgejo log in with Google. Create **one** Google Cloud
OAuth 2.0 Client (type: *Web application*) and register both redirect URIs:
- Grafana: `https://dashboard.thermograph.org/login/google`
- Forgejo: `https://git.thermograph.org/user/oauth2/google/callback`
- Forgejo: `https://dev.jinemi.com/user/oauth2/google/callback`
**Grafana:** put the client id/secret in `.env`, set `OAUTH_ENABLED=true`, and
`docker compose up -d`. Login is locked to pre-provisioned accounts