From 396b80cf919d08631fdbceb723043cbd12f739a4 Mon Sep 17 00:00:00 2001 From: Emi Griffith Date: Sat, 1 Aug 2026 09:01:59 -0700 Subject: [PATCH 1/3] forgejo: serve dev.jinemi.com alongside git.thermograph.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both hostnames go on a single Caddy site block. That is load-bearing rather than cosmetic: the /v2/* mesh-only matcher that keeps the OCI registry API off the public internet (hazard #15) is scoped per block, so a separate block for the new name would serve the same Forgejo with the registry open. git.thermograph.org stays canonical — Forgejo has one ROOT_URL and builds every absolute URL from it, so clone URLs, the OAuth callback and post-login redirects keep naming the .org host. Flipping FORGEJO_DOMAIN additionally requires the Google OAuth redirect URI, the registry host baked into image names and runner labels, and the mesh /etc/hosts pins; README documents those prerequisites. Config verified with `caddy validate` and by asserting against the adapted JSON that the /v2 403 guard matches both hostnames. --- infra/ACCESS.md | 2 +- infra/CLAUDE.md | 4 ++- infra/deploy/Caddyfile.vps1 | 3 ++ infra/deploy/forgejo/README.md | 47 +++++++++++++++++++++++++++++ infra/deploy/forgejo/caddy-git.conf | 35 +++++++++++++-------- 5 files changed, 77 insertions(+), 14 deletions(-) diff --git a/infra/ACCESS.md b/infra/ACCESS.md index 6713241..4ffec1e 100644 --- a/infra/ACCESS.md +++ b/infra/ACCESS.md @@ -26,7 +26,7 @@ rather than which environment happens to live there: | Host | Public IP | Mesh IP | Role | |------|-----------|---------|------| -| **vps1** | `75.119.132.91` | `10.10.0.2` | "Operational programs": Forgejo (git + CI + registry, `git.thermograph.org`), Grafana/Loki/Alloy (`dashboard.thermograph.org`), the `emigriffith.dev` portfolio, and the **dev** environment (mesh-only, `10.10.0.2:8137`, no public DNS/Caddy/TLS) | +| **vps1** | `75.119.132.91` | `10.10.0.2` | "Operational programs": Forgejo (git + CI + registry, `git.thermograph.org`, also served at `dev.jinemi.com` — migration in progress, `.org` still canonical), Grafana/Loki/Alloy (`dashboard.thermograph.org`), the `emigriffith.dev` portfolio, and the **dev** environment (mesh-only, `10.10.0.2:8137`, no public DNS/Caddy/TLS) | | **vps2** | `169.58.46.181` | `10.10.0.1` | "The deployed environment": **prod** and **beta**, as two separate Docker Swarm stacks, plus Centralis, Postfix, the backups, and the one shared TimescaleDB instance both stacks use | | **desktop** | — | `10.10.0.3` | AI-model hosting (voice-to-text, an upcoming-feature LLM) plus flex Swarm-worker capacity. Hosts **no** Thermograph environment — `make dev-up` still works there, but only as a laptop-local rehearsal | diff --git a/infra/CLAUDE.md b/infra/CLAUDE.md index 390e0e4..f8dc94b 100644 --- a/infra/CLAUDE.md +++ b/infra/CLAUDE.md @@ -12,7 +12,9 @@ Two VPS boxes plus the operator's desktop, on one WireGuard mesh, named by ROLE rather than by environment: - **vps1** — `75.119.132.91`, mesh `10.10.0.2`. "Operational programs": Forgejo - (git + CI + registry, `git.thermograph.org`), Grafana/Loki/Alloy + (git + CI + registry, `git.thermograph.org`, also served at `dev.jinemi.com` + — one Caddy site block, the `.org` name still canonical; migration in + progress, see `deploy/forgejo/README.md`), Grafana/Loki/Alloy (`dashboard.thermograph.org`), the `emigriffith.dev` portfolio site, and the **dev** environment (`/opt/thermograph-dev`, tracks `dev`), with its own Postgres container. Dev is mesh-only here — published on `10.10.0.2:8137`, diff --git a/infra/deploy/Caddyfile.vps1 b/infra/deploy/Caddyfile.vps1 index ebd6ac2..9c81727 100644 --- a/infra/deploy/Caddyfile.vps1 +++ b/infra/deploy/Caddyfile.vps1 @@ -3,6 +3,9 @@ # vps1 serves: # emigriffith.dev -> static portfolio site (from disk) # git.thermograph.org -> Forgejo (see deploy/forgejo/caddy-git.conf) +# dev.jinemi.com -> Forgejo, same site block — migration target for +# git.thermograph.org, both served, the .org one +# still canonical (that file's header explains) # dashboard.thermograph.org -> Grafana (see observability/caddy-grafana.conf) # # and hosts the **dev** environment at dev.thermograph.org (below). diff --git a/infra/deploy/forgejo/README.md b/infra/deploy/forgejo/README.md index 9e295c1..7c2a5ed 100644 --- a/infra/deploy/forgejo/README.md +++ b/infra/deploy/forgejo/README.md @@ -83,6 +83,53 @@ pattern as its other site blocks, same automatic-HTTPS. reach the registry over the mesh, not the public internet — see "Registry access from mesh clients" below. +## Migrating the domain (git.thermograph.org -> dev.jinemi.com) + +In progress. **Both hostnames are served**; `git.thermograph.org` is still the +canonical one. + +`caddy-git.conf` lists both names on a *single* site block. That is deliberate, +not cosmetic: the `/v2/*` mesh-only matcher is per-block, so a second block for +`dev.jinemi.com` would re-expose the registry API publicly and undo hazard #15. +Whatever else changes, keep the two names in one block. + +Serving a second hostname is the cheap half. Forgejo has exactly one +`ROOT_URL`, and it builds every absolute URL from it, so on the non-canonical +host today: + +- clone URLs on repo pages read `git.thermograph.org`, +- the OAuth callback and post-login redirect land on `git.thermograph.org` + (login is Google-SSO-only — see `docker-stack.yml`), +- webhook payload URLs and notification mail links use `git.thermograph.org`. + +Browsing, git-over-HTTP and the API all work on `dev.jinemi.com` regardless. + +Flipping canonical (`FORGEJO_DOMAIN=dev.jinemi.com`, then `docker stack deploy`) +needs these **first**, or it breaks login and CI rather than just renaming +things: + +1. **Google OAuth** — add `https://dev.jinemi.com/user/oauth2//callback` + as an authorized redirect URI in the Google Cloud console *before* the flip. + Not in this repo; nothing here can verify it. Miss it and no one can log in, + including to undo the flip. +2. **The registry host in image names** — images are named by registry host, and + the `git.thermograph.org/` prefix is baked into the runner labels + (`register-lan-runner.sh`, `runner-vps2/README.md`), the CI-runner image + (`ci-runner/Dockerfile`), `REGISTRY_HOST` in `infra/.env.example`, and every + already-pushed tag. Renaming the registry host is a separate migration from + renaming the web UI; the two need not happen together, and doing the web UI + alone is much the smaller change. +3. **Mesh `/etc/hosts` pins** — every node in "Registry access from mesh + clients" below pins `git.thermograph.org` to `10.10.0.2`. A node pulling + under a new hostname needs its own pin, or Caddy's `/v2/*` matcher sees a + public source IP and returns 403. +4. **Runner instance URL** — registered runners hold the instance URL they were + registered with. Existing runners keep working against the old name as long + as it resolves; a *re*-registration must use whichever name is canonical then. + +Until all four are handled, leaving `ROOT_URL` on `git.thermograph.org` is the +correct state, not an unfinished one. + ## Registry access from mesh clients Any node that needs `docker login`/push/pull against the registry (the CI diff --git a/infra/deploy/forgejo/caddy-git.conf b/infra/deploy/forgejo/caddy-git.conf index 55cb9c7..fdaefee 100644 --- a/infra/deploy/forgejo/caddy-git.conf +++ b/infra/deploy/forgejo/caddy-git.conf @@ -1,6 +1,6 @@ -# Only the block below (from "git.thermograph.org {") goes into the live -# Caddyfile — append just that part, not this instructional header, or it -# ends up as a confusing orphaned comment in production config with no +# Only the block below (from "git.thermograph.org, dev.jinemi.com {") goes into +# the live Caddyfile — append just that part, not this instructional header, or +# it ends up as a confusing orphaned comment in production config with no # surrounding context (docker-stack.yml isn't visible from there). # # Target: /etc/caddy/Caddyfile on vps1 (the box `forgejo` is pinned to — see @@ -9,19 +9,30 @@ # Forgejo publishes its web UI to 127.0.0.1:3080 only (host-local), and this # block is the only thing that ever talks to it. # -# Registry exposure (hazard #15, see docker-stack.yml's header comment): -# git.thermograph.org/v2/* is the built-in OCI registry API. It's blocked -# from anywhere except the WireGuard mesh (10.10.0.0/24) — reachable to CI -# runners and Swarm nodes over wg0, not from the public internet. Everything -# else (web UI, git-over-HTTP, PR pages) stays public like the rest of the -# site. +# TWO hostnames, ONE block, and that is load-bearing — see "Registry exposure" +# below. Both names must share a site block so the /v2/* restriction covers +# both. Splitting dev.jinemi.com into its own block serves the same Forgejo +# with the registry API open to the public internet. # -# Update the domain below to match whatever you actually pointed at vps1's -# IP, if not git.thermograph.org. +# dev.jinemi.com is the migration target for git.thermograph.org; both are +# served during the transition. Each gets its own Let's Encrypt cert +# automatically (HTTP-01) — both A records already point at vps1. +# +# git.thermograph.org stays CANONICAL until Forgejo's ROOT_URL is flipped +# (docker-stack.yml, FORGEJO_DOMAIN). Until then dev.jinemi.com serves every +# page, but Forgejo builds absolute URLs from ROOT_URL, so clone URLs, the +# OAuth callback and post-login redirects still name git.thermograph.org. +# Flipping it is NOT just this file — see README.md, "Migrating the domain". +# +# Registry exposure (hazard #15, see docker-stack.yml's header comment): +# /v2/* is the built-in OCI registry API. It's blocked from anywhere except +# the WireGuard mesh (10.10.0.0/24) — reachable to CI runners and Swarm nodes +# over wg0, not from the public internet. Everything else (web UI, +# git-over-HTTP, PR pages) stays public like the rest of the site. # --- copy from here down into /etc/caddy/Caddyfile --- -git.thermograph.org { +git.thermograph.org, dev.jinemi.com { encode zstd gzip @registry_external { From 5b5f28560d5955b5b5083c4e1b0c25d6262e7059 Mon Sep 17 00:00:00 2001 From: Emi Griffith Date: Sat, 1 Aug 2026 09:16:06 -0700 Subject: [PATCH 2/3] gitignore: ignore .claude/settings.local.json MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It was untracked but not ignored, so a single `git add -A` would have committed one operator's local permission allowlist — including which vault secrets they may decrypt — as shared repo config. `.claude/settings.json` stays tracked as the reviewed team config. --- .gitignore | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.gitignore b/.gitignore index 5aee0f7..f5ea9c0 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,9 @@ # Claude Code worktrees. These are other sessions' checkouts living inside this # repo; `git add -A` will otherwise commit them as embedded git repositories. .claude/worktrees/ + +# Per-operator Claude Code settings — local permission grants, not team config. +# Untracked but not ignored meant one `git add -A` would publish one operator's +# allowlist (including which secrets they may decrypt) to everyone. +# `.claude/settings.json` IS tracked and stays the shared, reviewed config. +.claude/settings.local.json From 3993b7552fccb2616b83398e276da11c0e92f795 Mon Sep 17 00:00:00 2001 From: Emi Griffith Date: Sat, 1 Aug 2026 09:06:11 -0700 Subject: [PATCH 3/3] lb: trust the host Caddy's X-Forwarded-Proto Every OAuth redirect URI the backend builds is currently http://, whatever scheme the user actually arrived with. There are two Caddy hops in front of the app: the host Caddy terminates TLS and sets X-Forwarded-Proto: https, then proxies to 127.0.0.1:8137 over plain HTTP; the stack LB is the second hop. Caddy preserves an incoming X-Forwarded-* header only when the immediate peer is a trusted proxy, and trusted_proxies was never set here -- so the LB overwrote the header with the scheme of the connection it had just received, which is http. accounts/oauth.py:_redirect_uri builds the callback from x-forwarded-proto, and that URI must match what is registered in the provider's console. Measured on vps2, same request to each hop: direct to web, XFP=https -> https://thermograph.org/api/v2/... through the LB, XFP=https -> http://thermograph.org/api/v2/... through the PATCHED LB -> https://thermograph.org/api/v2/... through the PATCHED LB, no XFP-> http://thermograph.org/api/v2/... The last line is the point of using trusted_proxies rather than hardcoding `header_up X-Forwarded-Proto https`: the LB still reports the truth when nothing in front of it claims otherwise. private_ranges covers 127.0.0.1/8 and the Docker bridge ranges, which is where the host Caddy reaches this container from. The LB binds loopback only, precisely so it cannot be reached un-fronted, so the only party that can set these headers is the host Caddy -- trusting the local peer widens nothing. Applied to beta's LB too. Beta is where a provider change gets tested before it reaches thermograph.org, so it needs the same behaviour or the test is not a test. --- infra/deploy/stack/lb/Caddyfile | 28 ++++++++++++++++++++++++++++ infra/deploy/stack/lb/Caddyfile.beta | 11 +++++++++++ 2 files changed, 39 insertions(+) diff --git a/infra/deploy/stack/lb/Caddyfile b/infra/deploy/stack/lb/Caddyfile index 46402ef..8ea7ffb 100644 --- a/infra/deploy/stack/lb/Caddyfile +++ b/infra/deploy/stack/lb/Caddyfile @@ -13,6 +13,34 @@ { auto_https off admin off + + # WITHOUT THIS, EVERY OAUTH REDIRECT URI THE BACKEND BUILDS IS http://. + # + # There are two Caddy hops in front of the app: the HOST Caddy terminates TLS + # and sets X-Forwarded-Proto: https, then proxies to 127.0.0.1:8137 over plain + # HTTP; this LB is the second hop. Caddy only preserves an incoming + # X-Forwarded-* header when the immediate peer is a TRUSTED proxy — otherwise + # it overwrites the header with the scheme of the connection it just received, + # which here is http. So the app saw http:// no matter how the user arrived. + # + # accounts/oauth.py:_redirect_uri builds the callback from x-forwarded-proto, + # and that URI must match what is registered in the provider's console. + # Discord tolerates the http:// form; GOOGLE REJECTS a non-HTTPS redirect URI + # for a web client outright, so Google sign-in could not work at all until + # this was fixed — independently of whether the credentials were configured. + # + # Measured on vps2 before the fix, same request to each hop: + # direct to web, XFP=https -> https://thermograph.org/api/v2/... + # through this LB, XFP=https -> http://thermograph.org/api/v2/... + # + # private_ranges covers 127.0.0.1/8 and the Docker bridge ranges, which is + # where the host Caddy reaches this container from. Nothing outside the box + # can connect here — the LB is bound to loopback precisely so it cannot be + # reached un-fronted (hazard #6, above) — so trusting the local peer does not + # widen anything: the only party that can set these headers is the host Caddy. + servers { + trusted_proxies static private_ranges + } } :8137 { diff --git a/infra/deploy/stack/lb/Caddyfile.beta b/infra/deploy/stack/lb/Caddyfile.beta index 756d21d..834f38b 100644 --- a/infra/deploy/stack/lb/Caddyfile.beta +++ b/infra/deploy/stack/lb/Caddyfile.beta @@ -17,6 +17,17 @@ { auto_https off admin off + + # Same reason as ./Caddyfile — see the long note there. Without it this second + # Caddy hop overwrites the host Caddy's X-Forwarded-Proto: https with http, + # and every OAuth redirect URI the backend builds comes out non-HTTPS, which + # Google rejects outright for a web client. + # + # Beta needs it as much as prod does, and arguably first: beta is where an + # OAuth provider change gets tested before it reaches thermograph.org. + servers { + trusted_proxies static private_ranges + } } :8137 {