2026-07-25 07:08:54 +00:00
|
|
|
# infra/
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
|
|
|
|
|
Infrastructure for [Thermograph](https://thermograph.org): Terraform host
|
2026-07-25 07:08:54 +00:00
|
|
|
provisioning, the SOPS+age secrets vault, WireGuard/Swarm networking, Forgejo,
|
|
|
|
|
Caddy, mail, and the deploy scripts that run the already-built app images on each
|
2026-07-26 06:56:38 +00:00
|
|
|
host. This is a domain of the `emi/thermograph` monorepo — a host's checkout
|
|
|
|
|
(`/opt/thermograph`, `/opt/thermograph-beta`, or `/opt/thermograph-dev`) is a
|
|
|
|
|
checkout of the whole monorepo, and `infra/` never builds app source; it only
|
2026-07-25 07:08:54 +00:00
|
|
|
runs published images.
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
|
|
|
|
|
- **`terraform/`** — provisions/configures hosts (SSH-driven by default; an
|
2026-07-25 07:08:54 +00:00
|
|
|
optional GCP-creating module is scaffolded, no live resources yet). See
|
|
|
|
|
`terraform/README.md`. No tfstate is persisted anywhere — treat `apply` as
|
|
|
|
|
executable documentation, not a routine operation.
|
|
|
|
|
- **`deploy/secrets/`** — the git-native SOPS+age secrets vault (every app secret,
|
|
|
|
|
encrypted at rest, rendered at deploy time). See `deploy/secrets/README.md`.
|
2026-07-26 06:56:38 +00:00
|
|
|
- **`deploy/swarm/`, `deploy/forgejo/`** — the WireGuard/Swarm mesh spanning
|
|
|
|
|
vps1, vps2 and the desktop, and Forgejo (git + CI + registry), pinned to
|
|
|
|
|
vps1. See `ACCESS.md`.
|
|
|
|
|
- **`deploy/env-topology.sh`** — the single source of truth for where each
|
|
|
|
|
environment (`dev`/`beta`/`prod`) lives: host, checkout path, branch, deploy
|
|
|
|
|
mode, stack/compose name, env file, LB ports, DB role/database, service-name
|
|
|
|
|
prefix. Every deploy path sources it and derives its behavior from it rather
|
|
|
|
|
than guessing from the host it happens to be running on — necessary since
|
|
|
|
|
vps2 alone now runs two environments.
|
|
|
|
|
- **`deploy/deploy.sh`** — the single deploy entry point for dev, beta and prod.
|
|
|
|
|
Takes `SERVICE=backend|frontend|all` plus `BACKEND_IMAGE_TAG`/`FRONTEND_IMAGE_TAG`
|
|
|
|
|
(and, on vps2, `THERMOGRAPH_ENV=beta|prod` to say which of the two checkouts
|
|
|
|
|
it's acting on), resets the host checkout, renders secrets, and routes to the
|
|
|
|
|
right orchestrator per `env-topology.sh`.
|
|
|
|
|
- **`deploy/deploy-dev.sh`** — a thin dev-specific wrapper around `deploy.sh`
|
|
|
|
|
(dev compose overlay, dev's secrets policy). See `DEPLOY-DEV.md`.
|
|
|
|
|
- **`deploy/stack/`** — the **Swarm** path, live on **vps2** for both prod and
|
|
|
|
|
beta: `thermograph-stack.yml` (prod: db, web, worker, lake, daemon, frontend,
|
|
|
|
|
autoscaler, autoscaler-lake) and `thermograph-beta-stack.yml` (beta: the same
|
|
|
|
|
service shape minus `db` and the autoscalers, every service name prefixed
|
|
|
|
|
`beta-`). `deploy-stack.sh`, `autoscale.sh` and `lb/` are shared by both.
|
|
|
|
|
Rolling updates are start-first, health-gated, with auto-rollback.
|
|
|
|
|
`STACK_TEST=1` rehearses the whole stack on throwaway volumes and ports.
|
|
|
|
|
- **`docker-compose*.yml`** — the **compose** path, live only on **dev** (vps1)
|
|
|
|
|
(db, backend, lake, daemon, frontend). `docker-compose.dev.yml` is dev's
|
|
|
|
|
mesh-only overlay; `docker-compose.openmeteo.yml` is the self-hosted
|
|
|
|
|
Open-Meteo overlay (prod only). `make dev-up` also runs this path locally as
|
|
|
|
|
a laptop convenience — that is not an "environment", just a local rehearsal.
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
|
2026-07-26 06:56:38 +00:00
|
|
|
Which path an environment takes is decided by `deploy/env-topology.sh`
|
|
|
|
|
(`TG_DEPLOY_MODE`, keyed by `dev`/`beta`/`prod`): dev is `compose`, beta and prod
|
|
|
|
|
are both `stack`. The old host-wide marker `/etc/thermograph/deploy-mode` still
|
|
|
|
|
exists as a fallback for a by-hand run with no explicit environment, but it
|
|
|
|
|
cannot describe vps2, which runs two environments in two different checkouts —
|
|
|
|
|
so it is no longer the thing that decides where files go. The workflows never
|
|
|
|
|
need to know which mode an environment runs; they only pass `THERMOGRAPH_ENV`.
|
2026-07-22 23:36:21 +00:00
|
|
|
|
|
|
|
|
## Branches & how changes reach each environment
|
|
|
|
|
|
2026-07-26 06:56:38 +00:00
|
|
|
- **`dev`** — deploys to **dev on vps1** (`/opt/thermograph-dev`).
|
|
|
|
|
- **`main`** — deploys to **beta on vps2** (`/opt/thermograph-beta`).
|
|
|
|
|
- **`release`** — deploys to **prod on vps2** (`/opt/thermograph`).
|
2026-07-22 23:36:21 +00:00
|
|
|
|
2026-07-26 06:56:38 +00:00
|
|
|
App code IS environment-staged this way (`dev`→`main`→`release` maps to
|
|
|
|
|
vps1/dev → vps2/beta → vps2/prod via image tags, one `Deploy` workflow keyed by
|
|
|
|
|
branch). **Infra itself is not environment-staged** the same way: `infra-sync.yml`
|
|
|
|
|
fires on a push touching `infra/**` — on `dev` it fast-forwards vps1's dev
|
|
|
|
|
checkout, on `main` it fast-forwards **both** of vps2's checkouts (beta and
|
|
|
|
|
prod) — re-rendering each environment's own env file from the vault. It
|
|
|
|
|
deliberately does **not** roll any service: image tags are the app domains'
|
|
|
|
|
axis, not infra's. A compose or stack change that must recreate containers
|
|
|
|
|
takes effect on the next app deploy, or a by-hand `SERVICE=all …
|
|
|
|
|
deploy/deploy.sh` (or `deploy-dev.sh`) run.
|
|
|
|
|
|
|
|
|
|
Note the asymmetry this leaves: dev's infra checkout tracks `dev`, the same
|
|
|
|
|
branch its app images are staged by. Beta's and prod's infra checkouts both
|
|
|
|
|
track `main` — prod's *app images* are staged by `release`, but prod's *infra
|
|
|
|
|
checkout* follows `main`, same as beta's.
|
