Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
# Thermograph — Terraform (host provisioning)
|
|
|
|
|
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
Terraform that **provisions and configures VPS hosts** and hands the app off to
|
|
|
|
|
`docker compose`, which pulls the published app image (built + pushed by the app
|
|
|
|
|
repo's CI) and runs it — no app source is ever built or checked out on a host.
|
|
|
|
|
By default it does *not* create servers: `modules/thermograph-host` is
|
|
|
|
|
SSH-provisioner-driven against a host that already exists. An optional
|
|
|
|
|
`modules/gcp-host` can additionally *create* the VM on GCP first (see "GCP
|
|
|
|
|
scaffold" below) — scaffold-only today, no live resources until you opt in.
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
|
|
|
|
|
## What it manages
|
|
|
|
|
|
|
|
|
|
One reusable module (`modules/thermograph-host`) is instantiated per host via
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
`for_each` (`main.tf`'s `local.all_hosts`, merging `var.hosts` — SSH-managed,
|
|
|
|
|
already-existing boxes — with any `var.gcp_hosts` Terraform created itself).
|
|
|
|
|
This config manages **two VPS hosts** today:
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
|
|
|
|
|
| key | role | VPS | branch | domain | notes |
|
|
|
|
|
|--------|--------|-------------------------|-----------|-------------------|------------------------------------|
|
2026-07-22 23:27:13 +00:00
|
|
|
| `prod` | prod | NEW 48 GB / 12-core box (`169.58.46.181`) | `release` (of the APP repos — see `backend_image_tag` / `frontend_image_tag`) | `thermograph.org` | Caddy TLS; sized up (8/8/4/16g) |
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
| `beta` | beta | old box `75.119.132.91` | `main` (of the APP repo) | `beta.thermograph.org` | Caddy TLS; also hosts Forgejo |
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
Each host's own checkout on disk (`app_dir`, `git_branch`) is **this infra
|
|
|
|
|
repo**, not the app repo — the "branch" column above is which app-repo tag a
|
2026-07-22 23:27:13 +00:00
|
|
|
host is meant to track conceptually; the actual pinned versions are
|
|
|
|
|
`var.hosts[*].backend_image_tag` / `frontend_image_tag` (e.g. `"sha-<12 hex>"` each —
|
|
|
|
|
the app is two separately-published images, `emi/thermograph-backend/app` and
|
|
|
|
|
`emi/thermograph-frontend/app`), since the host has no app checkout to derive a tag
|
|
|
|
|
from. The LAN dev server is **out of scope here** — it
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
builds from source via the app repo's `deploy/deploy-dev.sh` (a self-hosted
|
|
|
|
|
Forgejo Actions runner), not Terraform.
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
|
|
|
|
|
Per host, over SSH provisioners, Terraform:
|
|
|
|
|
|
|
|
|
|
- installs Docker + the compose plugin if missing;
|
|
|
|
|
- configures a `ufw` firewall (22/80/443 always; on a host with **no** domain it also
|
|
|
|
|
opens the app port `8137`);
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
- ensures the git checkout at `app_dir` exists (clones **this repo** on a fresh box)
|
|
|
|
|
and resets it to the host's `git_branch`;
|
|
|
|
|
- renders `/etc/thermograph-topology.env` from Terraform variables (non-secret sizing
|
|
|
|
|
only — `WORKERS`/`APP_CPUS`/`DB_CPUS`/`DB_MEMORY`/etc.), then runs
|
|
|
|
|
`deploy/render-secrets.sh` (from that same freshly-synced checkout) to render
|
|
|
|
|
`/etc/thermograph.env` from the SOPS+age vault — **Terraform itself never sees or
|
|
|
|
|
carries an app secret** (see "Secrets" below);
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
- for a host **with** a domain, installs a rendered Caddyfile and reloads Caddy;
|
2026-07-22 23:27:13 +00:00
|
|
|
- brings the stack up on the explicit per-service tags (exports `BACKEND_IMAGE_TAG` /
|
|
|
|
|
`FRONTEND_IMAGE_TAG` for the compose file): `docker login` to the registry,
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
`docker compose <-f each compose file> pull backend frontend`, then
|
|
|
|
|
`... up -d --remove-orphans`, running docker as root with both env files sourced;
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
- health-checks `http://127.0.0.1:8137/`.
|
|
|
|
|
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
A change to the rendered topology env, the compose files, the branch, the app image
|
|
|
|
|
tag, or the sizing flips the `null_resource` trigger and re-runs the provisioners on
|
|
|
|
|
next apply.
|
|
|
|
|
|
|
|
|
|
## GCP scaffold (no live resources)
|
|
|
|
|
|
|
|
|
|
`modules/gcp-host` can create a GCE VM (+ a dedicated VPC, subnet, and a firewall
|
|
|
|
|
opening 22/80/443) instead of assuming the host already exists — see `main.tf`'s
|
|
|
|
|
`module.gcp_vm`. It contributes *nothing* to provisioning: its only output
|
|
|
|
|
(`external_ip`) feeds into the exact same `thermograph-host` module every
|
|
|
|
|
SSH-managed host uses, so there's one provisioning path regardless of how a
|
|
|
|
|
host came to exist. `var.gcp_hosts` defaults to `{}`, so by default no
|
|
|
|
|
`google_*` resource is ever planned and the `google` provider is never
|
|
|
|
|
invoked — `terraform plan`/`validate` succeed with no GCP credentials
|
|
|
|
|
configured at all. To actually use it: populate an entry in
|
|
|
|
|
`terraform.tfvars` (see the commented example in
|
|
|
|
|
`terraform.tfvars.example`) and authenticate via `gcloud auth
|
|
|
|
|
application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`. This is the
|
|
|
|
|
same create-then-provision composition a future Proxmox module (the
|
|
|
|
|
architecture doc's longer-term target — see §6/§9 there) would use.
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
|
|
|
|
|
### Container sizing is env-driven
|
|
|
|
|
|
|
|
|
|
`docker-compose.yml` reads `WORKERS`, `APP_CPUS`, `DB_CPUS`, and `DB_MEMORY` from the
|
|
|
|
|
environment (defaults `4 / 4 / 2 / 8g`, identical to before). Terraform sets them per
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
host through `/etc/thermograph-topology.env`, so the big prod box can run larger caps
|
|
|
|
|
without a compose edit. The Postgres *internal* memory budget (`shared_buffers`,
|
Move the climate record from parquet to TimescaleDB hypertables (#227)
Replace the per-cell parquet cache with TimescaleDB hypertables as the
production backend for the raw daily climate record, and drop pg_duckdb.
Parquet stays the backend whenever THERMOGRAPH_DATABASE_URL is not a
Postgres URL (dev, tests, offline tooling), the same dialect switch the
accounts DB and derived store already use, so CI stays Postgres-free.
- data/climate_store.py: psycopg + polars bridge over climate_history
(a hypertable), climate_recent, and climate_sync (per-cell freshness).
Reads via pl.read_database, writes via COPY + ON CONFLICT upsert;
fail-soft to a cache miss so a DB hiccup degrades to upstream refetch.
- data/climate.py: route every cache/mtime touchpoint through a backend
dispatch. recent_stamp becomes int(recent_synced_at) on Postgres; the
stale-serve path still avoids bumping it, so derived-payload tokens
invalidate on exactly the same events as before.
- alembic 0002: CREATE EXTENSION timescaledb plus the hypertable schema
(compression policy on year-old chunks), guarded to no-op off Postgres.
- migrate_cache_to_pg.py (make migrate-cache): idempotent backfill of the
parquet cache into the hypertables, preserving file mtimes as sync
timestamps so recent_stamp is unchanged across cutover.
- db image -> stock timescale/timescaledb:latest-pg18; drop the custom
pg_duckdb Dockerfile, the read-only /parquet mount, and the duckdb
tuning GUC. Docs updated for the new backend and cutover.
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-20 20:15:55 +00:00
|
|
|
`effective_cache_size`, `work_mem`, `maintenance_work_mem`) is derived from the same
|
2026-07-20 14:33:09 +00:00
|
|
|
`DB_MEMORY` by `deploy/db/init/20-tuning.sh` — so raising `db_memory` scales the
|
Move the climate record from parquet to TimescaleDB hypertables (#227)
Replace the per-cell parquet cache with TimescaleDB hypertables as the
production backend for the raw daily climate record, and drop pg_duckdb.
Parquet stays the backend whenever THERMOGRAPH_DATABASE_URL is not a
Postgres URL (dev, tests, offline tooling), the same dialect switch the
accounts DB and derived store already use, so CI stays Postgres-free.
- data/climate_store.py: psycopg + polars bridge over climate_history
(a hypertable), climate_recent, and climate_sync (per-cell freshness).
Reads via pl.read_database, writes via COPY + ON CONFLICT upsert;
fail-soft to a cache miss so a DB hiccup degrades to upstream refetch.
- data/climate.py: route every cache/mtime touchpoint through a backend
dispatch. recent_stamp becomes int(recent_synced_at) on Postgres; the
stale-serve path still avoids bumping it, so derived-payload tokens
invalidate on exactly the same events as before.
- alembic 0002: CREATE EXTENSION timescaledb plus the hypertable schema
(compression policy on year-old chunks), guarded to no-op off Postgres.
- migrate_cache_to_pg.py (make migrate-cache): idempotent backfill of the
parquet cache into the hypertables, preserving file mtimes as sync
timestamps so recent_stamp is unchanged across cutover.
- db image -> stock timescale/timescaledb:latest-pg18; drop the custom
pg_duckdb Dockerfile, the read-only /parquet mount, and the duckdb
tuning GUC. Docs updated for the new backend and cutover.
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-20 20:15:55 +00:00
|
|
|
container cap and the tuning together (prod 16g → shared_buffers 4 GB). The tuning
|
|
|
|
|
applies on a fresh DB volume; on an existing volume re-run it by hand (see the script
|
|
|
|
|
header).
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
|
|
|
|
|
## Prerequisites
|
|
|
|
|
|
|
|
|
|
- Terraform >= 1.6 (v1.15 is installed).
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
- SSH key access to **both** hosts as a **sudo-capable** user (the live boxes use the
|
|
|
|
|
dedicated `agent` account, `~/.ssh/thermograph_agent_ed25519` — see
|
|
|
|
|
`deploy/provision-agent-access.sh`). Point `ssh_private_key_path` at that key (`~`
|
|
|
|
|
is expanded).
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
- The hosts are Debian/Ubuntu with `apt` and outbound internet (Docker/Caddy installs
|
|
|
|
|
pull from the network). Docker may already be present — installs are conditional.
|
|
|
|
|
- For the `prod` host: DNS for `thermograph.org` must point at the new box before apply,
|
|
|
|
|
or Caddy's first-request cert issuance will fail.
|
|
|
|
|
|
|
|
|
|
## Use
|
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
cd terraform
|
|
|
|
|
cp terraform.tfvars.example terraform.tfvars # then edit: real IPs + secrets
|
|
|
|
|
terraform init
|
|
|
|
|
terraform plan
|
|
|
|
|
terraform apply
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Target one host with `-target='module.host["beta"]'` if you want to apply to just one.
|
|
|
|
|
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
## Secrets
|
|
|
|
|
|
|
|
|
|
App secrets (`POSTGRES_PASSWORD`, `THERMOGRAPH_AUTH_SECRET`, VAPID keys,
|
|
|
|
|
`REGISTRY_TOKEN`, Discord/SMTP credentials, …) are **not** Terraform variables
|
|
|
|
|
— they live in the git-native SOPS+age vault at `../deploy/secrets/*.yaml`
|
|
|
|
|
(committed, encrypted) and are rendered into `/etc/thermograph.env` at deploy
|
|
|
|
|
time by `../deploy/render-secrets.sh`, which the provisioner's deploy step
|
|
|
|
|
runs from the freshly-synced checkout. Terraform only renders the non-secret
|
|
|
|
|
`/etc/thermograph-topology.env` (sizing/routing). See
|
|
|
|
|
`../deploy/secrets/README.md` to rotate a secret or add a new one — it's a
|
|
|
|
|
`sops edit` + commit + deploy, no Terraform apply involved.
|
|
|
|
|
|
|
|
|
|
`om_rclone_conf` (object-storage bucket credentials for the self-hosted ERA5
|
|
|
|
|
archive) is the one exception still supplied via Terraform, in
|
|
|
|
|
`terraform.tfvars` — folding it into the vault too is a reasonable future
|
|
|
|
|
step, not done here.
|
|
|
|
|
|
|
|
|
|
## Local state
|
|
|
|
|
|
|
|
|
|
The backend is **local**: `terraform.tfstate` is written next to the config.
|
|
|
|
|
It's gitignored (`terraform/.gitignore` and the root `.gitignore`) — keep it
|
|
|
|
|
off shared disks and back it up somewhere private. `terraform.tfvars` is
|
|
|
|
|
likewise gitignored (it may carry a `repo_url` credential and always carries
|
|
|
|
|
`om_rclone_conf`); only `terraform.tfvars.example` (dummy values) is
|
|
|
|
|
committed. `.terraform.lock.hcl` **is** committed so provider versions are
|
|
|
|
|
pinned across machines.
|
Have Terraform generate its own internal secrets, with sizing tiers (#239)
Terraform generates the secrets that have no external meaning
(POSTGRES_PASSWORD, AUTH_SECRET, METRICS_TOKEN, INDEXNOW_KEY) via the random
provider instead of requiring the operator to hand-generate and paste each
into terraform.tfvars. Each is pinned with a static keepers value (secrets.tf)
so apply never regenerates a value already in use - the exact incident class
this guards against: every session invalidated, the app<->DB password
mismatched. Rotation is now a deliberate keepers edit, never a side effect.
postgres_password/auth_secret move from required inputs to optional (default
"") - explicit var wins when supplied (seeding an EXISTING live secret during
a migration onto Terraform, hop-1 cutover runbook Stage 0), else Terraform
generates and owns it. metrics_token/indexnow_key are new: neither existed in
Terraform before, both previously left for the app's own fallback generation.
VAPID deliberately stays a required, non-generated input - an EC keypair
where regeneration breaks every existing push subscription outright, unlike
an opaque token.
Sizing tiers: a locals.sizes t-shirt map (nano/small/medium/large ->
{workers, app_cpus, db_cpus, db_memory}), toward the target Proxmox
sizing-tier model (architecture doc SS6) ahead of actually provisioning VMs -
Proxmox itself stays deferred; today a tier just sizes container caps on the
existing SSH-managed hosts. A host can reference one by name (hosts.<name>.
size) or keep hand-picking the four fields, so existing tfvars are
unaffected; prod's example now uses size = "large" (identical numbers),
beta keeps explicit numbers, and a commented uat example demonstrates the
shortcut for a future ephemeral host.
Strengthened terraform/README.md's local-state caveat: more Terraform-
generated secrets landing in tfstate raises the stakes of the existing
never-commit-cleartext-state guidance, not just the sizing.
Verified: terraform validate + fmt clean. A real `terraform plan` against
fake hosts (prod/beta/uat, mixing size="large"/explicit-numbers/size="nano")
resolved every sizing correctly (prod 8/8/4/16g, beta 4/4/2/8g, uat
1/1/1/1g) and planned exactly one instance of each random_password/random_id
resource. Applied just those four resources (real generation, -target to
avoid touching the fake SSH-only host resources) and re-planned: "No
changes" - confirming the keepers pinning holds. Adding an explicit
postgres_password override afterward left the random_password resource
itself completely untouched (0 replace/destroy), confirming the override
path never disturbs the generated resource.
2026-07-21 01:08:57 +00:00
|
|
|
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
## WARNING — applying against live prod
|
|
|
|
|
|
|
|
|
|
`terraform apply` runs `remote-exec` **on the server**: it resets the checkout to the
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
branch, renders topology config + secrets, and runs `docker compose pull && up -d`
|
2026-07-22 23:27:13 +00:00
|
|
|
(pulling the pinned `backend_image_tag` / `frontend_image_tag` and recreating
|
|
|
|
|
containers — a brief app restart).
|
Decouple Terraform from the app repo; add a GCP host scaffold
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
2026-07-22 04:46:05 +00:00
|
|
|
Against the live production host this is a real deploy. Review the plan, apply in a
|
|
|
|
|
maintenance window, and prefer `-target` to touch one host at a time.
|
Add Terraform to provision the VPS hosts (compose keeps running the app) (#223)
Terraform config under terraform/ manages the two existing VPS hosts and hands the
app to docker-compose, with local state:
- prod: the new 48GB/12-core VPS (release branch, thermograph.org), sized larger.
- beta: the old VPS 75.119.132.91 (main branch, testing tier), no public domain.
- The LAN dev box stays on deploy/deploy-dev.sh (dev branch) — out of Terraform.
A reusable module (modules/thermograph-host) SSHes each host to install docker/
compose/ufw (+ Caddy when a domain is set), sync the checkout to the host's branch,
render /etc/thermograph.env from Terraform variables (secrets pushed via provisioner
content, never on local disk), `docker compose up -d`, and health-check. Named
volumes are preserved on re-apply, so the Postgres data is never recreated.
Container resources are now env-driven in docker-compose.yml (APP_CPUS/DB_CPUS/
DB_MEMORY/WORKERS) with unchanged defaults, so Terraform can size each host.
2026-07-20 07:42:15 +00:00
|
|
|
|
|
|
|
|
This is **separate from the Postgres data cutover** in `deploy/POSTGRES-MIGRATION.md`.
|
|
|
|
|
Terraform provisions the host and starts the stack; it does **not** migrate the
|
|
|
|
|
SQLite→Postgres accounts data. Sequence them deliberately: for a first cutover on a
|
|
|
|
|
host, follow the migration doc's freeze/backup/copy steps around the point where
|
|
|
|
|
Terraform brings the stack up — don't let Terraform recreate containers mid-migration.
|
|
|
|
|
|
|
|
|
|
## Assumptions / notes
|
|
|
|
|
|
|
|
|
|
- **beta has no public domain by default.** With `compose_files = ["docker-compose.yml"]`
|
|
|
|
|
the app binds `127.0.0.1:8137` (loopback), so opening the port in `ufw` alone does not
|
|
|
|
|
expose it. Reach beta via an SSH tunnel, or set `domain = "beta.thermograph.org"` (adds
|
|
|
|
|
Caddy TLS) — or add the `0.0.0.0`-publishing dev overlay to `compose_files` — to make
|
|
|
|
|
it reachable. `COOKIE_SECURE` is auto-set to `0` when there's no domain (a Secure
|
|
|
|
|
cookie is never sent over plain HTTP) and `1` behind Caddy TLS.
|
|
|
|
|
- The rendered Caddyfile only reverse-proxies the app. The repo's `deploy/Caddyfile`
|
|
|
|
|
additionally serves the `emigriffith.dev` portfolio and legacy redirects; those are
|
|
|
|
|
host-specific and not templated here.
|
|
|
|
|
- Provisioner-based by design: the hosts already exist, so this is not a
|
|
|
|
|
create-from-scratch cloud config. Re-applying is idempotent (installs are guarded,
|
|
|
|
|
`git reset --hard`, compose `up` reconciles).
|
