0e3196fa05
Content-change pass following the extraction from the app monorepo (this repo
now stands alone, sourced via git filter-repo to preserve history):
- terraform/variables.tf, secrets.tf, modules/thermograph-host: remove every
app-secret Terraform variable (postgres_password, auth_secret, VAPID keys,
registry_token, Discord/SMTP creds, ...) and the random_password/random_id
generators. The SOPS+age vault (deploy/secrets/*.yaml) is now the sole
source of app secrets, rendered at deploy time by deploy/render-secrets.sh;
Terraform renders only a non-secret /etc/thermograph-topology.env (sizing,
routing) via the renamed thermograph-topology.env.tftpl template.
- hosts gains a required app_image_tag field: the host's own checkout is now
this infra repo, not the app repo, so there is no "current commit" to
derive an image tag from — every host pins one explicitly. repo_url now
points at this repo (private; typically needs an embedded read token).
- deploy.sh: IMAGE_TAG is now required from the environment instead of
derived via `git rev-parse HEAD` of the (now infra-repo) checkout, which
would have silently resolved to the wrong or a nonexistent tag.
- New terraform/modules/gcp-host: creates a GCE VM + minimal VPC/firewall
only, then feeds its IP into the same thermograph-host module every
SSH-managed host already uses — one provisioning path regardless of how a
host came to exist. var.gcp_hosts defaults to {}, so no google_* resource
is planned and the provider is never invoked without it (verified: plan
and validate succeed with no GCP credentials configured).
- terraform/README.md, ACCESS.md (renamed from INFRA.md), README.md: updated
for the new secrets model, the GCP scaffold, and this repo's own identity.
Verified: terraform fmt/validate/init clean; plan succeeds against realistic
dummy hosts (prod+beta shape) and against a populated gcp_hosts entry (plans
6 resources with no live credentials, confirming the composition wires
correctly end to end).
160 lines
8.9 KiB
Markdown
160 lines
8.9 KiB
Markdown
# Thermograph — Terraform (host provisioning)
|
|
|
|
Terraform that **provisions and configures VPS hosts** and hands the app off to
|
|
`docker compose`, which pulls the published app image (built + pushed by the app
|
|
repo's CI) and runs it — no app source is ever built or checked out on a host.
|
|
By default it does *not* create servers: `modules/thermograph-host` is
|
|
SSH-provisioner-driven against a host that already exists. An optional
|
|
`modules/gcp-host` can additionally *create* the VM on GCP first (see "GCP
|
|
scaffold" below) — scaffold-only today, no live resources until you opt in.
|
|
|
|
## What it manages
|
|
|
|
One reusable module (`modules/thermograph-host`) is instantiated per host via
|
|
`for_each` (`main.tf`'s `local.all_hosts`, merging `var.hosts` — SSH-managed,
|
|
already-existing boxes — with any `var.gcp_hosts` Terraform created itself).
|
|
This config manages **two VPS hosts** today:
|
|
|
|
| key | role | VPS | branch | domain | notes |
|
|
|--------|--------|-------------------------|-----------|-------------------|------------------------------------|
|
|
| `prod` | prod | NEW 48 GB / 12-core box (`169.58.46.181`) | `release` (of the APP repo — see `app_image_tag`) | `thermograph.org` | Caddy TLS; sized up (8/8/4/16g) |
|
|
| `beta` | beta | old box `75.119.132.91` | `main` (of the APP repo) | `beta.thermograph.org` | Caddy TLS; also hosts Forgejo |
|
|
|
|
Each host's own checkout on disk (`app_dir`, `git_branch`) is **this infra
|
|
repo**, not the app repo — the "branch" column above is which app-repo tag a
|
|
host is meant to track conceptually; the actual pinned version is
|
|
`var.hosts[*].app_image_tag` (e.g. `"sha-<12 hex>"`), since the host has no app
|
|
checkout to derive a tag from. The LAN dev server is **out of scope here** — it
|
|
builds from source via the app repo's `deploy/deploy-dev.sh` (a self-hosted
|
|
Forgejo Actions runner), not Terraform.
|
|
|
|
Per host, over SSH provisioners, Terraform:
|
|
|
|
- installs Docker + the compose plugin if missing;
|
|
- configures a `ufw` firewall (22/80/443 always; on a host with **no** domain it also
|
|
opens the app port `8137`);
|
|
- ensures the git checkout at `app_dir` exists (clones **this repo** on a fresh box)
|
|
and resets it to the host's `git_branch`;
|
|
- renders `/etc/thermograph-topology.env` from Terraform variables (non-secret sizing
|
|
only — `WORKERS`/`APP_CPUS`/`DB_CPUS`/`DB_MEMORY`/etc.), then runs
|
|
`deploy/render-secrets.sh` (from that same freshly-synced checkout) to render
|
|
`/etc/thermograph.env` from the SOPS+age vault — **Terraform itself never sees or
|
|
carries an app secret** (see "Secrets" below);
|
|
- for a host **with** a domain, installs a rendered Caddyfile and reloads Caddy;
|
|
- brings the stack up on the explicit `app_image_tag`: `docker login` to the registry,
|
|
`docker compose <-f each compose file> pull backend frontend`, then
|
|
`... up -d --remove-orphans`, running docker as root with both env files sourced;
|
|
- health-checks `http://127.0.0.1:8137/`.
|
|
|
|
A change to the rendered topology env, the compose files, the branch, the app image
|
|
tag, or the sizing flips the `null_resource` trigger and re-runs the provisioners on
|
|
next apply.
|
|
|
|
## GCP scaffold (no live resources)
|
|
|
|
`modules/gcp-host` can create a GCE VM (+ a dedicated VPC, subnet, and a firewall
|
|
opening 22/80/443) instead of assuming the host already exists — see `main.tf`'s
|
|
`module.gcp_vm`. It contributes *nothing* to provisioning: its only output
|
|
(`external_ip`) feeds into the exact same `thermograph-host` module every
|
|
SSH-managed host uses, so there's one provisioning path regardless of how a
|
|
host came to exist. `var.gcp_hosts` defaults to `{}`, so by default no
|
|
`google_*` resource is ever planned and the `google` provider is never
|
|
invoked — `terraform plan`/`validate` succeed with no GCP credentials
|
|
configured at all. To actually use it: populate an entry in
|
|
`terraform.tfvars` (see the commented example in
|
|
`terraform.tfvars.example`) and authenticate via `gcloud auth
|
|
application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`. This is the
|
|
same create-then-provision composition a future Proxmox module (the
|
|
architecture doc's longer-term target — see §6/§9 there) would use.
|
|
|
|
### Container sizing is env-driven
|
|
|
|
`docker-compose.yml` reads `WORKERS`, `APP_CPUS`, `DB_CPUS`, and `DB_MEMORY` from the
|
|
environment (defaults `4 / 4 / 2 / 8g`, identical to before). Terraform sets them per
|
|
host through `/etc/thermograph-topology.env`, so the big prod box can run larger caps
|
|
without a compose edit. The Postgres *internal* memory budget (`shared_buffers`,
|
|
`effective_cache_size`, `work_mem`, `maintenance_work_mem`) is derived from the same
|
|
`DB_MEMORY` by `deploy/db/init/20-tuning.sh` — so raising `db_memory` scales the
|
|
container cap and the tuning together (prod 16g → shared_buffers 4 GB). The tuning
|
|
applies on a fresh DB volume; on an existing volume re-run it by hand (see the script
|
|
header).
|
|
|
|
## Prerequisites
|
|
|
|
- Terraform >= 1.6 (v1.15 is installed).
|
|
- SSH key access to **both** hosts as a **sudo-capable** user (the live boxes use the
|
|
dedicated `agent` account, `~/.ssh/thermograph_agent_ed25519` — see
|
|
`deploy/provision-agent-access.sh`). Point `ssh_private_key_path` at that key (`~`
|
|
is expanded).
|
|
- The hosts are Debian/Ubuntu with `apt` and outbound internet (Docker/Caddy installs
|
|
pull from the network). Docker may already be present — installs are conditional.
|
|
- For the `prod` host: DNS for `thermograph.org` must point at the new box before apply,
|
|
or Caddy's first-request cert issuance will fail.
|
|
|
|
## Use
|
|
|
|
```sh
|
|
cd terraform
|
|
cp terraform.tfvars.example terraform.tfvars # then edit: real IPs + secrets
|
|
terraform init
|
|
terraform plan
|
|
terraform apply
|
|
```
|
|
|
|
Target one host with `-target='module.host["beta"]'` if you want to apply to just one.
|
|
|
|
## Secrets
|
|
|
|
App secrets (`POSTGRES_PASSWORD`, `THERMOGRAPH_AUTH_SECRET`, VAPID keys,
|
|
`REGISTRY_TOKEN`, Discord/SMTP credentials, …) are **not** Terraform variables
|
|
— they live in the git-native SOPS+age vault at `../deploy/secrets/*.yaml`
|
|
(committed, encrypted) and are rendered into `/etc/thermograph.env` at deploy
|
|
time by `../deploy/render-secrets.sh`, which the provisioner's deploy step
|
|
runs from the freshly-synced checkout. Terraform only renders the non-secret
|
|
`/etc/thermograph-topology.env` (sizing/routing). See
|
|
`../deploy/secrets/README.md` to rotate a secret or add a new one — it's a
|
|
`sops edit` + commit + deploy, no Terraform apply involved.
|
|
|
|
`om_rclone_conf` (object-storage bucket credentials for the self-hosted ERA5
|
|
archive) is the one exception still supplied via Terraform, in
|
|
`terraform.tfvars` — folding it into the vault too is a reasonable future
|
|
step, not done here.
|
|
|
|
## Local state
|
|
|
|
The backend is **local**: `terraform.tfstate` is written next to the config.
|
|
It's gitignored (`terraform/.gitignore` and the root `.gitignore`) — keep it
|
|
off shared disks and back it up somewhere private. `terraform.tfvars` is
|
|
likewise gitignored (it may carry a `repo_url` credential and always carries
|
|
`om_rclone_conf`); only `terraform.tfvars.example` (dummy values) is
|
|
committed. `.terraform.lock.hcl` **is** committed so provider versions are
|
|
pinned across machines.
|
|
|
|
## WARNING — applying against live prod
|
|
|
|
`terraform apply` runs `remote-exec` **on the server**: it resets the checkout to the
|
|
branch, renders topology config + secrets, and runs `docker compose pull && up -d`
|
|
(pulling the pinned `app_image_tag` and recreating containers — a brief app restart).
|
|
Against the live production host this is a real deploy. Review the plan, apply in a
|
|
maintenance window, and prefer `-target` to touch one host at a time.
|
|
|
|
This is **separate from the Postgres data cutover** in `deploy/POSTGRES-MIGRATION.md`.
|
|
Terraform provisions the host and starts the stack; it does **not** migrate the
|
|
SQLite→Postgres accounts data. Sequence them deliberately: for a first cutover on a
|
|
host, follow the migration doc's freeze/backup/copy steps around the point where
|
|
Terraform brings the stack up — don't let Terraform recreate containers mid-migration.
|
|
|
|
## Assumptions / notes
|
|
|
|
- **beta has no public domain by default.** With `compose_files = ["docker-compose.yml"]`
|
|
the app binds `127.0.0.1:8137` (loopback), so opening the port in `ufw` alone does not
|
|
expose it. Reach beta via an SSH tunnel, or set `domain = "beta.thermograph.org"` (adds
|
|
Caddy TLS) — or add the `0.0.0.0`-publishing dev overlay to `compose_files` — to make
|
|
it reachable. `COOKIE_SECURE` is auto-set to `0` when there's no domain (a Secure
|
|
cookie is never sent over plain HTTP) and `1` behind Caddy TLS.
|
|
- The rendered Caddyfile only reverse-proxies the app. The repo's `deploy/Caddyfile`
|
|
additionally serves the `emigriffith.dev` portfolio and legacy redirects; those are
|
|
host-specific and not templated here.
|
|
- Provisioner-based by design: the hosts already exist, so this is not a
|
|
create-from-scratch cloud config. Re-applying is idempotent (installs are guarded,
|
|
`git reset --hard`, compose `up` reconciles).
|