# Secrets — the git-native vault (SOPS + age)

The **single source of truth** for Thermograph's secrets is the set of
SOPS-encrypted YAML files in this directory. They're committed to the repo
**encrypted** (values only — keys stay readable so diffs are meaningful) and
**rendered into `/etc/thermograph.env` at deploy time** by
[`deploy/render-secrets.sh`](../render-secrets.sh), which the prod/beta
`deploy.sh` invokes.

Cycling a key is: **edit → commit → deploy**. No SSHing in to hand-edit a root file,
no per-host duplication.

## Files

| File | Holds | Encrypted? |
|------|-------|------------|
| `common.yaml` | The 16 values identical on every host — VAPID keypair, metrics token, IndexNow key, `REGISTRY_TOKEN`, S3 endpoint/bucket and both S3 keypairs, plus shared non-secret config | ✅ |
| `prod.yaml` | Prod's own — the three held-back credentials below, sizing (`APP_CPUS`, `DB_CPUS`, `DB_MEMORY`, `WORKERS`), `THERMOGRAPH_BASE_URL`, and the Discord + mail credentials that exist nowhere else | ✅ |
| `beta.yaml` | Beta's own — the three held-back credentials, sizing, `THERMOGRAPH_BASE_URL` | ✅ |
| `example.yaml` | Format reference / CI fixture (fake values) | ✅ |
| `../../.sops.yaml` | Which age recipient files are encrypted to (plaintext config) | — |

### Three credentials are deliberately NOT in `common.yaml`

`POSTGRES_PASSWORD`, `THERMOGRAPH_AUTH_SECRET` and `THERMOGRAPH_DATABASE_URL`
hold **identical values on prod and beta today**, so by the mechanical rule they
belong in `common.yaml`. They are kept per-host anyway.

These are the credentials that let one environment act as another: with them, a
foothold on beta is a foothold on prod's database and prod's session signing.
Beta is the *more* exposed box — it serves public Forgejo and Grafana. They match
only because beta was seeded from prod, not because the two are meant to be one
system.

Keeping them per-host costs nothing now (same values, one extra line each) and
buys the ability to diverge: rotating prod's database password stops implying
"and beta's too". Moving them into `common.yaml` would encode the equivalence as
intentional and turn breaking it into a migration rather than an edit.

If you ever *do* want them to differ, change one file. That is the whole point.

At deploy the renderer concatenates `common.yaml` then `<env>.yaml`, so a **host
value wins** (env_file / `source` take the last occurrence of a duplicate key).
`<env>` comes from `/etc/thermograph/secrets-env` on the box (`prod`/`beta`/`dev`).
`dev` has no real secrets and is intentionally not wired in — see `deploy-dev.sh`.

## The age key (the one thing that matters)

- The **private key is the single recovery root.** Lose it and every secret here is
  unrecoverable; leak it and every secret is compromised.
- It is **never in the repo.** It lives at:
  - `~/.config/sops/age/keys.txt` on the operator's machine (for editing), and
  - `/etc/thermograph/age.key` (`0400`) on each host that renders at deploy time.
- **Back it up** in the password manager. The public recipient is in `.sops.yaml`.

## Prerequisites (once per machine)

Install `sops` + `age` (single static binaries). On a host, use
[`deploy/provision-secrets.sh`](../provision-secrets.sh); on your laptop:

```sh
# see the pinned URLs in provision-secrets.sh, or:
go install github.com/getsops/sops/v3/cmd/sops@latest   # if you have Go
```

## Everyday: rotate a key

```sh
sops deploy/secrets/common.yaml        # opens DECRYPTED in $EDITOR; re-encrypts on save
git commit -am "rotate <thing>" && git push forgejo
# beta auto-deploys on push to main. prod (no CI) — one command:
ssh agent@169.58.46.181 'cd /opt/thermograph && git pull && deploy/deploy.sh'
```

A host-specific value (e.g. `POSTGRES_PASSWORD`) is the same, editing `prod.yaml` /
`beta.yaml` instead. **`POSTGRES_PASSWORD` is special**: the database only reads it on
a *fresh* volume, so also `ALTER ROLE thermograph PASSWORD '…'` inside the running
Postgres to match — the env change alone won't re-key an initialized DB.

## Add a new secret

1. `sops deploy/secrets/common.yaml`, add `THERMOGRAPH_NEW_THING: value`.
2. Add the app reader (`os.environ.get("THERMOGRAPH_NEW_THING")`).
3. Commit + deploy. It flows into `/etc/thermograph.env` automatically.

## Rotate the age identity itself (re-key)

```sh
age-keygen -o new.key                              # new identity
# add the new PUBLIC recipient alongside the old in .sops.yaml, then:
sops updatekeys deploy/secrets/*.yaml              # re-encrypt to BOTH keys
# distribute new.key to /etc/thermograph/age.key on every host + ~/.config/sops/age,
# deploy once so every host can decrypt with the new key, then remove the OLD
# recipient from .sops.yaml and:
sops updatekeys deploy/secrets/*.yaml              # drop the old key
git commit -am "rotate age identity" && push + deploy
```

## First-time bootstrap (seed from the live boxes)

Order matters — values must match the live env exactly so `AUTH_SECRET` / VAPID /
`POSTGRES_PASSWORD` don't rotate unintentionally:

1. Install `sops`+`age` and drop `/etc/thermograph/age.key` (0400) +
   `/etc/thermograph/secrets-env` on prod & beta (`provision-secrets.sh`).
2. Seed each host's file from its live env with the helper (pulls over SSH, encrypts,
   and verifies the render round-trips — run it on your own machine, it reads live
   secrets):
   ```sh
   deploy/secrets/seed-from-live.sh prod agent@169.58.46.181
   deploy/secrets/seed-from-live.sh beta agent@75.119.132.91
   ```
   That writes exact per-host copies (`prod.yaml`/`beta.yaml`); factor shared values
   into `common.yaml` later. Commit.
3. **Dry-run:** on the box, render to a temp file and `diff` against the live
   `/etc/thermograph.env` — must be byte-identical before cutover.
4. Merge the `deploy.sh` wiring; run one deploy; confirm the app is healthy.

## Safety

- CI (`.forgejo/workflows/secrets-guard.yml`) fails if any `*.yaml` here is committed
  unencrypted.
- Rendering is **presence-detected**: a host without the age key + marker keeps its
  existing `/etc/thermograph.env`, so nothing breaks before a host is migrated.
- Rendering happens **on the target box**, so CI/runners never see the age key or the
  plaintext.