openbao: add a dormant OpenBao backend alongside SOPS
Phase 1 of moving the secret store to OpenBao. Nothing is cut over:
TG_SECRETS_BACKEND defaults to sops for all three environments, so SOPS
remains authoritative until an environment is flipped in a reviewed PR.
The reason for the migration is one specific failure class. infra/CLAUDE.md
requires that vps1 never hold prod credentials, but that rule is currently
enforced by a caller-side shell flag (THERMOGRAPH_SECRETS_SKIP_COMMON) at
three call sites. A fourth call site, or one by-hand render, writes both S3
keypairs, the VAPID private key, REGISTRY_TOKEN and the IndexNow/metrics
tokens onto the Forgejo CI-runner box and exits 0. policies/tg-host-dev.hcl
makes that a 403 instead: dev's identity cannot read the common path at all.
Shape:
- vps2 only, native systemd (not a container: a Swarm service is circular,
and a plain docker run is one `prune -a` from gone), raft storage (2.6.0
deprecated the file backend), static auto-unseal so a reboot cannot block
deploys, mesh-only self-signed TLS (not ACME — that would put DNS in the
boot chain of the secret store).
- AppRole per environment, CIDR-bound, 5-minute tokens. Each environment's
secret-id is owned by its own deploy user, which is a boundary that does
not exist today since prod and beta both reach the same root-owned age key.
- Render still produces the same raw unquoted dotenv: Centralis compares
/etc/thermograph.env textually and the Swarm entrypoint parses it by line.
The backend dispatch in render-secrets.sh returns early rather than
refactoring the shared write path, so the SOPS path stays byte-identical.
Verified by rendering dev (12 keys) and prod (32 keys) through it, matching
the live hosts. The duplicated write path is noted for consolidation once
prod has been on OpenBao for a release cycle.
The OpenBao renderer rejects values containing a newline or a shell
metacharacter. /etc/thermograph.env is sourced by bash in six places, so
such a value is a command-execution path on the deploy host; the SOPS path
has always had this hazard and avoids it only because every current value
happens to be alphanumeric.
Also records that the age keypair is NOT retired by this migration: it is
the backup encryption key for every off-box dump (ops-cron.yml:142,197,
30-day retention), so deleting it with the vault files would silently
destroy a month of database recoverability.
verify-parity.sh is the cutover gate — it renders both backends and diffs
them, reporting key names and counts only, never values.
2026-07-30 04:35:20 +00:00
|
|
|
|
// OpenBao server config for the Thermograph estate. Lives at /etc/openbao/config.hcl
|
|
|
|
|
|
// on vps2. Verified against OpenBao v2.6.1 (2026-07-22).
|
|
|
|
|
|
//
|
|
|
|
|
|
// ============================================================================
|
|
|
|
|
|
// WHY THIS RUNS AS A NATIVE SYSTEMD SERVICE AND NOT A CONTAINER
|
|
|
|
|
|
// ============================================================================
|
|
|
|
|
|
// The estate is Docker-native and this is the one thing that must not be. The rule
|
|
|
|
|
|
// is that OpenBao must not depend on anything it secures.
|
|
|
|
|
|
//
|
|
|
|
|
|
// * A Swarm service is fatal: deploy-stack.sh renders secrets in order to deploy
|
|
|
|
|
|
// the stack, so the stack cannot contain the thing that holds them.
|
|
|
|
|
|
// * A standalone `docker run --restart=always` is merely bad: it makes the vault's
|
|
|
|
|
|
// availability a function of the same Docker daemon that deploys restart, and
|
|
|
|
|
|
// leaves it one careless `docker system prune -a` from gone. The root CLAUDE.md
|
|
|
|
|
|
// already lists `prune -a` as a thing that eats the rollback image; it would
|
|
|
|
|
|
// eat this too.
|
|
|
|
|
|
//
|
|
|
|
|
|
// A native binary with Restart=always has exactly one dependency: the local disk.
|
|
|
|
|
|
// See infra/openbao/openbao.service.
|
|
|
|
|
|
//
|
|
|
|
|
|
// Corollary that is easy to get wrong: the listener cert must NOT come from the
|
|
|
|
|
|
// Caddy/ACME path. That would put DNS and the public internet in the boot chain of
|
|
|
|
|
|
// the estate's secret store. It is self-signed, long-lived, and on disk.
|
|
|
|
|
|
// ============================================================================
|
|
|
|
|
|
|
|
|
|
|
|
ui = false
|
|
|
|
|
|
cluster_name = "thermograph"
|
|
|
|
|
|
log_level = "info"
|
openbao: fix bootstrap so it completes on 2.6.x
bootstrap.sh could not run to completion on OpenBao 2.6.1:
- Release asset names were wrong. `bao_<ver>_linux_amd64.tar.gz` and
`bao_<ver>_SHA256SUMS` both 404; the assets are `openbao_<ver>_...` and
`checksums.txt`. The tarball is now saved under its real name and the checksum
grep anchored to end-of-line, because checksums.txt also lists a .sbom.json and
`sha256sum -c` resolves each line by the filename inside it.
- `openssl rand -base64 32` appends a newline and the static seal reads the key
file raw, so the service failed with `Error configuring seal "static": unknown
encoding for AES-256 key`.
- The audit stanza relied on the block label being the device type. 2.6.1 requires
explicit `type` and `path` with device settings under `options`. Worth noting the
intermediate state: with type and path but a bare `file_path`, the server starts
and silently ignores the log location, which is the worse failure given a wedged
audit device stops OpenBao answering at all.
- `disable_mlock` is unsupported in 2.6.1 and warned on every start.
- Nothing opened port 8200 and ufw defaults to deny(incoming), so vps1 could not
reach the vault. dev renders on vps1, so this would have surfaced as a timeout
during cutover rather than here.
bootstrap-policies.sh installed beta's credentials as `deploy:deploy`, but vps2 has
no `deploy` user and both environments deploy as `agent` (env-topology.sh sets
TG_SSH_TARGET=agent@ for both; deploy.yml uses one VPS2_SSH_USER). install_creds
also printed a success line after a failed chown: the call site wrapped it in
`|| echo`, which suppresses `set -e` for everything inside the function, so it fell
through to chmod and reported an ownership it never applied. It now removes the
half-written file and returns non-zero.
Corrects the isolation rationale accordingly — separate credentials buy audit
attribution and a policy boundary against mistakes, not deploy-user isolation.
Documents the 2.6.0 root-token change (HCSEC-2026-08): `generate-root` now targets
an authenticated endpoint, so recovery keys cannot mint a replacement token and
revoking the last root token before a second admin identity exists is a one-way
door. Also corrects the runbook's `verify-parity.sh --all`, which cannot work from
a single host given the AppRole CIDR bindings.
2026-07-31 02:47:22 +00:00
|
|
|
|
// `disable_mlock` is deliberately absent: OpenBao 2.6.1 does not recognise it and
|
|
|
|
|
|
// logs `unknown or unsupported field disable_mlock` on every start. Carrying a
|
|
|
|
|
|
// setting the server ignores only trains the operator to skim startup warnings,
|
|
|
|
|
|
// which is where a real one will eventually hide. The corresponding note about
|
|
|
|
|
|
// AmbientCapabilities=CAP_IPC_LOCK in openbao.service is moot for the same reason.
|
openbao: add a dormant OpenBao backend alongside SOPS
Phase 1 of moving the secret store to OpenBao. Nothing is cut over:
TG_SECRETS_BACKEND defaults to sops for all three environments, so SOPS
remains authoritative until an environment is flipped in a reviewed PR.
The reason for the migration is one specific failure class. infra/CLAUDE.md
requires that vps1 never hold prod credentials, but that rule is currently
enforced by a caller-side shell flag (THERMOGRAPH_SECRETS_SKIP_COMMON) at
three call sites. A fourth call site, or one by-hand render, writes both S3
keypairs, the VAPID private key, REGISTRY_TOKEN and the IndexNow/metrics
tokens onto the Forgejo CI-runner box and exits 0. policies/tg-host-dev.hcl
makes that a 403 instead: dev's identity cannot read the common path at all.
Shape:
- vps2 only, native systemd (not a container: a Swarm service is circular,
and a plain docker run is one `prune -a` from gone), raft storage (2.6.0
deprecated the file backend), static auto-unseal so a reboot cannot block
deploys, mesh-only self-signed TLS (not ACME — that would put DNS in the
boot chain of the secret store).
- AppRole per environment, CIDR-bound, 5-minute tokens. Each environment's
secret-id is owned by its own deploy user, which is a boundary that does
not exist today since prod and beta both reach the same root-owned age key.
- Render still produces the same raw unquoted dotenv: Centralis compares
/etc/thermograph.env textually and the Swarm entrypoint parses it by line.
The backend dispatch in render-secrets.sh returns early rather than
refactoring the shared write path, so the SOPS path stays byte-identical.
Verified by rendering dev (12 keys) and prod (32 keys) through it, matching
the live hosts. The duplicated write path is noted for consolidation once
prod has been on OpenBao for a release cycle.
The OpenBao renderer rejects values containing a newline or a shell
metacharacter. /etc/thermograph.env is sourced by bash in six places, so
such a value is a command-execution path on the deploy host; the SOPS path
has always had this hazard and avoids it only because every current value
happens to be alphanumeric.
Also records that the age keypair is NOT retired by this migration: it is
the backup encryption key for every off-box dump (ops-cron.yml:142,197,
30-day retention), so deleting it with the vault files would silently
destroy a month of database recoverability.
verify-parity.sh is the cutover gate — it renders both backends and diffs
them, reporting key names and counts only, never values.
2026-07-30 04:35:20 +00:00
|
|
|
|
|
|
|
|
|
|
// STORAGE: raft, and this is now forced rather than chosen. OpenBao 2.6.0
|
|
|
|
|
|
// DEPRECATED the `file` backend for removal in v2.7.0, so picking `file` today buys
|
|
|
|
|
|
// a migration within a release. Raft also has the only first-class backup primitive
|
|
|
|
|
|
// (`bao operator raft snapshot save`), which is the disaster-recovery story.
|
|
|
|
|
|
//
|
|
|
|
|
|
// A single-node raft is the honest shape for a one-operator estate. Note that a
|
|
|
|
|
|
// TWO-node raft would be strictly worse than one: quorum of two means losing either
|
|
|
|
|
|
// node loses write availability. Three voters would be needed, and the desktop
|
|
|
|
|
|
// (10.10.0.3) is intermittent, so it cannot be a reliable third.
|
|
|
|
|
|
storage "raft" {
|
|
|
|
|
|
path = "/var/lib/openbao"
|
|
|
|
|
|
node_id = "vps2"
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Mesh-only. 10.10.0.1 is vps2's WireGuard address. There is deliberately no
|
|
|
|
|
|
// 0.0.0.0 listener and no public DNS record — vps2 is a public VPS, and reaching
|
|
|
|
|
|
// OpenBao should require being on the mesh, exactly like the Swarm control ports
|
|
|
|
|
|
// and the Loki endpoint.
|
|
|
|
|
|
listener "tcp" {
|
|
|
|
|
|
address = "10.10.0.1:8200"
|
|
|
|
|
|
tls_cert_file = "/etc/openbao/tls/bao.crt"
|
|
|
|
|
|
tls_key_file = "/etc/openbao/tls/bao.key"
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// Loopback, so a by-hand `bao` on the box works without the mesh address.
|
|
|
|
|
|
listener "tcp" {
|
|
|
|
|
|
address = "127.0.0.1:8200"
|
|
|
|
|
|
tls_cert_file = "/etc/openbao/tls/bao.crt"
|
|
|
|
|
|
tls_key_file = "/etc/openbao/tls/bao.key"
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
api_addr = "https://10.10.0.1:8200"
|
|
|
|
|
|
cluster_addr = "https://10.10.0.1:8201"
|
|
|
|
|
|
|
|
|
|
|
|
// ============================================================================
|
|
|
|
|
|
// SEAL: static auto-unseal. This is the crux of the whole design.
|
|
|
|
|
|
// ============================================================================
|
|
|
|
|
|
// The `static` seal (OpenBao v2.4.0+) auto-unseals from a local key with no cloud
|
|
|
|
|
|
// KMS and no HSM. OpenBao's own docs hedge it — "carefully evaluate use of Static
|
|
|
|
|
|
// Key Auto Unseal" — but that caveat is written for multi-tenant enterprises.
|
|
|
|
|
|
//
|
|
|
|
|
|
// The decisive argument here is that static seal is NOT A DOWNGRADE FROM THE STATUS
|
|
|
|
|
|
// QUO. Today /etc/thermograph/age.key is one file, 0400, root-owned, on each
|
|
|
|
|
|
// rendering host, which decrypts every secret in the estate forever with no audit
|
|
|
|
|
|
// trail. A static seal key at 0400 on vps2 is the IDENTICAL trust model: one file on
|
|
|
|
|
|
// one box whose compromise is total compromise. What changes is everything layered
|
|
|
|
|
|
// above it — per-consumer ACLs, an audit log, five-minute tokens, versioned rollback.
|
|
|
|
|
|
//
|
|
|
|
|
|
// WHY NOT SHAMIR: `bao operator init -key-shares=5 -key-threshold=3` means the vault
|
|
|
|
|
|
// is sealed after every process restart — reboot, package upgrade, OOM kill — and a
|
|
|
|
|
|
// human must type three shares. render-secrets.sh is on the deploy path for all three
|
|
|
|
|
|
// environments, so a sealed vault blocks every deploy and every hotfix. With one
|
|
|
|
|
|
// operator the N-of-M threshold is theatre: one person holds all the shares, so it
|
|
|
|
|
|
// buys nothing against the real threat while costing an availability property the
|
|
|
|
|
|
// estate currently has for free. Today a Contabo reboot needs no human at all.
|
|
|
|
|
|
// Introducing a human-in-the-loop requirement that does not exist today is a
|
|
|
|
|
|
// regression this estate cannot absorb.
|
|
|
|
|
|
//
|
|
|
|
|
|
// THE ONE NEW SINGLE POINT OF FAILURE this migration introduces: lose this key and
|
|
|
|
|
|
// the raft snapshots become unrestorable. It MUST exist in at least two places, one
|
|
|
|
|
|
// of them not on vps2. That is a hard operator obligation, not a suggestion.
|
|
|
|
|
|
//
|
|
|
|
|
|
// Rotation is supported n-1 via previous_key / previous_key_id.
|
|
|
|
|
|
seal "static" {
|
|
|
|
|
|
current_key_id = "20260801-1"
|
|
|
|
|
|
current_key = "file:///etc/openbao/unseal.key"
|
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
// ============================================================================
|
|
|
|
|
|
// AUDIT: two devices, deliberately.
|
|
|
|
|
|
// ============================================================================
|
|
|
|
|
|
// ⚠️ OpenBao STOPS ANSWERING REQUESTS ENTIRELY when no enabled audit device can
|
|
|
|
|
|
// record them. That is correct behaviour for an audit log and a catastrophic way to
|
|
|
|
|
|
// discover a full disk: a full /var on vps2 would block every deploy on the estate.
|
|
|
|
|
|
// Two independent devices mean one wedging is survivable.
|
|
|
|
|
|
//
|
|
|
|
|
|
// Values are HMAC-SHA256'd in the log, so it is safe to ship to Loki through the
|
|
|
|
|
|
// existing Alloy pipeline — which finally makes "who read which secret, when" a
|
|
|
|
|
|
// queryable question. Today reading age.key leaves no trace whatsoever.
|
|
|
|
|
|
//
|
|
|
|
|
|
// logrotate on this path MUST signal with SIGHUP, or bao keeps writing to an
|
|
|
|
|
|
// unlinked inode and the log silently stops growing.
|
openbao: fix bootstrap so it completes on 2.6.x
bootstrap.sh could not run to completion on OpenBao 2.6.1:
- Release asset names were wrong. `bao_<ver>_linux_amd64.tar.gz` and
`bao_<ver>_SHA256SUMS` both 404; the assets are `openbao_<ver>_...` and
`checksums.txt`. The tarball is now saved under its real name and the checksum
grep anchored to end-of-line, because checksums.txt also lists a .sbom.json and
`sha256sum -c` resolves each line by the filename inside it.
- `openssl rand -base64 32` appends a newline and the static seal reads the key
file raw, so the service failed with `Error configuring seal "static": unknown
encoding for AES-256 key`.
- The audit stanza relied on the block label being the device type. 2.6.1 requires
explicit `type` and `path` with device settings under `options`. Worth noting the
intermediate state: with type and path but a bare `file_path`, the server starts
and silently ignores the log location, which is the worse failure given a wedged
audit device stops OpenBao answering at all.
- `disable_mlock` is unsupported in 2.6.1 and warned on every start.
- Nothing opened port 8200 and ufw defaults to deny(incoming), so vps1 could not
reach the vault. dev renders on vps1, so this would have surfaced as a timeout
during cutover rather than here.
bootstrap-policies.sh installed beta's credentials as `deploy:deploy`, but vps2 has
no `deploy` user and both environments deploy as `agent` (env-topology.sh sets
TG_SSH_TARGET=agent@ for both; deploy.yml uses one VPS2_SSH_USER). install_creds
also printed a success line after a failed chown: the call site wrapped it in
`|| echo`, which suppresses `set -e` for everything inside the function, so it fell
through to chmod and reported an ownership it never applied. It now removes the
half-written file and returns non-zero.
Corrects the isolation rationale accordingly — separate credentials buy audit
attribution and a policy boundary against mistakes, not deploy-user isolation.
Documents the 2.6.0 root-token change (HCSEC-2026-08): `generate-root` now targets
an authenticated endpoint, so recovery keys cannot mint a replacement token and
revoking the last root token before a second admin identity exists is a one-way
door. Also corrects the runbook's `verify-parity.sh --all`, which cannot work from
a single host given the AppRole CIDR bindings.
2026-07-31 02:47:22 +00:00
|
|
|
|
// OpenBao 2.6.1 requires `type` and `path` explicitly — the block label is the
|
|
|
|
|
|
// device NAME, not its type — and device settings go in `options`. Written as
|
|
|
|
|
|
// `audit "file" { file_path = ... }` the server refuses to start ("audit type must
|
|
|
|
|
|
// be specified"); with type+path but a bare file_path it starts but silently
|
|
|
|
|
|
// IGNORES the path and mode, which is the worse failure of the two.
|
openbao: add a dormant OpenBao backend alongside SOPS
Phase 1 of moving the secret store to OpenBao. Nothing is cut over:
TG_SECRETS_BACKEND defaults to sops for all three environments, so SOPS
remains authoritative until an environment is flipped in a reviewed PR.
The reason for the migration is one specific failure class. infra/CLAUDE.md
requires that vps1 never hold prod credentials, but that rule is currently
enforced by a caller-side shell flag (THERMOGRAPH_SECRETS_SKIP_COMMON) at
three call sites. A fourth call site, or one by-hand render, writes both S3
keypairs, the VAPID private key, REGISTRY_TOKEN and the IndexNow/metrics
tokens onto the Forgejo CI-runner box and exits 0. policies/tg-host-dev.hcl
makes that a 403 instead: dev's identity cannot read the common path at all.
Shape:
- vps2 only, native systemd (not a container: a Swarm service is circular,
and a plain docker run is one `prune -a` from gone), raft storage (2.6.0
deprecated the file backend), static auto-unseal so a reboot cannot block
deploys, mesh-only self-signed TLS (not ACME — that would put DNS in the
boot chain of the secret store).
- AppRole per environment, CIDR-bound, 5-minute tokens. Each environment's
secret-id is owned by its own deploy user, which is a boundary that does
not exist today since prod and beta both reach the same root-owned age key.
- Render still produces the same raw unquoted dotenv: Centralis compares
/etc/thermograph.env textually and the Swarm entrypoint parses it by line.
The backend dispatch in render-secrets.sh returns early rather than
refactoring the shared write path, so the SOPS path stays byte-identical.
Verified by rendering dev (12 keys) and prod (32 keys) through it, matching
the live hosts. The duplicated write path is noted for consolidation once
prod has been on OpenBao for a release cycle.
The OpenBao renderer rejects values containing a newline or a shell
metacharacter. /etc/thermograph.env is sourced by bash in six places, so
such a value is a command-execution path on the deploy host; the SOPS path
has always had this hazard and avoids it only because every current value
happens to be alphanumeric.
Also records that the age keypair is NOT retired by this migration: it is
the backup encryption key for every off-box dump (ops-cron.yml:142,197,
30-day retention), so deleting it with the vault files would silently
destroy a month of database recoverability.
verify-parity.sh is the cutover gate — it renders both backends and diffs
them, reporting key names and counts only, never values.
2026-07-30 04:35:20 +00:00
|
|
|
|
audit "file" {
|
openbao: fix bootstrap so it completes on 2.6.x
bootstrap.sh could not run to completion on OpenBao 2.6.1:
- Release asset names were wrong. `bao_<ver>_linux_amd64.tar.gz` and
`bao_<ver>_SHA256SUMS` both 404; the assets are `openbao_<ver>_...` and
`checksums.txt`. The tarball is now saved under its real name and the checksum
grep anchored to end-of-line, because checksums.txt also lists a .sbom.json and
`sha256sum -c` resolves each line by the filename inside it.
- `openssl rand -base64 32` appends a newline and the static seal reads the key
file raw, so the service failed with `Error configuring seal "static": unknown
encoding for AES-256 key`.
- The audit stanza relied on the block label being the device type. 2.6.1 requires
explicit `type` and `path` with device settings under `options`. Worth noting the
intermediate state: with type and path but a bare `file_path`, the server starts
and silently ignores the log location, which is the worse failure given a wedged
audit device stops OpenBao answering at all.
- `disable_mlock` is unsupported in 2.6.1 and warned on every start.
- Nothing opened port 8200 and ufw defaults to deny(incoming), so vps1 could not
reach the vault. dev renders on vps1, so this would have surfaced as a timeout
during cutover rather than here.
bootstrap-policies.sh installed beta's credentials as `deploy:deploy`, but vps2 has
no `deploy` user and both environments deploy as `agent` (env-topology.sh sets
TG_SSH_TARGET=agent@ for both; deploy.yml uses one VPS2_SSH_USER). install_creds
also printed a success line after a failed chown: the call site wrapped it in
`|| echo`, which suppresses `set -e` for everything inside the function, so it fell
through to chmod and reported an ownership it never applied. It now removes the
half-written file and returns non-zero.
Corrects the isolation rationale accordingly — separate credentials buy audit
attribution and a policy boundary against mistakes, not deploy-user isolation.
Documents the 2.6.0 root-token change (HCSEC-2026-08): `generate-root` now targets
an authenticated endpoint, so recovery keys cannot mint a replacement token and
revoking the last root token before a second admin identity exists is a one-way
door. Also corrects the runbook's `verify-parity.sh --all`, which cannot work from
a single host given the AppRole CIDR bindings.
2026-07-31 02:47:22 +00:00
|
|
|
|
type = "file"
|
|
|
|
|
|
path = "file/"
|
|
|
|
|
|
options = {
|
|
|
|
|
|
file_path = "/var/log/openbao/audit.log"
|
|
|
|
|
|
mode = "0600"
|
|
|
|
|
|
}
|
openbao: add a dormant OpenBao backend alongside SOPS
Phase 1 of moving the secret store to OpenBao. Nothing is cut over:
TG_SECRETS_BACKEND defaults to sops for all three environments, so SOPS
remains authoritative until an environment is flipped in a reviewed PR.
The reason for the migration is one specific failure class. infra/CLAUDE.md
requires that vps1 never hold prod credentials, but that rule is currently
enforced by a caller-side shell flag (THERMOGRAPH_SECRETS_SKIP_COMMON) at
three call sites. A fourth call site, or one by-hand render, writes both S3
keypairs, the VAPID private key, REGISTRY_TOKEN and the IndexNow/metrics
tokens onto the Forgejo CI-runner box and exits 0. policies/tg-host-dev.hcl
makes that a 403 instead: dev's identity cannot read the common path at all.
Shape:
- vps2 only, native systemd (not a container: a Swarm service is circular,
and a plain docker run is one `prune -a` from gone), raft storage (2.6.0
deprecated the file backend), static auto-unseal so a reboot cannot block
deploys, mesh-only self-signed TLS (not ACME — that would put DNS in the
boot chain of the secret store).
- AppRole per environment, CIDR-bound, 5-minute tokens. Each environment's
secret-id is owned by its own deploy user, which is a boundary that does
not exist today since prod and beta both reach the same root-owned age key.
- Render still produces the same raw unquoted dotenv: Centralis compares
/etc/thermograph.env textually and the Swarm entrypoint parses it by line.
The backend dispatch in render-secrets.sh returns early rather than
refactoring the shared write path, so the SOPS path stays byte-identical.
Verified by rendering dev (12 keys) and prod (32 keys) through it, matching
the live hosts. The duplicated write path is noted for consolidation once
prod has been on OpenBao for a release cycle.
The OpenBao renderer rejects values containing a newline or a shell
metacharacter. /etc/thermograph.env is sourced by bash in six places, so
such a value is a command-execution path on the deploy host; the SOPS path
has always had this hazard and avoids it only because every current value
happens to be alphanumeric.
Also records that the age keypair is NOT retired by this migration: it is
the backup encryption key for every off-box dump (ops-cron.yml:142,197,
30-day retention), so deleting it with the vault files would silently
destroy a month of database recoverability.
verify-parity.sh is the cutover gate — it renders both backends and diffs
them, reporting key names and counts only, never values.
2026-07-30 04:35:20 +00:00
|
|
|
|
}
|
|
|
|
|
|
|
openbao: fix bootstrap so it completes on 2.6.x
bootstrap.sh could not run to completion on OpenBao 2.6.1:
- Release asset names were wrong. `bao_<ver>_linux_amd64.tar.gz` and
`bao_<ver>_SHA256SUMS` both 404; the assets are `openbao_<ver>_...` and
`checksums.txt`. The tarball is now saved under its real name and the checksum
grep anchored to end-of-line, because checksums.txt also lists a .sbom.json and
`sha256sum -c` resolves each line by the filename inside it.
- `openssl rand -base64 32` appends a newline and the static seal reads the key
file raw, so the service failed with `Error configuring seal "static": unknown
encoding for AES-256 key`.
- The audit stanza relied on the block label being the device type. 2.6.1 requires
explicit `type` and `path` with device settings under `options`. Worth noting the
intermediate state: with type and path but a bare `file_path`, the server starts
and silently ignores the log location, which is the worse failure given a wedged
audit device stops OpenBao answering at all.
- `disable_mlock` is unsupported in 2.6.1 and warned on every start.
- Nothing opened port 8200 and ufw defaults to deny(incoming), so vps1 could not
reach the vault. dev renders on vps1, so this would have surfaced as a timeout
during cutover rather than here.
bootstrap-policies.sh installed beta's credentials as `deploy:deploy`, but vps2 has
no `deploy` user and both environments deploy as `agent` (env-topology.sh sets
TG_SSH_TARGET=agent@ for both; deploy.yml uses one VPS2_SSH_USER). install_creds
also printed a success line after a failed chown: the call site wrapped it in
`|| echo`, which suppresses `set -e` for everything inside the function, so it fell
through to chmod and reported an ownership it never applied. It now removes the
half-written file and returns non-zero.
Corrects the isolation rationale accordingly — separate credentials buy audit
attribution and a policy boundary against mistakes, not deploy-user isolation.
Documents the 2.6.0 root-token change (HCSEC-2026-08): `generate-root` now targets
an authenticated endpoint, so recovery keys cannot mint a replacement token and
revoking the last root token before a second admin identity exists is a one-way
door. Also corrects the runbook's `verify-parity.sh --all`, which cannot work from
a single host given the AppRole CIDR bindings.
2026-07-31 02:47:22 +00:00
|
|
|
|
audit "syslog" {
|
|
|
|
|
|
type = "syslog"
|
|
|
|
|
|
path = "syslog/"
|
|
|
|
|
|
}
|