thermograph/infra/openbao/openbao.service
Emi Griffith 6bd07d33cb openbao: add a dormant OpenBao backend alongside SOPS
Phase 1 of moving the secret store to OpenBao. Nothing is cut over:
TG_SECRETS_BACKEND defaults to sops for all three environments, so SOPS
remains authoritative until an environment is flipped in a reviewed PR.

The reason for the migration is one specific failure class. infra/CLAUDE.md
requires that vps1 never hold prod credentials, but that rule is currently
enforced by a caller-side shell flag (THERMOGRAPH_SECRETS_SKIP_COMMON) at
three call sites. A fourth call site, or one by-hand render, writes both S3
keypairs, the VAPID private key, REGISTRY_TOKEN and the IndexNow/metrics
tokens onto the Forgejo CI-runner box and exits 0. policies/tg-host-dev.hcl
makes that a 403 instead: dev's identity cannot read the common path at all.

Shape:
- vps2 only, native systemd (not a container: a Swarm service is circular,
  and a plain docker run is one `prune -a` from gone), raft storage (2.6.0
  deprecated the file backend), static auto-unseal so a reboot cannot block
  deploys, mesh-only self-signed TLS (not ACME — that would put DNS in the
  boot chain of the secret store).
- AppRole per environment, CIDR-bound, 5-minute tokens. Each environment's
  secret-id is owned by its own deploy user, which is a boundary that does
  not exist today since prod and beta both reach the same root-owned age key.
- Render still produces the same raw unquoted dotenv: Centralis compares
  /etc/thermograph.env textually and the Swarm entrypoint parses it by line.

The backend dispatch in render-secrets.sh returns early rather than
refactoring the shared write path, so the SOPS path stays byte-identical.
Verified by rendering dev (12 keys) and prod (32 keys) through it, matching
the live hosts. The duplicated write path is noted for consolidation once
prod has been on OpenBao for a release cycle.

The OpenBao renderer rejects values containing a newline or a shell
metacharacter. /etc/thermograph.env is sourced by bash in six places, so
such a value is a command-execution path on the deploy host; the SOPS path
has always had this hazard and avoids it only because every current value
happens to be alphanumeric.

Also records that the age keypair is NOT retired by this migration: it is
the backup encryption key for every off-box dump (ops-cron.yml:142,197,
30-day retention), so deleting it with the vault files would silently
destroy a month of database recoverability.

verify-parity.sh is the cutover gate — it renders both backends and diffs
them, reporting key names and counts only, never values.
2026-07-29 21:35:20 -07:00

50 lines
2 KiB
Desktop File

# OpenBao, as a native systemd service on vps2. Install to
# /etc/systemd/system/openbao.service.
#
# Native, not containerised, on purpose — see the header of config/openbao.hcl. The
# whole point is that this unit's only dependency is the local disk: not the Docker
# daemon that deploys restart, not the Swarm it would otherwise be deployed by, not
# Caddy/ACME for its certificate, and not TimescaleDB.
#
# Note there is currently NO systemd unit of any kind loaded on either host for
# Thermograph (infra/deploy/thermograph.service exists in the repo but is not
# installed), so this is the first. That is deliberate: it is the one component that
# must survive the container layer being broken.
[Unit]
Description=OpenBao secret store
Documentation=https://openbao.org/docs/
# network-online rather than plain network: the mesh listener binds 10.10.0.1, and
# binding a WireGuard address before the interface is up fails the unit outright.
After=network-online.target
Wants=network-online.target
# Explicitly NOT After=docker.service. Adding it would recreate the dependency this
# design exists to avoid.
ConditionFileNotEmpty=/etc/openbao/config.hcl
[Service]
User=openbao
Group=openbao
ExecStart=/usr/local/bin/bao server -config=/etc/openbao/config.hcl
ExecReload=/bin/kill --signal HUP $MAINPID
KillSignal=SIGINT
# Restart always, which together with the static seal is what makes a host reboot
# self-healing. This pair is the reason deploys are not blocked by a reboot.
Restart=always
RestartSec=5
# mlock is disabled in the config (disable_mlock = true), so no IPC_LOCK capability
# is needed. If mlock is ever enabled, add: AmbientCapabilities=CAP_IPC_LOCK
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=full
ProtectHome=yes
# The three paths the service legitimately writes.
ReadWritePaths=/var/lib/openbao /var/log/openbao
# Raft + audit both do a lot of small writes; the default is too low for a store.
LimitNOFILE=65536
LimitCORE=0
[Install]
WantedBy=multi-user.target