Add ops/dbq.sh: read-only Postgres queries across all environments
All checks were successful
PR build (required check) / changes (pull_request) Successful in 7s
secrets-guard / encrypted (pull_request) Successful in 6s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 48s
PR build (required check) / gate (pull_request) Successful in 3s
All checks were successful
PR build (required check) / changes (pull_request) Successful in 7s
secrets-guard / encrypted (pull_request) Successful in 6s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 48s
PR build (required check) / gate (pull_request) Successful in 3s
Uniform read-only query access to the LAN dev, beta and prod databases, plus a thermograph_ro role in each to enforce it. None of the databases are exposed over TCP: each listens only on its private docker network, and prod's is a Swarm overlay the host cannot route to -- so ssh -L works for beta but is impossible for prod, and publishing 5432 would mean new ufw rules and a Swarm endpoint change on production. Running psql inside the db container works identically everywhere with no ports, tunnels or infra changes, so dbq.sh does that: local docker exec for dev, ssh for beta/prod. The prod container is a Swarm task whose name changes each redeploy, so it is resolved at call time rather than hardcoded. Queries connect as thermograph_ro (NOSUPERUSER, granted only pg_read_all_data), so a stray write fails with "permission denied" even against prod; the app's own superuser role is deliberately unused here. Extra args pass through to psql and stdin is forwarded. ops/README.md documents usage and the conventions Iceberg will follow when it lands as a sibling (env-keyed dispatch, run the engine where the data is reachable, dedicated read-only identity).
This commit is contained in:
parent
ebf5efadfb
commit
151cf37dfa
2 changed files with 129 additions and 0 deletions
61
infra/ops/README.md
Normal file
61
infra/ops/README.md
Normal file
|
|
@ -0,0 +1,61 @@
|
|||
# ops/ — operator + agent query tooling
|
||||
|
||||
Read-only query access to Thermograph data, uniform across environments.
|
||||
|
||||
## `dbq.sh` — Postgres, any environment
|
||||
|
||||
```sh
|
||||
infra/ops/dbq.sh dev "select count(*) from climate_history"
|
||||
infra/ops/dbq.sh prod -tA -c "select max(date) from climate_history"
|
||||
infra/ops/dbq.sh beta -c '\dt'
|
||||
echo "select 1" | infra/ops/dbq.sh prod -f -
|
||||
```
|
||||
|
||||
Environments: `dev` (LAN compose stack on the dev machine), `beta`
|
||||
(75.119.132.91), `prod` (169.58.46.181). Extra args pass straight through to
|
||||
`psql` (`-tA`, `-x`, `--csv`, `-f`, …); stdin is forwarded.
|
||||
|
||||
### Why it execs into the container instead of using a connection string
|
||||
|
||||
None of the databases are exposed over TCP — each listens only on its private
|
||||
docker network. Prod's is a Swarm **overlay** (`10.0.2.0/24`) that the host
|
||||
cannot route to, so `ssh -L` works for beta but is *impossible* for prod;
|
||||
publishing 5432 would mean new ufw rules plus a Swarm endpoint change on
|
||||
production. Running `psql` inside the db container works identically in all
|
||||
three environments with no ports, no tunnels, and no infra changes.
|
||||
|
||||
The prod container name is a Swarm task name that changes on every redeploy, so
|
||||
it is resolved at call time via `docker ps --filter name=…`, never hardcoded.
|
||||
|
||||
### Safety
|
||||
|
||||
Queries connect as **`thermograph_ro`** — a `NOSUPERUSER` role granted only
|
||||
`pg_read_all_data`. Read-only is enforced by Postgres, not by convention:
|
||||
|
||||
```
|
||||
$ infra/ops/dbq.sh prod -c "create table t(i int)"
|
||||
ERROR: permission denied for schema public
|
||||
```
|
||||
|
||||
The app's own `thermograph` role is a superuser and is deliberately **not** used
|
||||
by this tool. If the role is ever missing (fresh database), recreate it with:
|
||||
|
||||
```sql
|
||||
CREATE ROLE thermograph_ro LOGIN;
|
||||
GRANT CONNECT ON DATABASE thermograph TO thermograph_ro;
|
||||
GRANT pg_read_all_data TO thermograph_ro;
|
||||
```
|
||||
|
||||
## Iceberg (planned)
|
||||
|
||||
Iceberg lands as a sibling in this directory following the same three
|
||||
conventions, so the query surface stays uniform:
|
||||
|
||||
1. **Env-keyed dispatch** — `iceberg.sh <dev|beta|prod> "<sql>"`, same argument
|
||||
shape as `dbq.sh`.
|
||||
2. **Run the engine where the data is reachable** — the catalog/warehouse
|
||||
dictates it (a Trino/DuckDB/pyiceberg invocation, containerised the same way),
|
||||
rather than exposing new network endpoints.
|
||||
3. **A dedicated read-only identity** — the Iceberg equivalent of
|
||||
`thermograph_ro`, so the same "read-only enforced by the system, not by
|
||||
convention" property holds.
|
||||
68
infra/ops/dbq.sh
Executable file
68
infra/ops/dbq.sh
Executable file
|
|
@ -0,0 +1,68 @@
|
|||
#!/usr/bin/env bash
|
||||
# dbq -- run READ-ONLY SQL against any Thermograph Postgres (LAN dev / beta / prod).
|
||||
#
|
||||
# infra/ops/dbq.sh dev "select count(*) from climate_history"
|
||||
# infra/ops/dbq.sh prod -tA "select max(date) from climate_history"
|
||||
# infra/ops/dbq.sh beta -c '\dt'
|
||||
# echo "select 1" | infra/ops/dbq.sh prod -f -
|
||||
#
|
||||
# Why exec-into-the-container instead of a connection string:
|
||||
# none of the databases are exposed over TCP. Each listens only on its private
|
||||
# docker network -- and prod's is a Swarm *overlay* (10.0.2.0/24) that the host
|
||||
# itself cannot route to, so `ssh -L` works for beta but is impossible for prod.
|
||||
# Publishing 5432 would mean new firewall + Swarm endpoint changes on production.
|
||||
# Running psql *inside* the db container works identically in all three
|
||||
# environments with no ports, no tunnels and no infra changes -- locally for LAN
|
||||
# dev, over SSH for beta/prod.
|
||||
#
|
||||
# Safety: always connects as `thermograph_ro`, a NOSUPERUSER role granted only
|
||||
# pg_read_all_data. Read-only is enforced by Postgres itself, not by convention,
|
||||
# so a stray INSERT/DDL fails with "permission denied" even against prod. (The
|
||||
# app's own `thermograph` role is a superuser -- deliberately not used here.)
|
||||
#
|
||||
# Any extra arguments are passed straight through to psql, so -tA, -c, -f, -x,
|
||||
# --csv etc. all work. Stdin is forwarded, so `-f -` reads piped SQL.
|
||||
set -euo pipefail
|
||||
|
||||
KEYFILE="${THERMOGRAPH_AGENT_KEY:-$HOME/.ssh/thermograph_agent_ed25519}"
|
||||
DB_USER="${THERMOGRAPH_DB_QUERY_USER:-thermograph_ro}"
|
||||
DB_NAME="${THERMOGRAPH_DB_NAME:-thermograph}"
|
||||
|
||||
usage() {
|
||||
sed -n '2,9p' "$0" | sed 's/^# \{0,1\}//' >&2
|
||||
echo "environments: dev | beta | prod" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
env_name="${1:-}"
|
||||
[ -n "$env_name" ] || usage
|
||||
shift
|
||||
|
||||
case "$env_name" in
|
||||
dev) filter=thermograph-dev-db ; ssh_target= ;;
|
||||
beta) filter=thermograph-db-1 ; ssh_target=agent@75.119.132.91 ;;
|
||||
prod) filter=thermograph_db ; ssh_target=agent@169.58.46.181 ;;
|
||||
-h|--help) usage ;;
|
||||
*) echo "dbq: unknown environment '$env_name' (want: dev|beta|prod)" >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
[ $# -gt 0 ] || usage
|
||||
|
||||
# Quote the psql arguments so they survive the remote shell intact.
|
||||
psql_args=$(printf '%q ' "$@")
|
||||
|
||||
# The container name is resolved on the target host at call time: prod's Swarm
|
||||
# task name changes on every redeploy, so it can never be hardcoded.
|
||||
remote=$(cat <<REMOTE
|
||||
cid=\$(docker ps -qf name=${filter} | head -1)
|
||||
[ -n "\$cid" ] || { echo "dbq: no running db container matching '${filter}'" >&2; exit 1; }
|
||||
exec docker exec -i "\$cid" psql -U ${DB_USER} -d ${DB_NAME} -v ON_ERROR_STOP=1 ${psql_args}
|
||||
REMOTE
|
||||
)
|
||||
|
||||
if [ -z "$ssh_target" ]; then
|
||||
exec bash -c "$remote"
|
||||
else
|
||||
exec ssh -i "$KEYFILE" -o StrictHostKeyChecking=no -o ConnectTimeout=15 \
|
||||
"$ssh_target" "$remote"
|
||||
fi
|
||||
Loading…
Reference in a new issue