promote: dev → main (doc accuracy pass, runner unit fix) #158
10 changed files with 68 additions and 15 deletions
|
|
@ -29,8 +29,10 @@ concurrently. Read both before dispatching parallel work.
|
||||||
|
|
||||||
**`dev` is a first-class hosted environment, not just an integration branch.**
|
**`dev` is a first-class hosted environment, not just an integration branch.**
|
||||||
It runs on vps1 — the same box as Forgejo and the monitoring stack — with its
|
It runs on vps1 — the same box as Forgejo and the monitoring stack — with its
|
||||||
own Postgres container, reachable only on the WireGuard mesh
|
own Postgres container, reachable only from vps1 itself (`127.0.0.1:8137` —
|
||||||
(`10.10.0.2:8137`): no public DNS record, no Caddy site, no TLS. It is deployed
|
`infra/docker-compose.yml` binds the port to loopback, so the mesh does not
|
||||||
|
reach it either; use an SSH tunnel): no public DNS record, no Caddy site, no
|
||||||
|
TLS. It is deployed
|
||||||
by CI like beta and prod. The desktop hosts no Thermograph environment at all;
|
by CI like beta and prod. The desktop hosts no Thermograph environment at all;
|
||||||
`make dev-up` there is a laptop convenience for running the stack locally, not
|
`make dev-up` there is a laptop convenience for running the stack locally, not
|
||||||
a deployment target.
|
a deployment target.
|
||||||
|
|
|
||||||
|
|
@ -45,7 +45,7 @@ for systemd/CI callers. Entry target is `app:app`.
|
||||||
that disagree with the labels the API returns.
|
that disagree with the labels the API returns.
|
||||||
- **Fahrenheit country set** — `api/content_payloads.py`'s `F_COUNTRIES` is
|
- **Fahrenheit country set** — `api/content_payloads.py`'s `F_COUNTRIES` is
|
||||||
canonical; `frontend`'s `server/internal/format::FCountries` and
|
canonical; `frontend`'s `server/internal/format::FCountries` and
|
||||||
`static/units.js`'s `F_REGIONS` must stay identical to it. Both are asserted
|
`frontend/static/units.js`'s `F_REGIONS` must stay identical to it. Both are asserted
|
||||||
by tests in `frontend/server/internal/format/format_test.go` — but note the
|
by tests in `frontend/server/internal/format/format_test.go` — but note the
|
||||||
backend cross-check **skips in CI** (the frontend image's build context is
|
backend cross-check **skips in CI** (the frontend image's build context is
|
||||||
`frontend/`, so this file is unreachable from the builder stage, which is the
|
`frontend/`, so this file is unreachable from the builder stage, which is the
|
||||||
|
|
|
||||||
|
|
@ -69,7 +69,7 @@ cities_flavor.json bundled reference data (generated by gen_cities.py /
|
||||||
only builds and publishes its own container image
|
only builds and publishes its own container image
|
||||||
(`git.thermograph.org/admin_emi/thermograph-backend/app`) and hands infra a tag to
|
(`git.thermograph.org/admin_emi/thermograph-backend/app`) and hands infra a tag to
|
||||||
roll out (`SERVICE=backend` + `BACKEND_IMAGE_TAG` into infra's
|
roll out (`SERVICE=backend` + `BACKEND_IMAGE_TAG` into infra's
|
||||||
`deploy/deploy.sh`).
|
`infra/deploy/deploy.sh`).
|
||||||
- **`thermograph-docs`** holds the cross-repo architecture/runbook docs; this
|
- **`thermograph-docs`** holds the cross-repo architecture/runbook docs; this
|
||||||
README only covers what's local to this service.
|
README only covers what's local to this service.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -16,7 +16,7 @@ The `db` service runs the stock **`timescale/timescaledb:latest-pg18`** image
|
||||||
TTL, and the `recent_stamp` token embedded in derived-payload validity.
|
TTL, and the `recent_stamp` token embedded in derived-payload validity.
|
||||||
|
|
||||||
The schema is created by Alembic (`backend/alembic/versions/0002_climate_hypertables.py`,
|
The schema is created by Alembic (`backend/alembic/versions/0002_climate_hypertables.py`,
|
||||||
run at app boot via `deploy/entrypoint.sh`). The app reads/writes it through
|
run at app boot via `backend/deploy/entrypoint.sh`). The app reads/writes it through
|
||||||
`backend/data/climate_store.py` (psycopg + polars). See
|
`backend/data/climate_store.py` (psycopg + polars). See
|
||||||
`deploy/POSTGRES-MIGRATION.md` for the parquet→hypertable cutover.
|
`deploy/POSTGRES-MIGRATION.md` for the parquet→hypertable cutover.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,7 @@
|
||||||
Runs as `deploy/forgejo/docker-stack.yml` — the only Swarm-scheduled workload
|
Runs as `deploy/forgejo/docker-stack.yml` — the only Swarm-scheduled workload
|
||||||
this cluster carries (prod and beta's app stacks are separate `docker stack
|
this cluster carries (prod and beta's app stacks are separate `docker stack
|
||||||
deploy`s that happen to run on the manager node, vps2 — see
|
deploy`s that happen to run on the manager node, vps2 — see
|
||||||
`deploy/stack/README` context in `DEPLOY.md`). Pinned to the **vps1** node
|
`DEPLOY.md`). Pinned to the **vps1** node
|
||||||
(`75.119.132.91`) via the `role=forge` label from
|
(`75.119.132.91`) via the `role=forge` label from
|
||||||
`deploy/swarm/label-forge-node.sh`.
|
`deploy/swarm/label-forge-node.sh`.
|
||||||
|
|
||||||
|
|
@ -153,6 +153,28 @@ echo "10.10.0.2 git.thermograph.org" | sudo tee -a /etc/hosts
|
||||||
numbering — adjust if you assigned it differently.) The git/web UI keeps
|
numbering — adjust if you assigned it differently.) The git/web UI keeps
|
||||||
working normally for everyone else since only `/v2/*` is restricted.
|
working normally for everyone else since only `/v2/*` is restricted.
|
||||||
|
|
||||||
|
**Pin the canonical name too, not just the image host.** Forgejo derives the
|
||||||
|
registry's bearer-token realm from `ROOT_URL`, so a client that already
|
||||||
|
resolves `git.thermograph.org` over the mesh is still sent to
|
||||||
|
`https://<ROOT_URL host>/v2/token` to collect a token. When `ROOT_URL` became
|
||||||
|
`dev.jinemi.com`, that second request took the public route, the `/v2/*`
|
||||||
|
matcher above answered 403, and docker fell back to anonymous — every push then
|
||||||
|
failed with `unauthorized: reqPackageAccess`, which reads exactly like a
|
||||||
|
revoked token or a missing scope. It is neither. Pin both names:
|
||||||
|
|
||||||
|
```
|
||||||
|
echo "10.10.0.2 git.thermograph.org dev.jinemi.com" | sudo tee -a /etc/hosts
|
||||||
|
```
|
||||||
|
|
||||||
|
The realm moves whenever `ROOT_URL` does, so read it rather than assuming:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -sI https://git.thermograph.org/v2/ | grep -i www-authenticate
|
||||||
|
```
|
||||||
|
|
||||||
|
These `/etc/hosts` pins are host state. Nothing in this repo writes them, so a
|
||||||
|
reprovisioned node has to be pinned again before it can pull.
|
||||||
|
|
||||||
## Register the Actions runner
|
## Register the Actions runner
|
||||||
|
|
||||||
Once Forgejo answers at its domain:
|
Once Forgejo answers at its domain:
|
||||||
|
|
|
||||||
|
|
@ -93,6 +93,13 @@ cat > "$HOME/.config/systemd/user/forgejo-runner.service" <<EOF
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=Forgejo Actions runner (docker + thermograph-lan)
|
Description=Forgejo Actions runner (docker + thermograph-lan)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
|
# These bound the Restart=always loop below, so a genuinely broken config (bad
|
||||||
|
# token, docker unreachable) ends as a stopped unit rather than spinning
|
||||||
|
# forever. They belong in [Unit]: systemd accepts them in [Service] without
|
||||||
|
# complaint but IGNORES them, leaving the 10s default — \`systemctl --user show
|
||||||
|
# forgejo-runner -p StartLimitIntervalUSec\` is how you tell which one you got.
|
||||||
|
StartLimitIntervalSec=300
|
||||||
|
StartLimitBurst=5
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
WorkingDirectory=${RUNNER_DIR}
|
WorkingDirectory=${RUNNER_DIR}
|
||||||
|
|
@ -102,7 +109,16 @@ WorkingDirectory=${RUNNER_DIR}
|
||||||
# SOPS-configured) fails with "not installed" even though it plainly is.
|
# SOPS-configured) fails with "not installed" even though it plainly is.
|
||||||
Environment=PATH=%h/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
|
Environment=PATH=%h/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
|
||||||
ExecStart=${RUNNER_DIR}/forgejo-runner daemon -c config.yaml
|
ExecStart=${RUNNER_DIR}/forgejo-runner daemon -c config.yaml
|
||||||
Restart=on-failure
|
# \`always\`, NOT \`on-failure\` — the same distinction that took Forgejo down for
|
||||||
|
# 27 hours on 2026-07-29 (see docker-stack.yml's restart_policy comment): an
|
||||||
|
# always-on daemon that exits **0** is not "finished successfully", it is a
|
||||||
|
# daemon that stopped and must come back. \`on-failure\` cannot tell those apart,
|
||||||
|
# and forgejo-runner exits 0 on several paths — a lost instance connection it
|
||||||
|
# gives up on, or a SIGTERM from a Docker restart it interprets as a clean
|
||||||
|
# shutdown. The failure mode is silent: the unit sits \`inactive (dead)\`, the UI
|
||||||
|
# shows the runner offline, and every protected-branch merge blocks on a check
|
||||||
|
# that will never be produced.
|
||||||
|
Restart=always
|
||||||
RestartSec=5
|
RestartSec=5
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
|
|
|
||||||
|
|
@ -49,13 +49,24 @@ deploys.
|
||||||
|
|
||||||
So this directory is the source of truth; vps2 gets a copy:
|
So this directory is the source of truth; vps2 gets a copy:
|
||||||
|
|
||||||
|
The two files do **not** go to the same place, and getting it wrong fails in a
|
||||||
|
confusing way. `docker-compose.yml` mounts `./data:/data` and starts the daemon
|
||||||
|
with `--config /data/config.yaml`, so `/data` *is* `data/` — a `config.yaml`
|
||||||
|
sitting beside the compose file is invisible to the container. The daemon then
|
||||||
|
either refuses to start or silently runs on defaults (no `--add-host`, so every
|
||||||
|
registry push fails as if the credential were wrong).
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
install -d -m 0755 /opt/forgejo-runner
|
install -d -m 0755 /opt/forgejo-runner/data
|
||||||
cp /opt/thermograph/infra/deploy/forgejo/runner-vps2/docker-compose.yml \
|
cp /opt/thermograph/infra/deploy/forgejo/runner-vps2/docker-compose.yml \
|
||||||
/opt/thermograph/infra/deploy/forgejo/runner-vps2/config.yaml \
|
|
||||||
/opt/forgejo-runner/
|
/opt/forgejo-runner/
|
||||||
|
cp /opt/thermograph/infra/deploy/forgejo/runner-vps2/config.yaml \
|
||||||
|
/opt/forgejo-runner/data/
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`data/` is also where `register` writes `.runner`, so the runner's whole state —
|
||||||
|
config plus registration — lives in the one directory the compose file mounts.
|
||||||
|
|
||||||
Re-copy after changing either file here, then `docker compose up -d`. There is
|
Re-copy after changing either file here, then `docker compose up -d`. There is
|
||||||
no automation for that step and there should not be: a workflow that redeploys
|
no automation for that step and there should not be: a workflow that redeploys
|
||||||
the runner runs *on* the runner.
|
the runner runs *on* the runner.
|
||||||
|
|
|
||||||
|
|
@ -14,7 +14,7 @@ submission ($25 one-time) is deferred.
|
||||||
|
|
||||||
## What's in the repo
|
## What's in the repo
|
||||||
- `twa-manifest.json` — Bubblewrap config (reference values; see comment inside).
|
- `twa-manifest.json` — Bubblewrap config (reference values; see comment inside).
|
||||||
- `../../frontend/.well-known/assetlinks.json` — Digital Asset Links, served at
|
- `../../../frontend/static/.well-known/assetlinks.json` — Digital Asset Links, served at
|
||||||
`https://thermograph.org/.well-known/assetlinks.json`. Its fingerprint is a
|
`https://thermograph.org/.well-known/assetlinks.json`. Its fingerprint is a
|
||||||
**placeholder** until you generate the signing key (step 4).
|
**placeholder** until you generate the signing key (step 4).
|
||||||
|
|
||||||
|
|
@ -45,7 +45,7 @@ bubblewrap build
|
||||||
bubblewrap fingerprint list # or: keytool -list -v -keystore android-keystore.jks
|
bubblewrap fingerprint list # or: keytool -list -v -keystore android-keystore.jks
|
||||||
```
|
```
|
||||||
|
|
||||||
Copy the `SHA256` value into `frontend/.well-known/assetlinks.json`, replacing
|
Copy the `SHA256` value into `frontend/static/.well-known/assetlinks.json`, replacing
|
||||||
`REPLACE_WITH_TWA_SIGNING_KEY_SHA256_FINGERPRINT`, then **deploy the site** so the
|
`REPLACE_WITH_TWA_SIGNING_KEY_SHA256_FINGERPRINT`, then **deploy the site** so the
|
||||||
new file is live. Confirm:
|
new file is live. Confirm:
|
||||||
|
|
||||||
|
|
@ -80,6 +80,7 @@ install for every user. Store it out of the repo, backed up.
|
||||||
`.aab`; the store review for a well-formed TWA is light. Not part of this pass.
|
`.aab`; the store review for a well-formed TWA is light. Not part of this pass.
|
||||||
|
|
||||||
## Why no backend changes
|
## Why no backend changes
|
||||||
Android delegates the TWA's web notifications to the OS, so `backend/push.py`
|
Android delegates the TWA's web notifications to the OS, so
|
||||||
(VAPID) and `backend/notify.py` deliver push to the app unchanged. Nothing in
|
`backend/notifications/push.py` (VAPID) and `backend/notifications/notify.py`
|
||||||
|
deliver push to the app unchanged. Nothing in
|
||||||
`backend/` is touched by this track.
|
`backend/` is touched by this track.
|
||||||
|
|
|
||||||
|
|
@ -183,7 +183,8 @@ Terraform brings the stack up — don't let Terraform recreate containers mid-mi
|
||||||
`git.thermograph.org` reaches Forgejo's loopback port on vps1.
|
`git.thermograph.org` reaches Forgejo's loopback port on vps1.
|
||||||
`COOKIE_SECURE` is auto-set to `0` when there's no domain (a Secure
|
`COOKIE_SECURE` is auto-set to `0` when there's no domain (a Secure
|
||||||
cookie is never sent over plain HTTP) and `1` behind Caddy TLS.
|
cookie is never sent over plain HTTP) and `1` behind Caddy TLS.
|
||||||
- The rendered Caddyfile only reverse-proxies the app. The repo's `deploy/Caddyfile`
|
- The rendered Caddyfile only reverse-proxies the app. The repo's
|
||||||
|
`deploy/stack/lb/Caddyfile`
|
||||||
additionally serves the `emigriffith.dev` portfolio and legacy redirects; those are
|
additionally serves the `emigriffith.dev` portfolio and legacy redirects; those are
|
||||||
host-specific and not templated here.
|
host-specific and not templated here.
|
||||||
- Provisioner-based by design: the hosts already exist, so this is not a
|
- Provisioner-based by design: the hosts already exist, so this is not a
|
||||||
|
|
|
||||||
|
|
@ -147,7 +147,7 @@ LOKI_URL=http://10.10.0.2:3100/loki/api/v1/push \
|
||||||
docker compose -f docker-compose.agent.yml -f docker-compose.agent.beta.yml up -d
|
docker compose -f docker-compose.agent.yml -f docker-compose.agent.beta.yml up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
The mesh must already be up (see the main repo's `deploy/swarm/` / `INFRA.md`);
|
The mesh must already be up (see `infra/deploy/swarm/`);
|
||||||
Swarm is **not** required for the agent itself — it talks plain HTTP over wg0.
|
Swarm is **not** required for the agent itself — it talks plain HTTP over wg0.
|
||||||
|
|
||||||
<!-- TODO(cutover): desktop no longer runs a Thermograph app stack and ships no
|
<!-- TODO(cutover): desktop no longer runs a Thermograph app stack and ships no
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue