# Dev CI/CD → LAN server on this machine

Parallel to the prod pipeline (`main` → VPS, see `DEPLOY.md`), the **`dev`**
branch continuously deploys to a **LAN server running on this computer**. Git
hosting and CI are self-hosted **Forgejo** (`git.thermograph.org`, reachable
at `http://10.10.0.2:3080` over the WireGuard mesh) — GitHub is retired.

```
open PR ──▶ CI (build + boot/health, on the LAN Forgejo runner)
              │ green
              ▼
        merge into dev (explicit — Forgejo does not auto-merge on its own;
                         see "Landing a PR" below)
              │
              ▼
        LAN runner (thermograph-lan label) runs deploy/deploy-dev.sh
              │
              ▼
        ~/thermograph-dev updated ─▶ docker compose stack (backend + frontend +
                                     Postgres 18) brought up, backend on 0.0.0.0:8137
```

Reachable at `http://<lan-ip>:8137/` from any device on your Wi-Fi.

The dev server runs the **same containerized stack as prod** (`docker-compose.yml`),
overlaid with `docker-compose.dev.yml`: **uncapped** (no CPU limits, unlike prod's
backend=4 / frontend=2 / db=2) and backend published on `0.0.0.0:8137` for the
LAN (prod binds loopback behind Caddy; frontend has no published port here
either way, no Caddy to reach it directly, so it's only reached through
backend's own reverse-proxy fallback). Bring it up by hand with `make dev-up`.

## Why it's built this way

Forgejo has native branch protection + required-status-checks + auto-merge
(unlike GitHub on a private free-tier repo, where those are paywalled). But
"auto-merge" here still means opting a PR in — either clicking **"Auto merge
when checks succeed"** in the web UI, or merging explicitly once CI is green
via the API (`POST .../pulls/{n}/merge`, `{"Do": "squash"}`). Nothing merges
a ready PR for you automatically just because checks pass, unlike GitHub's
old in-workflow `ci-cd.yml` (retired along with the rest of `.github/`).

## Moving parts

| File | Purpose |
|------|---------|
| `.forgejo/workflows/pr-build.yml` | required status check for PRs into `dev` (calls `build.yml`) |
| `.forgejo/workflows/deploy-dev.yml` | build + deploy on pushes to `dev` (direct, or a PR merge) |
| `.forgejo/workflows/build.yml` | shared build gate: deps, backend tests, JS syntax check, boot/health check |
| `deploy/deploy-dev.sh` | pull `dev` into `~/thermograph-dev`, `docker compose up` the uncapped LAN stack, health check |
| `docker-compose.dev.yml` | dev overlay: no CPU limits, backend published on `0.0.0.0:8137` (frontend unpublished) |
| `deploy/provision-dev-lan.sh` | one-time sudo-free bootstrap of the checkout + runner |
| `deploy/forgejo/register-lan-runner.sh` | registers the Forgejo Actions runner on this machine |

`deploy-dev.yml`'s `deploy` job runs on the `thermograph-lan` label — **not**
`[self-hosted, thermograph-lan]` (the array form is what the original GitHub
version used; GitHub implicitly tags every self-hosted runner `self-hosted`,
but Forgejo's runner has only the labels it was explicitly registered with,
so the array form is permanently unschedulable there — a real bug found and
fixed once, worth knowing about if a "deploy" job ever silently stops
appearing in the Actions history again).

`deploy-dev.sh`'s git auth (`GH_TOKEN`/`GITHUB_TOKEN`, Forgejo's per-job
token exposed under that name for compatibility) is scoped to `REPO_URL`'s
own host via a per-command `http.<scheme>://<host>/.extraheader`, not a
hardcoded one — important if the Forgejo host ever changes, since a stale
hardcoded host would make git silently skip the header (no error) and fall
through to whatever ambient credentials happen to be available, not fail
loudly.

## The self-hosted runner

A Forgejo Actions **runner** (`forgejo-runner`) runs on this machine, labels
`docker` (containerized jobs, node:20-bookworm, with the host's docker.sock
automounted in — `container.docker_host: automount` in
`~/forgejo-runner/config.yaml` — so `docker build`/`push` work without
privileged Docker-in-Docker) and `thermograph-lan` (host-native jobs, for
`deploy-dev.sh`'s systemctl/docker-compose calls). Installed under
`~/forgejo-runner`, runs as the `forgejo-runner` systemd `--user` service.

```bash
# status / logs
systemctl --user status forgejo-runner
journalctl --user -u forgejo-runner -f

# re-register if the token/host ever changes
bash deploy/forgejo/register-lan-runner.sh <forgejo_url> <registration_token>
```

`git.thermograph.org`'s public DNS resolves to beta's **public** IP, which
the registry's mesh-only ACL (see `deploy/forgejo/README.md`) rejects for
`/v2/` paths — `docker build-push.yml` pushes run against the *host's*
automounted daemon, so it's this **host's** own resolver that needs to route
git.thermograph.org over the mesh, not anything settable from a job
container. If registry pushes ever start failing with an HTTP/TLS mismatch
or a 403 on `/v2/`, check `getent hosts git.thermograph.org` here — it needs
to resolve to beta's WireGuard IP (`10.10.0.2`), typically via a `/etc/hosts`
line, not the public one.

## The LAN dev service (Docker Compose)

Runs as a Docker Compose stack, not a bare systemd unit — `docker ps --filter
name=thermograph-dev` shows `thermograph-dev-backend-1`, `thermograph-dev-frontend-1`
(repo-split Stage 4 split the single "app" container in two — frontend has no
published port, reached only through backend's own reverse-proxy fallback) and
`thermograph-dev-db-1` (TimescaleDB). A legacy pre-container
`thermograph-dev.service` systemd --user unit may still exist from before this
cutover; it's stale and `deploy-dev.sh` stops/disables it on every run — don't
trust `systemctl --user status thermograph-dev` as a liveness check, use `docker
ps` instead.

```bash
docker ps --filter name=thermograph-dev                                          # is it up?
docker compose -f docker-compose.yml -f docker-compose.dev.yml logs -f backend frontend   # app logs (from ~/thermograph-dev)
```

Bootstrap (or rebuild) it by hand:

```bash
bash deploy/provision-dev-lan.sh
```

Config: port/base path are baked into the unit from `deploy/deploy-dev.sh`
(`PORT`, `THERMOGRAPH_BASE`). Change them there and redeploy. The dedicated
checkout at `~/thermograph-dev` is separate from your working tree, so a deploy
never touches uncommitted edits in `~/Code/Thermograph`.

## Before monitoring a PR to land

A PR needs an explicit merge once CI is green — but first confirm it's
actually set up to land, or you'll monitor a PR that can never merge:

1. **Not conflicting** — `GET /api/v1/repos/{owner}/{repo}/pulls/{n}` should
   show `"mergeable": true`. If not, merge `forgejo/dev` in and resolve now —
   don't monitor a PR that can't merge.
2. **Required check actually configured** — `GET
   /api/v1/repos/{owner}/{repo}/branch_protections/dev` should list
   `status_check_contexts` containing the *exact* string Forgejo reports for
   that PR's check (workflow display name + job name + event, e.g. `"PR build
   (required check) / build (pull_request)"` — not just the job name
   `"build"`). A mismatch here means no PR can ever satisfy the check even
   with fully green CI; this bit us once already.
3. **CI actually running** — `GET
   /api/v1/repos/{owner}/{repo}/commits/{sha}/status` shows a status, not an
   empty list. If empty, check the runner is up
   (`systemctl --user status forgejo-runner`).

Only start monitoring once all three pass. Once CI is green and mergeable is
true, merge explicitly: `POST /api/v1/repos/{owner}/{repo}/pulls/{n}/merge`
with `{"Do": "squash", "delete_branch_after_merge": true}`.

## Day-to-day

- **Ship:** open a PR into `dev`, confirm CI is green, merge it explicitly —
  the merge triggers the deploy here.
- **Manual redeploy:** trigger `deploy-dev.yml` via `workflow_dispatch`
  (`POST /api/v1/repos/{owner}/{repo}/actions/workflows/deploy-dev.yml/dispatches`,
  `{"ref": "dev"}`), or locally `APP_DIR=~/thermograph-dev bash deploy/deploy-dev.sh`.
- **Promote to prod:** merge/fast-forward `dev` into `main` and push — that
  triggers the VPS pipeline in `DEPLOY.md`.

## Monitoring

Logs and dashboards moved to a **separate project** — a central Grafana + Loki
stack fed by a Grafana Alloy agent on every node (including this LAN dev box),
over the WireGuard mesh (`thermograph-observability` on Forgejo; UI at
`https://dashboard.thermograph.org`). The dev node's Alloy agent ships its
container/app logs under `host="dev"`, so LAN dev shows up alongside prod and
beta in the same dashboards. This replaced the old `scripts/dashboard.py`.

For a quick terminal check without Grafana, the app still serves raw counters at
the gated `GET /api/v2/metrics` route (loopback-only; set
`THERMOGRAPH_METRICS_TOKEN` to read it over an SSH tunnel).