Auto-submit IndexNow on deploy when the URL set changes (#130)

Hook the production deploy (deploy/deploy.sh) to ping IndexNow after the health
check passes, so new/changed pages reach Bing/DuckDuckGo/Yandex without a manual
run. Best-effort — wrapped so it can never fail a deploy — and it sources
/etc/thermograph.env so its key matches the one the running service serves.

To avoid re-blasting ~14k URLs on every code push, indexnow.py gains a
--if-changed mode gated by a signature of the URL set (persisted to
data/indexnow_state.txt): code-only deploys skip; a deploy that adds or removes a
city submits. `make indexnow` still forces a full submit.
This commit is contained in:
Emi Griffith 2026-07-16 13:38:59 -07:00 committed by GitHub
parent 7492516a04
commit 1b4e2ec122
2 changed files with 16 additions and 1 deletions

View file

@ -20,6 +20,18 @@ warm_city_archives() {
</dev/null >"$APP_DIR/logs/warm-cities.log" 2>&1 & </dev/null >"$APP_DIR/logs/warm-cities.log" 2>&1 &
} }
# Notify IndexNow (Bing / DuckDuckGo / Yandex) of the site's URLs, but only when
# the set of pages actually changed (a new/removed city) — code-only deploys skip,
# so we don't re-blast ~14k URLs every push. Best-effort: never fails the deploy.
# Sourcing the env matches the key the running service serves at /{key}.txt.
ping_indexnow() {
echo "==> Pinging IndexNow (only if the URL set changed)"
( set -a; . /etc/thermograph.env 2>/dev/null || true; set +a
base="${THERMOGRAPH_BASE_URL:-https://thermograph.org}"
cd "$APP_DIR/backend" && "$APP_DIR/.venv/bin/python" indexnow.py --if-changed "$base"
) || echo "!! IndexNow ping failed (non-fatal)" >&2
}
echo "==> Fetching $BRANCH" echo "==> Fetching $BRANCH"
git fetch --prune origin "$BRANCH" git fetch --prune origin "$BRANCH"
git reset --hard "origin/$BRANCH" git reset --hard "origin/$BRANCH"
@ -43,6 +55,7 @@ for i in $(seq 1 15); do
if curl -fsS -o /dev/null "$url"; then if curl -fsS -o /dev/null "$url"; then
echo "==> OK: $url is serving" echo "==> OK: $url is serving"
warm_city_archives warm_city_archives
ping_indexnow
exit 0 exit 0
fi fi
sleep 1 sleep 1

View file

@ -23,5 +23,7 @@ THERMOGRAPH_BASE=/
# IndexNow key (Bing/DuckDuckGo/Yandex instant re-crawl). Auto-generated to # IndexNow key (Bing/DuckDuckGo/Yandex instant re-crawl). Auto-generated to
# data/indexnow_key.txt on first use; set here to pin a specific key. # data/indexnow_key.txt on first use; set here to pin a specific key.
#THERMOGRAPH_INDEXNOW_KEY= #THERMOGRAPH_INDEXNOW_KEY=
# Public site URL the IndexNow CLI submits (`make indexnow`). # Public site URL for IndexNow. The deploy hook auto-pings IndexNow after a
# successful deploy, but only when the URL set changed (a new/removed city), so
# code-only deploys don't resubmit. `make indexnow` forces a full submit.
THERMOGRAPH_BASE_URL=https://thermograph.org THERMOGRAPH_BASE_URL=https://thermograph.org