thermograph/deploy/provision-mail.sh

161 lines
6.7 KiB
Bash
Raw Normal View History

Rebuild the homepage as a distribution landing; add an SMTP seam (#178) The homepage was a bare tool: a find-bar and an empty panel reading "Find a location to begin." A visitor arriving from a search result or a shared link learned nothing about what the product does before deciding to leave. Rebuild it around the Weekly view, which is untouched: - Hero with the question as the h1, and a grade card showing a real graded example — the most unusual city we're currently tracking, either tail. The card's frame and text slots are server-rendered with reserved heights, so app.js re-pointing it at the visitor's own place shifts nothing. - "Unusual right now" strip, CSS scroll-snap, no JS carousel. A cold-tail city is force-included whenever one qualifies. - Stance line, how-it-works, explore cards, and 12 city chips linking into the ~1000-page /climate surface. - Monthly digest form, in the footer of every page. Serve / from Jinja instead of a static file with placeholder substitution, so crawlers and no-JS readers get the whole page as real HTML. home.html.j2 extends base.html.j2 and carries app.js's DOM contract over verbatim; the brand degrades to a <p> so the hero owns the sole h1. frontend/index.html is deleted rather than left behind the static mount, where it would keep serving indexable duplicate content. "Where is it most unusual right now" has no cheap answer at request time — percentiles live inside zlib-compressed payload blobs with no column to sort on. So homepage.py sweeps the warm cache and writes data/homepage.json, read by the template. The sweep is strictly cache-only (climate.load_cached_recent_forecast is new, the sibling of load_cached_history), so grading ~1000 cities costs zero upstream requests. It rides the notifier's timer behind an hourly guard rather than starting a second daemon, and also runs at the tail of warm_cities so a fresh deploy has a populated feed. Instrumentation: metrics gains a product-event counter keyed by (event, referrer domain, UTC day), behind an allowlist and a per-IP rate limit, fed by POST /api/v2/event. The referrer is taken from the request's own header, never from the client. The beacon gets its own inbound category that record_inbound ignores, so reporting an interaction doesn't also count as traffic. The dashboard grows an events block with per-referrer attribution. Email is scaffolded but sends nothing yet. mailer.py talks stdlib smtplib to a local Postfix null client on 127.0.0.1:25 (deploy/provision-mail.sh), so the choice between direct-to-MX and relaying through a provider stays a Postfix config change with no code change. The backend defaults to "console", which logs and sends nothing, so dev and tests exercise the whole signup path safely. Signups land in pending_digest unconfirmed; collecting the list shouldn't wait on delivery. Also adds /privacy, linked from the footer and kept out of the sitemap. The strip's classes are named unusual-* rather than record-*: .record-card is already the SEO records page's, and reusing it leaked layout rules onto /climate/<slug>/records.
2026-07-18 07:39:47 +00:00
#!/usr/bin/env bash
# Outbound email for Thermograph — run once on the VPS, as root.
#
# Installs Postfix as a SEND-ONLY NULL CLIENT: it listens on 127.0.0.1:25 only,
# accepts mail from this machine, and never receives mail from the internet.
#
# Why a local MTA instead of talking to a mail provider's API from Python:
#
# * The app's only mail config becomes "SMTP on localhost". Whether delivery
# then goes direct to the recipient's MX or through a relay is a Postfix
# setting — switchable without touching, redeploying, or retesting the app.
# * Postfix queues and retries. A request handler hands the message over in
# microseconds and returns; a slow or briefly-down upstream can't stall a
# web request or lose a signup.
# * No new Python dependency: stdlib smtplib talks to it (see backend/mailer.py).
#
# DELIVERABILITY — read before pointing this at real subscribers.
#
# Mail sent straight from a VPS IP is very often junked, regardless of Postfix
# config, because the IP has no sending reputation. Two options:
#
# A. RELAY through a transactional provider (recommended for real mail).
# Set RELAYHOST + RELAY_USER + RELAY_PASSWORD below. The provider handles
# SPF/DKIM alignment and reputation; you keep the loopback-SMTP seam.
#
# B. DIRECT to MX (no third party). Then you must also set up, in DNS:
# - SPF: TXT @ "v=spf1 a mx ip4:<VPS_IP> -all"
# - DKIM: install opendkim, publish the public key as a TXT record
# - DMARC: TXT _dmarc "v=DMARC1; p=none; rua=mailto:you@domain"
# - PTR / reverse DNS on the VPS IP -> mail.thermograph.org
# The PTR record is the one people forget, and its absence alone is enough
# for Gmail and Outlook to junk everything you send.
#
# Usage:
# sudo MAIL_DOMAIN=thermograph.org bash deploy/provision-mail.sh
# sudo MAIL_DOMAIN=thermograph.org RELAYHOST='[smtp.provider.com]:587' \
# RELAY_USER=apikey RELAY_PASSWORD=secret bash deploy/provision-mail.sh
set -euo pipefail
MAIL_DOMAIN="${MAIL_DOMAIN:-thermograph.org}"
MAIL_HOSTNAME="${MAIL_HOSTNAME:-mail.${MAIL_DOMAIN}}"
RELAYHOST="${RELAYHOST:-}"
RELAY_USER="${RELAY_USER:-}"
RELAY_PASSWORD="${RELAY_PASSWORD:-}"
if [[ $EUID -ne 0 ]]; then
echo "run as root (sudo)" >&2
exit 1
fi
echo "==> installing postfix (non-interactive)"
export DEBIAN_FRONTEND=noninteractive
# Preseed so the installer doesn't open its curses dialog.
debconf-set-selections <<EOF
postfix postfix/main_mailer_type select Internet Site
postfix postfix/mailname string ${MAIL_HOSTNAME}
EOF
apt-get update -qq
apt-get install -y -qq postfix libsasl2-modules
echo "==> configuring send-only null client"
postconf -e "myhostname = ${MAIL_HOSTNAME}"
postconf -e "myorigin = ${MAIL_DOMAIN}"
# Never listen on a public interface. This box sends only. The app runs in a
# Docker container, so it can't reach the host's loopback — it hands mail to
# Postfix over the compose bridge's gateway. So Postfix also listens on that
# gateway and accepts mail from the bridge subnet (both pinned in
# docker-compose.yml). Set DOCKER_MAIL_GATEWAY="" for a pure loopback-only null
# client (app running natively on the host, not in a container).
DOCKER_MAIL_GATEWAY="${DOCKER_MAIL_GATEWAY-172.19.0.1}"
DOCKER_MAIL_SUBNET="${DOCKER_MAIL_SUBNET-172.19.0.0/16}"
# Optional WireGuard-mesh listener: other mesh nodes (e.g. beta's Forgejo, whose
# mailer posts to 10.10.0.1:25 — see deploy/forgejo/docker-stack.yml) can relay
# through this box. Prod runs with MESH_MAIL_LISTEN=10.10.0.1 and
# MESH_MAIL_PEERS=10.10.0.2/32; both default OFF so a plain run stays a strict
# null client. Without these, re-running this script on prod would silently drop
# the mesh listener and break Forgejo's outbound mail — the live config was
# originally hand-applied and this script is the source of truth for it now.
MESH_MAIL_LISTEN="${MESH_MAIL_LISTEN-}"
MESH_MAIL_PEERS="${MESH_MAIL_PEERS-}"
Rebuild the homepage as a distribution landing; add an SMTP seam (#178) The homepage was a bare tool: a find-bar and an empty panel reading "Find a location to begin." A visitor arriving from a search result or a shared link learned nothing about what the product does before deciding to leave. Rebuild it around the Weekly view, which is untouched: - Hero with the question as the h1, and a grade card showing a real graded example — the most unusual city we're currently tracking, either tail. The card's frame and text slots are server-rendered with reserved heights, so app.js re-pointing it at the visitor's own place shifts nothing. - "Unusual right now" strip, CSS scroll-snap, no JS carousel. A cold-tail city is force-included whenever one qualifies. - Stance line, how-it-works, explore cards, and 12 city chips linking into the ~1000-page /climate surface. - Monthly digest form, in the footer of every page. Serve / from Jinja instead of a static file with placeholder substitution, so crawlers and no-JS readers get the whole page as real HTML. home.html.j2 extends base.html.j2 and carries app.js's DOM contract over verbatim; the brand degrades to a <p> so the hero owns the sole h1. frontend/index.html is deleted rather than left behind the static mount, where it would keep serving indexable duplicate content. "Where is it most unusual right now" has no cheap answer at request time — percentiles live inside zlib-compressed payload blobs with no column to sort on. So homepage.py sweeps the warm cache and writes data/homepage.json, read by the template. The sweep is strictly cache-only (climate.load_cached_recent_forecast is new, the sibling of load_cached_history), so grading ~1000 cities costs zero upstream requests. It rides the notifier's timer behind an hourly guard rather than starting a second daemon, and also runs at the tail of warm_cities so a fresh deploy has a populated feed. Instrumentation: metrics gains a product-event counter keyed by (event, referrer domain, UTC day), behind an allowlist and a per-IP rate limit, fed by POST /api/v2/event. The referrer is taken from the request's own header, never from the client. The beacon gets its own inbound category that record_inbound ignores, so reporting an interaction doesn't also count as traffic. The dashboard grows an events block with per-referrer attribution. Email is scaffolded but sends nothing yet. mailer.py talks stdlib smtplib to a local Postfix null client on 127.0.0.1:25 (deploy/provision-mail.sh), so the choice between direct-to-MX and relaying through a provider stays a Postfix config change with no code change. The backend defaults to "console", which logs and sends nothing, so dev and tests exercise the whole signup path safely. Signups land in pending_digest unconfirmed; collecting the list shouldn't wait on delivery. Also adds /privacy, linked from the footer and kept out of the sitemap. The strip's classes are named unusual-* rather than record-*: .record-card is already the SEO records page's, and reusing it leaked layout rules onto /climate/<slug>/records.
2026-07-18 07:39:47 +00:00
postconf -e "inet_protocols = ipv4"
listen="127.0.0.1"
networks="127.0.0.0/8 [::1]/128"
if [[ -n "$DOCKER_MAIL_GATEWAY" ]]; then
listen="${listen}, ${DOCKER_MAIL_GATEWAY}"
networks="${networks} ${DOCKER_MAIL_SUBNET}"
# ufw is default-deny incoming; a container connecting to the host's gateway IP
# hits the INPUT chain, so allow the bridge subnet to reach port 25.
command -v ufw >/dev/null 2>&1 && \
ufw allow from "${DOCKER_MAIL_SUBNET}" to any port 25 proto tcp \
comment 'app container -> host Postfix' || true
fi
if [[ -n "$MESH_MAIL_LISTEN" ]]; then
listen="${listen}, ${MESH_MAIL_LISTEN}"
networks="${networks} ${MESH_MAIL_PEERS}"
fi
if [[ "$listen" == "127.0.0.1" ]]; then
postconf -e "inet_interfaces = loopback-only"
else
postconf -e "inet_interfaces = ${listen}"
fi
postconf -e "mynetworks = ${networks}"
Rebuild the homepage as a distribution landing; add an SMTP seam (#178) The homepage was a bare tool: a find-bar and an empty panel reading "Find a location to begin." A visitor arriving from a search result or a shared link learned nothing about what the product does before deciding to leave. Rebuild it around the Weekly view, which is untouched: - Hero with the question as the h1, and a grade card showing a real graded example — the most unusual city we're currently tracking, either tail. The card's frame and text slots are server-rendered with reserved heights, so app.js re-pointing it at the visitor's own place shifts nothing. - "Unusual right now" strip, CSS scroll-snap, no JS carousel. A cold-tail city is force-included whenever one qualifies. - Stance line, how-it-works, explore cards, and 12 city chips linking into the ~1000-page /climate surface. - Monthly digest form, in the footer of every page. Serve / from Jinja instead of a static file with placeholder substitution, so crawlers and no-JS readers get the whole page as real HTML. home.html.j2 extends base.html.j2 and carries app.js's DOM contract over verbatim; the brand degrades to a <p> so the hero owns the sole h1. frontend/index.html is deleted rather than left behind the static mount, where it would keep serving indexable duplicate content. "Where is it most unusual right now" has no cheap answer at request time — percentiles live inside zlib-compressed payload blobs with no column to sort on. So homepage.py sweeps the warm cache and writes data/homepage.json, read by the template. The sweep is strictly cache-only (climate.load_cached_recent_forecast is new, the sibling of load_cached_history), so grading ~1000 cities costs zero upstream requests. It rides the notifier's timer behind an hourly guard rather than starting a second daemon, and also runs at the tail of warm_cities so a fresh deploy has a populated feed. Instrumentation: metrics gains a product-event counter keyed by (event, referrer domain, UTC day), behind an allowlist and a per-IP rate limit, fed by POST /api/v2/event. The referrer is taken from the request's own header, never from the client. The beacon gets its own inbound category that record_inbound ignores, so reporting an interaction doesn't also count as traffic. The dashboard grows an events block with per-referrer attribution. Email is scaffolded but sends nothing yet. mailer.py talks stdlib smtplib to a local Postfix null client on 127.0.0.1:25 (deploy/provision-mail.sh), so the choice between direct-to-MX and relaying through a provider stays a Postfix config change with no code change. The backend defaults to "console", which logs and sends nothing, so dev and tests exercise the whole signup path safely. Signups land in pending_digest unconfirmed; collecting the list shouldn't wait on delivery. Also adds /privacy, linked from the footer and kept out of the sitemap. The strip's classes are named unusual-* rather than record-*: .record-card is already the SEO records page's, and reusing it leaked layout rules onto /climate/<slug>/records.
2026-07-18 07:39:47 +00:00
# A null client delivers nothing locally; everything is relayed out.
postconf -e "mydestination ="
postconf -e "local_transport = error:local delivery is disabled"
# Use TLS opportunistically when talking to the next hop.
postconf -e "smtp_tls_security_level = may"
postconf -e "smtp_tls_loglevel = 1"
if [[ -n "$RELAYHOST" ]]; then
echo "==> configuring relay via ${RELAYHOST}"
postconf -e "relayhost = ${RELAYHOST}"
if [[ -n "$RELAY_USER" ]]; then
postconf -e "smtp_sasl_auth_enable = yes"
postconf -e "smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd"
postconf -e "smtp_sasl_security_options = noanonymous"
printf '%s %s:%s\n' "$RELAYHOST" "$RELAY_USER" "$RELAY_PASSWORD" \
> /etc/postfix/sasl_passwd
# The credential file must not be world-readable.
chmod 600 /etc/postfix/sasl_passwd
postmap /etc/postfix/sasl_passwd
chmod 600 /etc/postfix/sasl_passwd.db
fi
else
echo "==> no RELAYHOST set: delivering direct to MX"
echo " remember SPF + DKIM + DMARC + PTR, or expect the spam folder"
postconf -e "relayhost ="
fi
systemctl enable postfix
systemctl restart postfix
echo "==> verifying it listens on loopback only"
ss -lntp | grep ':25 ' || true
cat <<'NOTE'
==> next steps
1. Point the app at it, in /etc/thermograph.env:
THERMOGRAPH_MAIL_BACKEND=smtp
THERMOGRAPH_SMTP_HOST=127.0.0.1
THERMOGRAPH_SMTP_PORT=25
THERMOGRAPH_MAIL_FROM=Thermograph <no-reply@thermograph.org>
then: sudo systemctl restart thermograph
2. Send yourself a test message:
echo "test body" | mail -s "thermograph test" you@example.com
# or, exercising the app's own path:
# python -c "import sys; sys.path.insert(0,'/opt/thermograph/backend'); \
# import mailer; print(mailer.send('you@example.com','t','body'))"
3. Watch it leave: journalctl -u postfix -f (queue: mailq)
4. Check placement with https://www.mail-tester.com — it scores SPF, DKIM,
DMARC and rDNS in one shot and tells you exactly what's missing.
NOTE