Commit graph

2 commits

Author SHA1 Message Date
Emi Griffith
c38b42e0f1 Parallelize lake tile uploads
All checks were successful
PR build (required check) / changes (pull_request) Successful in 10s
secrets-guard / encrypted (pull_request) Successful in 8s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 56s
PR build (required check) / gate (pull_request) Successful in 2s
A tile is ~1.2k small parquet objects; sequential PUTs to the bucket are
round-trip-bound (measured 2.5 obj/s, ~8 min per tile — ~78 h for the city
stage). A 16-thread pool per tile turns that into well under a minute; boto3
clients are thread-safe and failures surface via future.result().
2026-07-23 14:30:57 -07:00
emi
370a4ffdc1 ERA5 lake: bucket-hosted history primary + SQL indexer service (#15)
All checks were successful
secrets-guard / encrypted (push) Successful in 10s
Deploy backend to LAN dev server / build (push) Successful in 1m13s
Build + push backend image (Forgejo registry) / build-push (push) Successful in 1m39s
Deploy backend to LAN dev server / deploy (push) Successful in 34s
2026-07-23 21:20:35 +00:00