Lake /query reads era5_daily through Iceberg metadata #44

Merged
admin_emi merged 1 commit from fix/lake-query-iceberg into dev 2026-07-24 19:09:23 +00:00

1 commit

Author SHA1 Message Date
Emi Griffith
38b11635a6 Lake /query reads era5_daily through Iceberg metadata
All checks were successful
secrets-guard / encrypted (pull_request) Successful in 5s
PR build (required check) / changes (pull_request) Successful in 8s
shell-lint / shellcheck (pull_request) Successful in 7s
PR build (required check) / build-frontend (pull_request) Has been skipped
PR build (required check) / validate-observability (pull_request) Has been skipped
PR build (required check) / build-backend (pull_request) Successful in 52s
PR build (required check) / gate (pull_request) Successful in 2s
The hive-glob view must LIST every object under era5/daily/ at view bind —
~1k objects per tile, which crossed the request timeout once the lake hit
390 tiles (seen live: prod /query timing out). Bucket mode now resolves the
newest Iceberg metadata JSON (one single-page LIST) and iceberg_scans it, so
the file list comes from table metadata; local mode (tests, small trees)
keeps the plain glob. The iceberg extension is baked next to httpfs, as the
runtime user.
2026-07-24 12:07:55 -07:00