Skip to content

Volume Budget Ledger

Phase 9 (Stream E — "the actual filling", docs/plan/phase-9-data-foundation.md) requires an expected raw + processed storage footprint per data pack, recorded before download, so the phase exit criterion can audit Hetzner disk usage against what was planned rather than discovering it after the fact. This file is that ledger.

Convention for the builders of #612–#620 — READ BEFORE EDITING

This table has one pre-stubbed row per wave-5 pack issue already. Nine issues (#612–#620) edit this file concurrently with each other and with any later pack issue. To avoid nine builders colliding on the same appended line:

  • Fill in your issue's existing row in place. Find the row whose Issue column matches your issue number and replace its placeholder cells with real values.
  • Do not append a new row. The row for your issue already exists below.
  • Do not reorder, renumber, or reformat other rows. Touch only your own line(s) — a table-wide reformat (e.g. realigning | columns) turns every other builder's small diff into a conflict.
  • Do not edit the Totals row's formula description — only its numbers, and only if you are genuinely the last pack to land (see "Totals" below). If you're unsure whether you're last, leave the totals row's numbers alone and just update your own row; whoever lands last recomputes it.
  • Every volume figure must trace to something read from the provider's own documentation, or be explicitly labelled an estimate with its basis stated inline. Never fill a cell with an invented number — an honest placeholder (TBD — pending ops download) is correct; a fabricated figure is not. This mirrors the provenance rule issue #621 established after this repo shipped mis-attributed flood curves (ADR-059): every number here must be traceable to a source you actually read.
  • Record the licence as the provider states it, not as assumed or copied from an unrelated document.

Columns

  • Issue / Pack — the GitHub issue and its one-line pack description.
  • Provider — the upstream data provider/publisher, as named in their own documentation.
  • Licence — the provider's own stated licence/reuse terms (SPDX id where one applies, otherwise the provider's own wording). Not assumed, not copied from a sibling pack.
  • Raw volume (est., transient peak GB)this column measures transient peak local-disk headroom during a pack's download-then-clip step; it is NOT object-storage cost, and the Totals row is not a storage bill. Per ADR-070 the raw zone is a transient staging concept, not an archive: every pack streams its source download to local disk, clips/processes it there, ingests the result into DB rows or the processed zone, then deletes the local copy. Nothing archives the download into the bucket's raw/ prefix, which holds only .keep + catalog.v1.json (~0.03 MB) — see the "Raw-zone reality check" below. The reproducibility artifact is the per-dataset source manifest at hazards/{dataset_id}/source_manifest.json (ADR-066), not a retained raw copy. Recorded before download per the phase exit criterion. Label anything not read from the provider's own published file listing as an estimate, with the basis stated in Notes.
  • Processed volume (est., GB) — expected size of the committed, ingest-pipeline output: the hazards/{dataset_id}/intensity.npz the chord commits (ADR-049) plus, where the pack also produces one, the processed/{org}/{dataset_id}/hazard.tif COG (ADR-029). Exposure packs have no hazard COG; note that in the Notes column instead of leaving the cell blank.
  • Statusplanned (budget recorded, not yet downloaded) / downloaded (raw zone populated, manifest written) / ingested (committed through the unified pipeline, catalogued) — update as your pack's ops work (post-merge, out of this builder's reach) progresses. Buildable work in this repo can only ever reach planned, since download/ops steps need the live Hetzner host.
  • Notes — basis for any estimate, caveats, source URL/citation pointer.

Totals

The Totals row sums the numeric raw/processed columns across every row above it that carries a real number (skip rows still at their pending placeholder). It is not kept live by automation — there is no CI job or script that recomputes it. Whoever edits the last remaining pending row in a merge wave should recompute and update the Totals row's numbers in that same PR. Until then the Totals row is understated by construction (it only sums what's filled in so far) — that is expected, not a bug; do not "fix" it by guessing at unfilled rows.

Raw-zone reality check — read this before treating the raw-volume column or the Totals row as a storage bill. Ratified as a deliberate design decision in ADR-070 (2026-08-10), which also amends Phase 9's exit criterion 3 so it no longer demands a raw-zone copy that does not exist for any of the ten packs. Verified live against the Hetzner bucket during the 2026-08-08 ops run (see docs/plan/phase-9-ops-plan.md OPS-5d): the raw/ prefix holds only two objects — .keep and catalog.v1.json, ~0.03 MB total — and the whole bucket was 0.325 GB, against this ledger's own raw-column sum of ≈24.6 GB (see the Totals row below). Re-measured 2026-08-10 after the NetCDF ops run and the trash/ purge: 158 objects / 359.87 MB (0.360 GB) in the bucket, against a raw-column sum of ≈24.87 GB — the same two-orders-of-magnitude gap, for the same reason. The raw/ prefix is still 2 objects / 0.04 MB. That is not a discrepancy to chase down; it is how this pipeline is built to behave. Every pack in this ledger streams its source download to local disk, clips/processes it there, and ingests the result straight into DB rows or the processed-zone artifact (hazards/{id}/intensity.npz) — then deletes the local download. Nothing in the current pipeline archives the raw download into the bucket's raw/ zone. The "Raw volume (est., GB)" column and the Totals row's raw figure therefore measure a transient — peak local-disk headroom needed during a pack's download-then-clip step — not bytes stored in the bucket, and neither should ever be read as a projection of object-storage cost.

Ledger

Issue Pack Provider Licence Raw volume (est., transient peak GB — not stored, see Columns + ADR-070) Processed volume (est., GB) Status Notes
#611 River flood — JRC Copernicus Emergency Management Service / European Commission Joint Research Centre — "River Flood Hazard Maps for Europe and the Mediterranean Basin" v3.1.1 CC-BY-4.0 (data.jrc.ec.europa.eu collection id-0054; recorded verbatim in scripts/historical_catalog_manifest.jsonperils.river_flood.source.license/.license_url) ≤ 3.7 (estimate, upper bound — see note a) TBD (see note a) ingested (a) Nine return periods (RP10..RP500). The upstream per-RP full-"European extended domain" GeoTIFF is documented at ~260–330 MB in this repo (scripts/ingest_historical_catalog.py module docstring, verified against the live source 2026-07-27) → ~2.3–3.0 GB for all 9 RPs combined, but that full-domain file is never downloaded whole: scripts.packs.river_flood_jrc.run_ingest_hazards (via fetch_clip) does a rasterio windowed HTTP-range read of only the Greece bbox [19.0, 34.5, 28.5, 42.0]. At the source's native 3 arc-second (~90 m) resolution that bbox is 11 400 × 9 000 px/RP (≈1.03×10⁸ px); an uncompressed single-band float32 clip is ≈0.41 GB/RP × 9 ≈ 3.7 GB — a calculated upper bound, not a measurement, and the figure recorded above. fetch_clip writes with compress="deflate", and flood-depth rasters are mostly nodata/zero outside inundated cells, so the real, compressed footprint actually landing in the raw/staging zone is expected to be meaningfully smaller than this bound; no live download ran in this build to measure it, so no smaller point figure is asserted — that would be a guess, not a measurement. Processed (hazards/{id}/intensity.npz + processed/{org}/{id}/hazard.tif) is the ingest chord's own encoding of the same clipped data and is not sized here for the same reason — TBD pending the first real ops ingest, which needs a live Hetzner host and is out of reach in this build (no live network/DB here). Update 2026-08-10 (ops close-out) — this pack's national baseline had been sitting in prod with no COG at all, and now has one. JRC river flood Greece (national, historical baseline), id 67854ef5-8052-435d-9828-1a1015146654, had been active since 2026-08-07 with its intensity.npz intact but cog_path NULL. It is the same dataset that originally triggered #754 (a classic-TIFF write overflowing 4 GB — 11,400×9,000 × 9 bands ≈ 3.7 GB uncompressed, which is exactly this row's own calculated raw bound arriving as a real failure). Re-dispatching build_dataset_cog from the retained NPZ succeeded in 141.6 s, 9 bands, zero GDAL errors, so #754's fix is now confirmed in production against the very dataset that broke it — not merely against a test fixture. This is the entire reason the bucket's processed/ zone grew 89.77 MB → 158.75 MB on 2026-08-10 (see the Totals note (c)); by difference the COG is ≈68.98 MB, which is a derived figure, not a directly measured object size. The row's processed cell stays TBD because that derivation covers the COG only, not this pack's intensity.npz.
#612 River flood alt — Aqueduct Floods v2 World Resources Institute — "WRI Aqueduct Floods Hazard Maps (version 2)" (scripts/scenario_hazards_manifest.jsonperils.river_flood.source.name) CC-BY-4.0 — "All the products, methodologies, and datasets that make up Aqueduct are available for use under the Creative Commons Attribution International 4.0 License", read live from wri.org/aqueduct in this build; matches what the manifest already recorded (.source.license/.license_url) and docs/concepts/eiopa-scenario-mapping.md's verification record (checked against WRI pages + the GEE catalog entry on 2026-07-27) ≤ 0.09 (estimate, upper bound — see note a) TBD (see note a) ingested (a) 7 datasets (2 scenarios × 3 horizons + 1 historical baseline) × 3 return periods (RP10/50/100) = 21 source files, never downloaded whole: scripts.packs.river_flood_aqueduct.run_ingest_hazards (via fetch_clip, shared with the JRC pack) does a rasterio windowed HTTP-range read of only the Greece bbox [19.0, 34.5, 28.5, 42.0]. docs/concepts/eiopa-scenario-mapping.md's verification record documents a real (not calculated) windowed test read of the RCP8.5/2080/RP100 file on 2026-07-27, returning a measured 1140×900 px tile at the source's native 0.008333°/~30 arc-second resolution; the other 20 files share the same bbox and resolution, so the same tile size is assumed uniform across them (not independently re-measured here). An uncompressed single-band float32 clip at that size is 1140×900×4 B ≈ 4.10 MB/file × 21 ≈ 86 MB ≈ 0.09 GB — a calculated upper bound from one measured tile, not a full download. fetch_clip writes with compress="deflate" and flood-depth rasters are mostly nodata outside inundated cells, so the real compressed footprint landing in the raw/staging zone is expected to be smaller than this bound; no live download ran in this build to measure it, so no smaller point figure is asserted. For reference, the full global per-RP GeoTIFFs (never downloaded) were HEAD-verified at ~88–93 MB each (docs/concepts/eiopa-scenario-mapping.md) → ~21×90 MB ≈ 1.9 GB if fetched whole, which this pack avoids entirely via the windowed clip. Processed (hazards/{id}/intensity.npz, no separate COG for this scenario-conditioned pack) is the ingest chord's own encoding of the same clipped data and is not sized here for the same reason as #611 — TBD pending the first real ops ingest, which needs a live Hetzner host and is out of reach in this build (no live network/DB here).
#613 Coastal flood — LISCOAST + Aqueduct coastal World Resources Institute — "WRI Aqueduct Floods Hazard Maps (version 2)", coastal variant (scripts/packs/coastal_flood.pyCOASTAL_SOURCE); JRC LISCOAST evaluated but NOT ingested (see note a) CC-BY-4.0 — Aqueduct only (WRI's own statement re-verified live 2026-08-06 against wri.org/aqueduct, unchanged from #612's recorded text) ≤ 0.09 (estimate, upper bound — Aqueduct coastal only, see note a) TBD (see note a) ingested (Aqueduct coastal half only — LISCOAST stays permanently out of scope per ADR-062, see note a) (a) LISCOAST is evaluated but deliberately NOT wired into ingest — the issue's own context line ("LISCOAST format likely NetCDF") does not hold for the real artifact: this build downloaded the JRC LISCOAST "Global Extreme Sea Level projections" archive whole (small enough under the no-multi-GB rule — https://cidportal.jrc.ec.europa.eu/ftp/jrc-opendata/LISCOAST/10012/LATEST/globalESLprojections.zip, 13,351,354 bytes, measured via both a live HTTP HEAD and the actual downloaded+inspected body in this build) and inspected it directly: its own README.txt states "The data are provided in csv files for all combination of parameter, year and RCP", and every one of its 24 CSVs is latitude,longitude,valueMedian,value5th,value95th — irregularly-spaced coastal transect points, not a raster. This platform's hazard-ingest formats (GeoTIFF; NetCDF per #593/ADR-051) both require a regular grid, so neither can ingest scattered points without a new point/vector IngestAdapter (Stream B scope, out of reach of this Stream E pack issue). Recorded as an honest evaluated-source (scripts/packs/coastal_flood.py::LISCOAST_EVALUATED_SOURCE), not silently dropped — mirrors the evaluated_sources shape the scenario manifest already uses for its wildfire/windstorm gaps. Decision final (ADR-062, 2026-08-07): LISCOAST is permanently out of scope for hazard ingest — coastal flood ships computable on Aqueduct coastal alone, so no point→grid ingest path is built; reversible by superseding ADR-062 if a point-published hazard source later becomes load-bearing. Aqueduct coastal (inuncoast_*), by contrast, ingests exactly like the riverine variant (#612): 7 datasets (2 scenarios × 3 horizons + 1 historical baseline) × 3 return periods (RP10/50/100) = 21 source files, every URL HTTP HEAD-verified live in this build (2026-08-06, all 200 image/tiff), never downloaded whole — scripts.packs.coastal_flood.run_ingest_hazards (via fetch_clip, the same helper #611/#612 already share) does a rasterio windowed HTTP-range read of only the Greece bbox [19.0, 34.5, 28.5, 42.0]. That windowed read was actually performed once in this build (not just calculated) against inuncoast_rcp8p5_wtsub_2080_rp0100_0.tif: a real measured 900×1140 px tile at the source's native 0.008333°/~30 arcsec resolution — identical grid to the riverine variant's own measured tile (#612), so the same per-file size is assumed uniform across the other 20 files (not independently re-measured here). An uncompressed single-band float32 clip at that size is 900×1140×4 B ≈ 4.10 MB/file × 21 ≈ 86 MB ≈ 0.09 GB — an upper bound from one measured tile, not a full download; fetch_clip writes with compress="deflate" and coastal-depth rasters are mostly nodata outside inundated cells, so the real compressed footprint is expected to be smaller. Subsidence variant fixed at wtsub (WRI's methodology documents both wtsub/nosub without a stated default — recorded as an [ASSUMPTION] in the pack module, not a missing option: nosub also HEAD-verified 200 in this build). Processed (hazards/{id}/intensity.npz) is not sized here for the same reason as #611/#612 — TBD pending the first real ops ingest on a live Hetzner host, out of reach in this build.
#614 Wildfire FWI — GEFF/ERA5 historical (EWDS) + C3S projections (CDS) — indicator-only, Refs #481: no EAI on FWI, ever ECMWF / Copernicus Emergency Management Service (CEMS), distributed via EWDS — cems-fire-historical-v1 (historical); Copernicus Climate Change Service (C3S) / ECMWF, distributed via CDS — sis-tourism-fire-danger-indicators (projections), both verified live against docs/DATA_SOURCES.html Sec 4.3 and docs/concepts/eiopa-scenario-mapping.md's own prior verification of the projections dataset EWDS historical: CC-BY family (its own landing page states "CC-BY family (Copernicus licence; per-dataset acceptance at download)" — docs/DATA_SOURCES.html Sec 4.3). CDS projections: TBD — see note a; its own landing page carries no licence line at all (unlike its EWDS sibling), so it is not assumed to match 0.0295 (measured 2026-08-10 — supersedes the ≤0.0002 calculated bound, which is kept visible in note a) 0.0000141 (measured 2026-08-10 — EWDS half only; the C3S half has no processed artifact because it is not ingested) partially ingested — EWDS historical ingested in prod 2026-08-10; C3S projections downloaded but NOT ingested (rotated-pole grid, see note a) (a) Neither dataset was downloaded in this build — both are authenticated, queue-based Retrieve-API stores with no anonymous HTTP HEAD path, so no live measurement is possible without a credentialed ECMWF account (matching this issue's own PROVENANCE constraint). Both figures below are calculated upper bounds from the datasets' own stated resolutions (docs/DATA_SOURCES.html Sec 4.3), clipped to the Greece bbox [19.0, 34.5, 28.5, 42.0], uncompressed float32, no compression credit taken (scripts/packs/_wildfire_fwi_common.py::estimate_raw_volume_bytes, unit-tested against synthetic fixtures). EWDS historical (scripts/packs/wildfire_fwi_ewds.py): ~0.25° grid → 38×30 = 1140 cells; this pack requests one reference date per invocation (not the full 1940→present daily series — see the pack's module docstring for why), so 1 slice × 1140 cells × 4 B ≈ 4.6 KB, effectively 0.0000045 GB. C3S projections (scripts/packs/wildfire_fwi_c3s.py): ~0.11° grid → 87×69 = 6003 cells; one retrieval covers all 3 scenarios (RCP2.6/4.5/8.5) — this pack's NetCDFDatasetSpec.supported_years is deliberately left undeclared (derived from the file, not hard-coded, since the exact year cadence to 2098 could not be verified without a live download) so this estimate assumes the platform's existing 3-horizon convention (2030/2050/2080, matching Aqueduct/#612) as a stated planning basis, not a confirmed file layout: 3 scenarios × 3 horizons × 6003 cells × 4 B ≈ 211 KB, effectively 0.0002 GB. Both totals stay well under 1 GB even if the real per-dataset time-step cadence turns out finer than assumed, because the Greece bbox clip is small at either resolution. Processed volume is TBD for the same reason as #611/#612 (no live ops ingest in this build) and because dispatch through the unified pipeline is currently blocked by a real platform gap this build found and filed as #706: resolve_ingest_adapter resolves every registered HazardType — including wildfire — to the GeoTIFF adapter regardless of the staged file's actual format, so a NetCDF FWI ingest cannot complete through the chord until that lands (both packs' run_ingest_hazards fail fast with an actionable error naming the issue rather than dispatching into a chord that would fail deep in the worker; this same gap is independently corroborated by #615's own row/PR below). Both packs are otherwise complete and unit-tested (tests/scripts/test_wildfire_fwi_ewds_pack.py, test_wildfire_fwi_c3s_pack.py, test_wildfire_fwi_common.py): NetCDF spec registration, request construction, and the CDS/EWDS retrieval driver (CdsApiClient/CdsRetrieval, exercised against a scripted httpx.MockTransport, no live credentials). Per-dataset licence acceptance (both stores) and the actual raw-zone download remain ops-only, out of this build's reach (no network/credentials here) — this issue's buildable acceptance checks do not require a completed ops ingest. Update 2026-08-10 (supersedes the 2026-08-08 line this row used to carry, which said "#761 is now CLOSED" — it is not). As of trunk d4b2cc5, no known CODE blocker remains: #706 (PR #717), #761's two worker-side gaps (ADR-064), #782 (PR #794), #791 (PR #798, ADR-067), #793 (PR #800, ADR-068) and #797 (PR #804, ADR-069) are all fixed. [SUPERSEDED 2026-08-10 (late), kept verbatim because this row's own convention is to show what was believed and when — both sentences below are now FALSE: "#761 itself is deliberately still OPEN, repurposed as the end-to-end evidence gate. The assert_netcdf_adapter_wired guard therefore still refuses every dispatch, by design — not for a missing fix, but because no NetCDF ingest has ever been staged, chunked and committed through the chord in any environment." What is true instead: #761 is CLOSED, and PR #825 (42f2a35) deleted layer 2 of assert_netcdf_adapter_wired — the evidence gate — along with its one-shot --first-end-to-end-run escape and the ADAPTER_GAP_ISSUE_URL constant. The guard that survives on trunk performs the plan-time adapter-resolution check only, and refuses nothing that resolves to the right reader. The evidence the gate demanded was produced by the first real end-to-end NetCDF ingest, recorded in the closing update at the end of this row.] Read NETCDF_INGEST_BLOCKERS in scripts/packs/_wildfire_fwi_common.py on trunk before citing a blocker from this row (ADAPTER_GAP_ISSUE_URL no longer exists to read). CDS/EWDS credentials ARE present in the worker (verified live 2026-08-08: CDSAPI_KEY, EWDSAPI_KEY, CDSAPI_URL, EWDSAPI_URL all set) — credentials are not what is blocking this row. Update 2026-08-09 (#802): the first successful retrieval this project ever ran was against the EWDS half of this rowcems-fire-historical-v1, jobID fd5475dd-c034-4ff5-a696-83e8314a3952, 201 → successful in ~24 s → application/netcdf, 28402 bytes measured for a 33×45-cell single-date Greece clip. That supersedes the EWDS estimate above with a measurement (~28 KB compressed vs the ~4.6 KB uncompressed float32 calculation — the delivered file carries CF metadata and a valid_time axis the estimate did not account for). The C3S projections estimate above is now known to be too low: sis-tourism-fire-danger-indicators declares no area request input, so it cannot be clipped server-side and the full European domain is delivered and clipped locally — the ~0.0002 GB Greece-clip figure describes the stored result, not the download. The download total stays TBD: sizing it needs gcm_model/period/time_aggregation/version, whose enums were not captured, so the pack refuses to build a request at all rather than guess. Update 2026-08-10 (#614) — supersedes the "TBD" immediately above, and the C3S half of this row is now MEASURED. All five missing enums were captured, the process's constraints.json was captured with them, and the first C3S retrieval this project has ever run was submitted and downloaded: job 520e1990-6fa4-451d-ba81-2b67450a48aa, 201 → successful, 29 435 326 bytes of application/zip. Request: time_aggregation=seasonal_indicators, product_type=multi_model_mean_case, variable=[seasonal_fire_weather_index], experiment=[rcp2_6,rcp4_5,rcp8_5], period=[2026_2030], version=v2_0, no gcm_model (every multi_model_* constraint block omits it). Note daily_fire_weather_index — what the pack asked for until now — is reachable only with single_model+daily_indicators and can never be combined with a multi-model product type. Download sizing: ~29.4 MB per (3-experiment × one 5-year band) retrieval, so the full 3-horizon pack is ~88 MB — three orders of magnitude above the 0.0002 GB Greece-clip figure, because there is no server-side area and the whole 424×412 European domain arrives. Two new blockers, both measured by opening the file, now stop the ingest regardless of #761: (1) the asset is a ZIP holding one NetCDF per experiment, not the single .nc the dispatch path passes to the worker — and there is no scenario dimension in any member, the scenario is a filename token; (2) the members are on a rotated-pole grid (grid_mapping_name=rotated_latitude_longitude, north pole 39.25 N / −162.0 E) and NetCDF ingest is EPSG:4326-only with no reprojecting read. Worse, before #614 that second one was silent: rlon/rlat are evenly spaced, so the file inspected cleanly as EPSG:4326 with its origin at lon −28.43 / lat 21.89 — the mid Atlantic. #614 added an explicit CF grid_mapping_name guard to core/ingest/netcdf.py that fails it closed, corrected the layout to the measured fwi-mean-jjas(time, rlat, rlon), and made wildfire_fwi_c3s.run_ingest_hazards refuse with UNINGESTABLE_REASON — a refusal the --first-end-to-end-run escape deliberately cannot waive. Unzipping and reprojection are the outstanding work for this row. Update 2026-08-10 (ops run, prod) — the EWDS half of this row is INGESTED, and every figure below is MEASURED on the production host, not calculated. hazard_datasets row wildfire-fwi-historical-greece-20230801, id c83be4f5-5bb1-4bef-82d8-a100061163cc, haz_type=wildfire, status=active, COG present — the first NetCDF ingest this project has ever completed end to end through the unified pipeline, and the evidence that retired the guard's evidence gate (see the SUPERSEDED block above). Raw download measured: 27,990 bytes, delivered as a plain NetCDF (data/wildfire_fwi_ewds/wildfire-fwi-historical-greece-20230801.nc) — not a ZIP, unlike every C3S delivery on this row and on #616. This is a different number from the 28,402 bytes recorded above, and neither supersedes the other: 28,402 B was the 2026-08-09 probe retrieval (#802, jobID fd5475dd-…), 27,990 B is the 2026-08-10 ops retrieval that was actually ingested — two separate submissions on two separate days, both recorded because the ~1.5% delta is real delivery variation, not a correction. Stored footprint measured: hazards/ 0.01 MB + processed/ 0.00 MB ≈ 14,125 bytes total — the figure now in this row's Processed cell. The C3S projections half is still NOT ingested, and exactly one blocker remains. The ZIP-per-experiment problem recorded above is FIXED — PR #822 (ffab274, ADR-071) makes the source layer unpack whatever archive a CDS retrieval actually delivers and dispatch one ingest per member, so the scenario-as-filename-token layout is no longer an obstacle. What still blocks it is the rotated-pole grid: every member declares CF grid_mapping_name=rotated_latitude_longitude (north pole 39.25 N / −162.0 E), and NetCDF ingest is EPSG:4326-only with no reprojecting read — PR #819 (02f0ee3) added the explicit grid_mapping_name guard that fails this closed rather than silently ingesting the mid-Atlantic misplacement described above. Reprojection is therefore the single outstanding item for the C3S half of this row — not the guard, not the ZIP, not the request vocabulary, all three of which are now resolved.
#615 Windstorm — C3S footprints + WISC synthetic Copernicus Climate Change Service (C3S) / ECMWF — Climate Data Store (cds.climate.copernicus.eu); sis-european-wind-storm-reanalysis ("Windstorm tracks and footprints derived from reanalysis over Europe between 1940 to present") + sis-european-wind-storm-synthetic-events ("Synthetic windstorm events for Europe from 1986 to 2011", the WISC set) CC-BY licence (see note a for the WISC caveat) 0.000055 (measured 2026-08-10, C3S historical as actually retrieved — supersedes the ≤0.04 per-footprint calculated bound, kept visible in note b) / ≤ 142.6 (WISC full synthetic set, upper bound — never retrieved, see note b) 0.000125 (measured 2026-08-10, C3S historical half — see note b) ingested (C3S historical half only — WISC synthetic not retrieved for Phase 9, see note b) (a) Both dataset ids and their "CC-BY licence" wording were read live 2026-08-06 from each dataset's own CDS landing page (cds.climate.copernicus.eu/datasets/<id>?tab=overview), not assumed or copied from a sibling pack. The WISC synthetic-events product's own Confluence "Product User Guide" (fetched live 2026-08-06) has a "Licence"/"Terms of Use" row in its overview table that is left blank in the document itself — the CDS landing page is the only live source naming a licence for WISC in this build. Per this issue's own instruction ("WISC ... has its own licence terms distinct from general Copernicus/C3S terms — read them, do not assume CC-BY"), ops must re-confirm the real click-through licence text on the CDS portal when accepting it (the existing ops checklist item "Per-dataset license accepted (portal)" already covers this) before treating CC-BY as final. Update 2026-08-10 — that ops action is MOOT for Phase 9 and should not be carried forward as an open item: WISC is not retrieved at all (see note b's 2026-08-10 update), so no click-through licence is ever accepted for it and there is nothing for ops to re-confirm. The instruction stands only if a future pass decides to retrieve WISC; the C3S historical half that was retrieved carries its own separately-read CC-BY line. (b) CDS has no static file URL to HEAD — every retrieval is an authenticated, queue-based submission (CopernicusCdsSource.download), so unlike JRC/Eurostat/GHSL (#611/#619/#620) there is no anonymous byte count to measure even in principle; both figures below are calculated upper bounds from documented grid dimensions, not measurements. C3S historical footprint: 0.016° grid over the dataset's own documented domain (25W-35E, 30N-70N, from the CDS overview page) = 3750×2500 = 9,375,000 cells; one uncompressed float32 band ≈ 37.5 MB per footprint — the total for "1940 to present" depends entirely on how many storms/years the operator's retrieval request spans (an operator scoping decision made at retrieval time, not fixed by this pack, and this build found no documented total storm count for this specific dataset id to multiply by), so the dataset-wide total is TBD, not calculated, pending that scoping decision. WISC synthetic set: the PUG's own documented domain (25W-40.5E, 34.4N-71.5N) at 4.4 km ≈ 0.0396°/pixel (111 km/degree, a stated approximation — the true per-pixel footprint likely differs slightly by latitude and this was not corrected for) ≈ 1654×937 ≈ 1.55M cells/footprint; one uncompressed float32 band ≈ 6.2 MB × the PUG's own documented count of 22,980 footprints ≈ 142.6 GB if the entire synthetic set were retrieved whole — given the issue's own "no multi-GB downloads" constraint, ops is expected to scope real retrievals far below this (a handful of test events, or an API-side area subset to Greece if the live CDS form supports one — unconfirmed in this build, see scripts/packs/windstorm_c3s_wisc.py's module docstring), so this is a ceiling on the unscoped case, not a plan to actually fetch it. Processed volume (hazards/{id}/intensity.npz) is TBD for the same reason as every other row in this ledger (no live ops ingest ran in this build) and is additionally blocked on a cross-cutting wiring gap this build found but did not fix: worker/ingest/adapters.py::resolve_ingest_adapter resolves ingest format purely from HazardType (every entry typed BaseGeoTIFFIngestor), so STORM_EUROPE's NetCDF footprints cannot reach the unified pipeline's NetCDF reader (core/ingest/netcdf.py, already registered by this pack via register_netcdf_layouts()) until that resolution is wired to route on format, not just haz_type — the same gap the wildfire pack (#614, also CDS/EWDS NetCDF) will hit; see the PR for #615. Update 2026-08-10 (supersedes the 2026-08-08 line, which wrongly said "#761 is now CLOSED"): as of trunk d4b2cc5 no known CODE blocker remains — #706, #761's two worker-side gaps, #782, #791, #793 and #797 are all fixed (see #614's row for the fix table). [SUPERSEDED later the same day, kept verbatim: "#761 stays OPEN as the end-to-end evidence gate, and the assert_netcdf_adapter_wired guard still refuses every dispatch by design, because no NetCDF ingest has ever run end to end." Both halves of that are now FALSE — #761 is CLOSED and PR #825 (42f2a35) deleted the guard's evidence-gate layer, its --first-end-to-end-run escape and the ADAPTER_GAP_ISSUE_URL constant; only the plan-time adapter-resolution check remains, and it refuses nothing that resolves to the right reader. The evidence that satisfied the gate was #614's EWDS ingest — see that row.] Read NETCDF_INGEST_BLOCKERS on trunk, not this row. CDS/EWDS credentials ARE present in the worker (verified live 2026-08-08: CDSAPI_KEY, EWDSAPI_KEY, CDSAPI_URL, EWDSAPI_URL all set) — credentials are not what is blocking this row. Update 2026-08-10 (ops run, prod) — the C3S historical half of this row is INGESTED, and its volumes are now MEASURED rather than calculated. hazard_datasets row windstorm-c3s-footprints-hist, id 3f4f864d-01ab-485c-9f39-23c8856a0452, haz_type=storm_europe, status=active, COG present. One event committed: scenario NULL, year=2007, event_name=1463@2007, no frequency — a year-labelled, frequency-less single-footprint dataset, exactly the shape ADR-068/ADR-069 were written for. Grid 241×161 with 13,174 non-zero cells, 1 band. Raw download measured: the CDS delivery is a ZIP of 54,965 bytes holding one member of 54,713 bytes. That is three orders of magnitude below this row's calculated ≤0.04 GB (37.5 MB) per-footprint upper bound, and the bound is not wrong — it is answering a different question. It sized an unscoped whole-domain (25W–35E, 30N–70N) uncompressed float32 band at 3750×2500 cells; the retrieval ops actually submitted is scoped to a single storm, so what arrives is one small compressed footprint, not the domain-wide grid the bound assumed. The bound is retained above as the ceiling on the unscoped case; the 54,965-byte figure is what a real scoped retrieval costs. Stored footprint measured: hazards/ 0.08 MB + processed/ 0.04 MB = 0.12 MB (124,719 bytes) — the figure now in this row's Processed cell. WISC synthetic is DROPPED for Phase 9, on measured evidence — and the hope this row recorded is now confirmed FALSE. Note b above speculated that the ≤142.6 GB full-set figure could be scoped down "by ... an API-side area subset to Greece if the live CDS form supports one — unconfirmed in this build". It does not support one. The process schema was read live from GET {cds_root}/retrieve/v1/processes/sis-european-wind-storm-synthetic-events and publishes exactly four inputs — variable, version_id, year, month. There is no spatial input, no area, nothing geographic at all. The only scoping levers WISC offers are temporal, so any WISC retrieval fetches whole-domain footprints and cannot be brought under this plan's no-multi-GB rule at retrieval time. This is recorded as a measured negative, not a permanent decision: no ADR was written and none is implied — WISC is simply not pursued for Phase 9 because it cannot be scoped, and a future pass that is willing to fetch whole-domain footprints (or that finds a subsetting path this schema read did not show) is free to revisit it. The C3S historical half alone is what makes this row ingested.
#616 Heatwave — EURO-CORDEX + ERA5-HEAT alt Copernicus Climate Change Service (C3S) — primary: "Heat waves and cold spells in Europe derived from climate projections" (CDS dataset sis-heat-and-cold-spells); alt: "Climate indicators for Europe from 1940 to 2100 derived from reanalysis and climate projections" (CDS dataset sis-ecde-climate-indicators) — see note a CC-BY (both datasets — read live from each dataset's own CDS overview page 2026-08-06, not assumed or copied from a sibling Copernicus product) 0.238 (measured 2026-08-10 — 238,249,300 B of delivered ZIPs; replaces the TBD, see note b) 0.203 (measured 2026-08-10 — 203.33 MB across all three datasets, see note b) ingested (both datasets, all scenarios — with one open caveat, #827, see note b) (a) Both datasets are served through the Copernicus CDS's authenticated, queue-based Retrieve API (src/climate_lama/ingest/sources/copernicus_cds.py, issue #598), not a plain HTTPS URL — unlike #611/#612/#618-#620, there is no file listing an HTTP HEAD can size, and this build holds no CDS credentials to submit a retrieval. scripts/packs/heatwave_packs.py builds the two datasets' CdsRetrievalSpec-shaped registry entries (id/store/dataset/request/title) and round-trips them through the real copernicus_cds.parse_retrieval_specs (unit-tested, not a re-implementation) so they are provably consumable by the production adapter once pasted into the operator-declared DATASET_SOURCE_COPERNICUS_RETRIEVALS registry — that registration is a config concern (per this issue's scope fence), not a code change. The CDS request keys whose values this build could not verify (variable, experiment/period, format — chosen on the dataset's own JS-rendered, login-gated "Download" tab) are left as an explicit _UNRESOLVED_CONFIRM_FROM_CDS_DOWNLOAD_FORM sentinel rather than a guessed value (ADR-059); heatwave_packs.unresolved_fields() reports them so ops cannot silently paste an incomplete request. Update 2026-08-10 — those sentinels are RESOLVED. Both datasets' variable/experiment/period/format values were captured and the retrievals were actually submitted and delivered (see note b), so the unresolved-field discussion above is history rather than an outstanding owner input. [ASSUMPTION — PROMOTED TO A CONFIRMED PICK 2026-08-10, validated by execution rather than by re-reading a page; see note b. The reasoning below is retained in full, because it is still the record of why derived-utci-historical was rejected, and that rejection is what execution confirmed.] "ERA5-HEAT" (this issue's title; docs/plan/phase-9-data-foundation.md's Stream E table row "ERA5/ERA5-HEAT (CDS) — historical truth | 1940→ | CC-BY") is read here as sis-ecde-climate-indicators's own "Heatwave days" indicator (same "days" unit as the EURO-CORDEX primary, ERA5-reanalysis-based 1940→2100 span matching the plan doc), not the literal ERA5-HEAT paper dataset (Di Napoli et al. 2021, CDS id derived-utci-historical), which publishes a different physical quantity (UTCI thermal comfort, °C) — see heatwave_packs.py's module docstring for the full reasoning; ops/product confirmation is no longer outstanding — the 2026-08-10 ops run retrieved and ingested the sis-ecde-climate-indicators pick and it yields heatwave days in days over 1940–2025, the same physical quantity and unit as the EURO-CORDEX primary, which is exactly what the assumption predicted. CMIP6 extreme indices (the plan doc's other alternative, flagged non-standard licence) is not implemented by this pack — the issue title names ERA5-HEAT, and the licence-flag instruction is conditional on CMIP6 being the one picked. (b) No download ran in this build (no CDS credentials, and CDS retrievals cannot be sized by HTTP HEAD the way #611/#612/#618-#620's plain-HTTPS files were) — raw and processed volume are genuinely unknown, not merely estimated, until ops runs a real retrieval; recording TBD rather than a fabricated figure is the correct answer here per this issue's own instruction and ADR-059. Processed volume (hazards/{id}/intensity.npz) is not sized for the same reason. Registering the NetCDF ingest layout (core.ingest.netcdf.register_netcdf_dataset) needs the file's actual variable/dimension names, which exist only inside a downloaded sample — also out of reach here; that registration and the first real ingest through the unified pipeline are this issue's ops-only acceptance checks. Update 2026-08-10 (supersedes the 2026-08-08 line, which wrongly said "#761 is now CLOSED"): as of trunk d4b2cc5 no known CODE blocker remains — #706, #761's two worker-side gaps, #782, #791, #793 and #797 are all fixed (see #614's row for the fix table). [SUPERSEDED later the same day, kept verbatim: "#761 stays OPEN as the end-to-end evidence gate, and the assert_netcdf_adapter_wired guard still refuses every dispatch by design, because no NetCDF ingest has ever run end to end." Both halves are now FALSE — #761 is CLOSED and PR #825 (42f2a35) deleted the guard's evidence-gate layer, its --first-end-to-end-run escape and the ADAPTER_GAP_ISSUE_URL constant; the plan-time adapter-resolution check is all that remains. The satisfying evidence was #614's EWDS ingest.] Read NETCDF_INGEST_BLOCKERS on trunk, not this row. CDS/EWDS credentials ARE present in the worker (verified live 2026-08-08: CDSAPI_KEY, EWDSAPI_KEY, CDSAPI_URL, EWDSAPI_URL all set) — credentials are not what is blocking this row. Update 2026-08-10 (ops run, prod) — this row is INGESTED, both datasets, all scenarios, and every figure here is MEASURED on the production host. This replaces the TBD — pending ops download that stood in both volume cells since the row was authored. Three hazard_datasets rows, all status=active, all with a COG: (i) heatwave-euro-cordex-europe-ssp5-8.5, id 5857653d-4a63-40cc-95ce-3c2c19e54a01 — 100 events over 1986–2085, grid 599×425, 100 bands, stored hazards/ 57.92 MB + processed/ 43.33 MB = 101.25 MB; (ii) heatwave-euro-cordex-europe-ssp2-4.5, id 591da928-cee9-4693-a39d-6a4afbcf0f9f — 100 events over 1986–2085, grid 599×425, 100 bands, stored 57.87 + 42.95 = 100.82 MB; (iii) heatwave-ecde-europe, id 32fcd25e-59ec-4558-8b98-33365203cebd — 86 events over 1940–2025, grid 271×185, 86 bands, stored 0.70 + 0.56 = 1.26 MB. Processed total = 203.33 MB ≈ 0.203 GB. Raw downloads measured, both delivered as ZIPs: EURO-CORDEX returned 203,731,118 bytes holding two members of 101,849,863 bytes eachHWD_EU_health_rcp45_mean_v1.0.nc and HWD_EU_health_rcp85_mean_v1.0.nc (note the upstream filenames say rcp45/rcp85 while this catalog names the datasets ssp2-4.5/ssp5-8.5; the mapping is deliberate and the filenames are the provider's, not a mislabel on our side); ECDE returned 34,518,182 bytes holding one member of 34,512,676 bytes (09_heat_waves_climatological-reanalysis-yearly-grid-1940-2025-v2.0.nc). Raw total = 238,249,300 bytes ≈ 0.238 GB, the figure now in this row's Raw cell and folded into the Totals row's note (c). The two-members-in-one-ZIP delivery is handled by ADR-071/PR #822 — one ingest per member — which is why one retrieval produced two catalogued datasets. Known caveat — do NOT read this row as fully complete. #827 (filed 2026-08-10, OPEN): derive_dataset_axis discards the per-member scenario label at commit time, so both EURO-CORDEX datasets carry their scenario in the dataset name but have scenario IS NULL in the column. Consequence, stated plainly: heatwave cannot yet answer a scenario-labelled matrix cell — the data is ingested and the intensities are correct, but the scenario axis it was retrieved for is not addressable until #827 lands.
#617 Earthquake — ESHM20 hazard maps (EFEHR); ESRM20 vulnerability already seeded by #604, not re-derived here (see note a) EFEHR (European Facilities for Earthquake Hazard and Risk) / Swiss Seismological Service (SED), ETH Zurich — "The 2020 update of the European Seismic Hazard Model (ESHM20)" (Danciu et al. 2021, doi:10.12686/a15) CC-BY-4.0 — confirmed three independent ways, live, in this build 2026-08-06: the EFEHR GitLab efehr/eshm20 repo's own LICENSE file ("This project is licensed under the Creative Commons Attribution 4.0 International License"); the ESHM20 WMS service's own GetCapabilities (<Fees>CC-BY-4</Fees>); and the EFEHR REST API's own openapi.json description ("License: CC BY 4.0"). Not copied from the broader hazard.efehr.org portal page's own generic Dublin-Core tag, which states the differing "CC BY-SA 4.0" for the portal as a whole — per ADR-059 the artefact's own three statements govern, not the sibling/portal-wide notice (see scripts/packs/earthquake_eshm20.py module docstring). ≤0.002 (estimate, upper bound — see note b) ≤0.0002 (estimate, upper bound — see note b) ingested (a) Two real, live findings changed this pack's source from what the issue assumed — see scripts/packs/earthquake_eshm20.py's module docstring for the full trail. First, the EFEHR GitLab eshm20 repo (file tree read live via the GitLab API) holds only LICENSE/README.md/ESHM20_CITATION.cff plus oq_computational/ (OpenQuake input configs) and additional_materials/ (plots) — no precomputed hazard-map rasters; producing one from these inputs needs OpenQuake, out of scope per #605. Second, the EFEHR map-rendering endpoint (WMS and WCS, efehrmaps.ethz.ch/cgi-bin/mapserv?map=/var/www/mapfile/eshm20data.map, layer/coverage seismic-hazard) is render-only: a live GetMap and a live GetCoverage request against it in this build both returned a 3-band uint8 RGB image (values 153–255, confirmed via rasterio), not the underlying PGA floats, despite DescribeCoverage advertising a plausible single-band range — neither WMS nor WCS on this mapfile registers any output format beyond rendering formats (wcs:formatSupported lists only png/gif/jpeg/tiff-as-image, live-read). Using it would have silently ingested rendered pixel colours as PGA values. The pack instead uses the EFEHR Seismic Hazard Services REST API (efehr-services.ethz.ch/api/v1, OpenAPI-documented, linked from hazard.efehr.org/en/web-services/hazard-curve-data/, built under the EU Geo-INQUIRE project): /v1/maps/area returns real numeric PGA values (unit g, live-confirmed via /v1/models/81/imts, matching EarthquakeIngestor.intensity_unit from #604 exactly) on a genuine 0.1° regular grid — live-verified with a real 1×1° test query over the Peloponnese (121 points, values 0.29–0.61 g, a physically plausible range). Model id 81 = "European Seismic hazard Model 2020 (ESHM20)", soil type rock_vs30_800ms-1 (the only one this model publishes at map level, matching Vs30=800 m/s reference rock), aggregation = arithmetic mean — all live-confirmed. The five (probability, timespan) pairs read live from /v1/models/81/poe?imt=PGA give nominal (round(1/probability)) return periods 50/476/976/2500/5000y — within 0–1% of the commonly-cited engineering labels 50/475/975/2475/5000y that core/ingest/earthquake.py's #604 docstring uses (largest gap: 2500 vs 2475, ~1%); this pack dispatches the exact live probability/timespan pair, and treats the nominal year as a cosmetic label only, not a re-derivation. ESRM20 vulnerability/fragility (the other half of this issue) was already fetched, licence-checked (CC-BY-4.0, from the esrm20_vulnerability/esrm20_exposure GitLab repos' own LICENSE files at their pinned release tags) and seeded by #604 — see scripts/packs/derive_esrm20_earthquake_curves.py and scripts/packs/esrm20_2021_pga_vulnerability.json, both read but not re-derived or duplicated here. (b) No live fetch of the full Greece bbox was attempted in this build — not a volume constraint (see below) but a timing one: the real 1×1° (121-point) test query took ~7.8s, i.e. server-side per-point computation dominates response time, not payload size (~47 bytes/point measured: 5688 B / 121 pts). The full Greece bbox (lon 19.0–28.5, lat 34.5–42.0) at the confirmed 0.1° native grid is ≈96×76 = 7296 points per return-period map; extrapolating the measured per-point latency across 5 return periods implies roughly 30–40 minutes of cumulative request time — ops should budget for this, and may want to request sub-tiles in parallel rather than one bbox-wide call per RP. Raw-volume estimate: 7296 pts × 5 RPs × ~47 B/pt (JSON, the wire format actually fetched) ≈ 1.71 MB ≈ 0.0017 GB, rounded up to ≤0.002 GB for margin — a calculated bound from one measured request, matching #611/#612/#613's own methodology, not a guess. Processed-volume estimate: the ingest chord's own single-band float32 GeoTIFF encoding of the same grid, 7296 px × 4 B × 5 RPs ≈ 146 KB ≈ 0.00015 GB before compression, rounded up to ≤0.0002 GB; fetch_map_grid_geotiff's compress="deflate" output (and the chord's own hazards/{id}/intensity.npz) are expected to be smaller still. Both figures are far below any multi-GB concern — no live download of the full grid was needed to establish that.
#618 Exposure — LitPop default asset values ETH Zurich — Institute for Environmental Decisions, Weather and Climate Risks Group (Eberenz, S., Stocker, D., Röösli, T., Bresch, D. N.); dataset archived at the ETH Research Collection, https://doi.org/10.3929/ethz-b-000331316 (DOI resolution verified live in this build — redirects to research-collection.ethz.ch, not a CLIMADA-branded host) CC-BY-4.0 — independently CONFIRMED 2026-08-08 (see note b): the item's own registered DataCite metadata carries rightsList "Creative Commons Attribution 4.0 International", rightsIdentifier cc-by-4.0 on the SPDX scheme, publisher ETH Zurich ≤0.02 (UNVERIFIED estimate — see notes c and d) N/A (see note c) ingested (Path B, --source climada-api — see note d) (b) The companion paper (Eberenz et al., Earth Syst. Sci. Data, 12, 817–833, 2020, https://doi.org/10.5194/essd-12-817-2020) was read directly (PDF): its own license notice (page 1) reads "This work is distributed under the Creative Commons Attribution 4.0 License." Its reference list gives the dataset's own citation: "Eberenz, S., Stocker, D., Röösli, T., and Bresch, D. N.: LitPop: Global Exposure Data for Disaster Risk Assessment, ETH Research Collection, https://doi.org/10.3929/ethz-b-000331316, 2019." Update 2026-08-08 — the licence is now independently confirmed, and this row's earlier access finding was wrong. (i) DataCite (https://api.datacite.org/dois/10.3929/ethz-b-000331316, read live) returns the item's OWN registered metadata: rightsList "Creative Commons Attribution 4.0 International", rightsIdentifier cc-by-4.0, SPDX scheme; publisher ETH Zurich; publicationYear 2019. That record is registered by ETH Zurich as the publisher, so it is the item's own rights field — the "not independently confirmed" hedge this row previously carried is retired, and ops no longer needs to re-check it. (ii) The item page is not a JS-rendered app returning 500: research-collection.ethz.ch returns HTTP 403 "Access Restricted — due to a high volume of automated traffic (scraping), access ... is temporarily restricted from your location / your provider". That is an IP-level anti-scraping control, and this project deliberately does not evade it (ADR-063). (iii) The earlier "deliberately NOT sourced from CLIMADA's own data server" stance is superseded by ADR-063: the CLIMADA Data API is now an opt-in second source (--source climada-api), not the default — see note d. ADR-024 fences CLIMADA's code and packaging, not data fetched over HTTPS. (c) LitPop's published format is a per-country CSV (value, latitude, longitude, region_id) at 30 arcsec (~1 km) resolution (secondary-sourced: CLIMADA docs' LitPop tutorial page; not independently confirmed against the item page). No file was downloaded in this build (no live network access) — the raw-volume figure is a calculated upper bound from Greece's land area (~131,957 km², a verifiable geographic fact) at ~1 km²/cell (≈130,000–190,000 rows including coastal/island cells) times an assumed ≤60 bytes/row (value+lat+lon+region_id as CSV text) ≈ ≤11 MB, rounded up to ≤0.02 GB for margin — a calculation, not a measurement; the real per-country file could be a single combined download rather than land-only, in which case this figure is TBD pending the live ops download. Processed volume is N/A in the object-storage sense the other rows use (hazards/{id}/intensity.npz / processed/.../hazard.tif): exposure packs land as exposure_datasets/exposures DB rows via the existing raster exposure adapter (#594), not a committed storage artifact — see the ledger's own column note for exposure packs. The ≤0.02 GB figure is now flagged UNVERIFIED and possibly far too low: DataCite reports the whole Research Collection release as 16.63 GB across text/plain + text/csv + application/x-tar (the formats field of the same record read in note b). Nothing read so far establishes the per-country share of that 16.63 GB, so the calculated bound above stands only as a bound on the land-area arithmetic, not on the real file. The pack (scripts/packs/litpop_exposure.py) still has no default download URL for this source: ops resolves it from the live handle page in a browser and passes it via --download-url — never a guessed URL. (d) Alternative source, added 2026-08-08 (ADR-063, issue #618). The CLIMADA Data API (https://climada.ethz.ch/data-api/v1/dataset/?data_type=litpop&country_iso3alpha=GRC) is a plain, public, credential-free REST file server publishing the same authors' LitPop as per-country HDF5. It is a different product, recorded as such and catalogued under its own dataset name: 150 arcsec (~4.6 km, screening-grade) versus this row's 30 arcsec, and value epoch 2018 versus 2014. Licence per the API record's own license field: "Attribution 4.0 International (CC-BY-4.0)" — matching DataCite's reading of the RC release. Raw volume is a real measurement, not an estimate: LitPop_150arcsec_GRC.hdf5 (exponents (1,1), fin_mode pc, uuid d9840877-2347-4247-9fc9-ad914cc53fd0) is 1,678,480 bytes = 0.0017 GB, downloaded and verified against its declared md5:06f9bd0a29cfc0d1faa2a9acf3f103f8 byte-exact. Processed volume is N/A for the same reason as the RC path (exposure packs write DB rows, no storage artifact). This alternative is not counted in the Totals row: it is an opt-in second source for the same pack, not an additional pack, and only one of the two lands per ingest. Update, 2026-08-08 ops run (PR #781): ingested via Path B, dataset_id=69e03607-8d99-4870-9560-6f998537f9d3, 7,896 points, total value 1,067,141,118,804 USD(2018) confirmed identical to the source dataset — see docs/plan/phase-9-ops-plan.md OPS-3b for the verification query.
#619 Exposure — GHSL built-up + population European Commission, Joint Research Centre (Global Human Settlement Layer) — GHS-BUILT-S R2023A, GHS-BUILT-V R2023A, GHS-POP R2023A, all epoch 2020, 3 arc-second (~100 m) grid, EPSG:4326 CC-BY-4.0 (see note a — read live from each product's own bundled copyright.txt, not assumed from a sibling GHSL product) 20.13 (measured, not an estimate — see note a) N/A — see note a ingested (a) Licence: each product's own copyright.txt (e.g. .../GHS_BUILT_S_GLOBE_R2023A/GHS_BUILT_S_E2020_GLOBE_R2023A_4326_3ss/V1-0/copyright.txt, fetched live 2026-08-06 for all three products — BUILT-S, BUILT-V, POP) states verbatim: "Any copyright and/or sui generis right on the dataset is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence." Recorded per-artefact rather than copied from the JRC data-catalogue landing pages' more generic "European Commission reuse notice" wording (issue #621/ADR-059's rule). Resolution: GHS-POP R2023A's own JRC catalogue page (data.jrc.ec.europa.eu, verified live 2026-08-06) states the 2023 update ships "at 100m resolution in World Mollweide projection (or 3 arc-seconds in WGS84)" — the native 100 m grid is EPSG:54009; RasterExposureAdapter (#594) requires EPSG:4326 and does not reproject, so scripts/packs/ghsl_exposure.py uses each product's _4326_3ss variant, confirmed live to exist at epoch 2020 for all three products by fetching each product's own JEODPP directory index (see the PR body for the listings read). Raw volume 20.13 GB (3.21 + 4.36 + 12.55 GB) is a real HTTP HEAD measurement, not an estimate: Content-Length read live 2026-08-06 for all three whole-globe zips — BUILT-S 3 208 625 400 B, BUILT-V 4 362 815 348 B, POP 12 554 406 149 B (no body fetched, matching the "no multi-GB download in this build" constraint). These are whole-globe archives: like Eurostat's Census-GRID (#620) and unlike JRC's river-flood GeoTIFFs (#611/#612), GHSL's zip endpoint has no server-side windowed-clip option, so the full per-product archive must land in the raw/staging zone once before scripts/packs/ghsl_exposure.py::clip_and_scale_local clips the Greece bbox [19.0, 34.5, 28.5, 42.0] locally, streamed in row-blocks (never the whole extracted raster in memory) — matching how ingest_historical_catalog.py::fetch_exposure_raster already handles GHSL's existing 30ss/~1km GHS-BUILT-V product. The m3-to-currency fix this issue exists for: GHS-BUILT-S (built-up surface, m2) is converted to EUR using a real, cited factor — Huizinga et al. 2017 (JRC105688, doi:10.2760/16510), sheet "MaxDamage-Residential", column B ("Max Damage Structure, EUR/m2, 2010 price level"), row 77 (Greece): 440.5788419051012 EUR/m2 — the spreadsheet downloaded in this build hashes to the exact sha256 already recorded in scripts/packs/huizinga_2017_europe_damage_functions.json's source.xlsx_sha256 (the same byte-identical file this repo already trusts). GHS-BUILT-V ships alongside it unconverted (value_unit="m3"): no citable EUR/m3 factor, or citable average-building-height factor to derive one from the EUR/m2 figure, could be sourced/verified in this build — GHSL's own methodology derives GHS-BUILT-V as GHS-BUILT-S × GHS-BUILT-H (a per-cell height grid, not a documented single average), so no constant height factor exists to cite either; this is stamped as an explicit [ASSUMPTION] on the dataset's properties, not silently resolved (see the PR body's provenance section for the exact searches run). GHS-POP ships as value_unit="persons" (a headcount, not a currency). Update, 2026-08-08 ops run: all three products ingested (dataset ids 65eb047e/6fe049d3/5e6ed880, BUILT-S/BUILT-V/POP respectively). BUILT-S's first attempt failed on a real defect (#774): its citation inlined the full Huizinga provenance note, overflowing exposure_datasets.citation's VARCHAR(1024) — fixed by pointing at properties.conversion_source instead, which already carries the same text verbatim (JSONB, unbounded). Processed volume is N/A, not blank, for the same reason as #618/#620: exposure ingest (core/exposure_service.ingest_exposure_from_source) writes Exposure/ExposureDataset DB rows only, no hazards/{id}/intensity.npz or processed/{org}/{id}/hazard.tif COG. scripts/packs/ghsl_exposure.py is standalone (own CLI), matching #618/#620's precedent that ingest_historical_catalog.py is a serialised hotspot other concurrent Phase 9 pack issues touch — it does not replace that file's existing GHS-BUILT-V (30ss/~1km) ingest path, which is additive/untouched by this pack.
#620 Exposure — Eurostat Census 2021 grid Eurostat (European Commission), GISCO — "Population and housing census 2021 — population grid" (Census-GRID 2021, V3, published 2026-05-30) UNRESOLVED — see note b 0.566 (measured, not an estimate — see note b) N/A — see note b ingested (b) Provenance/licence check per issue #621's rule (ADR-059): the GISCO population-distribution page (verified live 2026-08-06, https://ec.europa.eu/eurostat/web/gisco/geodata/population-distribution) states, specifically for "Eurostat Census Grid 2021": "EU copyright rules apply, while the licence would be under CC-BY 4.0." Eurostat's general copyright notice (verified live 2026-08-06, https://ec.europa.eu/eurostat/help/copyright-notice), however, separately lists "population grids" among the categories excluded from the default Commission Decision 2011/833/EU CC-BY-4.0-for-any-purpose policy — restricted instead to non-commercial reuse — and the predecessor GEOSTAT 2011/2006 grids carry an explicit no-redistribution/no-commercial-use notice. Which regime governs the 2021 grid's population attribute (TOT_P_2021) was not resolved in this build; the real GISCO-ConditionsOfUse.pdf ships only inside the archive itself, which was not downloaded here (see below). This ambiguity is stamped verbatim onto the ingested dataset's license column by scripts/packs/eurostat_census_grid.py::LICENSE_NOTE so it travels with the data. Raw volume 0.566 GB (566 125 577 bytes) is a real measurement, not an estimate: an HTTP HEAD request against the V3 zip (https://gisco-services.ec.europa.eu/census/2021/Eurostat_Census-GRID_2021_V3.zip, verified live 2026-08-06, Content-Length: 566125577) — no body was fetched, matching the "no multi-GB download in this build" constraint. This is the whole-EU zip; unlike JRC's river-flood files, Eurostat's zip endpoint does not support a server-side windowed clip, so the full archive must land in the raw/staging zone once before the Greece bbox [19.0, 34.5, 28.5, 42.0] clip happens locally (matching how fetch_exposure_raster already handles GHSL's zip in ingest_historical_catalog.py). Every format Eurostat ships this grid in (GeoTIFF, GeoPackage, CSV/Parquet) is natively EPSG:3035 (ETRS89-LAEA) — confirmed on the same page — with no documented EPSG:4326 option, so RasterExposureAdapter (#594, which requires EPSG:4326 and does not reproject) cannot ingest it directly; scripts/packs/eurostat_census_grid.py::reproject_population_to_wgs84 reprojects the extracted raster to EPSG:4326 with rasterio.warp.reproject(resampling=Resampling.sum) first, chosen specifically to conserve the population total under the CRS change (unit-tested against a synthetic fixture — nearest/bilinear would invent or lose people). Processed volume is N/A, not blank: exposure ingest (core/exposure_service.ingest_exposure_from_source) writes Exposure/ExposureDataset DB rows only — there is no hazards/{id}/intensity.npz or processed/{org}/{id}/hazard.tif COG for an exposure pack (per this file's Processed-volume column convention). The zip's exact internal member layout (which .tif holds TOT_P_2021) was not verified by downloading it in this build; select_population_raster_member picks a uniquely-named .tif member by convention and raises rather than guessing if that assumption doesn't hold against the real archive. Update, 2026-08-08 ops run: it didn't hold — the real archive ships one combined 14-band GeoTIFF (band descriptions T/M/F/age groups/..., no per-variable file at all), fixed in #767 (select_population_raster_band matches the band by description, verified against the archive's own read.me: T = Total population, band 1). Ingested 2026-08-08 (dataset id 4ce9317b-c417-4024-9456-4160c954e90f, 52,347 points).
Totals ≈ 24.87 (mixed — calculated upper-bound estimates plus 5 real measurements; now includes #616, excludes a separate ≤142.6 GB WISC full-synthetic-set hypothetical from #615; see note c) ≈ 0.203 partial (measured across #614/#615/#616 only — #611/#612/#613/#617 still TBD, exposure packs N/A; see note c) (c) Recomputed 2026-08-10 after the NetCDF ops run, superseding the ≈24.64 GB figure computed 2026-08-07 on trunk f65cc59. Three cells changed and the change is entirely in what is measured rather than calculated: #614's raw went from the ≤0.0002 GB calculated Greece-clip bound to 0.0295 GB measured (EWDS 27,990 B ingested + C3S 29,435,326 B downloaded-not-ingested), #615's from the ≤0.04 GB per-footprint bound to 0.000055 GB measured (the real retrieval is scoped to one storm, not the whole 25W–35E/30N–70N domain the bound assumed), and #616 from TBD to 0.238 GB measured (238,249,300 B of delivered ZIPs). Both superseded upper bounds stay visible in their own rows — they are not deleted, because they remain the correct ceiling for the unscoped case each was computed for. New sum: #611 ≤3.7 + #612 ≤0.09 + #613 ≤0.09 + #614 0.0295 (measured) + #615 0.000055 (measured) + #616 0.238 (measured) + #617 ≤0.002 + #618 ≤0.02 + #619 20.13 (measured) + #620 0.566 (measured) ≈ 24.87 GB. This total still mixes calculated upper-bound estimates (#611, #612, #613, #617, #618) with real measurements (#614, #615, #616 by download; #619, #620 by HTTP HEAD) — it is not itself a measurement and should not be read as one. #615's own row separately documents a ≤142.6 GB upper bound for fetching WISC's entire 22,980-footprint synthetic set whole; that figure stays deliberately excluded from this headline total — and as of 2026-08-10 it is not merely an un-scoped hypothetical but a not-pursued one: the WISC process schema publishes no spatial input at all, so it cannot be scoped under the no-multi-GB rule (see #615's note b). If it were included anyway, the combined raw-volume exposure would be ≈167.5 GB. The processed column is now partially summable for the first time: #614 0.0000141 + #615 0.000125 + #616 0.203 ≈ 0.203 GB measured, against #611/#612/#613 still TBD — those packs are ingested, so their processed bytes are already sitting in the bucket and are measurable, they have simply never been attributed per-pack — #617 at a ≤0.0002 GB estimate, and #618/#619/#620 N/A (exposure packs write DB rows, not a storage artifact). Treat the 0.203 GB as a floor, not a total. This ≈24.87 GB is NOT a storage bill — see the "Raw-zone reality check" above the Ledger table. The live Hetzner bucket measured 359.87 MB (0.360 GB) across 158 objects on 2026-08-10, after that day's ops run and after trash/ was purged (superseding both the 0.325 GB measured on 2026-08-08 and the 528.30 MB / 185-object intermediate reading taken earlier on 2026-08-10, before the purge; raw/ is still only .keep + catalog.v1.json, 2 objects / 0.04 MB): every pack here streams its source download to local disk, clips/ingests it, then deletes it, so nothing in this column is ever archived to the bucket. Per-zone, final 2026-08-10 reading: hazards/ 55 obj / 200.72 MB, processed/ 47 obj / 158.75 MB, raw/ 2 obj / 0.04 MB, reports/ 7 obj / 0.24 MB, results/ 30 obj / 0.12 MB, staging/ 16 obj / 0.01 MB, tiles/ 1 obj / 0.00 MB, trash/ 0 obj (purged). (Those per-zone figures add to 359.88 MB against a reported total of 359.87 — a 0.01 MB artifact of rounding each zone independently, not a missing object.) Two movements explain the delta from the pre-purge reading: trash/ went 28 obj / 237.40 MB → 0 (the trash/2026-08-08-ops5b/ keys were deleted one by one after confirming all four referenced UUIDs were absent from hazard_datasets, that none was a live ingest_jobs row, and that no dataset had a cog_path under trash/), and processed/ grew 46 obj / 89.77 MB → 47 obj / 158.75 MB because #611's JRC national baseline finally got its COG built (see that row's own update). The bucket is now almost entirely live data: hazards/ + processed/ = 359.47 MB of the 359.87 MB total, of which #616 alone is 203.33 MB.