Skip to content

Phase 9 — Data foundation: fill the lake, unify the pipeline

Status: Defined — ratified 2026-07-30; entry conditions cleared 2026-08-02; doc refreshed 2026-08-03 (post-ratification drift reconciled: the #436 ingest-scale chain and the Phase-8 exit fix wave landed most of Stream B3; B5 rewritten per #481's closure; audit re-verified against code). All six decisions resolved by owner; recommendations approved as written, including the reassessment refinements. Produced from a planning session: full codebase dataset-consumption audit + web-verified open-data source scan (July 2026). Ready for /phase-split now. Predecessor: Phase 8 (milestone 21/21 closed 2026-08-02; open tail absorbed here — of it, the #436/#441/#443–#446 ingest-scale chain is now fully closed, #405 landed 2026-07-30; what remains open is #412 WF/WS sourcing and the four mid-review walkthrough defects, see entry/exit). License posture: unchanged — per-dataset license is RECORDED on every ingested artifact but license resolution stays deferred (standing rule: nothing publishes publicly before the licensing decision).

Goal — the Full-Catalog walkthrough

At phase end, from a clean stack pointed at the Hetzner object store, in one sitting, manually, through the real UI and API:

  1. Open the catalog → browse ≥6 hazard types (river flood, coastal flood, wildfire, windstorm, heatwave, earthquake; drought and TC stretch), multiple exposure datasets, and seeded impact functions — every entry carrying source / license / citation / version / resolution / scenario metadata.
  2. For at least two hazards, pick between ≥2 alternative datasets for the same peril (e.g. river flood: JRC vs Aqueduct; heatwave: EURO-CORDEX vs CMIP6-derived) and see the choice reflected in results + provenance.
  3. Run an impact computation on every computable peril; see indicator/score layers for every indicator peril (distinction defined below).
  4. Run the scenario matrix on scenario-conditioned data for ≥2 hazards.
  5. Every dataset on the map has working tiles; every lookup answer names the dataset it came from.
  6. All of it ingested through one pipeline (no side-door scripts writing directly to the DB), from documented, re-runnable source manifests.

Computable vs indicator perils (honest framing, from the vulnerability-curve scan): RF/WS/TC/EQ have machine-readable damage curves → full EAI/EP compute. Heatwave and drought have no published structural-damage curves — they enter as hazard-indicator perils: ingested, tiled, scored, shown on the score card, excluded from EAI. Amended 2026-08-19 (#856, decision 6 below): their literatures do publish functions for other consequences (heat → health/labour-productivity, drought → crop yield), so the exclusion is a deliberately deferred monetisation, not an absent literature — read decision 6 before restating this line anywhere. Wildfire is conditional (#481, closed 2026-08-01): damage curves exist (the seeded Lüthi brightness-temperature sigmoids) but every picked open source is FWI — a weather-only fire-danger index no damage function can consume in principle. WF therefore ships as an indicator peril on the FWI packs, and becomes computable only if intensity-grade footprints (brightness temperature / FRP) are sourced under #412 (stretch, Stream B5b). The UI says so explicitly.

Entry conditions

  • [x] Phase 8 exits — cleared 2026-08-02: milestone 21/21, Mid-Review ran, exited under option A (amend and exit) — the walkthrough ran against prod (1/6 moments passed) and the criterion was deliberately waived, with the four blocking defects carried here (#548/#549 closed 2026-08-03; #558/#550 fixes deployed on v0.6.0 + ui v0.3.0, closing on the owner walkthrough — see exit criteria).
  • [x] CDS + EWDS access provisioned — cleared 2026-08-02, tokens + endpoints stored as deployment secrets. Verified 2026-07-30: one ECMWF account signs into both stores (no separate registration); what is per-store is the API endpoint (https://cds.climate.copernicus.eu/api vs https://ewds.climate.copernicus.eu/api), the Personal Access Token copied from each store's profile page, and the per-dataset license acceptance (manual, at the bottom of each dataset's download form). Tokens + endpoints stored as deployment secrets. Week 0; nothing in streams C/E for wildfire/windstorm/heatwave/drought moves without this.

Codebase audit — where we actually stand (2026-07-30, re-verified 2026-08-03)

Full audit in the planning session; load-bearing facts. Re-verified against code 2026-08-03 after the #436 chain + Phase-8 exit fix wave landed — two items flipped to fixed (marked), the rest still hold:

  • 4 hazard types, closed enum; only river flood has real data (JRC full-Greece historical + Aqueduct scenario pack via scripts). WF ingest is blocked by a category mismatch, not a missing curve (#481): the ingestor declares FWI, a danger index no damage function can consume in principle — the unit gate is correctly rejecting it, the seeded Lüthi Kelvin curves are fine. WS has neither data nor a v98 raster. TC has layer specs but no ingest path for tracks.
  • No exposure_datasets table — exposure "datasets" are a shared UUID + per-row duplicated sha256; ExposureType.BUILDINGS and value_unit="USD" are hardcoded on upload (the GHSL pack stores m³ labelled USD).
  • ADR-029 is largely unimplemented: zero manifest writers, no catalog.v1.json producer (/v1/admin/catalog 404s on every deployment), and the real keys (hazards/, results/, staging/, chunks/, raw/{hostname}/…) are off-spec.
  • ~~#442 grid-membership assignment is dead code in prod~~ — FIXED 2026-08-03: the ingest chord now stamps grid_transform/grid_width/ grid_height + per-centroid pixel_index (aggregate_and_commit.py), and all 8 pre-v2 prod datasets were backfilled via scripts/backfill_grid_geometry.py (24/24 bit-exact).
  • ~~COGs only happen when raw_raster_key is passed~~ — FIXED 2026-08-03 (PR #568): every chord now ends in _schedule_cog_build() building the COG from the committed NPZ; producers can no longer pass raw_raster_key (the old path survives only as the legacy convert_to_cog route). Prod COGs backfilled 8/8.
  • Two datasets per peril can coexist but not be chosen (still true): /v1/risk/* picks "newest with a COG" (no dataset_id param), the matrix resolver — now scenario/supported_years-conditioned — is still newest-wins within a group with no provider dimension, and risk-surface keys (SurfaceKey) omit dataset_id (two same-peril datasets produce colliding surfaces). Result cache and centroid assignments are the only multi-dataset-safe pieces.
  • IngestAdapter ABC (Phase 7.1 / ADR-033) does not exist; the three DatasetSource polling adapters point at invented manifest URLs and have never run against a live provider.
  • Dataset metadata gaps: no resolution, bbox, temporal coverage, upstream version, GCM identity, scenario taxonomy (a frozenset in the API layer), region vocabulary, or dataset status/supersession.
  • Phase-8 walkthrough defects (post-ratification, carried in per plan.md): #548 (COG never built) and #549 (exposure layer wiring) closed; #558 (matrix centroid preflight) and #550 (uploaded portfolio rows in the rollup, ADR-047) have their fixes deployed on prod (v0.6.0 + ui v0.3.0, 2026-08-03) and close when the owner walkthrough passes — see exit criteria.

Source decisions per dataset type

License column is recorded risk, not a blocker (deferral rule). ✔ = primary pick; ◇ = user-selectable alternative; ✗ = rejected/deferred with reason. Greece-first coverage, Europe where the same download gives it for free.

Hazards

Peril Pick Dataset (access) Scenarios License note
River flood ✔ JRC river flood Europe/Med 100 m (direct, already ingested for Greece) baseline, 9 RPs CC-BY
◇ WRI Aqueduct Floods v2 ~1 km (GEE/download; already scripted) 2030/2050/2080 × RCP4.5/8.5 open, attribution
◇ Paprotny/4TU pan-EU ~100 m (direct ZIP, ~5 GB) RCP-driven to 2100 CC0
Coastal flood ✔ JRC LISCOAST extreme sea levels (direct) + Aqueduct coastal depth grids RCP4.5/8.5 ≈CC-BY
Wildfire ✔ GEFF/ERA5 FWI historical 0.25° (EWDS, authed) 1940→present CC-BY
✔ C3S FWI projections 0.11° v2.0 (CDS, authed) RCP2.6/4.5/8.5 → 2098 CC-BY
◇ EFFIS burnt areas (WMS/request) — event overlay, not RP hazard — CC-BY
note All picked WF sources are FWI → indicator-only (#481: no damage curve can consume FWI in principle). Computable WF needs intensity-grade footprints (brightness temp / FRP) sourced under #412 — stretch, Stream B5b
Windstorm ✔ C3S windstorm tracks + footprints 1940→present 0.25°/0.016° (CDS) historical CC-BY
✔ WISC synthetic set, 22,980 footprints 4.4 km (CDS) — probabilistic complement present climate CC-BY
note No scenario-conditioned WS footprints exist anywhere — deriving from CORDEX winds is future in-house work, recorded as a gap (closes #412's scenario half honestly)
Heatwave ✔ C3S heat waves & cold spells EURO-CORDEX 0.1° (CDS) RCP4.5/8.5, 1986–2085 CC-BY
◇ CMIP6 extreme indices + heat stress (CDS) 4 SSPs non-standard licence — flag
◇ ERA5/ERA5-HEAT (CDS) — historical truth 1940→ CC-BY
✗ E-OBS — non-commercial licence; research-mode only
Earthquake ✔ ESHM20 hazard maps/curves (EFEHR GitLab + web services) static (RPs) CC-BY
✗ GEM global maps — NC/SA clauses; fallback only, never in the insurer product path
Drought (stretch) ✔ ERA5-Drought SPI/SPEI monthly (CDS) historical CC-BY
◇ SPEIbase v2.11 (direct); ISIMIP3b (only scenario-capable option) SSPs CC-BY / CC0
TC (stretch, non-Greek†) ✔ CLIMADA Data API 150-arcsec footprints (REST, HDF5) RCP-scaled 2040/60/80 CC-BY
◇ STORM v4 + STORM-CC (4TU, CC0) — needs windfield derivation RCP8.5 CC0

† Verified 2026-08-12, not just a scope choice: the CLIMADA Data API does return country_iso3alpha=GRC-tagged tropical_cyclone records (both the 150as IBTrACS random-walk family and the 300as STORM family), but every sampled HDF5 file has an empty intensity/data array for Greece — real events/centroids, zero non-zero wind intensity. "Non-Greek" here is a verified data constraint (no track's footprint reaches Greece in either product), not merely a peril-breadth preference. See #624.

Exposure

Pick Dataset Role
✔ LitPop (ETH) default asset-value exposure (what EIOPA's CLIMADA-App also defaults to)
✔ GHSL R2023A BUILT-S/BUILT-V + GHS-POP 100 m (direct, no auth) building-stock + population proxies; fix the m³→currency story with a documented €/m³ factor
✔ Eurostat Census 2021 grid 1 km / 100 m pop (GISCO) Greek socio-economic truth layer
◇ WorldPop 100 m population alternative
◇ Microsoft ML Building Footprints + OSM/Geofabrik Greece building footprints (EUBUCCO weak in Greece; Google Open Buildings absent in Europe) — ODbL, share-alike recorded
✗ GEM Global Exposure open release Admin-1 aggregation only + NC-SA

Impact / vulnerability functions

Pick Source Note
✔ JRC/Huizinga 2017 depth-damage xlsx (direct) re-derive parameters from the published xlsx/papers, not from CLIMADA's GPL-3.0 code — keeps the engine-isolation and license boundary clean
✔ ESRM20 vulnerability/fragility CSV+XML (EFEHR GitLab, CC-BY) earthquake curves incl. Greek taxonomies
✘ ~~FWI-calibrated wildfire curve (literature)~~ — no such curve exists See #481. FWI is a fire-danger index (weather-only, dimensionless), not a hazard intensity a structure experiences, so a damage function cannot be calibrated on it in principle. OS-Climate physrisk surveyed the same literature and states this explicitly, landing on the same Lüthi Kelvin sigmoid this repo already seeds. The WF unit block is fixed on the ingestor side (ingest brightness temperature / FRP), not by finding a curve.
◇ Nirandjan 2024 infrastructure curve DB (Zenodo, 1,510 curves) critical-infrastructure expansion
— Windstorm: implement Klawa-Ulbrich/Schwierz parameters from papers no curated open file exists

Curated collections worth ingesting from directly

OS-Climate physrisk Zarr on AWS (s3://os-climate-physical-risk, no-sign-request, per-resource licenses), ISIMIP (CC0-leaning), World Bank CCKP S3, CDS ECDE indicators. These are sources, not architecture — we re-host what we ingest.

Storage & metadata design (the "how we store" decisions)

  1. Inventory-as-contract (pattern copied from physrisk): the catalog is a machine-readable inventory where every dataset entry carries hazard_type/category, provider, version, path template with {scenario}/{year} placeholders, scenarios→years map, resolution, bbox, CRS, units, license, attribution, viz metadata. Ours lives in Postgres (not a JSON file) with the ADR-029 per-leaf manifest.json as the object-store mirror — implemented at last, or the ADR is amended honestly.
  2. Dataset identity columns on hazard_datasets (+ new exposure_datasets table): provider/source id, upstream version, native resolution, bbox, temporal coverage, GCM/ensemble, data_type (fluvial/pluvial/coastal…), status (active/superseded). Scenario taxonomy becomes a table with the EIOPA↔RCP/SSP mapping from Phase 8.9's doc; region gets a vocabulary linked to admin_boundaries.
  3. Raw zone on Hetzner: original files land at raw/{source}/{data-type}/{version}/{dataset-id}/ with manifest, checksum, license — re-ingestable without re-downloading (Copernicus queues are slow; the raw zone is the cache). Processed stays npz+COG; Zarr is deliberately deferred (revisit if/when multi-scenario chunked point reads become the bottleneck — record as parked, not rejected).
  4. Key-layout reconciliation — legalize or migrate hazards/, results/, staging/, chunks/; fix raw/{hostname}/… to ADR-029 form. Not covered by ADR-045, which is scoped to ingest-scale compute/storage decisions and says nothing about MinIO key layout; this remains open for a Stream A2 ADR.
  5. ThinkHazard-style precedence rule recorded per peril for overlapping datasets: quality score → local-over-global → recency (replaces silent newest-wins).

Streams

Stream A — Contracts (schema + storage identity) — first, everything depends on it

  • A1: dataset identity migration(s) + exposure_datasets table + scenario/region taxonomies; ADR. (tier 4)
  • A2: manifest writers + catalog.v1.json producer + key reconciliation; needs its own ADR — ADR-045 does not cover key layout. (tier 3–4)
  • A3: multi-dataset safety: dataset_id into risk-surface keys, explicit dataset selector on /v1/risk/* and the matrix resolver (provider dimension + precedence rule), unit-aware impact-fn resolution. Design point: when a user picks a dataset, the matrix resolver must pin the provider family across cells (a JRC baseline must resolve to JRC scenario siblings, not silently mix in Aqueduct cells) — falling back across providers only explicitly, per the precedence rule. (tier 4)

Stream B — Pipeline unification (ETL)

  • B1: IngestAdapter ABC + registry (Phase 7.1 pulled forward, private, no PyPI); refactor GeoTIFF modes 1/2 + exposure CSV/XLSX behind it. ADR-033. (tier 4)
  • B2: new adapters: NetCDF (unlocks every CDS/EWDS dataset above), raster→exposure (GHSL/WorldPop path out of scripts), point/track (TC) as stretch. (tier 4)
  • B3: ADR-045 remnant — most of the original chord-completion scope landed before the split (grid definition + pixel_index stamped at ingest and COG built inside the chord, both fixed 2026-08-03; absorbed #436/#443/#444/#445 all closed 2026-08-01→03). Topic 1 (out-of-core aggregation — streaming disk-backed union/CSR-assembly, gating pan-EU #447) landed 2026-08-05 via #595. Topic 2 landed 2026-08-06 via #596 (ADR-057): both write sites retired, the core/centroid_assignment.py membership-join redesign done (dense → grid arithmetic, sparse → the artifact's pixel-index array, legacy → the unchanged join), and hazard_datasets.footprint added as the dataset-level extent. What remains is ops, not code: backfilling legacy datasets onto v2 and then deleting the existing ~4.29 GB of rows. (tier 3–4)
  • B4: real DatasetSource adapters — CDS/EWDS API client (authed), JRC open-data, Zenodo/4TU, AWS Open Data S3 — replacing the three fictional manifest URLs; polling → review → approve exercised against a live provider end-to-end once. (tier 3)
  • B5: wildfire posture per #481's closure (2026-08-01: no FWI-calibrated damage curve exists in principle — FWI is a weather-only danger index; do not re-open that search, two investigations already dead-ended there). Two halves: B5a — FWI packs enter as score/indicator layers via score_schemes, riding C1's indicator-peril mechanism (no EAI on FWI, UI-labelled); B5b (stretch) — source intensity-grade footprints (brightness temperature / FRP) under #412 and pair them with the already-seeded Lüthi curves — no new coefficients needed; this is what makes WF computable. Absorbs the #412 wildfire half. (tier 3)

Stream C — Hazard expansion

  • C1: new HazardType members + per-type ingestor/curve/score-scheme/LayerSpec wiring, plus the indicator-peril mechanism (peril without curve → no EAI, score-card only, UI labelled). One issue per peril: coastal flood, heatwave, earthquake (ESHM20 rasters through the existing engine — RP grids × ESRM20 curves; OpenQuake stays wave 2 for the modelled seismic story), drought (stretch), TC data (stretch). Migration caveat for the checklist: Postgres ALTER TYPE ... ADD VALUE on hazard_type_enum cannot be used in the same transaction that adds it — the enum-add and any seed rows referencing the new value must be separate Alembic revisions (or use autocommit blocks). (tier 3–4 each)
  • C2: windstorm data-in (C3S footprints + WISC) + Klawa-Ulbrich parameters; closes #412's windstorm half. (tier 3)
  • C3: OpenQuake scoping spike (docs-only, timeboxed — decision 3's anti-deferral anchor): (a) integration scoping — model selection, semantic mapping to our hazard/exposure contracts, sidecar posture, effort estimate — authored as a Phase 10 entry condition for wave 2/#366; (b) verdict on whether screening-grade ESHM20-based EQ EAI is defensible or earthquake stays indicator-only until OQ lands. (tier 3)

Stream D — Product surface (multi-source choice)

  • D1: UI dataset picker per hazard (listing endpoint already filterable) + dataset provenance chip on results/lookups. UI-repo issues. (tier 3)
  • D2: catalog browser surface upgrade (metadata-rich admin + user-facing catalog). (tier 3)
  • D3: SDK parity — the Python SDK (Phase 5 artifact) gains the dataset selector params and catalog metadata fields, so the dual-track (portal + API/SDK, identical semantics) story holds. (tier 2)

Stream E — The actual filling (data packs on Hetzner)

  • E1..En: one issue per pack (peril × source), acceptance = downloaded to raw zone with manifest → ingested via the unified pipeline → catalogued → tiled → computable/scored → visible in UI. Greece-first; EU-wide where the same artifact covers it. Sequenced after the A/B items each pack needs, but sourcing/downloading starts day 1 (slowest lead times — CDS queues, account provisioning). The first pack issue also creates the volume-budget ledger (expected raw + processed GB per pack, recorded in the pack manifests before download) — the artifact the exit criterion checks. The initial pack list, from the ✔/◇ picks above (split may re-batch, not silently drop — the two ◇ packs are load-bearing for the ≥2-hazards×≥2-datasets exit criterion):
  • E1 river flood — JRC (re-ingest the existing Greek data through the unified pipeline; retires the side-door scripts)
  • E2 river flood ◇ — Aqueduct Floods v2 scenario pack
  • E3 coastal flood — LISCOAST extreme sea levels + Aqueduct coastal grids
  • E4 wildfire — GEFF/ERA5 FWI historical (EWDS; indicator, B5a)
  • E5 wildfire — C3S FWI projections (CDS; indicator, B5a)
  • E6 windstorm — C3S tracks + footprints + WISC synthetic set (CDS)
  • E7 heatwave — EURO-CORDEX heat waves (CDS; indicator)
  • E8 heatwave ◇ — ERA5-HEAT historical and/or CMIP6 indices (license flag)
  • E9 earthquake — ESHM20 maps/curves + ESRM20 vulnerability (EFEHR)
  • E10 exposure — LitPop default asset values
  • E11 exposure — GHSL BUILT-S/BUILT-V + GHS-POP (with the documented €/m³ factor)
  • E12 exposure — Eurostat Census 2021 grid (GISCO)
  • E13 vulnerability — JRC/Huizinga depth-damage re-derivation (from the published xlsx, not CLIMADA code) + Klawa-Ulbrich WS parameters
  • E14 (stretch) drought — ERA5-Drought SPI/SPEI (CDS; indicator)
  • E15 (stretch) TC — CLIMADA Data API footprints (non-Greek† — verified data constraint, see above and #624)
  • E16 (stretch) wildfire intensity — BT/FRP footprints (B5b, #412)
  • E-final: demo/seed migration — once the real Greek river-flood pack is in, seed_demo re-points at real catalog data (or its synthetic tif is kept only as the explicit offline fallback, labelled as such); the demo stops being the only data story. (tier 2)

Decisions (all six ratified 2026-07-30 — recommendations approved as written, except decision 6: its posture stands and is now DECIDED, but its rationale was refuted and amended 2026-08-19 per #856)

  1. Geographic envelope — DECIDED 2026-07-30: Greece-first packs, EU-wide only for coarse (≥1 km) datasets. Pan-EU full-resolution expansion is tracked in #447, gated on implementing ADR-045's out-of-core aggregation decision (topic 1; #446 itself is now resolved as ADR-045 — direction decided, implementation deferred to Stream B3).
  2. Copernicus CDS + EWDS access (operational prerequisite): wildfire, windstorm, heatwave, drought sources are all behind authed (free) access. Corrected after live verification (2026-07-30): one ECMWF account covers both stores — CDS and EWDS are sibling portals behind ECMWF sign-on; the per-store parts are the API endpoint, the Personal Access Token from each store's profile page, and per-dataset license acceptance. Tokens are issued to an individual — usable as deployment secrets, bus-factor-noted in the ops doc. Retrieval is queue-based: packs batch their requests, and the raw zone is the once-only cache. Recommendation: wire up both, week 0.
  3. Earthquake v1 posture: ESHM20 RP rasters through the existing engine now (CC-BY, closes the Greek peril-mix gap) vs waiting for OpenQuake wave 2. Caveat surfaced by reassessment: mapping PGA rasters onto ESRM20 fragility (natively Sa(T)/taxonomy-keyed; OpenQuake does that convolution properly) is a real simplification — v1 EAI is screening-grade and must be labelled so; the EMS-98/macroseismic form of the ESRM20 curves is the cleaner v1 match to raster intensities. Recommendation: ship v1 now as score card + screening-grade EAI behind the honest label, and add an OpenQuake scoping spike (docs-only, timeboxed) to Phase 9 with two deliverables: (a) integration scoping (model selection, semantic mapping, sidecar posture, effort) as a Phase 10 entry condition — the anti-indefinite-deferral anchor for wave 2/#366; (b) a verdict on whether screening-grade EQ EAI is defensible or earthquake stays indicator-only until OQ.
  4. Zarr adoption: defer (parked) vs adopt for scenario stacks now. Recommendation: defer; keep npz+COG; copy only physrisk's metadata pattern. Two sharpening notes: (a) our point-serving plane is already the Phase 8 answer layer (H3 RiskCells + COG point reads) — adopting Zarr would add a third plane that overlaps it, which strengthens the deferral; (b) deferring Zarr as our storage format does not preclude reading Zarr sources at ingest time (physrisk's AWS holdings) via an xarray/zarr dep in the ETL — that is just another source format behind the IngestAdapter.
  5. Absorption — DECIDED 2026-07-30: Phase 9 absorbs #412, the #436 chain (#443/#444/#445/#446), and Phase 7.1; Phase 7 re-cut shrinks to public-facing items only. #442 and #405 landed same day (PRs #449, #448) before absorption mattered; #446 landed 2026-07-30 as ADR-045 (direction decided, implementation deferred to Stream B3). Status 2026-08-03: the entire #436/#441/#443/#444/#445 chain is closed pre-split — only #412 and the ADR-045 implementation (B3 remnant) carry forward as open work.
  6. Heatwave/drought honesty rule — DECIDED 2026-08-19 (#856, option (a)): heatwave and drought ship as indicator perils with no EAI path; health/productivity monetisation stays deferred. The posture stands, but the 2026-07-30 rationale below it is amended, because a revalidation pass on 2026-08-14 refuted both of its premises against the primary literature:
  7. ~~"no published building-damage curves"~~ (this doc's framing paragraph above, and the old core/eai_eligibility.py opener's blunter "no published curve at all") — what does not exist for heat is a structural-damage curve. The field models heat as a health and labour-productivity peril, and the platforms that monetise it zero structural damage outright. Impact functions for those consequences are published: OS-Climate physrisk maps a days-above-threshold indicator (unit days/year) through piecewise-linear curves to fractional production disruption, and Stalhandske et al. 2022 (NHESS) builds heat mortality and labour-productivity impact on CLIMADA — this platform's own engine lineage. The honest statement is therefore deliberate deferral of a monetisation, not an empty literature. (The same correction applies in kind to drought, whose crop-yield functions this entry already acknowledged.)
  8. ~~"matches what ThinkHazard/score-card products do"~~ — ThinkHazard scores every peril and computes EAI for none, so it validates nothing heat-specific. Indicator-only matches screening-tier tools; risk-quantification peers (physrisk, S&P Climanomics, XDI, Jupiter, Moody's) do monetise heat, as disruption or health loss. With P1 the regional-insurer persona, that is the relevant peer set — which is why this is recorded as a scope choice, not as a claim about the science.

Weighed and not taken now: a days-index productivity EAI (physrisk-style piecewise days-above-threshold → fractional disruption against a wage-bill/production-value exposure) is the credible next step and is parked as a Phase 10 candidate behind the P1 insurer checkpoint. It needs an absolute-threshold days product — a re-request of the existing C3S heatwave pack, whose ingested variants are definition-relative (#919) — and a decided compute mode for annual expected values on scenario × year datasets, which ADR-065/ADR-069 refuse structurally today: a product decision, not a removed guard. Full mortality + productivity EAI is not pursued; property-damage heat EAI never will be, having no fragility basis at all.

Mechanism (unchanged, and the reason this stays cheap to reverse): "indicator peril" is derived from curve availability per (peril × exposure type), not a hardcoded peril list — a peril with no seeded curve that consumes its unit simply gets no EAI path, and the UI renders the score/indicator card. The curves that do exist in these families (drought crop-yield for agricultural exposure; heat mortality/productivity for population) stay reachable: seed one and every affected dataset flips with no migration and no backfill. Today's three blocking mechanisms — ADR-069's frequency gate (#797), the #804 refusal, and the absent curve — stay untouched.

Exit criteria

ALL MET — Phase 9 closed 2026-08-25. The owner ran the Full-Catalog walkthrough end to end against prod v0.17.0 (a5a8404) / UI v0.7.0 and all six steps passed; the coastal half of step 4 completed its grid in production the same day. The phase therefore exits on its stated criterion met, superseding the "closed by supersession" framing recorded at the 2026-08-24 goal reset. Per-criterion verdicts are inline below; the sitting record is in phase-9-full-catalog-walkthrough.md. One criterion — the live poll→ingest cycle — was dispositioned, not executed; it is called out as such below rather than counted as met.

  • Wave 0, before the data waves: ~~the Phase 8 six-moment Showcase walkthrough passes against production, closing #558 and #550.~~ CLEARED 2026-08-10 — the owner reports the walkthrough passed against prod; #558 was already closed 2026-08-04 and #550 is closed on that report. The Showcase claim is no longer gated. ~~This does not close Phase 9: the exit is the Full-Catalog walkthrough plus the ops track, both below.~~ (Superseded: the Full-Catalog walkthrough and the ops track are both done — the phase closed 2026-08-25.)
  • ✅ MET 2026-08-25. The Full-Catalog walkthrough (Goal) runs manually end-to-end — API and UI. Walked by the owner against prod v0.17.0 (a5a8404) / UI v0.7.0, Alembic 0086; all six steps passed, including step 4's reinstated two-hazard form — the coastal 3 × 3 grid completed in production at 12:12 UTC (six scenario cells; job ids and AAI recorded in the walkthrough doc) next to eleven river-flood results the same day. This is what closes the phase.
  • ✅ MET. Every dataset in the catalog has: source manifest, license + attribution recorded, working tiles, dataset-attributed lookups/results — audited across walkthrough steps 1, 5 and 6, all of which passed. Amended 2026-08-10 by ADR-070 — this criterion originally read "raw-zone copy + manifest". The raw zone is a transient staging concept, not an archive: every pack streams its download to local disk, clips/ingests, then deletes it, so no raw-zone copy exists for any of the ten packs and none ever will under the current design (verified 2026-08-08 — raw/ holds only .keep + catalog.v1.json, ~0.03 MB). The reproducibility artifact is the per-dataset source manifest at hazards/{dataset_id}/source_manifest.json (ADR-066); pre-ADR-066 datasets have theirs at staging/{ingest_job_id}/manifest.json. The Full-Catalog walkthrough audits that, not a raw/ copy.
  • ✅ MET. ≥2 hazards offer ≥2 user-selectable datasets with the precedence rule documented — audited in walkthrough step 2, which passed. The catalog holds 21 active datasets across 6 hazard types: river flood ×8, coastal flood ×7, heatwave ×3, and one each of wildfire, storm_europe and earthquake (counted in prod 2026-08-25).
  • ✅ MET. Zero side-door ingests remain — audited in walkthrough step 6, which passed. Both surviving scripts/ingest_*.py are thin callers into scripts/packs/: ingest_scenario_hazards.py delegates to packs/river_flood_aqueduct.py (#612) and ingest_historical_catalog.py to packs/river_flood_jrc.py (#611); what stays in them is shared plumbing (manifest load/validate, windowed HTTP-range clip, dispatch-and-wait).
  • ⚠️ DISPOSITIONED, NOT EXECUTED. One live polling→review→approve→ingest cycle demonstrated against a real provider. Three legs are proven live and the fourth reaches real work (9 JRC rasters staged and validated on 2026-08-12), but it stopped at plan_chunks' typed size refusal. #865 fixed that and is deployed — yet the cycle was never re-run: the prod queue still shows the one jrc_flood row stuck at approved from 2026-08-12, and the catalog is still 21 datasets. The 2026-08-24 goal reset dispositioned this criterion (carrying only the coastal proof forward, as #970) and the owner confirmed that on 2026-08-25. Recorded here as dispositioned so no later reader mistakes it for a completed run. Re-running it is available ops work, not a Phase 9 debt.
  • ✅ MET. OpenQuake scoping doc merged (C3) — the Phase 10 entry condition for wave 2/#366 exists in writing: docs/plan/exploration/06-openquake-scoping.md. (Wave 2/#366 is itself parked with the End-State Vision at the 2026-08-24 goal reset; the written entry condition survives the parking.)
  • ✅ MET 2026-08-08, re-measured 2026-08-10. Storage bill on Hetzner checked against a recorded volume budget: 158 objects / 359.87 MB in the bucket against a recomputed ledger raw sum of ≈24.87 GB. The gap is expected — the raw zone is transient by design — and is ratified by ADR-070.

Risks

  • Copernicus lead times (queues, account provisioning) — start Stream E sourcing day 1, in parallel with contracts.
  • Volume creep on the box — the raw zone is a cache with a budget line per pack, not a mirror of the internet; packs record expected GB before download.
  • License findings pile up (ODbL share-alike family, CMIP6 non-standard licence, ELSTAT unverified) — recorded per dataset now; a single license-review session after the catalog exists is the plan, per the deferral rule.
  • Enum-driven hazard coupling makes each new peril a cross-cutting change — C1's first issue should carve the per-peril wiring into one documented checklist (ingestor, curve, score scheme, LayerSpec, chunk budget, engine code) so subsequent perils are mechanical.
  • Windstorm scenario gap is unfixable with open data — the phase ships WS historical + synthetic and says so; deriving CORDEX-based footprints is its own future project.