Phase 12 — Second Country + Second Engine¶
Defined 2026-08-24 at the goal reset (plan.md); matured for
/phase-split on 2026-08-27 — grounded against trunk f7bde49, prod
backbone v0.17.2 + UI v0.8.0, then adversarially verified (/reassess
fast: two refuters + a completeness critic; 9 of 14 premises confirmed, 5
corrected, 5 items added — all folded in below). Serves Goal A (data
end-to-end beyond Greece — the pipeline parameterizes) and completes
Moment 3 of the showcase (step 6, the only showcase step whose delivering
phase is P12). Runs after the Phase 13 issues it shares files with (see
Sequencing).
What it is. Two independent proofs on one platform:
- Cyprus — a comparable-size EU country lands through the same packs, manifests, boundary loader, curve seeder, reference-surface builder and smoke walk that serve Greece, with the country as a pack parameter and the platform's dataset selection made coverage-aware so two countries can coexist in one catalog. Success is measured by the walk, not the row count.
- CLIMADA live on
/agreement— the second engine the backbone has been registering since #378 is deployed on prod, and one reference scenario runs through both engines with a divergence the page can explain.
What it is not. Not pan-EU (#447 stays parked, gated on the out-of-core work ADR'd in #446); not a country column on impact functions; not a published sidecar image; not a third engine; not cross-hazard matrix runs (parked, revisit after P12).
Business value (why this phase earns its slot)¶
| Workstream | Value | Evidence it is cheap now |
|---|---|---|
| Cyprus | Turns "Greece-first" (ADR-014) into "country-agnostic, Greece-first" — the claim every future customer conversation and the parked pan-EU track rest on. Also removes a latent defect: the first same-peril dataset with a smaller extent than Greece's — a second country, a regional re-ingest — would today deterministically shadow Greece in /lookup (design record D1) |
Packs already take --bbox; smoke_e2e.py already takes --region/--boundary; hazard_datasets.footprint is already stamped at ingest and GiST-indexed; the country-specific artifacts are three reproducible tables |
| CLIMADA sidecar | The platform's headline credibility argument (ADR-042: "same scenario through both engines, here is the divergence") and the last target: P12 step in the showcase script. Engine plurality is a product commitment (CLAUDE.md) that today renders as a one-engine degenerate case |
Everything except the deploy path and a roster endpoint exists: image, server, adapter, registry, error codes, compose service on dev and prod, UI picker + agreement view; the parity harness reads 0.000 % on the reference fixture |
Ground truth (verified 2026-08-27; the doc's 2026-08-24 premises, corrected)¶
- The engine#33 gate is resolved. Closed 2026-08-26 (engine
v0.5.1, PR engine#39): the engine keeps its[0, 1]clamp on MDD/PAA; CLIMADA clamps at[0, ∞). The divergence appears only when a measure pushes MDD past 1. In range the two are the same calculation atrtol=1e-6(notebook 08 scenario 12a).POST /v1/compute/impact/compare'sCompareRequest(api/v1/compute.py:901-910) carries hazard/impact-function/exposure ids,save_mat,deductible,cover,engines— no measures; every cell is built by_build_impact_paramswith no measure argument (:1002-1035), andmeasure_idsexists only on the separate adaptation path (:1111/1143,worker/tasks.py:1503/1527). Deductible/cover are post-impact insurance terms and cannot push MDD past 1. Gate passed; nothing to display as a bug. - The sidecar is built, not "adapter only".
docker/climada-sidecar/(Dockerfile, CLIMADA 5.0.0 pinned; stdlibserver.pywithGET /health,POST /v1/impact, forwardinghaz_typeverbatim, default"RF",:170,251),worker/models/climada_adapter.py(HTTP client), registry entryclimadawith a config-based availability probe (registry.py:202-239),Settings.climada_sidecar_url/_timeout_seconds(config.py:135,138— 900 s compute timeout), theclimadacompose profile in bothdocker-compose.yml(:207-225, URL injected at:285,:332) anddocker-compose.prod.yml(:283-295,build:only, noimage:tag — the ADR-042 deviation from ADR-038),.env.example:158-167, UIEnginePicker+AgreementView. What is missing: prod.env.prodhas noCLIMADA_SIDECAR_*key (tests/test_env_template_parity.py:126-136,150pins the spine at 33 keys and forbids extras), no sidecar image was ever built on the host,deploy-prod.ymlnever passes a profile (every invocation usesP="-f docker-compose.prod.yml -f docker-compose.hetzner.yml --env-file .env.prod",:343,:428,:720; the roll isup -d --remove-orphansat:383, so a profile omitted on one call tears the sidecar down; rollback model:47-49= "previous images still on the host"; prune keep-list:676is tag-keyed),docker-compose.hetzner.ymlhas no sidecar override,scripts/verify-deploy.shhas no sidecar probe,docs/DEPLOYMENT.mdanddocs/RUNBOOK.mdcontain zero mentions, and there is no engine-roster endpoint —ui/src/lib/engines.tshardcodes['climate_lama','climada']and says so.ClimadaEngineAdapter.supported_hazards(climada_adapter.py:157-158) is["river_flood","tropical_cyclone","wildfire","storm_europe"]— noearthquake, nocoastal_flood— while the default engine's list includes earthquake (engine_adapter.py:72-78); nothing inengine_comparison_service.pyreconciles the two, so an earthquake comparison today yields a failed engine, not a divergence (the engine#30 failure class). - Greece is a constant in more places than the 2026-08-24 doc counted.
(a)
GREECE_BBOX = (19.0, 34.5, 28.5, 42.0)declared in seven packs —litpop_exposure.py:176,coastal_flood.py:158,earthquake_eshm20.py:192,ghsl_exposure.py:133,eurostat_census_grid.py:146,heatwave_packs.py:227,_wildfire_fwi_common.py:131(imported only bywildfire_fwi_ewds.py;wildfire_fwi_c3s.pyandwindstorm_c3s_wisc.pyare bbox-free by product design — noareainput — and need no country change) — plus an eighth consumer,scripts/backfill_source_manifests.py:177,209with its ownregion="Greece"/ bbox literals (:189,233,244), and a second Greece-bbox fixture generator,scripts/gen_test_hazards.py:35,152. (b) Dataset names that bypassdataset_identity.name_prefix(:276-330) entirely:earthquake_eshm20.py:339,eurostat_census_grid.py:424,ghsl_exposure.py:356,395,437(+ the cross-reference string at:426),coastal_flood.py:325,328,ingest_scenario_hazards.py:115(DEMO_EXPOSURE_NAME = "LitPop Greece demo"). (c)litpop_exposure.py:493,978,1015defaultcountry_iso3="GRC". (d) Inside the platform, not the scripts:config.py:388-389dataset_source_clip_bbox = "19.0,34.5,28.5,42.0"/dataset_source_clip_region = "Greece"and:346dataset_source_worldpop_iso3 = "GRC", consumed byworker/dataset_ingest.py:269andcore/ingest/clip.py— the in-product catalog approve→ingest handoff clips to Greece and stamps"Greece"from process-wide settings. The ingest scripts already take--manifest; the two manifests (scripts/historical_catalog_manifest.json,scripts/scenario_hazards_manifest.json) carryregion: "Greece", a bbox andgreece-…ids. - Three artifacts are genuinely country-derived, all reproducible from
published tables: (1) the Huizinga
MaxDamage-ResidentialEUR/m² row (ghsl_exposure.py:60-80,185-195— row 77 = Greece, 440.58 EUR/m²); (2) the ESRM20 curve selection (impact_function_seeder.py:366-380ranksExposure_Model_Greece_Res.csvby replacement cost; the ESRM20 exposure repository publishesExposure_Model_Cyprus_{Res,Com,Ind}.csvat the samev1.0tag — GitHub API listing, 2026-08-27); (3) the heatwave score-band edges —migrations/versions/0086_heatwave_score_schemes_v2.pyadopts[3,4,5,7,10,14,20,28,40]and justifies each band's reachability from the pixel distribution of the Greece-clipped cubes; a Cyprus pin inherits them unvalidated. One sub-regional constant:_WS_V98_REF = 20.0(impact_function_seeder.py:115-118, "representative of central/western Europe", spatially-varying v98 deferred by ADR-023). Everything else seeded is Europe-wide (Huizinga fluvial + coastal, column EUROPE) or global (TCGLB, Lüthi WF). - Pack idempotency is keyed on the dataset name.
ingest_scenario_hazards.py:527-578(_existing_dataset_idmatchesorg_id+name),heatwave_packs.py:717,769-771,798,ingest_historical_catalog.py:37,476; the repository's only name accessor isget_by_name_and_type(hazard_repository.py:224-237); no content-hash key exists. Renaming a Greek dataset re-ingests it or orphans its prod row. - Dataset selection is not coverage-aware — the hidden defect.
core/lookup_service.pick_dataset(:710-777) applies four rules — pinned id, exact scenario label, COG-bearing pool, thendataset_precedence.select_dataset— and its signature takes no lat/lon.select_dataset's key is(lifecycle, -quality, locality, extent_fraction ↑, -recency)(dataset_precedence.py:243-265): two JRC datasets tie on quality and both rank LOCAL, so the one with the smaller native bbox wins deterministically — a Cyprus JRC dataset beats the Greek one for every pin, whichever was ingested first (recency only breaks a tie when both bboxes are absent).HazardRepository.list_datasetsfiltersregionby exact string equality only (:387),regionis free text with no vocabulary at read time, the risk API'sregiondefaults toNone(api/v1/risk.py:120-123), and the UI'sriskLookupsends none (ui/src/api/client.ts:733-742).hazard_datasets.footprint(PostGIS POLYGON,models/hazard.py:98-110; stamped bycore/ingest/base.py:1749asST_MakeEnvelope(grid.envelope); already GiST-indexed —migrations/versions/0075_hazard_dataset_footprint.py:65-71, the model'sspatial_index=Falseonly suppresses a duplicate implicit index) exists for exactly this and is read only bycore/centroid_assignment.py:294andcore/centroid_geometry.py:127— never on the selection path. It is nullable with no backfill (0075 docstring:23): pre-#596 rows are NULL. The portfolio rollup is safe by construction — it reads surface planes per H3 cell and picks nothing by precedence (rollup_service.py:1045-1060); matrix resolution is scoped to the base dataset's(haz_type, region)(compute_batch_service.py:478-479,hazard_repository.py:475-477) — but that scope matchesregion IS NULLtoo, so two region-less datasets in different countries would collide there. - The rollup never reads a COG (
rollup_service.py:214): a Cyprus portfolio scores empty until reference surfaces exist for the Cyprus datasets. Surfaces are built from a manifest exposure block flaggeddesignated_for_reference_surfaces(ingest_historical_catalog.py:230→core.surface_writer.build_reference_surface/worker.tasks.build_reference_surfaces). - Impact-function auto-resolution is silent on ambiguity.
core/impact_function_service.py:98-104:defaults = [f … if f.is_default]; none →ValidationError; otherwisedefaults[0].id— no more-than-one branch. The seeder marks exactly one EQ curve default (impact_function_seeder.py:653, among:645,657,668,679); the DB's only uniqueness is(org, haz_type, name); user-created functions are forced non-default (:163). A second EQ default would make every unpinned earthquake compute resolve to whichever country's curve lists first, with no error and no provenance flag. Also: thenot defaultsmessage (:100-103) says "Multiple impact functions exist … but none is marked as default" even when exactly one non-default function exists. - The smoke walk is country-parameterised in shape, single-leg in
structure.
scripts/smoke_e2e.pytakes repeatable--boundaryfixtures,--region,SDK_SMOKE_HAZARD_TIF,--exposure-csv(:1104-1116,1167) and fails fast on a region/boundary mismatch (:439-462); butHAZARD_NAME/EXPOSURE_NAMEare module constants (:115-116) matched withnext(row … == HAZARD_NAME)(:859,:889— first match wins, so a second leg on the same org would bind to the Greek rows and pass vacuously),READY_PREFIXis emitted once (:137,:1072-1074) andsdk-smoke.yml:193reads it withgrep '^E2E_READY ' | head -1, the workflow stages one raster (:89-92) into a scalarSDK_SMOKE_HAZARD_TIF(:163).scripts/gen_reference_datasets.py:38BBOXis a module constant with no argparse; the Aegean tile(23.0, 37.0, 24.0, 38.0)cannot be reused for Cyprus (~32.2–34.6 E / 34.5–35.7 N).scripts/extract_admin_boundaries.py:87-98FIXTURESis GRC-only. - UI state (
climate-lama-ui):src/lib/engines.tshardcodes the list;src/components/agreement/EnginePicker.tsx:17,40offersclimadaunconditionally and has a free-text add-an-engine input (:57-73);src/components/agreement/AgreementView.tsx:156,173-175rendersdivergence_pctandfailed_enginesonly —excluded_engines(src/types/index.ts:1279,{engine, reason}; fixtured insrc/test/matrixClient.test.ts:51) is typed, parsed and rendered nowhere; no per-engine version, no parity link. - Docs of record still carry the old gate:
showcase-script.md:80-81andplan.md(Phase 12 row, status table, banner) say "gated on engine#33";docs/DECISIONS.md:453ADR-014 is "Greece as MVP Region" (latest ADR is 077; Phase 13 reserves 078). - Prod today (2026-08-27,
docker pson the host): api/worker/beatv0.17.2, UIv0.8.0, postgres, redis, titiler, caddy — noclimada-sidecarcontainer, no sidecar image. Host is a CX43.
Design record (decisions made 2026-08-27 — owner may veto any before the split)¶
| # | Decision | Choice | Why (evidence) |
|---|---|---|---|
| D1 | Two-country coexistence | Coverage-aware selection server-side: pick_dataset gains a rule between rule 2 and rule 3 — coverage before answerability: a candidate whose footprint is non-NULL and does not intersect the pin is dropped; NULL footprints are always admitted ("makes no claim"). list_datasets gets an optional covering=(lat, lon) predicate — footprint IS NULL OR ST_Intersects(footprint, point) — the idiom exposure_repository.py:156 already uses. No migration: the GiST index exists (0075) |
"Always pass region" is a client convention over an unguarded server default and a free-text label; the footprint is geometry stamped by the ingest itself. Additive: a Greece-only stack has one candidate per peril either way. The regression test must be written against the extent term (two same-provider datasets, disjoint footprints, the smaller-extent one must not win a pin outside it), not insertion order |
| D2 | Country as a pack parameter, not a platform knob | New scripts/packs/country_profile.py: CountryProfile(iso3, name, bbox, gadm_fixtures, huizinga_max_damage_row, esrm20_exposure_file, litpop_iso3, manifest_ids), GREECE + CYPRUS, one --country CLI helper; every bbox holder in (a) above and every name literal in (b) read it; per-country manifest files fed through the existing --manifest. The platform's dataset_source_clip_* settings (d) become defaults that a catalog approval may override per request (D14) |
Packs are operator-side; the platform sees region labels + footprints. The one platform-side Greece constant is handled by D14 rather than by a global switch |
| D3 | Greek names are frozen | Every Greek dataset name — the dataset_identity prefixes and the module-level literals in (b) — plus manifest ids, identity triples, curve names and every is_default flag are byte-identical after the refactor; a regression test pins the literal strings |
Idempotency is keyed on name (ground truth); prod's 21 active datasets carry those names |
| D4 | ESRM20 Cyprus curves + a guard | derive_esrm20_earthquake_curves.py --country CYP selects the top-4 classes from Exposure_Model_Cyprus_Res.csv (same method, same --check exactness), seeded is_default=False, names ESRM20 EQ CYP …; the Cyprus walk pins its curve. And resolve_impact_function_id gains a cardinality guard (len(defaults) > 1 → typed ValidationError naming the curves), the wrong "none is marked" message is fixed, and a partial unique index (org_id, haz_type) WHERE is_default makes a second default unwritable (one gated migration) |
Today two defaults resolve silently to defaults[0] — a wrong-country curve applied with no signal. A country column on impact_functions stays out of scope with a named follow-up |
| D5 | Cyprus construction cost | CountryProfile.huizinga_max_damage_row for Cyprus read from the same JRC105688 MaxDamage-Residential sheet the repo already hashes; row number + value [VERIFY at build] |
Same provenance rule (ADR-059) as Greece; no new source |
| D6 | GADM scope | Load CYP only (admin-0 + one admin-1 district); GADM's separate XNC (Northern Cyprus) admin-0 is not loaded; docs/architecture/admin-boundaries.md's disputed-borders list gains a Cyprus entry saying exactly that |
Same "editorial state of a third-party dataset, not an endorsement" posture the doc takes for Crimea/Kashmir/Western Sahara |
| D7 | Cyprus exposure for the walk | LitPop via --source climada-api (ADR-063 Path B — credential-free, per-country HDF5) as the default-value exposure; GHSL BUILT-S with the D5 row as the currency-valued proxy; Eurostat census grid as population. ETH RC 30″ (Path A) optional — manual browser step per country |
Reproducible by an operator without owner input |
| D8 | Smoke | CountryLeg(boundaries, region, hazard_tif, exposure_csv, hazard_name, exposure_name, iso3); the Greek leg keeps today's literals; next(…) lookups replaced by unique-match assertions on leg.hazard_name/leg.exposure_name; one READY line per leg (E2E_READY_GRC …, E2E_READY_CYP …) and sdk-smoke.yml requires both; both tiles staged; SDK_SMOKE_HAZARD_TIF honoured for the Greek leg only. Cyprus leg runs after the Greek leg on the same stack |
One extra leg costs seconds; a second job costs a billed floor + boot. The per-leg names are what stop the second leg binding to the first leg's rows |
| D9 | Sidecar on prod | Built on the host under the climada profile (ADR-042: never published) — but pre-built out-of-band by a dedicated deploy step (docker compose $P --profile climada build climada-sidecar) that runs before the api/worker roll and is gated on CLIMADA_SIDECAR_URL being set; the service gains a local image: tag (climate-lama-climada-sidecar:local) so rollback and the tag-keyed prune keep it; when enabled, the profile is passed on every $P invocation (the --remove-orphans roll included); .env.prod gains the two keys and the parity spine goes 33 → 35 in the same PR; verify-deploy.sh probes /health in-network |
An unpinned CLIMADA+GDAL build must not sit on the critical path of every deploy, and a build:-only service has no rollback artefact today |
| D10 | No sidecar in CI | The comparison is proven locally (compose climada profile) and on prod by a reusable scripts/verify_engine_agreement.py; sdk-smoke.yml does not build the CLIMADA image |
Multi-minute build per run against a ~2000-min/month pool already at ~88 % in August |
| D11 | Engine roster | New GET /v1/engines → [{name, available, default, description, supported_hazards}] from registered_engines() / available_engines() (already imported by the API layer at api/v1/compute.py:66-67) + each adapter's supported_hazards; the UI picker reads it (hardcoded list stays as offline fallback), disables unavailable engines with the reason, disables per peril what an engine does not support, defaults to all available; the free-text add-engine input is removed |
"Two live engines" must be a fact the page reads; per-peril support is what turns a failed_engines entry into a disabled checkbox |
| D12 | "Explained divergence" | AgreementView renders excluded_engines (name + reason — the "why" already reaches the client and is discarded), per-engine engine_version from result metadata (climate_lama_engine 0.5.1 / climada 5.0.0), and a provenance line linking the engine repo's parity matrix (https://github.com/CortoMaltese3/climate-lama-engine/blob/main/docs/CLIMADA_PARITY.md); expected reference divergence < 1 % EAD (parity harness tolerance, tests/parity/test_parity_river_flood.py:33) |
The page has the numbers but no why; the parity matrix is the why |
| D13 | Showcase evidence for Cyprus | Moments 1 and 2 gain a one-line "walked on Cyprus" evidence note per moment rather than new steps; step M3-6 flips to live on exit |
A second country changes the data behind the steps, not the steps |
| D14 | Platform-side clip | The catalog approve→ingest path accepts an optional clip_bbox + clip_region (+ worldpop_iso3) per approval request, defaulting to today's settings values — the settings become documented defaults, not the only source |
Otherwise "the pipeline parameterizes" is true of the scripts and false of the product path; changing the defaults would break prod's Greek approvals |
| D15 | Cyprus reference surfaces | The Cyprus historical manifest carries a designated_for_reference_surfaces exposure block; surfaces are built and verified on prod per Cyprus peril before the walk |
The rollup answers only from surface planes; without them Moment 2 on Cyprus is empty |
| D16 | Decisions of record | New ADR (next free number after Phase 13's 078) for the coverage-before-answerability rule + a dated amendment to ADR-014 ("Greece as MVP region" → "Greece-first, country-parameterised"); the four "gated on engine#33" clauses in plan.md/showcase-script.md are struck in the PR that lands this doc |
The phase falsifies two claims the docs of record still make |
Guardrails — every Phase 12 issue¶
Inherited from phase-11-showcase.md guardrails 1–5, plus:
- ADR-024 byte-level.
import climadaappears only indocker/climada-sidecar/server.pyandtests/parity/sidecar_runner.py; CLIMADA is never added topyproject.toml; no fallback between engines. - Greece is byte-identical (D3): names, manifest ids, identity triples,
curve names,
is_defaultflags unchanged; the Greek smoke leg's assertions unchanged; a Greek pin answers from the same dataset before and after. - Coverage rule is additive (D1): NULL-footprint datasets are never
dropped; a pinned
dataset_idstill wins (rule 1); the rule runs before answerability and quality. - Every Cyprus dataset carries a non-NULL
regionand a non-NULLfootprint— the matrix'sregion IS NULLmatch and the coverage rule's NULL admission both depend on it. - Real data through the real pipeline (ADR-077): every Cyprus dataset
is fetched by a pack from the primary provider with a
docs/data/volume-budget.mdrow (licence verbatim) before download and a manifest row; nothing hand-uploaded or copied between orgs. - No sidecar image leaves the host (ADR-042) and no CI job builds it (D10).
- Exactly one default curve per peril, enforced (D4) — the partial
unique index is the phase's one expected migration; it lands as its own
gated PR (
migrations-gate.yml), typed ops only.
Scope — waves and items¶
Dependency line: W0 → {W1, W2} → W3 → exit; W4 runs beside them
(only W4.2 is file-independent of Phase 13 — see Sequencing). Sizes are
drafts; /phase-split grounds and re-sizes each item against the code.
W0 — Country as a pack parameter (scripts)¶
| Item | Delta (grounding) | Acceptance |
|---|---|---|
W0.1 country_profile.py + name templating + freeze test |
MISSING (checked scripts/packs/, scripts/*.py); name literals in dataset_identity.py:276-330 and earthquake_eshm20.py:339, eurostat_census_grid.py:424, ghsl_exposure.py:356,395,437,426, coastal_flood.py:325,328, ingest_scenario_hazards.py:115; litpop_exposure.py:493,978,1015 "GRC" |
CountryProfile + GREECE, CYPRUS, profile_for(iso3), add_country_argument(parser); every listed literal templated on the profile; tests/scripts/test_greek_name_freeze.py pins the current literal strings for Greece (dataset names, manifest ids, identity triples) and fails on any drift; stdlib-only like dataset_identity.py |
| W0.2 hazard packs + backfill + generators read the profile | GREECE_BBOX in coastal_flood.py:158, earthquake_eshm20.py:192, heatwave_packs.py:227, _wildfire_fwi_common.py:131, scripts/backfill_source_manifests.py:177,189,209,233,244, scripts/gen_test_hazards.py:35,152; _pseudo_manifest(region="Greece"); _greece.tif cache names. wildfire_fwi_c3s.py/windstorm_c3s_wisc.py are bbox-free — no change |
--country on each CLI (default GRC); grep -rn GREECE_BBOX scripts/ → 0; cache filenames carry the iso3; regions/names from the profile for every pack that has a country-varying retrieval — which excludes heatwave_packs.py (issue #1122: neither CDS collection declares a server-side area and the pack performs no local clip, so its --country is validated-only and its names and region="Europe" are identical for every country, pinned by tests/scripts/test_greek_name_freeze.py); freeze test green |
| W0.3 exposure packs read the profile | GREECE_BBOX in litpop_exposure.py:176, ghsl_exposure.py:133, eurostat_census_grid.py:146; GHS_BUILT_S_EUR_PER_M2 + evidence text ghsl_exposure.py:185-195 |
--country; GHSL reads profile.huizinga_max_damage_row (value, row, evidence string per country — Cyprus row [VERIFY], first step); LitPop --country-iso3 defaults from the profile; freeze test green |
| W0.4 Cyprus manifests | Manifests Greece-bound; tests/scripts/test_historical_catalog_manifest.py pins the schema; designated_for_reference_surfaces block at ingest_historical_catalog.py:230 |
scripts/cyprus_historical_catalog_manifest.json + scripts/cyprus_scenario_hazards_manifest.json: region: "Cyprus", Cyprus bbox, HEAD-verified JRC + Aqueduct tile URLs (JRC coverage [VERIFY]), an exposure block flagged designated_for_reference_surfaces (D15); schema tests parametrised over both countries; volume-budget.md rows for every Cyprus source |
| W0.5 GADM Cyprus fixtures | extract_admin_boundaries.py:87-98 GRC-only; fixtures README; tests/scripts/test_admin_boundary_fixtures.py |
gadm41_CYP_0.json.gz + gadm41_CYP_1_<district>.json.gz tracked (deterministic gzip, --check green); README rows; docs/architecture/admin-boundaries.md disputed-borders list gains Cyprus/XNC (D6); GADM CYP vs XNC split [VERIFY] |
W1 — Backbone core¶
| Item | Delta (grounding) | Acceptance |
|---|---|---|
W1.1 coverage rule in pick_dataset + covering predicate — after Phase 13 #1022 |
lookup_service.pick_dataset:710-777 (no lat/lon in scope); hazard_repository.list_datasets:271-389 (no spatial predicate); dataset_precedence.py:243-265 (extent term); footprint GiST-indexed (0075) — no migration |
list_datasets(covering=(lat, lon)) → footprint IS NULL OR ST_Intersects(footprint, ST_SetSRID(ST_Point(lon, lat), 4326)); build_score_card passes the pin; rule documented as "2b — coverage before answerability"; tests: (i) two same-provider datasets with disjoint footprints, the smaller-extent one outside the pin → the covering one wins; (ii) a NULL-footprint legacy dataset still competes; (iii) pinned id unchanged; docs/CLIMATE_LAMA.md lookup section; ADR per D16 |
| W1.2 footprint audit + backfill | 0075 :23 — no backfill for pre-#596 rows; prod NULL count [VERIFY] |
scripts/backfill_dataset_footprint.py: --dry-run prints the NULL rows, --apply derives the envelope from the stored grid (hazards/{id}/intensity.npz or centroid extent) and stamps it; RUNBOOK entry; prod: SELECT count(*) FROM hazard_datasets WHERE footprint IS NULL AND status='active' → 0 (needs-prod-proof) |
| W1.3 default-curve guard + partial unique index | impact_function_service.py:98-104 (defaults[0]), :100-103 (wrong message); no uniqueness on is_default |
len(defaults) > 1 → ValidationError naming the curves; message fixed; migration adds UNIQUE (org_id, haz_type) WHERE is_default (gated PR, typed ops); tests: two defaults → typed error / IntegrityError; exactly one default per peril after a full seed |
| W1.4 per-approval clip override | config.py:346,388-389; worker/dataset_ingest.py:269; core/ingest/clip.py |
Approval request accepts optional clip_bbox, clip_region, worldpop_iso3; settings remain the defaults; the region passes assert_known_region; tests: default path byte-identical, override stamps the given region + footprint; docs/CLIMATE_LAMA.md |
W2 — Cyprus data run (ops; W2.2a is a spike, W2.3 is a /build target, the rest are bootstrap)¶
| Item | Delta (grounding) | Acceptance |
|---|---|---|
| W2.1 load Cyprus boundaries on prod | load_admin_boundaries.py --geojson single-feature mode |
"Cyprus" in the region vocabulary on prod; an ingest with region: "Cyprus" is accepted; Greek regions unaffected — DONE 2026-08-31: CYP\|0\|Cyprus, CYP\|1\|Larnaca (GADM 4.1) loaded, #1048 comment |
| W2.2a source-coverage probe (spike, no downloads) | Each source's Cyprus coverage is unverified: JRC v3.1.1 tiles, Aqueduct (global), ESHM20 WCS, Eurostat Census 2021, EURO-CORDEX EUR-11 (Cyprus at the domain's eastern edge), ERA5 FWI | A recorded coverage table (source → probe command → result → go/no-go) in this doc; every [VERIFY] in the ledger below that names W2.2a executed |
| W2.2b flood + coastal run | river_flood_jrc.py, river_flood_aqueduct.py, coastal_flood.py with the Cyprus manifests |
Datasets active on prod with region="Cyprus", non-NULL footprint, COGs built, identity triples stamped, volume-budget rows with measured sizes — DONE 2026-08-31/09-02: 7 Aqueduct river_flood + 7 Aqueduct coastal_flood datasets active, region="Cyprus", org-wide NULL-footprint count 0/37 active rows (#1048 comment 2026-08-31, volumes measured 2026-09-02) |
| W2.2c seismic + exposure run | earthquake_eshm20.py, litpop_exposure.py --source climada-api, ghsl_exposure.py, eurostat_census_grid.py |
Same acceptance shape as W2.2b; the LitPop dataset is the walk's default-value exposure — DONE 2026-08-31: ESHM20 earthquake (1 dataset) active; all three exposures active (LitPop 306 pts, GHSL 2478 pts, Eurostat 2771 pts), LitPop's off-box convert + in-container ingest recipe run and verified byte-for-byte (#1048 comment) |
| W2.2d heat + fire run + band check | heatwave_packs.py, wildfire_fwi_ewds.py; band edges from migration 0086 |
Split per command (issue #1122). Wildfire (wildfire_fwi_ewds.py:281 sends a real server-side area): same acceptance shape as W2.2b, including region="Cyprus". Heatwave: region="Cyprus" is unreachable by design and is not asked for — both CDS collections declare no area, the pack performs no clip, and the three datasets are pan-European rows already active on prod, so a --country CYP run correctly reports "already in catalog, skipping". What remains, and is the real check: the per-band pixel histogram over the Cyprus window of the heatwave grid shows every band ≥ 2 reachable and a non-degenerate green/amber/red split — run scripts/score_band_reachability.py (see the evidence row below) — and if not, a follow-up issue for a country-scoped scheme row is filed (never silently shipped) — DONE 2026-08-31/09-02: wildfire (EWDS FWI, c6691db1…) active, region="Cyprus"; heatwave --country CYP reported "already in catalog, skipping" for all three pan-European rows (pass condition); band reachability run 2026-09-02 with the PR #1137 fix — ssp5-8.5 health v1 exit 0 PASS, ssp2-4.5 health v1 exit 1 (band 9 Severe 0 cells), ECDE climatological v2 exit 1 (bands 8–10 under floor) — per this row's own rule the two GATE FAILs are filed as #1139, never shipped silently (#1048 comment) |
W2.3 ESRM20 Cyprus curves (/build) |
derive_esrm20_earthquake_curves.py:126-135 pins the Greek CSV + sha; seeder holds 4 GRC curves |
--country CYP reads Exposure_Model_Cyprus_Res.csv (v1.0 tag, sha recorded), --check exact; 4 ESRM20 EQ CYP … entries seeded non-default; test: exactly one EQ default before and after; POST /v1/impact-functions/seed on prod adds them; ADR-060 dated note — DONE 2026-08-31: 19 impact functions seeded on prod; 4 ESRM20 EQ CYP … curves present, all non-default; exactly one EQ default org-wide (ESRM20 EQ GRC RC infill low-code H6) (#1048 comment) |
| W2.4 Cyprus reference surfaces on prod | build_reference_surfaces task; rollup reads surfaces only |
Surfaces exist for every Cyprus peril; a Cyprus pin's /v1/risk/lookup and a Cyprus asset's rollup both answer; needs-prod-proof — DONE 2026-09-03: surfaces + pin lookup 2026-08-31/09-02, and the portfolio rollup PASSING 3 / 3 after the remediation recorded at the end of this row. Diagnosis history kept because the failure mode outlives it — org_risk_surfaces holds Cyprus river_flood 6 + coastal_flood 6, all ready; Cyprus earthquake and wildfire surfaces were not built (neither Cyprus manifest designates those perils for designated_for_reference_surfaces — recorded gap, not an impossibility, Greece has only one earthquake org surface too); cross-country pin proof passed — Cyprus pin 34.89, 33.61 resolves the Aqueduct baseline (region=Cyprus, RP100 0.2542 m), Greek pin 40.60, 22.80 resolves the JRC Greece dataset (RP100 1.2580 m). The portfolio rollup ran and scored 0 of 1 members ("Cyprus rollup proof" book acd702b1…, every hazard "No risk surface in scope" at baseline and SSP5-8.5/2050): the book's only member (LitPop 90df0ce5…, 34.6958 N/33.0208 E, Limassol) sits in surface cell 873f4dad0ffffff, which exists in the Cyprus surfaces but carries eai 0.0 and no flood_depth_m metric (Aqueduct Cyprus is valid on ~0.15 % of on-land cells); worse, there is no baseline/0 Cyprus surface at all — the 12 Cyprus rows are ssp2-4.5/ssp5-8.5 × 2030/2050/2080 only, while the underlying river (41040218…) and coastal (d7a29d22…) datasets have no baseline row and Greece does. A member inside a valued cell exists (LitPop 07f35dfa…, 34.9042 N/33.6042 E, cell 872da449affffff, flood_depth_m 0.249 in the ssp2-4.5/2030 surface) but is not in the book. Root cause, established 2026-09-03: the rollup scores a member only through the res-8 H3 cell the member's own coordinates fall in (core/rollup_service.py, risk_lookup_resolution = 8, no parent-cell fallback), and an org surface only carries cells where the exposure dataset that built it had a point — the 12 Cyprus surfaces were built over GHSL built-up volume Cyprus (2478 points), so a LitPop point 500 m from any GHSL point is unscorable whatever the raster says there. Verified both ways: the LitPop member near the demo pin scored 4 against the two LitPop-built surfaces that exist (baseline/2030 and /2050, 612 cells) and nothing against every GHSL-built one. Remediation — DONE 2026-09-03 on prod v0.21.0, no schema or code change, every write through the public API: (1) the two missing baseline surfaces built through the real matrix endpoint over GHSL Cyprus with scenario_labels ["baseline"], horizon_years [0] — river 41040218… → batch 08b03314…, cell job 65c21af3…, result 9690725e…; coastal d7a29d22… → batch 0167a744…, job 14d032d7…, result 8784bbc8… — each landing an org_risk_surfaces row river_flood / coastal_flood × Cyprus × baseline × horizon 0, status ready, 3902 cells (created 07:11:52 / 07:12:01 UTC); (2) portfolio acd702b1-4784-4a38-bda5-0020d7a0fa11 re-membered with three GHSL Cyprus rows inside valued baseline cells — 8537e916… (34.6579 N 33.0071 E, cell 883f4d32cdfffff), bd13d3d7… (34.9246 N 33.6404 E, 882da6b35dfffff), 60d19c58… (34.9079 N 33.6237 E, 882da6b34dfffff) — the two LitPop members no GHSL-built surface can score removed. GET /v1/risk/portfolios/acd702b1… now answers 3 / 3 scored in all three views: baseline (default) headline 4 Moderate-low river_flood, aggregate 0.2542 m; ?scenario=ssp2-4.5&horizon=2030 band 3, 0.2495 m; ?scenario=ssp5-8.5&horizon=2050 band 4, 0.2605 m — coastal_flood metric_absent_from_cells in every view (no coastal depth at those three cells), exposed value 16,769,833 USD throughout, worst members 7,043,304 / 5,050,948 / 4,675,581 USD each at the headline band. Cross-country lookup re-read in the same session: Cyprus pin 34.89, 33.61 → river flood band 4, 0.2542 m; Greek pin 40.60, 22.80 → band 7, 1.258 m. Recorded API trap: the rollup parameter is horizon, not horizon_year — a hand-written ?scenario=…&horizon_year=… silently re-reads baseline/0; the UI sends horizon correctly. Trail: #1048 (2026-08-31 comment, 2026-09-02 comment, 2026-09-03 diagnosis and pass comments); the res-8 / per-member-reason finding is #1154 |
W3 — Cyprus walk (smoke)¶
| Item | Delta (grounding) | Acceptance |
|---|---|---|
| W3.1 Cyprus reference tile + exposure | gen_reference_datasets.py:38 BBOX constant, no argparse; reference_datasets/README.md one geometry; test_ead_plausibility.py drives the Greek tile |
--bbox + --suffix; tracked river_flood_cyprus.tif + exposure_cyprus.csv (three assets inside a Cyprus tile, e.g. (33.0, 34.8, 33.5, 35.3) [VERIFY on land]); README row; plausibility test parametrised over both tiles |
| W3.2 smoke Cyprus leg — after Phase 13 #1029 | smoke_e2e.py:115-116,137,859,889,1072-1074; sdk-smoke.yml:89-92,163,193 |
CountryLeg per D8; Greek leg byte-identical in assertions; unique-match assertions; per-leg READY lines and the workflow requires both; both tiles staged; the Cyprus leg asserts a saved scenario with a non-empty result and the cross-pin check (Cyprus pin → Cyprus dataset, Greek pin → Greek dataset — W1.1's proof in the walk); runtime delta in the PR |
W4 — CLIMADA live on /agreement¶
| Item | Delta (grounding) | Acceptance |
|---|---|---|
W4.1 GET /v1/engines — after Phase 13 #1026 |
No /engines route (19 modules under api/v1/); registered_engines/available_engines already reachable from api/v1/compute.py:66-67; router registration touches api/v1/__init__.py + main.py (#1026 hotspots) |
Router returns [{name, available, default, description, supported_hazards}], authenticated, org-agnostic; core/errors.py untouched; tests for both availability states; docs/CLIMATE_LAMA.md |
W4.2 UI reads the roster + explains (climate-lama-ui) — independent |
src/lib/engines.ts; src/components/agreement/EnginePicker.tsx:17,40,57-73; AgreementView.tsx:156,173-175; types/index.ts:1279 |
Picker lists /v1/engines, disables unavailable engines with the reason and unsupported perils per engine, defaults to all available, falls back to KNOWN_ENGINES; free-text add-engine input removed; AgreementView renders excluded_engines (test: the reason string is in the DOM), per-engine version, parity link (D12); tests |
| W4.3a sidecar deploy path — after Phase 13 #1018 | deploy-prod.yml:343,383,428,720; docker-compose.prod.yml:283-295 (build: only); docker-compose.hetzner.yml:24-60; tests/test_env_template_parity.py:126-136,150 |
Pre-build step gated on CLIMADA_SIDECAR_URL, before the roll; image: climate-lama-climada-sidecar:local tag; profile on every $P when enabled ([VERIFY] --profile vs COMPOSE_PROFILES under --env-file); hetzner memory limit for the sidecar ([VERIFY] CX43 headroom); parity test admits the two keys and the spine count goes 33 → 35 in the same PR; verify-deploy.sh probes /health through the worker network when enabled |
W4.3b sidecar live on prod + docs (bootstrap) |
.env.prod (hand-mutation ledger); no docs |
.env.prod keys set; image built on host; container healthy; DEPLOYMENT.md "Enabling the CLIMADA engine" incl. rollback/prune notes for an untagged-by-registry service; RUNBOOK triage entry; needs-prod-proof |
W4.4 scripts/verify_engine_agreement.py + reference run |
tests/parity/test_parity_river_flood.py:33 proves 1 % in isolation; nothing runs the platform's /compare with two engines |
Script authenticates, runs POST /v1/compute/impact/compare (engines omitted ⇒ all available), polls, prints the divergence table, exits non-zero if failed_engines non-empty or abs(divergence_pct.ead) ≥ 1.0; run locally under the climada profile on the smoke reference dataset, then on prod against the JRC Greece baseline; result recorded here; exit criterion 3 |
W4.5 CLIMADA supported_hazards revalidation |
climada_adapter.py:157-158 omits earthquake, coastal_flood; sidecar forwards haz_type verbatim; CLIMADA has no EQ class (free-form tag) and CF is a petals convention |
Local sidecar runs on the earthquake (ESHM20) and coastal (Aqueduct) reference datasets; the adapter list is extended for each peril that computes within 1 % of the default engine, otherwise the roster's supported_hazards keeps it out and the picker disables it (D11); test pins the list against a recorded parity run |
Owner tasks (not /build targets)¶
- O1 walk on prod — Done 2026-09-02/09-03: Moment 1 on a
Cyprus pin and Moment 2 on a Cyprus portfolio; add the D13 evidence notes to
showcase-script.md; exit criterion 2. Moment 1 walked and evidenced 2026-09-02. Moment 2 was first walked 2026-09-03 and failed (0 of 1 members scored); the two causes were diagnosed and remediated the same day (see the W2.4 row), and the book now answers 3 / 3 with headline band 4 and exposed value 16,769,833 USD through the API. The owner reopened/portfolios/acd702b1…on 2026-09-03 and confirmed the page renders those numbers, band 3 appearing on the ssp2-4.5/2030 toggle. Exit criterion 2 met. - O2 flip Moment 3 step 6 to
liveafter W4.4 passes on prod; exit criterion 3. Done 2026-09-02/09-03 —verify_engine_agreement.pypassed on prod 2026-08-31, the owner's/agreementwalk confirmed the UI half 2026-09-02, and the showcase step is flipped in this PR.
Conflict hotspots (files touched by >1 item, or shared with Phase 13)¶
scripts/packs/country_profile.py (W0.1 owns; W0.2/W0.3/W2.3 consume);
scripts/packs/dataset_identity.py (W0.1 only — put all templating there so
W0.2/W0.3 are consumers); scripts/smoke_e2e.py + .github/workflows/sdk-smoke.yml
(W3.2 — also Phase 13 #1029); src/climate_lama/db/repositories/hazard_repository.py,
src/climate_lama/core/lookup_service.py, src/climate_lama/core/compute_batch_service.py
(W1.1 — also Phase 13 #1022); src/climate_lama/api/v1/__init__.py +
src/climate_lama/main.py (W4.1 — also Phase 13 #1026); src/climate_lama/config.py,
docker-compose.yml, docker-compose.prod.yml, .env.example (W4.3a, W1.4 —
also Phase 13 #1018); src/climate_lama/core/impact_function_seeder.py
(W2.3; Phase 13 #1021 reads seeded_units_for — additive both sides);
src/climate_lama/core/impact_function_service.py (W1.3 only);
docs/data/volume-budget.md (W0.4, W2.2b–d — also #1028/#1030; append-only);
.github/workflows/deploy-prod.yml, docker-compose.hetzner.yml,
tests/test_env_template_parity.py (W4.3a only); UI src/lib/engines.ts,
src/components/agreement/EnginePicker.tsx, src/components/agreement/AgreementView.tsx
(W4.2 only).
Sequencing¶
- Start now: W0 (scripts only) and W4.2 (UI repo) — the only items that share no file with an open Phase 11/13 issue.
- After Phase 13: W1.1 after #1022; W3.2 after #1029; W4.1 after
#1026; W4.3a and W1.4 after #1018.
/phase-buildhonours these as cross-phaseDeps:. - W2 is operator work on prod after W0 + W1.1 + W1.3 ship in a backbone release — the coverage rule must be live before the first Cyprus dataset goes active, or Greek pins break (guardrail 7).
- W2.4 (surfaces) before O1; W4.5 before W4.4's earthquake run (the river-flood reference run does not need it).
Exit criteria¶
- Smoke:
sdk-smoke.ymlgreen on trunk with both legs — the Greek leg's assertions unchanged, the Cyprus leg saving a scenario linked to a non-empty result, and the cross-pin assertion passing. - Walk: the owner walks Moment 1 and Moment 2 on prod on a Cyprus pin and a Cyprus portfolio (reference surfaces present — W2.4); the showcase script carries the evidence notes.
- Agreement:
scripts/verify_engine_agreement.pypasses on prod — both enginescompleted,failed_engines == [],|divergence_pct.ead| < 1 %— and/agreementshows two engines with versions, reasons and the parity link; showcase Moment 3 step 6 flips tolive. - Guardrails hold:
grep -rn GREECE_BBOX scripts/ src/→ 0; the literal19.0,34.5,28.5,42.0appears insrc/only as the documentedconfig.pydefaults; the Greek name-freeze test passes;import climadagrep → the two permitted files; the default-curve uniqueness index exists.
Verify-at-build ledger ([VERIFY] items — each is the first step of its issue)¶
| Claim | How to verify | Item |
|---|---|---|
| ~~JRC river-flood v3.1.1 tiles cover Cyprus~~ FALSIFIED — see the W2.2a result table below | ~~HEAD the tile URLs~~ — HEAD-verified ≠ covered: nine RP tiles return 200 with zero valid Cyprus cells. A HEAD cannot answer a coverage question; only the windowed read could. This falsifies already-landed W0.4 acceptance, not just a W2.2a step |
W0.4 / W2.2a — EXECUTED 2026-08-28, gap |
| ESHM20 serves Cyprus | ~~one WCS GetCoverage~~ — the checklist was wrong: the efehrmaps.ethz.ch WCS is render-only (3-band uint8 RGB, not PGA floats) per scripts/packs/earthquake_eshm20.py's module docstring, finding 2, and is deliberately not the pack's source. Probe the EFEHR REST API the pack actually reads |
W2.2a — EXECUTED 2026-08-28, ok |
~~EURO-CORDEX EUR-11~~ EURO-CORDEX (CDS sis-heat-and-cold-spells, 0.1° regular grid — not the 0.11° rotated-pole EUR-11) clip over Cyprus is non-empty |
domain containment against the delivered grid recorded in scripts/packs/heatwave_packs.py; count valid land pixels at the first real clip |
W2.2a — CLOSED 2026-09-02: the one-cell eastern margin held — the Cyprus window is a 14x28 px grid at 0.1°, entirely inside the raster: 198 land cells read, 194 absent (sea), 0 out of extent — #1048 comment |
| ERA5 FWI + Eurostat Census 2021 cover Cyprus | one windowed read each | W2.2a — EXECUTED 2026-08-28, both ok |
Huizinga MaxDamage-Residential has a Cyprus row |
open the hashed xlsx; record row + value in the profile evidence string | W0.3 |
GADM 4.1 CYP vs XNC geometry split |
download both admin-0 JSONs; inspect extents | W0.5 |
| Prod NULL-footprint count | SELECT count(*) … footprint IS NULL AND status='active' as superuser |
W1.2 — EXECUTED 2026-09-02, 0 (37 active rows, all northlane) — #1048 comment |
| Heatwave band reachability over Cyprus | python scripts/score_band_reachability.py --dataset-id <heatwave dataset id> --country CYP --org-slug northlane — the tool committed by issue #1122 (src/climate_lama/core/band_reachability.py holds the logic). Exit 0 every expected band ≥ --min-cells and no band ≥ --max-band-share; exit 1 the gate failed, with the full histogram and every failing condition printed; exit 2 the question could not be asked (no active scheme, NULL cog_path, cell_metric scheme, RLS-invisible) — never read a 2 as a banding finding. Add --json to record the row verbatim |
W2.2d — EXECUTED 2026-09-02 with the PR #1137 fix: c1f3c898… ssp5-8.5 health v1 exit 0 PASS; a1f804a6… ssp2-4.5 health v1 exit 1 (band 9 Severe 0 cells); 32fcd25e… ECDE climatological v2 exit 1 (bands 8-10 under the floor, band 1 at 100 %) — follow-up filed as #1139 — #1048 comment. 2026-09-03 decision on #1139 (decision comment): --country GRC control run (default gate) — c1f3c898… health v1 ssp5-8.5 exit 0 over both CYP and GRC; a1f804a6… health v1 ssp2-4.5 exit 1 over GRC too (band 9 Severe 0 cells, band 8 78 cells) — a scenario ceiling, not a ladder defect; 32fcd25e… ECDE climatological v2 over GRC exit 1 on band_share only (band 1 = 100 % of 1140 cells; bands 8/9/10 reached with 60/110/20 cells, no floor failure). With the documented --max-band-share 1.01 remedy: ECDE v2 exits 0 over GRC and 1 over CYP (band_floor 8/9/10 = 1/0/0 cells) — a Cyprus window-size / coverage fact. Decision: option 2 — record the finding; no country-scoped scheme row, no re-band, no gate-default change. The must-exit-0 set is health v1 ssp5-8.5 over CYP + GRC and ECDE v2 over GRC with the share remedy; ssp2-4.5 and ECDE-over-CYP are recorded facts and keep exiting 1 as the standing evidence |
CLIMADA computes EQ/CF through the sidecar within 1 % |
local sidecar runs on the ESHM20 + Aqueduct coastal datasets | W4.5 — STILL OPEN, recorded negative-until-measured (per #1051's closing comment). Run 33237777433 (2026-08-29) passed both perils <1 % — but on a synthetic 4×5 fixture through the stdin/stdout parity sidecar, not on ESHM20/Aqueduct rasters and not through docker/climada-sidecar/server.py. #1051 closed 2026-09-02 without discharging this row: ClimadaEngineAdapter.supported_hazards stays at four perils, exactly what the /agreement picker showed the owner on the ESHM20 Cyprus walk. If earthquake/coastal-flood support on climada is wanted, it needs its own new issue — this row is not it |
| CX43 headroom for CLIMADA import | free -m on host + docker stats after sidecar boot |
W4.3a |
--profile vs COMPOSE_PROFILES under --env-file |
try both on the dev stack | W4.3a |
| Cyprus reference tile bbox is on land | plot the tile; three assets inside | W3.1 |
W2.2a Cyprus source-coverage probe — result table (executed 2026-08-28, issue #1046)¶
Every row below is reproducible with the committed probe
scripts/probe_country_source_coverage.py (--country CYP, Cyprus bbox
(32.0, 34.4, 34.8, 35.8) and the Greece control bbox (19.0, 34.5, 28.5, 42.0)
both read from scripts/packs/country_profile.py). The raster probes are
rasterio windowed reads over HTTP range requests — byte-for-byte the mechanism
scripts/ingest_scenario_hazards.py::fetch_clip uses — counting cells that are
neither nodata nor <= 0. Nothing was ingested and no archive was fetched whole.
Rows are phrased per probe command, not per peril. coastal_flood.py,
wildfire_fwi_ewds.py and heatwave_packs.py are standalone CLIs that no manifest
gates, so a manifest gap for a peril does not cancel that peril's CLI step in
the W2.2b/#1048 ops run.
| Source | Exact probe command | Observed result | ok/gap | Approx. download volume |
|---|---|---|---|---|
JRC river flood v3.1.1 (CEMS-EFAS id-0054), all nine RP tiles — the river_flood block of scripts/cyprus_historical_catalog_manifest.json |
python scripts/probe_country_source_coverage.py --country CYP --control GRC --only jrc |
All nine tiles HEAD 200 (272,286,610 – 349,928,950 B). Windowed read of the Cyprus window (1680 × 3360 px): 0 valid cells in every one of the nine. Greece control on the same files: RP10 2,097,768 (2.045 %) · RP20 2,221,588 · RP30 2,281,023 · RP40 2,318,743 · RP50 2,345,936 · RP75 2,391,369 · RP100 2,420,695 · RP200 2,485,392 · RP500 2,558,502 (2.494 %) |
gap | n/a (gap). The nine full-domain tiles are 2,804,642,262 B remote; the windowed clip transfers only the country window |
WRI Aqueduct Floods v2 riverine — scenario (inunriver_rcp4p5_00000NorESM1-M_2030_rp00010.tif) and historical baseline (inunriver_historical_000000000WATCH_1980_rp00100.tif) |
python scripts/probe_country_source_coverage.py --country CYP --control GRC --only aqueduct |
Scenario: HEAD 200, 88,733,128 B; 358 valid cells (0.634 %), max 0.278 m (Greece control 2.221 %). Historical: HEAD 200, 92,162,308 B; 368 valid cells (0.652 %), max 0.827 m (Greece control 2.377 %) |
ok | 88.7 / 92.2 MB remote per tile (Content-Length); the clip transfers only the 168 × 336 px Cyprus window (225,792 B float32 uncompressed per tile) |
WRI Aqueduct Floods v2 coastal (inuncoast_rcp4p5_wtsub_2030_rp0100_0.tif) — reached via scripts/packs/coastal_flood.py --country CYP, a standalone CLI, not via the scenario manifest |
python scripts/probe_country_source_coverage.py --country CYP --control GRC --only coastal |
HEAD 200, 47,109,700 B; 34 valid cells (0.060 %) at RP100, max 4.747 m (Greece control 0.206 %) |
ok | 47.1 MB remote per tile; the clip transfers the 168 × 336 px Cyprus window only |
| ESHM20 — EFEHR Seismic Hazard Services REST API, model id 81 (not the WCS the checklist named) | python scripts/probe_country_source_coverage.py --country CYP --only eshm20 |
GET https://efehr-services.ethz.ch/api/v1/maps/area?…&modelid=81&imt=PGA&poe=0.0010249×pan=1&soiltype=rock_vs30_800ms-1&aggregationtype=arithmetic&aggregationlevel=0.5&format=json → HTTP 200 in 58–93 s, 14,337 B, 287 grid points at 0.1°, PGA 0.138 – 0.872 g over the whole bbox; the point nearest the bbox centroid, (33.4 E, 35.1 N), reads 0.301 g |
ok | 14,337 B per return period → 71,685 B for all five the pack ingests |
| Eurostat Census 2021 population grid (GISCO Census-GRID V3, 1 km) | python scripts/probe_country_source_coverage.py --country CYP --only eurostat |
Archive 566,125,577 B, 9 members, no per-country split; the single needed member Eurostat_Census-GRID_2021_V3/ESTAT_Census_2021_V3.tiff (30,762,675 B compressed, 32,327,795 B raw) streamed out over HTTP ranges. EPSG:3035, 14 bands, band 1 = T. Cyprus window (227 × 285 cells) → 2,521 populated cells / 923,554 people (matches the real 2021 census of the Republic-controlled area) |
ok | 30,762,675 B — one member out of a 566,125,577 B archive; the archive is never fetched whole |
EURO-CORDEX heatwave cubes (CDS sis-heat-and-cold-spells), reached via scripts/packs/heatwave_packs.py --country CYP — a standalone CLI |
python scripts/probe_country_source_coverage.py --country CYP --only heatwave (evidence row — authenticated CDS Retrieve API, no anonymous probe exists) |
Delivered grid measured on a real successful retrieval (jobID cb35635a-f6ec-4228-8e39-ea1f35282523, 203,731,118 B; recorded in heatwave_packs.py): 425 × 599 cells at 0.1°, 30.1 N–72.5 N by 24.9 W–34.9 E. The Cyprus bbox is inside that domain with one 0.1° cell of margin at the eastern edge (34.8 E vs 34.9 E) |
ok, conditional | 203,731,118 B for the whole delivered domain (this collection declares no server-side area) |
ERA5 FWI historical (EWDS cems-fire-historical-v1), reached via scripts/packs/wildfire_fwi_ewds.py --country CYP — a standalone CLI |
python scripts/probe_country_source_coverage.py --country CYP --only fwi (evidence row — authenticated EWDS Retrieve API) |
wildfire_fwi_ewds.py documents a server-side area input on this ~0.25° collection, so the retrieval is clipped upstream, not locally; docs/data/volume-budget.md's #614 row records the EWDS historical half prod-ingested 2026-08-10 at 0.0295 GB for the whole Greece bbox |
ok | ≤ 0.0295 GB (the measured Greece figure; the smaller Cyprus bbox is smaller still) |
| (distinct from the row above) manifest-driven wildfire RP-footprint source | — | Stays gap in both Cyprus manifests, unchanged by this probe, for the reasons those files already state — see perils.wildfire.reason in scripts/cyprus_historical_catalog_manifest.json and scripts/cyprus_scenario_hazards_manifest.json (METHOD: fire-danger time series, not return-period footprints; ACCESS: authenticated Retrieve API) |
gap | n/a |
Three corrections to #1046's own checklist, each of which would have produced a wrong or meaningless answer if followed literally:
- JRC. The checklist asked only for a
HEAD200 per tile. The manifest already recorded exactly that on 2026-08-27 — and aHEADreturns 200 for a tile that is empty over Cyprus. Only the windowed read could detect the gap. - ESHM20. The checklist asked for a WCS
GetCoverage. Theefehrmaps.ethz.chWCS endpoint is render-only — a liveGetCoveragereturns a 3-band uint8 RGB picture, not PGA floats — andscripts/packs/earthquake_eshm20.py's module docstring (finding 2) records that it is deliberately not the pack's source. - Heatwave. The checklist's Context says "the heatwave packs do no server-side
bbox". True for the CDS heatwave pack (
heatwave_packs.py— neither collection declares anareainput), false for the FWI pack it groups with them (wildfire_fwi_ewds.pydoes declare one). It also names the grid "EURO-CORDEX EUR-11": naming EURO-CORDEX is right, the EUR-11 (0.11° rotated-pole) grid label is wrong.
Fallbacks and go/no-go for the #1048 ops run
- JRC river flood → retire the leg. The fallback is the Aqueduct historical WATCH/1980 riverine baseline (row 2), which measures 368 valid Cyprus cells at RP100.
- Heat run's first step: count valid land pixels over the island at the first clip — the one-cell eastern margin is the residual risk that step exists to close.
- No minimum-valid-cell guard exists anywhere on the ingest path (verified across
scripts/packs/andsrc/climate_lama/core/ingest/), so a 0.06 %–0.65 %-density Cyprus clip ingests silently. W2.4's reference surfaces and exit criterion 2 need a named Cyprus demo pin that falls inside a valid Aqueduct cell before they can be called done.
Out of scope (deliberate)¶
Pan-EU / continent-scale ingest (#447, gated on #446's out-of-core work); a
country/region column on impact_functions (follow-up when a second
country's curves need auto-resolution); a country-specific _WS_V98_REF
(the windstorm curve's central/western-Europe reference is applied unchanged
to Cyprus — ADR-023's deferred spatially-varying v98); publishing the sidecar
image; running CLIMADA in CI; OpenQuake or any third engine
(exploration/06-openquake-scoping.md);
measures on /agreement (the one place the engine#33 clamp divergence would
show — a deliberate future scenario); cross-hazard matrix runs;
building-level exposure; Northern Cyprus boundaries (XNC).
Issues (split 2026-08-27 — /phase-split, sized by /analyze)¶
Milestone "Phase 12 -- Country + Engine" in both repos, label milestone:phase-12.
| Wave | Backbone | UI (climate-lama-ui) |
|---|---|---|
| W0 | #1035 country profile + identity templating + Greek name-freeze test (foundation) · #1036 hazard packs + backfill + generator · #1037 exposure packs + Cyprus MaxDamage row · #1038 Cyprus manifests + volume-budget rows · #1039 GADM CYP fixtures + disputed-borders note | — |
| W1 | #1047 coverage rule 2b + covering= + ADR-079 (after #1022) · #1049 backfill_dataset_footprint.py · #1050 default-curve guard + model-declared index (after #1019/#1024) · #1052 per-approval clip override (after #1018) |
— |
| W2 | #1046 source-coverage probe (spike, not a build target) · #1040 ESRM20 Cyprus curves · #1048 ops tracker — Cyprus data run on prod (needs-prod-proof, not a build target) |
— |
| W3 | #1041 Cyprus reference tile + exposure · #1045 smoke Cyprus leg (after #1029, #1039, #1041, #1047) | — |
| W4 | #1054 GET /v1/engines (after #1026) · #1042 sidecar deploy path (after #1018) · #1043 DEPLOYMENT/RUNBOOK sections · #1044 verify_engine_agreement.py · #1055 CLIMADA supported_hazards revalidation + engine_version in the compare view · #1051 ops tracker — sidecar live on prod (not a build target) · #1053 UI coordinator (not a build target) |
ui#255 EnginePicker reads the roster · ui#256 AgreementView explains divergence |
Sizing (/analyze, 2026-08-27): every buildable issue is Sonnet 5 · xhigh
except #1038 → Opus 5 · max (live HEAD-verification of real tile URLs,
must-not-fabricate). Dependency line as filed: #1035 → {#1036, #1037, #1038,
1040}; #1039, #1041, #1049, #1044, #1055, ui#255, ui#256 independent;¶
1047 ← #1022; #1050 ← #1019, #1024; #1052 ← #1018; #1042 ← #1018;¶
1054 ← #1026; #1043 ← #1042, #1044; #1045 ← #1029, #1039, #1041, #1047.¶
Ops trackers #1048/#1051 run after their waves are released. Land with
/phase-build phase-12 in each repo.
Build status (2026-08-29 — every wave landed; the phase now waits on a release)¶
20 of 23 issues closed, ending at trunk 0ff4420. W0 (#1035–#1039) and W1
(#1047, #1049, #1050, #1052) landed 2026-08-27; W2's #1046 probe, its two
follow-ups #1083/#1084, and #1040 landed 2026-08-28; W3 (#1041, #1045) and W4's
backbone lane (#1042, #1043, #1044, #1054, #1055) landed 2026-08-28/29. Both UI
children (ui#255, ui#256) merged 2026-08-28. Still open: #1048 and #1051
(needs-prod-proof operator trackers) and #1053 (the UI coordinator that closes
behind #1051's /agreement evidence). None is a /build target.
Update 2026-09-02: #1051 and #1053 both closed — the sidecar's prod proof
(verify_engine_agreement.py exit 0, /agreement UI walk) and the UI
coordinator behind it, respectively. 22 of 23 issues are now closed. The
one remaining open issue is #1048, carrying the Cyprus portfolio-rollup
remediation the Moment 2 walk's failure surfaced (see the exit-criteria table
below and the W2.4 row above); none is a /build target.
Update 2026-09-03: #1048 closed on the passing rollup proof (W2.4 row). All 23 backbone issues (#1035–#1055, #1083, #1084) and both UI children (ui#255, ui#256) are closed. Mid-review: mid-review-phase-12.md — verdict Ready for next phase. Phase 12 CLOSED 2026-09-03 on the owner's browser re-walk of the Cyprus book (exit criterion 2, the last open one); all four exit criteria met.
The four cross-phase Deps: on Phase 13 were inverted, deliberately.
1042 ← #1018, #1054 ← #1026 and #1052 ← #1018 were **file-collision orderings¶
only (the compose / .env.example hotspots; main.py router registration),
so Phase 12 went first on the same precedent the owner set in W1. #1045 ← #1029
was not a file collision — it was a genuine design fork over which phase
defines the smoke-leg abstraction, and the owner resolved it by inverting: #1045
now defines CountryLeg (a NamedTuple in scripts/smoke_e2e.py, the country
axis of a country × hazard matrix) for Phase 13's #1029 to hold** rather than
subclass. Premise-correction comments are posted on all four issues; their line
numbers are stale.
Exit criteria as they stand:
| # | State | Evidence |
|---|---|---|
| 1 — Smoke | MET | sdk-smoke.yml green on 0ff4420 (run 33238895155) with both legs. The log carries check [X] Cross-pin: each leg's tile centre resolves that leg's own dataset passing for both GRC and CYP, and distinct E2E_READY_GRC / E2E_READY_CYP lines whose hazard_ids differ and each match that leg's own cross-pin resolution — W1.1's coverage rule proven end-to-end with no region filter. (The ids themselves are per-run: the smoke stack is built and torn down each time, so the assertion is the durable evidence, not the UUIDs) |
| 2 — Walk | MET | Data prerequisites met via #1048 on prod v0.21.0 (closed 2026-09-03). Moment 1 walked 2026-09-02 on both a Cyprus pin (34.89, 33.61 → Earthquake 6/10, River flood 4/10) and a Greek pin (40.60, 22.80 → River flood 7/10), D13 evidence in showcase-script.md. Moment 2 first walked 2026-09-03 and scored 0 of 1 members; both causes — no baseline/0 Cyprus surface, and the rollup's res-8-cell requirement against a GHSL-built surface — were diagnosed and remediated the same day. The re-membered book now answers 3 / 3 scored: headline 4 Moderate-low river flood at baseline, 3 at ssp2-4.5/2030, 4 at ssp5-8.5/2050, exposed value 16,769,833 USD (W2.4 row). The owner re-walked /portfolios/acd702b1… in the browser on 2026-09-03 and confirmed 3 / 3 scored, headline 4, band 3 on the ssp2-4.5/2030 toggle, coastal flood as metric absent — criterion met |
| 3 — Agreement | MET | scripts/verify_engine_agreement.py ran on prod 2026-08-31 (comment of 2026-08-31 12:19): exit 0, both engines completed, failed_engines == [], |divergence_pct.ead| = 0.0000 %. The owner's /agreement walk (2026-09-02, UI v0.12.0) confirmed the UI half: two engines listed and selected by default, EAD/AAI 409,851,434 on both, 0 % badges, climate_lama marked reference, parity link rendered, and switching to ESHM20 earthquake - Cyprus disables climada with the inline reason "(not supported for Earthquake)" (#1053, closed). Showcase Moment 3 step 6 flips to live. On W4.5: ClimadaEngineAdapter.supported_hazards still stays at four perils (river_flood, tropical_cyclone, wildfire, storm_europe) — the earthquake/coastal-flood question below stays open on its own row, but does not gate this criterion, which only requires the default roster to agree within 1 %. Caveat recorded on #1053: no per-engine version string was observed in the completed table — [ENHANCE-NEXT], not a blocker |
| 4 — Guardrails | MET (re-run 2026-08-29 on 0ff4420) |
grep -rn GREECE_BBOX scripts/ src/ → 0; 19.0,34.5,28.5,42.0 in src/ only as config.py:417's documented default and the clip.py:22 docstring citing it; tests/scripts/test_greek_name_freeze.py present; import climada → zero real call sites (the sole textual hit is climada_adapter.py:5's own "must never import climada" docstring); ONE_DEFAULT_PER_ORG_HAZ_TYPE_INDEX declared on the model |
Closeout sweep (2026-08-29) — what the release gates caught¶
Dispatching both gates before the release found two defects Phase 12 had shipped, neither reachable from the backbone's own CI. Both are fixed; the sweep is recorded because the reason they were invisible outlives them.
Fixed:
- #1091 (
b6e7406) —ingest_historical_catalog.py::cmd_verifyHEADed the data URLs but never re-read thecopyright.txtitslicense_notecites, so a well-formed but fabricated citation passed the gate (proven by mutation during #1088's review). Now GETs it and re-checks the pinned sentence. - #1102 (
bd9ba9a) — the parity harness passed vacuously when both engines computed zero (harness.py:56,denom = max(abs(climada_ead), 1e-12)),parity.ymlexited 0 when the sidecar was absent and every test skipped, and a pass recorded no divergence number. Run 33250141990 now printsrel_diff=0.000%for all three perils at EAD ≈ 5e6 — non-zero, so the earlier result was real agreement, not a vacuous pass. - #1103 (
9f13080) —docs/CLIMATE_LAMA.md:1347-1349asserted CLIMADA was not a runtime dependency and thatclimada_adapter.pywas removed in Phase 0, contradicting both the shipped adapter and the same file's own engine roster at:365. - #1109 (
52750a5) —smoke_e2e.pycould no longer walk one country. W3.2 (#1045) madeDEFAULT_LEGSunconditional with no leg-selection flag, andrun()validates every leg's fixtures before any HTTP. Any caller staging only Greek fixtures died in ~1.5 s on aFileNotFoundError— which is what had the UI repo's e2e gate red. Fixed by--legs; omitting it still walks both, sosdk-smoke.ymland exit criterion 1 were untouched.climate-lama-uiPR #280 passes--legs GRC. - #1111 (
efefc97) — the roster claimed CLIMADA was available where it cannot run. Three individually-defensible changes composed into a broken default: W4.3a (#1042) hardcodedCLIMADA_SIDECAR_URLin compose while the sidecar service isprofiles: [climada]; W4.1 (#1054) reportsavailablefrombool(that url), a config check that only equalled a deployment check while the URL was set where the sidecar ran; and ui#255's picker auto-selects every available engine. So every sidecar-less stack defaulted to a two-engine comparison whose CLIMADA half always failed. This had a prod failure mode: #1051 has the operator set that key in.env.prod, after which any sidecar outage would have half-failed every default comparison. Compose now uses${CLIMADA_SIDECAR_URL:-}.
Still open:
- #1104 — parity
measures
tests/parity/sidecar_runner.py, a second CLIMADA implementation; production runsdocker/climada-sidecar/server.py. The adapter's CSR intensity encoding, the insured-loss branch and the payload size cap are exercised by nothing. This is the structural reason W4.5 cannot close on a parity run alone.
The lesson worth keeping. Both shipped defects lived in the seam between
repos, and both were invisible to the backbone's own gate: ci.yml and
sdk-smoke.yml were green on every commit that introduced them. The UI's
e2e-smoke.yml is workflow_dispatch-only and had not run since 55af1f07
(2026-08-27) — before either Phase 12 UI child merged. A cross-repo phase needs
the sibling's gate dispatched before its release, not just its own.
The W2 rows above are flipped to done, evidenced by the #1048 comments dated 2026-08-31, 2026-09-02 and 2026-09-03 — the portfolio-rollup half of W2.4, that issue's last checkbox, passed 3 / 3 on 2026-09-03 and closed it.
Corrections the split surfaced (2026-08-27 — the doc above is amended by these)¶
- Guardrail 12 / D4: the partial unique index
(org_id, haz_type) WHERE is_defaultalready exists (migrations 0011 → 0053). #1050 adds the cardinality guard, fixes the messages and declares the index on the model (migrations-gate.ymlstill fires viamodels/**). The phase expects no new migration. - D12: the compare response carries no per-engine
engine_versiontoday (serialize_comparison_engines,engine_comparison_service.py:135-166); #1055 adds it, ui#256 renders it forward-compatibly until then. - D9 / W4.3a: the
.envparity spine needs no change — both CLIMADA keys already sit below the tail marker in.env.exampleand.env.example.prod; the prune is image-ID-keyed and${IMAGE_NS}-scoped, so the:localtag is for deterministic naming, not prune survival.deploy-prod.ymlruns every step on the host inside an ssh heredoc, so theCLIMADA_SIDECAR_URLgate is read from.env.prodon the host in each of the three$Psteps. - W0.2:
river_flood_jrc.py:106andriver_flood_aqueduct.py:134also hardcode_greece.tifcache names while reading bbox from the manifest — a Cyprus manifest would clip into a Greece-named cache file; both are in #1036.gen_test_hazards.pyalready has--bbox(its own inset default). - W3.2:
E2E_READYis a published contract with six consumers, includingtests/scripts/test_smoke_e2e.py:61, which asserts the workflow's literal grep — #1045 carries the sweep (or the non-breakingleg=key alternative). - W1.1: the lookup seam is
lookup_point(lookup_service.py:1358), not abuild_score_card; the new ADR is ADR-079 (078 is Phase 13's). - W1.4: the approval endpoint is
POST /v1/admin/pending-ingests/{id}/approve(api/v1/admin/pending_ingests.py:245-266, no request body today);worldpop_iso3is consumed at catalog-poll time, not approval — dropped from #1052. - W4.1:
api/v1/__init__.pyis empty; routers register inmain.py:192-215only.