Phase 12 — Mid-Review¶

Date: 2026-09-03 Reviewer: Claude (desk pass over the prod evidence) and the owner (/lookup and /agreement walks, 2026-09-02) Status: closed 2026-09-03 — the owner's Moment 2 re-walk confirmed the rollup in the browser Artifact walked: the live production stack — https://climate-lama.online (UI v0.12.0) and https://api.climate-lama.online (backbone v0.21.0), against the MVP slice in CLAUDE.md and Moments 1–3 of showcase-script.md. Framework: per ~/.claude/collaboration-framework.md, Cluster 1 ritual. Evidence pack: phase-12-country-engine.md and the closing comments on #1048, #1051, #1053 and #1139.

Verdict¶

Verdict: Ready for next phase. The owner's Moment 2 re-walk on the Cyprus book ran 2026-09-03 and confirmed the page.

Every one of the phase's 23 backbone issues (#1035–#1055, plus the #1083/#1084 coverage follow-ups) and both UI children (ui#255, ui#256) are closed as of 2026-09-03. All four exit criteria are met. The fourth — the walk — was met at the API layer first and confirmed in the browser the same day: the Cyprus portfolio rollup, which scored 0 of 1 members when the owner first walked it on 2026-09-03, now scores 3 / 3 with a headline band and a real exposed value after the two defects behind that zero were diagnosed and remediated the same day. The owner's re-walk of /portfolios/acd702b1… on 2026-09-03 confirmed it in the browser: 3 of 3 scored, headline 4, band 3 on the ssp2-4.5/2030 toggle, coastal flood as metric absent. Nothing is owed.

The phase's honest headline is that the walk found what the tests could not. Both root causes of the empty rollup were invisible to every gate the project runs: the surfaces existed, the datasets were ingested, the RLS bound, the COGs read, and each individual assertion passed. What failed was the composition — a member is scorable only if the surface-building exposure happened to have a point in the same res-8 H3 cell, and nothing anywhere states that.

#	Check	State
1	Artifact health	✅ prod stack healthy, sidecar included
2	Happy path (Moments 1–3)	✅ all three moments walked on prod; Moment 2 re-walked 2026-09-03 after the rollup remediation
3	Unhappy paths	✅ every miss reported honestly, never as a zero
4	UI/UX presentability	🟨 no embarrassments; two legibility gaps (F-2, F-3)
5	User-path completeness	✅ every Phase 12 surface reachable by clicking
6	Promised-but-unbuilt	🟨 three recorded gaps, all deliberate or measured

Check 1 — Artifact health ✅¶

Prod runs backbone v0.21.0 + UI v0.12.0, deployed 2026-09-02 06:41 UTC, Alembic head 0088. climate-lama-climada-sidecar:local is healthy at 276 MiB RSS against the 4 GiB limit #1042 set (6.7 %), and survived two consecutive deploys without being torn down by --remove-orphans (runs 33386494542 and 33387177394). Host headroom at the reading: 11.3 GiB available of 15.6 GiB.

The Cyprus data plane is real and small: 16 datasets, 173,431 B total (≈ 0.17 MB) across both object-store prefixes — three orders of magnitude under the calculated upper bounds in docs/data/volume-budget.md, because a 0.6 % valid-cell density deflate-compresses to almost nothing. Zero active hazard datasets have a NULL footprint (37 rows checked as superuser).

Check 2 — Happy path ✅¶

Moment 1 — walked 2026-09-02, passes. Cyprus pin 34.89, 33.61 → Earthquake 6/10 Elevated (ESHM20 Cyprus) and River flood 4/10 Moderate-low (Aqueduct Cyprus historical baseline 1980 (WATCH), RP10 0.11 m / RP50 0.21 m / RP100 0.25 m). Greek pin 40.60, 22.80 → River flood 7/10 High (JRC Greece, RP100 1.258 m). The cross-country property — a pin resolves its own country's dataset with no region filter — holds in the browser, not just in the smoke walk.

Moment 2 — re-walked 2026-09-03, passes. The first walk scored 0 of 1 members with every hazard labelled "No risk surface in scope", at baseline and at SSP5-8.5/2050. Two independent causes, both diagnosed read-only against org_risk_surfaces / org_risk_cells:

No baseline/0 Cyprus surface existed at all. The 12 Cyprus rows were {ssp2-4.5, ssp5-8.5} × {2030, 2050, 2080} only, while Greece had baseline rows. The rollup's default scope is baseline/0, so the default view could never answer for Cyprus.
The member sat in a cell the surface does not carry. The rollup scores a member only through the res-8 H3 cell the member's own coordinates fall in (core/rollup_service.py, risk_lookup_resolution = 8, no parent-cell fallback), and an org surface only has cells where the exposure dataset that built it had a point. The 12 Cyprus surfaces were built over GHSL built-up volume Cyprus (2478 points), so a LitPop point 500 m from any GHSL point is simply not in the surface — whatever the raster says there.

Both were remediated through the real API, no schema or code change. The two missing baseline surfaces were built through the matrix endpoint over GHSL Cyprus (river 41040218… → result 9690725e…; coastal d7a29d22… → result 8784bbc8…), each landing 3902 cells, ready, at 07:11:52 / 07:12:01 UTC. The book acd702b1-4784-4a38-bda5-0020d7a0fa11 was re-membered with three GHSL Cyprus rows sitting inside valued baseline cells. Result:

View	Scored	Headline	River flood	Coastal flood	Exposed value (USD)
baseline (default)	3 / 3	4 Moderate-low, river_flood	band 4, aggregate 0.2542 m	`metric_absent_from_cells`	16,769,833
`?scenario=ssp2-4.5&horizon=2030`	3 / 3	3, river_flood	band 3, 0.2495 m	`metric_absent_from_cells`	16,769,833
`?scenario=ssp5-8.5&horizon=2050`	3 / 3	4, river_flood	band 4, 0.2605 m	`metric_absent_from_cells`	16,769,833

Worst members in all three views: 7,043,304 / 5,050,948 / 4,675,581 USD, each at the headline band. The owner re-walked /portfolios/acd702b1… in the browser on 2026-09-03 and the page rendered exactly those numbers, band 3 appearing on the ssp2-4.5/2030 toggle.

Moment 3 — walked 2026-09-02, passes. /agreement lists climada and climate_lama, both selected by default from GET /v1/engines; EAD and AAI both 409,851,434 with 0 % divergence badges, climate_lama marked reference, parity link rendered. Switching to ESHM20 earthquake - Cyprus disables climada with the inline reason "(not supported for Earthquake)" — the roster's supported_hazards drives the picker, exactly as D11 specified.

Check 3 — Unhappy paths ✅¶

Three misses were exercised, and all three reported honestly rather than as a zero — the property the phase's guardrails exist to protect:

Metric absent where the surface exists. Coastal flood on the Cyprus book returns metric_absent_from_cells, not band 1. The three cells carry no coastal depth; the payload says so.
Peril not supported by an engine. climada is disabled for earthquake with a reason string, not silently dropped or shown as agreeing.
Band ladder that does not fill. The heatwave reachability gate exits 1 with a full per-band histogram rather than shipping a degenerate banding. The --country GRC control proved the ssp2-4.5 band-9 miss is a scenario ceiling, not a Cyprus defect: Greece misses it too, while ssp5-8.5 reaches it with 1902 cells.

Check 4 — UI/UX presentability 🟨¶

No cosmetic embarrassments; the two gaps are legibility, not polish. "0 of 1 members scored" with every hazard labelled "No risk surface in scope" gave the owner no way to tell no surface from surface present, member has no value here — the single observation that cost the most diagnostic time this phase (F-1). And the /agreement completed table shows metric values and divergence but no per-engine version, so two engines agreeing says nothing about which builds agreed (F-2).

Check 5 — User-path completeness ✅¶

Every surface this phase touched is reachable by clicking: /lookup and /agreement are in the primary nav, /portfolios/:id is a nested route reached from the portfolio list. No Phase 12 surface requires URL-typing.

One API-shape trap is recorded rather than fixed: the rollup query parameter is horizon, not horizon_year. A hand-written ?scenario=…&horizon_year=… silently re-reads baseline/0 instead of erroring. The UI itself sends horizon correctly, so this bites operators and scripts, never users.

Check 6 — Promised-but-unbuilt 🟨¶

Three gaps, each measured rather than assumed:

Cyprus earthquake / wildfire / heatwave org surfaces do not exist. Only river_flood and coastal_flood surfaces were built. Neither Cyprus manifest designates the other perils for designated_for_reference_surfaces. Greece has one earthquake org surface, so this is a recorded gap, not an impossibility (F-3).
W4.5 — CLIMADA on earthquake / coastal flood — is unanswered, deliberately. ClimadaEngineAdapter.supported_hazards stays at four perils, which is exactly what the picker showed the owner. This is the negative-until-measured outcome the tracker asked for (F-4).
Country-scoped score-scheme rows: dropped. Decided on #1139 after a /reassess pass refuted the original three-amendment draft (F-6).

Findings¶

F-1 · The rollup cannot score a member unless the surface-building exposure shared its res-8 cell, and says nothing about it — `[ENHANCE-NEXT]`¶

Two compounding defects behind one symptom. Mechanically: core/rollup_service.py scores a member only through its own res-8 H3 cell, with no parent-cell fallback, so a member 500 m from the nearest surface point is unscorable even though the raster is valid there. Presentationally: the payload and the UI answer no_cells_in_scope for both "there is no surface" and "there is a surface, this member has no value in it", which are entirely different situations for a user deciding whether to trust the number.

This is what turned a data question into a full day of read-only forensics against org_risk_surfaces. A per-member unavailable_reason would have made it a one-glance diagnosis.

Recommended action: ship #1154 early in the next phase — parent-cell fallback and the per-member reason. The fallback is the correctness half; the reason string is the half that pays for itself the next time a book reads empty.

F-2 · `/agreement` shows no per-engine version — `[ENHANCE-NEXT]`¶

The completed comparison table renders metric values and divergence badges but no engine version string. engine_comparison_service.py:182 does emit engine_version from result metadata, so either the UI does not render it or the metadata is unstamped on prod. Two engines agreeing to 0.0000 % is a much weaker claim when the reader cannot see which two builds agreed — and engine plurality is a product commitment, not a detail.

Recommended action: ship ui#299. Establish first which of the two causes it is — an unrendered field costs a line of TSX; unstamped metadata on prod is a backbone fix.

F-3 · Cyprus has no earthquake, wildfire or heatwave org surface — `[BACKLOG]`¶

org_risk_surfaces holds 6 river_flood + 6 coastal_flood Cyprus rows (now 7 + 7 with the baselines), and nothing else. A Cyprus pin still answers for earthquake — Moment 1 returned 6/10 from ESHM20 — because the lookup path reads the dataset directly; it is the rollup plane that is empty for those perils. So a Cyprus book will report "no surface in scope" for earthquake forever, while a Cyprus pin one metre away answers.

Recommended action: leave backlogged. Building them is a matrix run per peril, not new code, and no showcase moment needs them today. Revisit when a Cyprus book is a demo target rather than a proof artifact.

F-4 · W4.5 — CLIMADA earthquake / coastal flood on real data — unmeasured — `[BACKLOG]`¶

Whether CLIMADA computes earthquake and coastal_flood within 1 % through the production sidecar on real data is unanswered. It cannot be settled by the parity harness, structurally: #1104 established that parity measures tests/parity/sidecar_runner.py — a second CLIMADA implementation — while production runs docker/climada-sidecar/server.py, whose CSR intensity encoding, insured-loss branch and payload cap are exercised by nothing.

Recommended action: file its own issue before adding either peril to the roster, with the executable check named up front (a prod-sidecar comparison run on a real Cyprus/Greek dataset per peril, divergence recorded as a number). Do not expand supported_hazards on the strength of a parity green.

F-5 · `verify_engine_agreement.py` and `smoke_e2e.py` cannot run in a shipped image — `[BACKLOG]`¶

Both scripts import requests, which is a dev-only dependency, so neither runs inside a released container. Every prod proof this phase produced therefore ran from an operator workstation or a staged script — workable, but it means the tools that verify a release are not in the release.

Recommended action: #1141 stays backlogged. Fold it into the next release-tooling pass rather than handling it alone; the workaround is documented and cheap.

F-6 · Country-scoped score-scheme rows — `[DROP]`¶

Proposed on #1139 to fix the Cyprus heatwave gate failures. Dropped on 2026-09-03 after the --country GRC control run and a /reassess pass (2 of 6 sub-claims confirmed, 2 corrected, 2 refuted):

The failure is not country-specific — health v1 ssp2-4.5 reaches band 9 with 0 cells over Greece too, so a Cyprus-keyed row would fix nothing.
The serving path has no region dimension: models/score_scheme.py has no region column and core/scoring.py's select_active_scheme keys on hazard_type + metric + metric_unit. Option 1 was a migration plus a serving-path change plus a precedence rule, for a defect it does not address.
EXPECTED_REACHABLE_BANDS is keyed by scheme_ref alone, so exempting band 9 of health v1 would also exempt ssp5-8.5, where the band is reached.

Recommended action: none — the decision is recorded as option 2 (record the facts, change no gate default) in the phase doc's [VERIFY] ledger. The must-exit-0 set is health v1 ssp5-8.5 over CYP + GRC and ECDE v2 over GRC with the documented --max-band-share 1.01 remedy; ssp2-4.5 and ECDE-over-CYP keep exiting 1 as the standing honest evidence.

Exit criteria tracking¶

#	State	Evidence
1 — Smoke	MET	`sdk-smoke.yml` run 33238895155 green with both country legs, the cross-pin check passing for `GRC` and `CYP`; re-confirmed on `fe05e17`
2 — Walk	MET	Moment 1 walked 2026-09-02 on both country pins; Moment 2 proven through the API 2026-09-03 (3 / 3 scored, headline band 4, exposed value 16,769,833 USD) and re-walked by the owner in the browser the same day, confirming the rendered numbers and the horizon toggle
3 — Agreement	MET	`verify_engine_agreement.py` exit 0 on prod 2026-08-31, both engines `completed`, `failed_engines == []`, 0.0000 % EAD divergence; owner `/agreement` walk 2026-09-02 (#1053)
4 — Guardrails	MET	Re-run 2026-08-29 on `0ff4420`: no `GREECE_BBOX`, zero real `import climada` call sites, `test_greek_name_freeze.py` present, one-default index declared

Findings summary¶

ID	Finding	Tag	Issue
F-1	Rollup res-8 cell requirement + no per-member reason	`[ENHANCE-NEXT]`	#1154
F-2	No per-engine version on `/agreement`	`[ENHANCE-NEXT]`	ui#299
F-3	Cyprus EQ / WF / heatwave org surfaces absent	`[BACKLOG]`	— (recorded here)
F-4	W4.5 CLIMADA EQ / CF real-data revalidation	`[BACKLOG]`	— (needs filing)
F-5	`requests` dev-only blocks in-image script runs	`[BACKLOG]`	#1141
F-6	Country-scoped score-scheme rows	`[DROP]`	#1139 (decided)

Carried into the next phase¶

Only F-1 is worth scheduling deliberately: it is the one finding that changed a user-visible answer from "0 of 1" to "3 / 3", and the one whose absence cost real time. F-2 is a small, high-leverage credibility fix. F-3 through F-5 are genuine backlog — measured, bounded, and blocking nothing. F-6 is closed.

The durable lesson is narrower than "walk the demo": a surface is only as useful as the exposure that built it, and nothing in the system says which exposure that was. Every gate in this repo passed while the composed answer was empty.