Phase 13 — Mid-Review¶
Date: 2026-09-03 Reviewer: Claude (desk pass over the prod replay evidence) and the owner (Moment 2 steps 8–9 walk on both replay books, 2026-09-03) Status: closed 2026-09-03 — the owner walked both replay books in the browser Artifact walked: the live production stack —
https://climate-lama.online(UIv0.12.0) andhttps://api.climate-lama.online(backbonev0.21.0), live mode on since 2026-09-02, against Moment 2 steps 8–9 of showcase-script.md. Framework: per~/.claude/collaboration-framework.md, Cluster 1 ritual. Evidence pack: phase-13-event-replay.md § Build status, and the closing comments on #1146 and #1030.
Verdict¶
Verdict: Ready for next phase. Backbone, UI and prod replays are live, and the owner walked Moment 2 steps 8–9 on both replay books on 2026-09-03.
Every Phase 13 backbone issue (#1018–#1031) and every UI issue (ui#240–ui#245)
is closed. The code shipped as backbone v0.21.0 + UI v0.12.0 on 2026-09-02,
live mode has been on prod since the same day, and on 2026-09-03 both
replays ran on production against real books through the public API — a real
2020 earthquake and a real 2021 wildfire, each ingested as an event snapshot,
computed on the deterministic-event lane, and reported as a screening-grade
event loss with no annual figure anywhere in the payload.
The last gap closed on 2026-09-03 when the owner looked at it in a browser: the Samos card opened to its header, the life-safety line, the 585,548,574 USD screening-grade loss, the ranked table and the ShakeMap raster; the Evia card played its six-tick scrubber with observed-at and loss changing per day and peaking on 6 August; a book without an event showed no Events heading; a page search for "alert" found nothing. Phase 12's identical gap is where its only real defect hid, which is why this pass stayed open until a human confirmed the screen rather than rounding up from the payloads.
| # | Check | State |
|---|---|---|
| 1 | Artifact health | ✅ prod on v0.21.0/v0.12.0, live mode on, replays resident |
| 2 | Happy path (Moment 2 steps 8–9) | ✅ API-proven end to end and walked by the owner 2026-09-03 |
| 3 | Unhappy paths | 🟨 dark landing and guardrails hold; one 502 (F-2) |
| 4 | UI/UX presentability | 🟨 one real wart — 100 identically-named assets (F-1) |
| 5 | User-path completeness | ✅ Events reached by clicking; dark elsewhere by design |
| 6 | Promised-but-unbuilt | ✅ complete; perils.wildfire.status correctly still gap |
Check 1 — Artifact health ✅¶
Prod runs backbone v0.21.0 (fe05e17) + UI v0.12.0 (c35b1dc), deployed
2026-09-02 06:41 UTC, Alembic head 0088, live mode enabled via .env.prod
reaching the running api and worker containers.
One precondition had to be fixed first, and it is the phase's most
transferable finding. Prod's region vocabulary held only GRC | Attica | 1,
CYP | Cyprus | 0 and CYP | Larnaca | 1 — there was no GRC | Greece | 0
row — so an ingest declaring region: "Greece" was refused with
E_VALIDATION {"region": "Greece"} although every live Greek dataset on the
box already carried region="Greece". The vocabulary and the data had drifted
apart with nothing detecting it. Fixed by loading the tracked GADM 4.1 admin-0
fixture (gadm41_GRC_0.json.gz, sha256 b6febf95…, 145,741 B) through
scripts/load_admin_boundaries.py at 07:15:36 UTC (F-4).
Check 2 — Happy path ✅¶
Both replays are resident on prod, built from real provider data through the real ingest API, on real LitPop Greece books — no invented assets.
Samos–İzmir M7.0, 2020-10-30 (W2)¶
| Book | portfolio 9de7a920-3aed-4aa4-b06e-08b46c989068 — the 100 LitPop Greece rows inside 25.8–27.2 E, 37.3–38.6 N (Samos, Ikaria, Chios) |
| Live event | 2e3fd696-b326-442c-b72c-c21363730e5e, earthquake, open, portfolio_id bound, source "USGS ShakeMap, atlas product us7000c7y0 v1" |
| Snapshot | dataset 47564d9e-e13d-404b-b6b2-bd4ca58c13e6, valid_at 2020-10-30T11:51:27Z, intensity_unit g, ingested in single-event mode (job 3bcbd98b…, 6 s) |
| Compute | job 76e9ec1d-8d89-45d7-9a08-4473ff6bb2fb, completed in 111 s |
| Result | event_loss 585,548,574 USD, grade screening; top members 134,898,561 / 78,285,453 / 74,662,151 USD; no ead/aai key in the payload |
| Layers | raster LayerSpec, source_url /v1/tiles/cog/47564d9e…/{z}/{x}/{y}.png, legend unit g, domain 0.0024–0.129 g, band max 0.526 g (read by titiler from the COG) |
North Evia wildfire, 3–10 Aug 2021 (W3)¶
| Book | portfolio dbe2a065-1f7b-47e5-8ee1-1f9e789a8087 — the 216 LitPop Greece rows inside 22.8–24.1 E, 38.5–39.1 N |
| Live event | da01cb7a-fa36-4510-8fcb-5357134f2508, wildfire, open, portfolio_id bound, source "NASA FIRMS VIIRS S-NPP 375 m, VIIRS_SNPP_SP (Collection 2)" |
| Snapshots | six, one per acquisition day, all intensity_unit K, indicator=false, ~6 s each: 35d9724f… 08-05T00:31Z · 795a32a1… 08-06T00:12Z · ce94fa69… 08-07T01:33Z · 77ab86d9… 08-08T01:15Z · 84de6c15… 08-09T00:56Z · 8db5cecb… 08-10T00:37Z |
| Curve | impact_function_id omitted on every compute — the seeded Luthi 2021 WF buildings 1km default (b671d3e2…, unit K) auto-resolved; every job completed |
| Results | 08-05 926,127 · 08-06 21,451,558 · 08-07 15,095,527 · 08-08 13,491,984 · 08-09 5,470,912 · 08-10 15,714,748 USD — grade screening, no ead/aai |
| Layers | raster, tile route, legend unit K |
The 08-06 peak matches the fire's worst day — the series is not merely non-zero, it has the shape the event actually had. That is the strongest single signal in this phase that the lane is measuring something real.
The pack itself ran against the live FIRMS archive: 2 requests (2021-08-03
day_range 5, 2021-08-08 day_range 3), both HTTP 200, 314,404 B raw, 4,092
rows, brightness column pinned as bright_ti4 (Collection 2 archive).
Six per-day GeoTIFFs, 113 × 67 EPSG:4326 float32, 11,606 B total. The vendored
fixture is a real extract (rows[::16] of 2021-08-06), fixture_is_synthetic
= false.
Walked 2026-09-03: the owner opened /portfolios/9de7a920… and
/portfolios/dbe2a065…, entered the Events section and read steps 8–9 in the
browser — the observed footprint on the map in place of the return-period
layer, the event loss stated with its observed-at, its age at render and its
source version, the Evia scrubber changing the loss per tick and peaking on
6 August, and no Events heading on a book without an event.
Check 3 — Unhappy paths 🟨¶
The dark landing works as designed: GET /v1/live-events?portfolio_id= returns
exactly one event for each replay book and nothing for any other, so the UI's
Events section mounts on precisely those two portfolio pages and stays absent
everywhere else — no flag layer needed, degrading correctly on any stack.
Guardrail 8 holds on prod: GET /v1/hazards?haz_type=earthquake lists only the
two ESHM20 datasets and ?haz_type=wildfire only the two FWI datasets. Seven
event snapshots are resident and none of them is visible to the
probabilistic readers — precisely the hijack the design record predicted a
1 km ShakeMap would cause in dataset_precedence if left unhidden.
One genuine unhappy path is wrong: an out-of-bounds tile request answers
502 UPSTREAM_TILE_ERROR rather than an empty 204/404 (F-2). Panning a map
past a snapshot's footprint is normal user behaviour, not an upstream failure.
Check 4 — UI/UX presentability 🟨¶
One real wart, seen in the payload that feeds the ranked-asset table: every ranked row is named after its exposure dataset ("LitPop -- Greece default asset-value exposure…"), because LitPop rows carry no per-row name. The Samos table therefore reads as 100 identical names with different numbers — which is the opposite of what a ranked list is for (F-1). This is the single most visible defect standing between the current build and a presentable Moment 2 walk.
Check 5 — User-path completeness ✅¶
The event view is reached by clicking: /portfolios/:id → Events section →
/portfolios/:id/events/:eventId. No header entry, by design — Phase 11's
guardrail 4 targets invisible surfaces, and /portfolios/:id is itself a
nav-less nested route reached by click.
Check 6 — Promised-but-unbuilt ✅¶
Nothing the phase promised is missing. perils.wildfire.status is still
"gap" in scripts/historical_catalog_manifest.json on trunk — correctly, and
deliberately: an observed footprint is not #412's return-period footprint, and
letting the replay flip that flag would have been the phase's easiest and worst
mistake.
The volume-budget row for FIRMS is now MEASURED rather than estimated (raw 0.000314 GB, processed 0.0000116 GB).
Guardrail results (exit criterion 3, verified on trunk 7cc61db)¶
| Guardrail | Result |
|---|---|
6 — no ead/aai on any event row or payload |
HOLDS — absent from all seven prod result payloads |
7 — _SCREENING_GRADE_PERILS byte-identical |
HOLDS — unchanged vs v0.20.0; the diff adds only EVENT_LANE_GRADE_NOTE beside it (core/eai_eligibility.py:156 and :192), never an edit to the mapping |
| 8 — snapshots invisible to probabilistic readers | HOLDS — verified on prod against the live catalog (Check 3) |
| 9 — no "alert" in user-facing copy; life-safety line everywhere | HOLDS — the UI's live namespace and live components contain no non-ARIA "alert"; the life-safety line is pinned by src/test/LiveEventPage.test.tsx |
| 10 — real data through the real API only | HOLDS — both replays fetched from the primary provider, volume-budget and manifest rows present, fixture a real extract |
— perils.wildfire.status still "gap" |
HOLDS |
One security note, recorded because the class outlives the incident: the
FIRMS pack's first live run printed the MAP_KEY through httpx's INFO
request log, because the key is a URL path segment. Fixed in PR #1153
(_silence_transport_url_logging, with a prove-it-fails test). The key reached
only a build agent's local transcript — never a commit, an issue or CI.
Findings¶
F-1 · Ranked event assets are all named after their exposure dataset — [ENHANCE-NEXT]¶
LitPop rows carry no per-row name, so the ranked-asset table renders 100 (Samos) or 216 (Evia) rows sharing one long identical label. A ranked list whose rows cannot be told apart defeats its own purpose, and this is the table Moment 2 step 8 asks the owner to read.
Recommended action: ship #1156 before the owner's walk if the schedule allows — it is the difference between a demo that reads well and one that needs apologising for. Dataset short name + coordinates is enough; no schema change.
F-2 · Tile proxy answers an out-of-bounds tile with 502 — [ENHANCE-NEXT]¶
Requesting a tile outside a snapshot's footprint returns 502
UPSTREAM_TILE_ERROR instead of an empty 204/404. Panning past the edge of a
1 km ShakeMap is ordinary map interaction; reporting it as an upstream failure
both misleads the reader and poisons error-rate monitoring with routine
behaviour.
Recommended action: ship #1155. Small, well-bounded, and it makes every map surface — not just the event one — quieter.
F-3 · Moment 2 steps 8–9 have no machine-checkable twin — [ENHANCE-NEXT]¶
Every other showcase step that reached live did so behind an e2e spec. Steps
8–9 have none: the smoke stack does not run with LIVE_MODE_ENABLED=true, so
no automated walk ever enters the Events section. Exit criterion 1 covers the
lane (sdk-smoke.yml run 33597653760, event_loss=49450.5251 across 3
assets), not the screen. Without this, the phase's user-facing half rests
permanently on a one-time human walk.
Recommended action: ship ui#298 — events replay spec plus live mode in the smoke stack. This is the finding that keeps the phase closed once it closes.
F-4 · Prod's region vocabulary had drifted from the data it governs — [BACKLOG]¶
Prod refused an ingest declaring region: "Greece" while every live Greek
dataset on the same box already carried region="Greece". The admin_boundaries
table arms the region vocabulary (#721), and nothing anywhere reconciles it
against the regions actually in use — so the drift was silent until an ingest
hit it, mid-run, on a phase-closing data run. Fixed by hand; the fixture was
tracked and the load took one command.
Recommended action: leave backlogged, but record the check rather than the
incident — a startup or deploy-time assertion that every distinct
hazard_datasets.region has a matching admin_boundaries row would have caught
this at deploy, not at ingest. Worth a small issue when the next region is
added; the load command is now in RUNBOOK.md.
Exit criteria tracking¶
| # | State | Evidence |
|---|---|---|
| 1 — Smoke | MET | sdk-smoke.yml run 33597653760 green on fe05e17, the release gate for v0.21.0: event_loss=49450.5251 assets=3 valid_at=2020-10-30T11:51:27Z grade=screening |
| 2 — Walk | MET | Both replay books exist on prod with computed losses (Check 2); the owner walked Moment 2 steps 8–9 on /portfolios/9de7a920… and /portfolios/dbe2a065… on 2026-09-03 — Samos header, life-safety line, screening badge, ranked table and raster; Evia six-tick scrubber peaking 6 August; no Events heading on a book without an event; zero "alert" hits |
| 3 — Guardrails | MET | Every guardrail verified on trunk 7cc61db and against prod — see the guardrail table above |
Findings summary¶
| ID | Finding | Tag | Issue |
|---|---|---|---|
| F-1 | Ranked event assets all share one dataset-derived name | [ENHANCE-NEXT] |
#1156 |
| F-2 | Tile proxy 502 on an out-of-bounds tile | [ENHANCE-NEXT] |
#1155 |
| F-3 | No e2e spec / live mode for Moment 2 steps 8–9 | [ENHANCE-NEXT] |
ui#298 |
| F-4 | Prod region vocabulary drifted from the data it governs | [BACKLOG] |
— (recorded here; fixed by hand) |
Carried into the next phase¶
F-1 and F-3 are the first post-closure items (the walk ran without them). F-2 is small and can ride any wave. F-4 is a class, not an incident — worth one assertion, not a project.
The phase's own claim stands up well under this pass: a hazard that happened runs through the same pipeline as one that is probable, with the annual figure structurally absent rather than merely unrendered, and with the observed series showing the shape the real fire had. The owner's walk on 2026-09-03 confirmed that the screen says so.