Phase 15 — Mid-Review¶
Date: 2026-09-17 Reviewer: the owner (the phase-end walk on prod, 2026-09-17) and Claude (planning session
lama-plan-phase-15, desk pass over the issue, gate, release and prod-log evidence) Status: open — exit criteria 1 and 4 outstanding at writing (see the tracking table); the walk itself (criterion 2) is met Artifact walked: the live production stack —https://climate-lama.online(UIv0.15.1) and the backbonev0.25.0for Moment 4 steps 1–7, thenv0.26.0from 16:09Z (a Phase 16 release deployed mid-walk, run 35244603330; Alembic head0094at both tags, so no migration rode it and nothing walked was invalidated),features.live_mode: true, against Moment 4 steps 1–8 and Moment 2 steps 8–9 of showcase-script.md Framework: per~/.claude/collaboration-framework.md, Cluster 1 ritual. Evidence pack: phase-15-live-risk.md § "Exit criteria", § W3, § W4.2 and its/phase-analyzestate block; the walk comment and the agent pass on #1299 and #1300.
Verdict¶
Verdict: the feature works on prod and the phase can close on two mechanical
items. Moment 4 was walked end to end on the production stack and every step
passed: a replay feed released six archived observations on its own clock, the
map, scrubber, headline, delta panel and ranked drawer all moved with it, and a
watch was created and closed from the UI. Moment 2 steps 8–9 re-walked green on
the new /live/:eventId route. Two near-real-time watches (FIRMS, USGS
ShakeMap) are open on prod and polling without error; their 24-hour proof
(criterion 4) matures on 2026-09-18.
The walk found three real defects, none of them in the poll loop or the numbers — all three are in how the surface reports itself: a success message styled as an error, a stale chip on the watch list, and a cadence the deployment cannot keep being echoed back as if it could. That distribution is the phase's main signal: the machinery Phase 15 built is sound, and the residue is in its self-description.
Check — the walk, with ids¶
| Step | Result | Evidence |
|---|---|---|
| M4.1 nav + watch list | pass | nav = 9 with Live mounted; rail lists the two Phase 13 replay watches with chip, freshness, latest loss, sparkline |
| M4.7a create a watch | pass (with F-1, F-2) | watch d2d04091-d117-4e73-bcdf-d2a64605e609, feed a2ed14e2, replay @ 120 s, created 2026-09-17T13:40:30Z |
| M4.2 select + map + scrubber | pass | map flew to the AOI; Follow latest on; watched-area and event-loss layers rendered |
| M4.3 an observation arrives untouched | pass (with F-3) | 6 observations released by the scheduler, live_feed.replay.series_exhausted after the sixth; ticks at the deployment's 300 s interval, not the feed's 120 s |
| M4.4 the two numbers | pass | Aug 6 snapshot: $21,451,558, 21 assets exposed, exposed value $21,451,558, Screening-grade EAI, observed-at + age + source + life-safety line all present |
| M4.5 what changed | pass | "Since the previous observation" vs Aug 5: +$20,525,432, +17 assets; largest changes and newly exposed populated |
| M4.6 drawer + timeline | pass | Assets (100) drawer sorted by event loss and by name; stepping back moved the ranking |
| M4.7b close the watch | pass | walk watch closed after the series exhausted |
| M4.8 the real wires | polling | FIRMS watch d33c553d / feed 1f4de1a9 first poll 19:10:00Z; USGS watch aafa30f7 / feed 6d0fba09 first poll 19:15:00Z; last_error null on both — opened on v0.26.0, with live-worker and beat restarted by that deploy |
| M2.8 event opens on the new route | pass | the book's Events card lands on /live/:eventId; Evia plays its six-tick scrubber |
| M2.9 screening-grade figure | pass | Samos reads $585,548,575 with the screening badge and observed-at 2020-10-30T11:51:27Z; no annual figure, no band |
Walk precondition, recorded because it was not free. Prod had no replay
series staged: live_feeds held 0 rows and no manifest.json existed under the
worker's /data mount. A 6-observation series was built on the host at
/data/live_series/evia_2021_daily from the Phase 13 daily FIRMS rasters
(data/firms-evia/<date>/*_bt.tif, each valid_at read from that day's
source_manifest.json). The vendored fixture in the repo carries one
observation, which would have left the delta panel with nothing to compare (see
F-5).
Exit criteria tracking¶
| # | Criterion | State |
|---|---|---|
| 1 | Build issues at 0 open on both milestones | not met — #1267 (owner decision 10) and #1298 (closeout docs, this record's successor) open; UI milestone at 1 open (ui#461, waiting on the next release tag) |
| 2 | One phase-end walk on prod after the release | met — this record |
| 3 | Series smoke green on trunk with the series leg | met — sdk-smoke.yml 34566448950 at 5f8d163, N = 1 per decision 7 |
| 4 | A real wire proven once — 24 h of poll health | pending — both NRT feeds polling since 19:10Z / 19:15Z, last_error null; read due 2026-09-18 on #1300 |
| 5 | Guardrails 6–20 hold on trunk and prod | met — FIRMS_MAP_KEY absent from every prod log (0 hits); life-safety line rendered on both surfaces; no annual number on an event figure |
| 6 | Surfaces — nav = 9, Live gated, redirect, no provider-derived chip | met with one defect — all four hold; the chip is correct but is rendered from a stale object (F-2) |
| 7 | This record with the two rework counters | met — below |
The circuit breaker (D15) — rework at close¶
scripts/rework_rate.py --since 2026-09-08 --follow-ups, run 2026-09-17 on both
repos, in the same window:
| Repo | escape rate | partial rate | coverage | unsized builds |
|---|---|---|---|---|
| climate-lama | 0.64 (48/75) | 0.24 (18/75) | 0.78 | 28 |
| climate-lama-ui | 0.41 (21/51) | 0.65 (33/51) | 0.90 | 4 |
Both repos are over the 25 % breaker, and this is the second consecutive phase
that is true (Phase 14 closed at 0.33 backbone / 0.44 UI on the older
follow-ups-per-build measure). The counter was rewritten on 2026-09-17 (#1438:
PR-footer fallback, escape and partial counters, trial tiers), so these numbers
are larger than Phase 14's not because rework grew but because the old measure
missed Refs-footered rework. The honest reading: roughly two builds in three
leave something behind, and 28 backbone builds still cannot be attributed to a
tier at all.
Per-tier, same window:
| Tier | builds | fixes/build | f-ups/build | partial |
|---|---|---|---|---|
| backbone Opus · xhigh | 26 | 0.50 | 0.58 | 13 |
| backbone Sonnet · xhigh | 8 | 0.00 | 0.12 | 2 |
| backbone Sonnet · xhigh · 2a-trial | 5 | 0.20 | 0.20 | 2 |
| UI Opus · xhigh | 20 | 0.30 | 0.20 | 16 |
| UI Sonnet · xhigh · 2a-trial | 11 | 0.00 | 0.00 | 8 |
~/.claude/scripts/usage_scan.py --since 2026-09-08 --by-build, same window:
builder agents cost $79 average on Opus (n = 83) against $13 on Sonnet
(n = 30) at the same average turn count (182 vs 194).
Tier-2a Sonnet trial verdict: adopt (the trial window would have closed
2026-09-24). Across 24 trial and plain Sonnet · xhigh builds in two repos the
Sonnet tiers produced less attributed rework than Opus · xhigh at one sixth of
the cost. The trial issues were ui#396, ui#397, #1314, #1315, #1405 and #1421 —
all merged, all verified, none reverted. Sonnet · xhigh becomes the default tier
for S/M work; Opus is reserved for structural change and composed seams,
which is where its rework concentration also sits.
Findings¶
F-1 · Success toasts render as errors — [ENHANCE-NOW]¶
Every Live toast uses AuthContext.showToast, whose only renderer is
hard-coded bg-red-600 (App.tsx:180) and documented as the error channel
(RunProvider.tsx:38). "Watch created" and "Poll queued" therefore read as
failures. Filed ui#481 with a tone on showToast and a prove-it-fails box.
F-2 · A new replay watch is chipped Near-real-time — [ENHANCE-NOW]¶
The backbone defaults kind to near_real_time until a feed exists;
CreateWatchDialog hands LivePage the pre-feed watch and the rail never
refetches, so a replay watch wears the wrong chip until a reload. The detail
page and the database are both correct. Filed ui#482.
F-3 · A cadence the deployment cannot keep is echoed back — [ENHANCE-NEXT]¶
live_feed_tick_seconds is 300 on prod; the API and the UI accept 120 (floor
60) and then state "Every 2 minutes" and a "Next poll" instant already past.
The poll loop is right, the contract is not. Filed #1444 — advertise the
effective interval on /v1/info, then refuse or normalise a sub-tick cadence.
F-4 · An exhausted replay feed is re-polled forever — [BACKLOG]¶
Once a replay cursor passes its last observation, every tick still selects the
feed and logs series_exhausted (36 occurrences in one 3-hour window on prod).
Cheap and harmless, but it is log noise that hides a real feed's silence. No
issue; drain it if a second series makes it visible.
F-5 · No documented way to stage a replay series on prod — [ENHANCE-NEXT]¶
docs/RUNBOOK.md documents staging only for the smoke stack. The walk had to
derive the host recipe from the validator's allowed roots. Folded into #1298
(closeout docs) rather than filed separately: one paragraph in
docs/DEPLOYMENT.md naming the mount, the layout and the cadence/tick relation.
F-6 · Version: Not stated on a replay snapshot — [DROP]¶
The replay manifest names no provider version, so the snapshot panel says so rather than inventing one. That is ADR-059 behaviour working as intended; it only reads like a gap. No action.
F-7 · A third of backbone builds are still tier-unattributable — [ENHANCE-NEXT]¶
28 of 75 backbone builds are unsized because a partial-scope PR uses Refs
and never populates closingIssuesReferences. #1438 added a footer fallback and
recovered most of the UI's; the backbone's remaining gap makes any per-tier
verdict weaker than it looks. Tracked with the token/closure work, not as a
Phase 15 issue.
Findings summary¶
| # | Finding | Tag | Channel |
|---|---|---|---|
| F-1 | Success toasts render as errors | [ENHANCE-NOW] |
ui#481 |
| F-2 | New replay watch chipped Near-real-time | [ENHANCE-NOW] |
ui#482 |
| F-3 | Sub-tick cadence accepted and misreported | [ENHANCE-NEXT] |
#1444 |
| F-4 | Exhausted replay feed re-polled every tick | [BACKLOG] |
this record |
| F-5 | No prod recipe for staging a replay series | [ENHANCE-NEXT] |
#1298 |
| F-6 | Version: Not stated on a replay snapshot |
[DROP] |
— |
| F-7 | A third of backbone builds tier-unattributable | [ENHANCE-NEXT] |
token/closure work |
What the next phase inherits¶
- Three walk findings carrying
found-in:phase-15: ui#481, ui#482, #1444 — drained by the next/phase-split, not by Phase 15. - The live lane's measured cost:
live-worker155 MiB andbeat111 MiB on prod (~0.27 GiB together), replacing the D13 RAM[ASSUMPTION]. - N = 1 replay series in the repo. #1297 (a second real FIRMS series) is off the milestone and blocked on the FIRMS credential; until it lands, a walk that wants a moving map has to build its series from archived rasters as this one did.
- Sonnet · xhigh as the default build tier for
S/Mwork, with Opus kept for structural change and composed seams. - Two NRT watches open on prod, deliberately: they are criterion 4's evidence and should not be closed before #1300 records the 24-hour read.