Climate-Lama -- Complete Project Reference

Purpose: Document for the backbone platform. Everything needed to understand, extend, and deploy Climate-Lama.

Repository: github.com/CortoMaltese3/climate-lama (private) License: Apache 2.0 (confirmed, review before making repo public) Python: 3.12 (pinned for CLIMADA/Docker compatibility) Status: PoC complete (River Flood, Greece, Buildings, EAD, GeoJSON). Control panel UI on main.

---

1. What This Is

Positioning superseded 2026-07-26 by the goal reset in docs/plan/plan.md (Extended End Goal): personas are now ranked (P1 regional/mid-market insurer, conditional — see plan), the business model is commercial white-label instances, and licensing is deferred pending validation. The paragraphs below predate that reset; treat plan.md as authoritative where they disagree. Technical reference content in this file remains valid.

Climate risk intelligence platform built for extensibility. Backend-first, API-driven, self-hostable.

Target users: researchers, government agencies (GIZ, UNU, EIOPA-type), NGOs, development banks, smaller insurers -- organizations that need transparency, reproducibility, and self-hosting.

Not competing with: enterprise SaaS (delta-climate, Jupiter Intelligence). Serves the development/research/regulatory market where open-source, auditable methods matter.

What it provides: - Unified data access to hazard, exposure, and vulnerability datasets - Model-agnostic compute with climate-lama-engine as the default (pluggable via ModelInterface) - Clean REST API for downstream consumers (web apps, desktop apps, BI tools, AI agents) - Offline-capable architecture for sensitive data environments

---

2. Architecture

Two deployable units

                  HTTP
    Clients -----> Core Service (FastAPI monolith, port 8000)
                       |           |           |
                       v           v           v
                  PostgreSQL   MinIO     Redis
                  + PostGIS    (S3)      (queue/cache)
                       ^
                       |  Celery tasks
                  Compute Worker

Core Service (FastAPI): - API layer (REST, /v1/ versioned) - Ingest module (data acquisition, ETL) - Data module (normalization, validation) - Catalog module (search, metadata)

Compute Worker (Celery): - climate-lama-engine adapter (~200MB container) - ModelInterface abstraction (pluggable engines) - Async job processing

Storage: - PostgreSQL 16 + PostGIS 3.4 -- normalized domain data, spatial queries - MinIO -- raw files, intensity matrices (NPZ), result arrays (S3-compatible) - Redis 7.x -- Celery broker/backend, caching

Source tree

src/climate_lama/
  api/v1/              # HTTP layer only -- no business logic
    exposures.py       # Upload CSV, list, CRUD
    hazards.py         # Ingest trigger, list, centroid assignment
    compute.py         # Impact calculation trigger
    jobs.py            # Job status polling
    results.py         # Result retrieval, GeoJSON generation
    impact_functions.py # Seed and list impact functions
  core/                # Business logic -- no HTTP, no DB queries
    hazard_ingest.py   # GeoTIFF parsing, NPZ creation, centroid bulk insert
    centroid_assignment.py  # KDTree nearest-neighbor, bulk SQL update
    exposure_service.py     # CSV parsing, dedup, create exposures
    impact_function_seeder.py  # Seed JRC flood impact functions
  models/              # Pydantic + SQLAlchemy
    hazard.py          # HazardDataset, HazardEvent, HazardCentroid
    exposure.py        # Exposure (with geometry, centroid_idx)
    job.py             # Job (status, celery_task_id, timestamps)
    result.py          # ImpactResult (ead, aai, impact_per_event, eai_exp_path)
    impact_function.py # ImpactFunction (mdd_x/y, paa_x/y curves)
    enums.py           # HazardType, ExposureType, JobStatus, JobType
  db/
    session.py         # Async (asyncpg) + sync (psycopg2) session factories
    base.py            # SQLAlchemy declarative base
    repositories/
      hazard_repository.py    # CRUD + get_by_name_and_type (dedup)
      exposure_repository.py  # CRUD + exists_by_name (dedup)
      job_repository.py
      result_repository.py
      impact_function_repository.py
  worker/
    celery_app.py      # Celery config, queue routing
    tasks.py           # compute_impact, ingest_hazard (+ centroid assignment)
    models/
      base.py          # ModelInterface ABC + ImpactResult dataclass
      engine_adapter.py  # EngineAdapter (wraps climate-lama-engine)
  storage/             # MinIO/S3 abstraction
  config.py            # Pydantic Settings (env vars)
  main.py              # FastAPI app, router registration, lifespan
docker/
  core.Dockerfile      # API container
  worker.Dockerfile    # Worker container (installs climate-lama-engine)
migrations/            # Alembic
tests/                 # pytest + pytest-asyncio

Dependency rules

• api/ calls core/, never db/ directly
• core/ calls db/repositories/ for data access
• worker/ is independent -- communicates via Redis queue and shared DB
• Engine imports only in worker/models/engine_adapter.py
• No circular imports -- dependency flows downward

---

3. Tech Stack

Layer	Choice	Version
Language	Python	3.12
API Framework	FastAPI	0.115+
Async Server	Uvicorn	latest
Task Queue	Celery	5.4+
Message Broker	Redis	7.x
Database	PostgreSQL + PostGIS	16 + 3.4
Object Storage	MinIO	latest
ORM	SQLAlchemy	2.x (async)
Migrations	Alembic	latest
Validation	Pydantic	2.x
Geospatial	GeoAlchemy2, Shapely	latest
Containerization	Docker + Compose	latest
Linting/Formatting	Ruff	latest
Testing	pytest + pytest-asyncio	latest
Package manager	uv	latest
Engine	climate-lama-engine	0.1.0+

---

4. API Design

Versioning and format

All endpoints: /v1/ prefix. Breaking changes require /v2/.

Success response:

{ "data": { ... }, "meta": { "request_id": "...", "timestamp": "..." } }

Error response:

{ "error": { "code": "VALIDATION_ERROR", "message": "...", "details": {} }, "meta": { ... } }

Exceptions: GET /v1/results/{id}/geojson returns a raw FeatureCollection, and GET /v1/results/{id}/report returns a raw summary object, CSV, or PDF bytes depending on ?format= — all without the envelope. Errors from both still use it.

Endpoints

Method	Path	Description
GET	/health	Health check
POST	/v1/exposures	Upload CSV exposure file
GET	/v1/exposures	List exposures
POST	/v1/hazards/ingest	Trigger hazard GeoTIFF ingestion (async job)
GET	/v1/hazards	List hazard datasets — omitting ?status= now excludes superseded rows by default (issue #873); pass ?status=superseded explicitly to see them. Each row carries event_axis/computable (issue #920) — see below
POST	/v1/hazards/{id}/assign-centroids	Trigger centroid assignment (async job)
POST	/v1/impact-functions/seed	Seed default impact functions
GET	/v1/impact-functions	List impact functions (filterable by ?haz_type=)
POST	/v1/impact-functions	Create a user-defined impact function (Phase 4+, #69)
DELETE	/v1/impact-functions/{id}	Delete a user-defined function; builtin functions are protected (Phase 4+, #69)
POST	/v1/compute/impact	Trigger impact calculation (async job)
POST	/v1/compute/impact/matrix	Submit a scenario x horizon matrix batch (async group, issue #380)
GET	/v1/compute/impact/matrix/{batch_id}	Batch status + per-cell result refs (partial-failure semantics)
POST	/v1/compute/impact/matrix/{batch_id}/report	Queue a template-driven PDF render for a scenario-matrix batch (async job, issue #415) — the batch-shaped sibling of POST /v1/results/{id}/report
GET	/v1/compute/impact/matrix/{batch_id}/report	Fetch the stored PDF pack for a scenario-matrix batch
POST	/v1/compute/impact/compare	Run one scenario through N engines (async group) — see below
GET	/v1/compute/impact/compare/{id}	Side-by-side per-engine metrics + divergence %
GET	/v1/jobs/{id}	Poll job status
GET	/v1/results/{id}	Get result summary (EAD, AAI, per-event)
GET	/v1/results/{id}/geojson	Get result as GeoJSON FeatureCollection
POST	/v1/results/{id}/report	Queue a template-driven PDF render (async job, ADR-039)
GET	/v1/results/{id}/report	Export the result — ?format=json\|csv\|pdf; PDF serves the stored document
GET	/v1/risk/lookup	Multi-hazard score card for one coordinate (?lat&lon[&scenario&horizon&region&hazards&dataset_id]) — see below

event_axis / computable — the catalog's compute-gate prediction (issue #920)

GET /v1/hazards and GET /v1/hazards/{id} both carry two fields a client builds its "can I run this?" affordance from:

Field	Type	Meaning
event_axis	"return_period" | "scenario_year" | null	Which axis this dataset's hazard_events rows sit on (ADR-065). null means the dataset holds no events at all — an unfinished or failed ingest, which is not a posture.
computable	bool	What core.compute_service.validate_impact_refs would decide about this dataset, on the unit and event axes together.

computable: false means POST /v1/compute/impact naming this dataset is refused before anything is queued — with E_HAZARD_DATASET_NO_EVENT_FREQUENCIES (no annual frequency to integrate, ADR-069) or E_INDICATOR_DATASET_NO_EAI (no seeded curve consumes the intensity unit, issue #600). eai_eligibility.reason names which. indicator is the same posture read the other way round and can never disagree with it.

Two boundaries worth stating:

• computable does not cover the curve/exposure-type refusal (ADR-061). That one is keyed on the impact function a job names, which no catalog row knows, so a computable dataset can still be refused for aiming a built-in buildings curve at population exposure. The catalog reports that axis as the eai_eligibility.by_exposure_type matrix rather than folding it into a boolean it cannot honestly compute.
• The axis is read from the events, never inferred from the dataset row. return_periods IS NULL looks like a free answer and ADR-069's alternative 5 rejected it for the compute gate: the column is a derived summary written only by the post-ADR-067 commit step and has been nullable since migration 0002, so a legacy row that never recorded one reads as a cube. The catalog must reach the same verdict as the gate, so it reads the same thing the gate reads.

Cost: the list endpoint resolves the whole page's axes in one GROUP BY dataset_id aggregate (HazardRepository.get_event_axes), not one probe per row; the detail endpoint keeps ADR-069 §2's single bounded row (HazardRepository.get_event_axis). See core/dataset_computability.py.

Before #920 the catalog classified a dataset by its intensity unit alone, so a single-scenario multi-year cube with a consumable unit (the #797 class) advertised indicator: false and then 422'd on Run — the dead end these fields exist to prevent.

The /v1/risk/* score card had the same defect on its own indicator field and was closed the same way by issue #930 — same classifier, same two arguments, so the two surfaces and the gate are one verdict rather than three derivations. core.eai_eligibility.classify_dataset_eai and is_indicator_dataset now require event_axis with no default, which is what makes leaving the question unasked a TypeError rather than a silently wrong answer.

X-Engine — request-time engine selection (issue #377, phase 8.15)

Engine plurality is a product commitment, so every compute submission — POST /v1/compute/impact, POST /v1/compute/impact/matrix (applies to every cell of the matrix) and POST /v1/compute/cost-benefit — accepts an optional X-Engine header naming the engine to run on.

The selector is a header, not a body field: it picks the runtime rather than describing the calculation, so it stays out of the request schemas and out of anything that round-trips a saved scenario. Selectors are normalised (case and -/_ are interchangeable), so X-Engine: Climate-Lama == climate_lama.

• Omitted → the default engine (climate_lama, the in-process climate-lama-engine adapter). Behaviour is unchanged from before the header existed.
• Unknown engine → 422 with E_ENGINE_UNKNOWN; the error details list the registered engines. Nothing is queued.
• Registered but not deployed here → 400 with E_ENGINE_UNAVAILABLE.

The resolved name is threaded into the job params and stamped on the result provenance as engine_name (next to engine_version), and — for non-default engines — folded into the result-cache key so two engines never share a cache entry. worker/models/registry.py owns the name → ModelInterface mapping; it holds lazily-invoked factories and an availability probe per engine, so an engine running behind a process boundary registers exactly like an in-process one. There is no fallback: a job that names an engine the backbone cannot serve fails rather than being rerouted (ADR-024).

/v1/compute/impact/compare — multi-engine agreement view (issue #379, phase 8.15)

Runs one scenario through N registered engines and puts the answers side by side. Multi-engine agreement is a validation signal: engines that independently land on the same EAD corroborate the number; a wide divergence flags something worth investigating. It is not a ranking, and there is no blending or weighted consensus.

POST fans the inputs out to one ordinary compute_impact job per engine and returns 202 with a comparison_id; GET .../compare/{comparison_id} serves the group. Group tracking reuses the compute_batches row that the scenario matrix (#380) uses — params.kind discriminates engine_comparison from scenario_matrix, and a group is never served through the other's route.

Contract points that callers depend on:

• engines is optional. Omitted → every engine available on this deployment; registered-but-undeployed engines are skipped and reported in params.excluded_engines. Named explicitly → an unregistered engine is 422 (E_ENGINE_UNKNOWN) and one that cannot run here is 400 (E_ENGINE_UNAVAILABLE), both before anything is queued. Asking for "whatever you have" never fails over an engine the caller never named; asking for a specific engine never silently drops it.
• A group of one is valid. Until a second engine is deployed, a comparison reports the single engine's metrics with zero divergence. Degenerate, not an error.
• Partial views beat failures. An engine whose job failed is named in divergence.failed_engines (with its error_message on its entry) and excluded from the maths; engines still running are in pending_engines. The group still rolls up to completed once every job is terminal, and no engine failure turns the view into a 500.
• Divergence is relative to one reference — the default engine when it is in the group, else the first engine submitted — as a signed percentage, (value - reference) / |reference| * 100, per scalar metric (ead, aai, insured_loss) and per frequency-curve return period. A divergence is null where it is undefined (zero reference against a non-zero value); two zeros agree exactly.
• No cache short-circuit. Unlike POST /v1/compute/impact, a comparison always dispatches a job per engine so the group has a cell per engine.

GET /v1/risk/lookup — point score card (issue #382, phase 8.8)

The answer plane's read path: no job, no engine, no compute. For each hazard it reads the org-scoped risk cell covering the coordinate (falling back to the org-less reference plane), reads the hazard COG's intensity at the point through titiler /cog/point, and bands the result via the versioned score schemes (ADR-043). Lat/lon only — address resolution is the geocoder's job.

Contract points that callers depend on:

• Versioned payload. schema_version (currently 1.0) is bumped on any breaking change; additive fields do not bump it.
• No silent omission (RFC amendment A3.6). A hazard with no surface and no readable COG is returned with available: false, a machine-readable unavailable_reason, and provenance.surface.surface == "not_built_here". The response is 200 — "nothing was built here" is an answer, not an error.
• Band precedence (RFC amendment A3.3, narrowed by ADR-073). Every wired peril bands from the COG point intensity — at the design return period (RISK_LOOKUP_BAND_RETURN_PERIOD, default 100 y) on a return-period raster, or at the resolved scenario × year slice on a cube. Windstorm, tropical cyclone and heatwave were classified to the cell metric until #924: they are indicator perils with no seeded damage curve, so no impact run and therefore no cell metrics are ever written for them, and every one of those ladders was dead. There is no cross-source fallback either way.
• Dataset selection is stated, and overridable (issue #590, ADR-056). With several datasets catalogued per peril, each peril's dataset is chosen by quality score → local-over-global → recency, not by insertion order, and provenance.dataset.selection reports the score, the locality rank and the rule that decided. An optional dataset_id (accepted on all four /v1/risk/* rungs) pins one dataset for its own peril only; every other peril still resolves by precedence. A dataset_id that is not visible to the caller's org is a 404; one whose peril is not among hazards is a 422.
• Feature-flag aware. With SCORE_BANDS_ENABLED off the structure is unchanged and every continuous metric is still served; only band is null, with band_unavailable_reason: "score_bands_disabled".
• Bounded band read (issue #838). A titiler point read costs one internal tile fetch per band, so the read asks for only the bands the card can use, via bidx. A return-period vector no deeper than RISK_LOOKUP_MAX_POINT_BANDS (default 16; the deepest catalogued raster has 9) is read whole and publishes its full by_return_period curve; a deeper one narrows to the design return period's band; a dataset with no return-period vector (ADR-067's scenario × year slice axis) reads the slice horizon addresses, or the first band when it addresses none. Labels follow the band index read, never the position in the response.
• A cube states which slice answered (issue #844). On a scenario × year dataset, horizon is resolved to a band through that dataset's own hazard_events rows — event_index + 1 is the raster band carrying that event's year, so the mapping is a join, not an inference from supported_years. The answering entry carries a slice block naming year, scenario (the event's label, else the dataset's — ADR-071 puts it on the dataset row), the event's event_name and the band read, and the sample is labelled with that event_name rather than a bare band number. A horizon the events do not carry is refused, not substituted: unavailable_reason: "horizon_not_in_dataset" (or "horizon_ambiguous" when two events claim one year) with slice.available_years naming what the dataset can answer. Before this, horizon=2050 against the 1986-2085 heatwave cube returned the 1986 band's value with nothing saying so. Every other dataset shape is untouched — a return-period raster's events carry no year, so it is never addressed and its slice is null.
• A timeout is its own reason. A read that exhausts RISK_LOOKUP_COG_TIMEOUT_SECONDS reports unavailable_reason: "cog_read_timeout", distinct from cog_read_failed — one means the read is too big, the other that the raster or tile server is broken.
• indicator is the compute gate's verdict, event axis included (issue #930). Each entry's indicator is decided the way core.compute_service.validate_impact_refs decides it — the dataset's intensity unit and its event axis — so a card can never advertise a runnable peril the Run button is then refused for. Before this the card read the unit alone, which is the same defect #920 fixed on GET /v1/hazards: a frequency-less cube with a consumable unit (the #797 class) showed indicator: false and then 422 E_HAZARD_DATASET_NO_EVENT_FREQUENCIES. An entry with no dataset still falls back to the peril-level question ("does the platform hold any curve for this peril?"), which is all it can answer. Cost: at most one GROUP BY dataset_id aggregate for the whole card (never one probe per peril), skipped entirely when no chosen dataset's verdict the axis could move, and issued inside the COG stage's window rather than added to it — reported as elapsed_ms.axes so that stays checkable.
• An absent band says which kind of absent (issue #933). When band is null, band_unavailable_reason is one of score_bands_disabled, no_catalogued_dataset, no_bandable_reading, no_scheme_for_hazard, band_metric_unavailable or value_unbandable. The first two of those matter most to branch on, and used to be one string: no_catalogued_dataset means nothing is catalogued for the peril, so no coordinate, return period or horizon can ever band it and a retry is futile; no_bandable_reading means a dataset answered and this address has no bandable value in it — a return-period raster that does not publish RISK_LOOKUP_BAND_RETURN_PERIOD, or a scenario × year cube whose slice horizon did not address — so another query may well succeed. Reporting both as band_metric_unavailable told a caller "missing data for your query" about a peril that has no data at all. tropical_cyclone and storm_europe are the live examples: both carry seeded ladders and neither has a catalogued dataset, so both report no_catalogued_dataset today. That is read off whether this request resolved a dataset, never off a table of which perils have data — either peril can be catalogued through POST /v1/hazards/ingest or /v1/hazards/upload without a seed pack, and its reason changes on the next request when it is. Adding a reason is additive and does not bump schema_version; treat an unrecognised code as opaque rather than failing on it.
• A count indicator's absent reading is the value zero (issue #935, ADR-074). The platform stores only the cells worth storing, so a heat-wave COG carries nodata = 0.0 and titiler answers null at a pixel that recorded no heat-wave day. For a count index that null is a number — "no heat-wave day here that year" is 0 days — so on the perils listed in core/scoring.py's _ABSENT_READS_AS_ZERO (heatwave today) a resolved slice with no stored value bands as the ladder's Negligible rung instead of reporting no_bandable_reading. Without it, band 1 of every count ladder was unreachable by construction and a reference-period year answered "band unavailable" over ~95% of the ECDE grid. Two consequences a client must read correctly:
• intensity.band_value_source says "read" or "absent_as_zero" (null when there was no band input at all), and it is the authoritative marker. band.metric_source names the input class the scheme bands from and still reports point_intensity here — it is a property of the ladder, not a claim that a pixel answered.
• The raw reading is not rewritten: intensity.band_value is 0.0 while intensity.by_return_period[…].value stays null. That asymmetry is deliberate — the convention supplies a band input, it does not invent a measurement.

Continuous-intensity perils are not in the table: an absent flood depth is dry land outside the footprint, and the stated non-answer is still the honest card there. The /v1/risk/assets/{id} rung projects the card verbatim, so it inherits the band, an aggregate_value of 0.0 and a matching provenance.band_value_source; the portfolio and admin-unit rungs read cell aggregates and are unaffected. - The climatological heat-wave ladder bands at version 2 (issue #925, migration 0086). heatwave/heatwave_days_climatological uses edges [3, 4, 5, 7, 10, 14, 20, 28, 40]; the ECDE index counts days in a run of at least three consecutive days, so values of 1 and 2 cannot exist and band 2 ([3, 4)) is exactly one minimum-length event, while band 1 is the Negligible rung a genuine zero-day reading lands in under the convention above. The health ladder (heatwave_days_health) stays at version 1 with edges [2, 5, 10, 15, 20, 30, 45, 60, 90] — a June-August window caps that index at 92 days, so its >= 90 band is an intentional physical-cap sentinel. Both sets are self-set: no published banding exists for either index, and each row's citation says so and names what the edges were derived from. Clients must read band.scheme_version (surfaced as scheme_ref, e.g. heatwave/heatwave_days_climatological@v2) rather than assume a fixed ladder — a stored band keeps the version it was computed under and is not re-banded retroactively. - Stated latency budget. 100 ms cells (indexed) + 1200 ms COG (one round trip per hazard, issued concurrently) + 50 ms assembly, 1500 ms p95 end to end. Echoed as budget_ms, measured per request as elapsed_ms (slices, axes, cells, cog, assembly), and exported as the climate_lama_risk_lookup_seconds histogram. Asserted in tests/test_api/test_risk_lookup.py.

Rollup reason codes — /v1/risk/{assets,portfolios,admin-units}/{id}

The three rollup rungs report an unaggregated hazard the way the point card does: available: false plus a machine-readable unavailable_reason, on a 200. This is the rollup's own reason set, distinct from the card's band_unavailable_reason above — that one explains a missing band on a hazard that did answer. Four codes, and they are not interchangeable:

• no_members_to_aggregate — nothing was in scope to aggregate over.
• no_cells_in_scope — members (or a boundary) were considered, and no answer-plane cell covers any of them. Nothing was built here.
• metric_absent_from_cells — cells were found, and none of them carries a value for this scheme's metric. A cell that exists but carries no value for this metric is a different operator signal from no cell at all: the surface was built, just not with this scheme's metric on it. The portfolio rung states this exactly as the admin-unit rung does (issue #937); before that it answered no_cells_in_scope while sitting on a full set of cells, which points at building a surface that already exists.
• no_scheme_for_hazard — no versioned score scheme names a metric for this peril, so there is nothing to aggregate.

Rollup payload — what aggregate_value and worst_value are

Both numbers, and the bands derived from them, come from one place: the metrics blob of the answer-plane cells in scope, read under the active scheme's metric key. Until issue #932 no production writer produced any scheme's key, so every rollup reported metric_absent_from_cells for every peril. ADR-075 closes that on the run path:

• What the cell carries. A run-backed surface now stores, per cell, the mean design-return-period hazard intensity over the exposure points inside that cell, plus <metric>_points — how many valued points that mean was taken over. Both are written by core/surface_writer.build_cells; a cell none of whose points had an intensity gets neither key, so metric_absent_from_cells keeps meaning exactly what it says.
• It is the hazard where the book is. A surface covers exposure locations, not a grid, so this is a book-conditioned aggregate — the same conditioning the cell's EAI already carries. An unconditioned hazard field for a district is a different question and a different (deferred) build.
• Portfolio rung — aggregate_value is the exposed-value-weighted mean of the members' cell values, worst_value the largest. Unchanged by #932, and still free of COG reads.
• Admin-unit rung — aggregate_value is the points-weighted mean over the cells whose centroid falls inside the boundary, weighting each cell by its <metric>_points (a cell with no such count weighs 1, which is right for a legacy or hand-written cell). worst_value is the largest cell average, not the worst point. provenance.aggregation says points_weighted_cell_mean.
• One surface per hex. The boundary aggregate pins neither region nor the engine's identity, so several READY surfaces can satisfy one lookup key — production holds six for one river-flood identity. The aggregate now picks the newest surface per h3_index before aggregating, reported as provenance.surface_selection = "newest_surface_per_h3_index".
• Which perils this reaches. Only perils an impact run can be computed for, whose dataset publishes an event at that peril's design return period and whose intensity_unit matches the scheme's metric_unit. River flood bands at the global RISK_LOOKUP_BAND_RETURN_PERIOD (default 100). Earthquake bands at 476, not the global default: core/scoring.py's design_return_period_for is the single resolver both the address-level card and this writer call, and its per-hazard override table names HazardType.EARTHQUAKE: 476.0 — the catalogued ESHM20 dataset publishes nominal return periods [50, 476, 976, 2500, 5000] and no 100-year event, so the global default could never match (issue #942). 476 is ESHM20's own nominal label for the ~475-year (10%-in-50-years) reference seismic design return period, matched exactly, not within a tolerance. Consequence for clients: intensity.band_return_period is hazard-varying (100 on a flood card, 476 on an earthquake card, null on a cube slice) — it is provenance about which slice answered, and a client should not assume one fixed value across perils. Heat-wave and wildfire/FWI are indicator perils with no run at all, and stay unaggregated pending issue #944.
• Surfaces built before #932 carry no metric. Cells are replaced in place, so re-running a scenario is what fills them in; a bulk backfill is issue #943.
• /v1/risk/lookup's cell.metrics gains the same two keys, additively, for surfaces written after this lands. eai, eai_density_km2, points and any *_points key are a reserved namespace no score scheme may claim.

Status codes

200 (GET/PUT), 201 (POST create), 202 (async job submitted), 400, 401, 403, 404, 422, 500.

Naming

Resources: plural nouns (/exposures, /hazards). Actions: POST + verb (/compute/impact). URLs: kebab-case. JSON bodies: snake_case.

---

5. Data Models

HazardDataset

Column	Type	Description
id	UUID	PK
name	str	Dataset name (dedup key with haz_type)
haz_type	HazardType enum	river_flood, tropical_cyclone, etc.
source	str	Data source (e.g., "JRC")
region	str	Geographic region
scenario	str	Climate scenario label
intensity_unit	str	"m", "m/s"
frequency_type	str	"marginal" or "occurrence"
npz_path	str	MinIO path to intensity matrix NPZ
n_centroids	int	Centroid count
return_periods	JSON list or None	Original RPs (e.g., [10, 50, 100, 200, 500])
crs	str	Coordinate reference system (default "EPSG:4326")
bbox	JSONB or None	Upstream extent before any ingest-time clip
grid_transform, grid_width, grid_height	float[6], int, int	Regular-grid definition — the primary centroid representation (ADR-045 topic 2)
footprint	Polygon (PostGIS) or None	EPSG:4326 envelope of the ingested grid; GiST-indexed dataset extent (#596)

HazardEvent

Column	Type	Description
id	UUID	PK
dataset_id	UUID	FK -> HazardDataset
event_name	str	"RP-10", "RP-100", etc.
rp	float	Return period value (e.g., 10, 100, 500)
exceedance_freq	float	Exceedance frequency (1/RP)
marginal_freq	float	Computed marginal frequency (used by engine)

Important: No explicit array_index column. Events map to intensity matrix rows by insertion/query order. Workers must query with ORDER BY consistent with ingestion order to guarantee index alignment with the NPZ matrix.

HazardCentroid — legacy only since #596 (ADR-057)

No ingest path writes these rows any more. A hazard dataset's centroid geometry is the grid_transform/grid_width/grid_height triple on HazardDataset plus the pixel_idx array in its v2 intensity.npz — eight numbers and one array instead of one row per pixel (the table was 4294 MB of a 4.32 GB prod database). Rows written before #596 are still read: get_centroids, admin aggregation, and the membership assignment join all keep a fallback for datasets that have not been backfilled onto a v2 artifact.

Column	Type	Description
id	UUID	PK
dataset_id	UUID	FK -> HazardDataset
array_index	int	Column index in intensity matrix
pixel_index	bigint or None	Row-major cell index in the dataset grid
lat, lon	float	Coordinates
geom	Point (PostGIS)	Spatial geometry with GiST index

Exposure

Column	Type	Description
id	UUID	PK
name	str	Dataset name (dedup key)
dataset_id	UUID	Optional grouping FK
geometry	Point (PostGIS)	Location
value	float	Asset value
value_unit	str	Currency
exposure_type	ExposureType	buildings, population, etc.
centroid_idx	int or None	Assigned centroid index (set by centroid assignment job)
centroid_dataset_id	UUID or None	Which hazard dataset the centroid belongs to

ImpactFunction

Column	Type	Description
id	UUID	PK
name	str	"JRC Flood - Europe - Residential"
haz_type	HazardType	river_flood, tropical_cyclone, wildfire, storm_europe
intensity_unit	str	"m", "m/s", "FWI", "K"
mdd_x, mdd_y	list[float]	MDD curve sample points (x = intensity, y ∈ [0, 1])
paa_x, paa_y	list[float]	PAA curve sample points (x = intensity, y ∈ [0, 1])
source	str	"builtin" (seeded, protected from deletion) or "user" (user-created)

User-defined curves (Phase 4+, #69): The data model already supports arbitrary user curves via the source = "user" flag. The POST /v1/impact-functions endpoint (Phase 4+) will accept any piecewise curve with monotonically increasing x and y ∈ [0, 1]. Builtin functions cannot be deleted. In Phase 2+ (multi-tenancy), a org_id column will scope user-defined curves per organisation. The compute endpoint requires no changes — it already accepts any impact_function_id UUID without distinguishing source.

Job

Column	Type	Description
id	UUID	PK
job_type	JobType	data_ingest (covers hazard ingest + centroid assignment), impact_calc
status	JobStatus	pending, running, completed, failed
celery_task_id	str	Celery task ID
params	JSON	Input parameters
result_id	UUID or None	FK -> ImpactResult (set on completion)
error_message	str or None	Set on failure
created_at, started_at, completed_at	datetime	Timestamps

ImpactResult

Column	Type	Description
id	UUID	PK
job_id	UUID	FK -> Job
exposure_id	UUID	FK -> Exposure dataset
hazard_dataset_id	UUID	FK -> HazardDataset
impact_function_id	UUID	FK -> ImpactFunction
ead	float	Expected Annual Damage
aai	float	Average Annual Impact
impact_per_event	JSON	{event_name: value}
eai_exp_path	str	MinIO path to per-exposure NPZ
metadata_	JSON	Engine version, valid/excluded points

---

6. Key Implementation Patterns

Hazard Ingest Flow

1. API receives POST /v1/hazards/ingest -> creates Job (PENDING) -> dispatches Celery task
2. Worker reads GeoTIFF(s) via rasterio, extracts intensity bands
3. Converts return periods to marginal frequencies (ADR 018)
4. Stores sparse intensity matrix as NPZ in MinIO
5. Bulk inserts centroids via SQL unnest() (ADR 022 -- 630K in ~1 sec vs 30+ min with ORM)
6. Creates HazardDataset + HazardEvent rows
7. Marks Job COMPLETED

Async Ingest Pipeline

The hazard ingest path uses a Celery chord so the per-chunk NPZ writes run in parallel. The full chord is assembled dynamically because the number of chunks (N) is only known after plan_chunks runs.

Chord structure

chain(
    stage_source              ← preface 1: copy local files into MinIO
    validate_source           ← preface 2: metadata sanity checks
    plan_chunks               ← preface 3: deterministic chunk list
    _dispatch_chunk_writes    ← dispatcher: builds and fires the inner chord
        └── chord(
                group(write_chunk × N),   ← parallel body
                aggregate_and_commit      ← callback
            )
)

The linear preface runs in series on a single worker. The dispatcher is itself a Celery task; it issues apply_async() for the inner chord and returns immediately. aggregate_and_commit is the callback that fires once all write_chunk tasks complete.

State machine

pending → running → chunking → writing → aggregating → committed → succeeded
                ↘           ↘          ↘            ↘           ↘
              failed/    failed/     failed/      failed/     failed/
              cancelled  cancelled   cancelled    cancelled   cancelled

failed and cancelled are reachable from every non-terminal state. Terminal states (succeeded, failed, cancelled) have no outgoing edges.

Resume guarantee

plan_chunks produces a deterministic chunk list from the staged descriptor (same input → same chunk boundaries). write_chunk is idempotent: it skips any chunk whose row already has status = 'succeeded'. Together these two properties make it safe to re-dispatch the chord on a worker crash — already-completed chunks are no-ops and only the remaining chunks are re-run.

Determinism is conditional on the same inputs, and the request's bbox is one of them: since #445 the planner sizes windows, the hazard_ingest_max_grid_cells fail-fast and the window boundaries against the bbox-clipped extent. The clip is therefore persisted on ingest_jobs.bbox (migration 0062) rather than living only in the original chord's Celery message, and both planning entry points (plan_chunks, select_chunks_for_resume) resolve it from the row — their bbox parameter is a pure override. Without that, a resume would re-plan against the raw file and write a different window into a chunk slot that ingest_chunks addresses by chunk_idx alone (#453).

The request's resolution (degrees/pixel) is the same class of input, one layer down: it does not feed the plan, but every write_chunk read derives its GridSpec from it — grid_width / grid_height, and therefore the pixel_idx space each chunk's NPZ slice is expressed in. It is persisted on ingest_jobs.resolution (migration 0064) and resolved from the row by select_chunks_for_resume, its parameter likewise a pure override. Without that, a resume of a resampled ingest wrote native-resolution slices into a dataset whose already-succeeded chunks were resampled, and aggregate_and_commit unioned two different grid bases (#482). NULL on both columns means "un-clipped" / "native resolution" — the correct value for every historical row.

The same rule applies to plan_chunks's other post-#445 parameter, for the same reason: a chord dispatched before a deploy carries the old three-argument message, so an omitted job_id must not mean "there is no legacy Job row" — a rejected plan would leave the row that GET /v1/jobs/{id} polls stuck at running. It is instead recovered from the jobs.params->>'ingest_job_id' back-link that both ingest producers stamp (#454). The reverse skew (a new, longer message reaching an old worker) is a TypeError that no amount of defaulting fixes — see Deployment → In-flight Celery messages.

Cancel semantics

POST /v1/hazards/ingest-jobs/{id}/cancel writes status='cancelled' on the ingest_jobs row. Cancellation is chunk-boundary only: each write_chunk task polls the row at the start before doing any work and exits as a no-op if the parent is cancelled. aggregate_and_commit performs the same poll at the start of the callback. In-flight writes are always allowed to finish; there is no mid-chunk interrupt.

Log buffer

Each ingest_jobs row carries a logs text column. append_log caps the buffer at 256 KiB. On overflow the oldest bytes are dropped and a truncation marker ...[truncated]... is prepended so readers can tell the buffer was trimmed. The cap is enforced on every write in IngestJobRepository.append_log (db/repositories/ingest_job_repository.py).

Chunk-size table

Chunk sizes are configured in Settings.hazard_ingest_chunk_size_bytes_per_haz_type (config.py). The defaults reflect the per-event memory footprint of each hazard type:

Hazard type	Chunk size
river_flood	256 MiB
tropical_cyclone	512 MiB
wildfire	256 MiB

Tropical-cyclone is larger because its per-event footprint is roughly half that of the other types, so a 512 MiB chunk fits the same worker RAM budget while halving chord fan-out overhead. Unlisted types fall back to hazard_ingest_chunk_size_bytes_default (256 MiB). Chord width is bounded by hazard_ingest_max_parallel_chunks (default 8) to keep aggregate memory bounded under burst. The decisions behind this design are captured in ADR-027, ADR-029, and ADR-034.

Approved-Dataset Ingest Handoff

The weekly DatasetSource poll (issue #321) queues new/changed external items into pending_ingests as awaiting_review. When a platform admin approves a row, POST /v1/admin/pending-ingests/{id}/approve dispatches the ingest_approved_dataset Celery task, which runs the handoff into the async ingest pipeline above: it resolves the item's ingest descriptor, downloads the rasters, and dispatches ingest_hazard.

Ingest-descriptor contract

The queue row carries only source_id / item_id and the item's opaque metadata — the backbone never interprets that metadata while polling or reviewing. To make an item auto-ingestable on approve, a source publishes an ingest object inside that item's metadata, in one of two raster layouts:

{
  "id": "jrc-flood-eu",
  "title": "JRC EU river flood",
  "download_url": "https://.../flood_hazard/",
  "ingest": {
    "haz_type": "river_flood",
    "files": [
      { "rp": 10,  "url": "https://.../Europe_RP10_filled_depth.tif" },
      { "rp": 100, "url": "https://.../Europe_RP100_filled_depth.tif" }
    ]
  }
}

Field	Requirement
haz_type	A HazardType value (river_flood, tropical_cyclone, wildfire, storm_europe). Absent/unknown ⇒ not auto-ingestable.
files	Mode 2. Non-empty list of {rp, url} — one single-band raster per return period, return periods unique and positive.
return_periods	Mode 1. Non-empty list of positive years, one per band of the item's single multi-band raster at download_url.

Exactly one of files / return_periods may be given; supplying both is refused rather than silently ranked. Mode 2 exists because it is what real hazard publishers serve — the JRC river-flood distributions publish one *_RP<N>_*.tif per return period in a directory, not a stacked raster. Modelling only mode 1 was why no source could satisfy this contract (#830).

Descriptors are derived, not hand-maintained. JrcFloodSource reads the Apache autoindex its DCAT download URL resolves to and turns each *RP<years>*.tif it lists into one {rp, url} pair, so the return periods come from the provider. Derivation never guesses: an unreachable listing, a non-HTML body, or a distribution that publishes its return periods as tiled sub-directories (the global GLOFAS package does) yields no descriptor and the item stays reviewable-but-not-auto-ingestable.

Because the queue row's metadata is a poll-time snapshot, a row queued before its source learned to publish a descriptor carries none. The handoff therefore re-resolves the descriptor against the live source (DatasetSource.ingest_descriptor) before giving up, so an old row still ingests without waiting for a re-poll.

The contract and its parser live in ingest/sources/base.py (IngestDescriptor, IngestFile, ingest_descriptor_from_metadata). Only the ingest block is read; everything else in the item metadata stays opaque.

Sources that can never be auto-ingested say so

A source whose items are structurally not hazard rasters sets a class-level auto_ingest_unsupported_reason. WorldpopSource does (it publishes gridded population — exposure, not hazard intensity). GET /v1/admin/pending-ingests surfaces this per row as auto_ingestable + auto_ingest_unsupported_reason so the review queue marks those rows review-only instead of letting a reviewer discover it from a failed approval, and the handoff reports handoff: auto_ingest_unsupported with that reason.

Failure is recorded, not silent

A non-ingestable approved item — no ingest block (and none derivable live), a non-hazard descriptor, a malformed descriptor, or a failed download — flips the row to the terminal ingest_failed state (migration 0051) and logs the reason at ERROR. The pipeline is dispatched only for valid hazard descriptors; nothing is guessed. Because the platform-level queue carries no org, the resulting hazard dataset and its jobs are created in the approving admin's org (threaded from the approve request).

No decision is a dead end for the dataset (issue #870)

The poll writes a dataset_source_catalog last-seen marker for every item it sees, not just the ones it queues. That is what keeps a declined item from nagging admins on every poll — but it also means a rejected or ingest_failed row is skipped by every later poll until its upstream content changes, so a mis-click would otherwise put a dataset permanently out of the platform's reach. POST /v1/admin/pending-ingests/{id}/re-arm is the supported way back: it deletes that item's catalog marker so the next poll sees the item as new and queues a fresh awaiting_review row. Platform-admin only, audited as pending_ingest.rearmed, and idempotent — the response's last_seen_cleared says whether a marker was actually there. The decided row is left untouched as the record of the decision, and polling behaviour is unchanged: an item nobody re-arms is still never re-queued. Re-arm brings an item back through review, never around it — it dispatches no ingest. Only dead-end rows are re-armable; awaiting_review (already reachable) and approved (its dataset exists or is being built) are a 422.

Re-arm is self-completing and batched (issue #905)

Clearing the marker only unblocks the next poll, and the beat is weekly — a recovery whose effect is invisible for up to seven days is close enough to no recovery at all. So both re-arm endpoints dispatch a scoped poll after committing: poll_dataset_source(source_id, item_ids, notify=False), one task per affected source, narrowed to exactly the re-armed items. Scoping is what makes it affordable — dispatching the platform-wide poll_dataset_sources was rejected in #870 because it polls every source and notifies every admin — and a scoped poll deliberately does not catalog items outside its scope, so it can never mark an unrelated item as seen. The response's polls_dispatched reports what went out.

POST /v1/admin/pending-ingests/re-arm takes {"ingest_ids": [...]} (max 500) for recovering from a bulk sweep. Partial success is the contract: it answers 200 with rearmed / failed / counts, where an id that does not exist or a row that is not a dead end lands in failed with a not_found / not_rearmable code while every other id is still re-armed — so read counts, not the status code. Each re-armed row is audited individually.

Both responses also carry source_registered, and a warning when it is false. A source can be de-registered from the build while its rows survive in the queue; re-arming one of those still clears the marker but no poll will ever re-surface it. Registration is reported, never enforced — the source may be registered again later, and refusing the re-arm would leave nothing to re-arm when it is. docs/RUNBOOK.md has the operator procedure.

Centroid Assignment

1. API receives POST /v1/hazards/{id}/assign-centroids -> Job -> Celery task
2. Worker loads all hazard centroids for dataset, loads all exposures
3. Builds KDTree, finds nearest centroid per exposure point
4. Bulk updates Exposure.centroid_idx and centroid_dataset_id

Assignment also runs automatically at compute time (#558): when a compute job finds no junction rows for its (hazard dataset, exposure dataset) pair, the worker assigns them in place — under a pg_advisory_xact_lock so concurrent scenario-matrix cells on one dataset do not collide — and retries the load before failing. The endpoint above stays the explicit, pollable path (the UI's Dashboard preflight uses it), but no client can skip assignment by not calling it.

Impact Calculation Flow

1. API receives POST /v1/compute/impact with {hazard_dataset_id, impact_function_id, save_mat: true}
2. Creates Job (PENDING) and commits -> dispatches compute_impact to compute queue (see Job submission ordering)
3. Worker loads: hazard (from DB + MinIO NPZ), exposures (from DB), impact function (from DB)
4. Converts to engine data contracts (dicts) -> passes to EngineAdapter
5. Adapter builds Hazard, Exposures, ImpactFuncSet -> runs ImpactCalc.impact()
6. Stores eai_exp array as compressed NPZ in MinIO
7. Creates ImpactResult row with EAD, AAI, per-event impacts
8. Records the cache key in result_cache (issue #283 — see below)
9. Marks Job COMPLETED

Job submission ordering

Every API path that queues a job goes through core/job_dispatch.py (create_and_dispatch_job, or stage_job + commit_and_dispatch for batches). Two rules, both enforced by tests/test_api/test_submit_dispatch_ordering.py:

1. Commit before dispatch. The jobs row (and anything the task needs to read, e.g. the matching ingest_jobs row or the compute_batches row) is durable before apply_async is called. A worker can never be handed an id it cannot read.
2. The API never writes status. The Celery task id is pre-generated and passed to apply_async(task_id=...), so it lands in the same pre-dispatch INSERT and no post-dispatch write is needed. pending -> running -> terminal belongs to the worker alone — JobRepository deliberately has no set_running.

Issue #578 is what happens without them: the submit path dispatched inside its open transaction and wrote status='running' afterwards, so a job that failed in the worker in 8 ms had its failed overwritten by the API's commit ~150 ms later and sat at running forever.

The one write after dispatch is the failure path: if apply_async raises, the committed row is marked failed (in a fresh transaction) before the error propagates, since nothing will ever run it. Committing mid-request ends the transaction the RLS app.current_org_id GUC was set on, so the helper re-issues it right after the commit.

Result Cache (issue #283)

POST /v1/compute/impact checks result_cache before dispatching the Celery task. The cache is content-addressed by a deterministic SHA256 over: exposure_dataset_id, exposure_dataset_sha256, hazard_dataset_id, hazard_dataset_sha256, impact_function_id, impact_function_version, year, the canonical-JSON of the saved scenario's inputs, discount_rate, and growth_rate. Measures are excluded — they apply as a post-cache multiplier on the cached result, so caching with measures would explode the key space without value.

Dual response shape on the same endpoint:

• Cache hit -> 200 OK

  {"data": {"cache_hit": true, "result_id": "<uuid>", "job_id": null}, "meta": {...}}
  

• Cache miss -> 202 Accepted

  {"data": {"cache_hit": false, "job_id": "<uuid>", "result_id": null}, "meta": {...}}
  

Invalidation contract. The cache key includes impact_functions.version, so bumping the version retires every entry derived from the old version (the new key is different — old rows just become unreachable). Bug fixes that change result values MUST bump the version; otherwise the cache will serve stale numbers.

DB-level guard (issue #485). A BEFORE UPDATE trigger on impact_functions (migration 0063) RAISE WARNINGs whenever any of mdd_x/mdd_y/paa_x/paa_y changes without version also changing. It is advisory — the UPDATE still succeeds and the trigger never bumps version itself (an automatic bump would silently invalidate cached results out from under a running system) — so bulk paths like restores, reseeding migrations, and core/impact_function_seeder.py's upsert-on-conflict stay unblocked. Treat the warning in the Postgres log as a prompt to check whether the edit should have bumped version.

Cross-org safety: cache rows have no org_id; lookup re-checks RLS on the underlying impact_results row before treating a row as a hit, so two orgs with identical inputs share the same key but only the org that owns the result sees the hit.

Maintenance: prewarm_result_cache (Celery beat, 03:00 UTC) reads config/prewarm.yaml and dispatches missing entries; purge_orphan_cache_entries (04:00 UTC) deletes rows whose result_id no longer exists. Hit/miss counts are emitted as climate_lama_result_cache_hits_total{result=...}.

GeoJSON Result Retrieval

1. GET /v1/results/{id}/geojson loads ImpactResult + Exposure rows + eai_exp NPZ from MinIO
2. eai_exp holds one entry per exposure point, ordered by core.eai_alignment.EXPOSURE_COMPUTE_ORDER_BY — array position is not a centroid array_index. The endpoint re-runs the worker's exposure join in that order and rolls the values up onto the centroid each asset snapped to (ADR-024 era bug, fixed in #370)
3. Returns FeatureCollection with [lon, lat] coordinates (GeoJSON standard) and eai_exp, array_index, point_count properties — one Feature per centroid that carries exposure; unexposed centroids are omitted
4. Returns 409 E_RESULT_STALE when the exposure dataset has changed since the job ran, so no safe attribution exists. GET /v1/results/{id}/report behaves the same way

Async/Sync Boundary (ADR 021)

Celery tasks are synchronous. DB operations use async (asyncpg). Solution: wrap async code in asyncio.run() inside the task, create engine + session inside the coroutine, never at module level. Dispose engine in finally block.

Bulk Spatial Inserts (ADR 022)

ORM add_all() with geometry triggers per-row INSERT. Solution: raw SQL with unnest() + ST_SetSRID(ST_MakePoint()). Result: 630K centroids in ~1 second.

Deduplication

• Hazard: get_by_name_and_type(name, haz_type) check before ingestion. Returns existing dataset if found.
• Exposure: exists_by_name(name) check before CSV parsing. Returns [] if exists (hard skip to preserve centroid assignments).

CSV Upload MIME Handling

Windows browsers send .csv as application/vnd.ms-excel or application/octet-stream. Expanded allowed types + filename extension fallback:

if ct not in ALLOWED_TYPES:
    if not filename.lower().endswith(".csv"):
        raise HTTPException(400, ...)

---

7. Docker Compose

services:
  postgres:   postgis/postgis:16-3.4-alpine    # port 5432
  redis:      redis:7-alpine                    # port 6379
  minio:      minio/minio:latest                # ports 9000/9001
  api:        ./docker/core.Dockerfile           # port 8000, hot reload
  worker:     ./docker/worker.Dockerfile         # Celery, compute queue

The UI is a standalone deployable in the climate-lama-ui repo and is not part of this compose file.

Key env vars: DATABASE_URL (asyncpg), DATABASE_SYNC_URL (psycopg2), REDIS_URL, CELERY_BROKER_URL, MINIO_ENDPOINT.

Worker queue routing:

task_routes = {
    "climate_lama.worker.tasks.ingest_hazard": {"queue": "compute"},
    "climate_lama.worker.tasks.assign_centroids": {"queue": "compute"},
    "climate_lama.worker.tasks.compute_impact": {"queue": "compute"},
}

---

8. Code Conventions

• Formatter/Linter: Ruff (replaces Black, isort, flake8)
• Type hints: Required on all public functions
• Docstrings: Google style on public API
• Line length: 100 characters
• Naming: snake_case files/functions, PascalCase classes, SCREAMING_SNAKE_CASE constants, kebab-case URLs
• Error handling: Custom exceptions in core/exceptions.py, API layer catches + transforms, never expose internals
• Logging: Python logging (structlog planned for Phase 2)

Git conventions

Branch: <type>/<issue-number>-<short-description> (e.g., feat/42-exposure-upload) Commit: type(scope): description (imperative, no period, max 72 chars) Types: feat, fix, chore, docs, refactor, test, ci Scopes: api, worker, core, db, storage, ingest, catalog, ci, docs Final commit on branch: Resolves #N or Refs #N in footer.

---

9. Security Defaults

• Auth (MVP): API key via Authorization: Bearer <key>, hashed in DB
• Input validation: All via Pydantic. File uploads: MIME + extension check, size limits. Spatial: coordinate bounds.
• Secrets: Environment variables only, .env.example as template, Docker secrets for production
• Data access: Repository pattern, no raw SQL in API/core, parameterized queries (SQLAlchemy)

---

10. MVP Scope

Dimension	Choice	Rationale
Hazard	River Flood (JRC European Flood data)	Freely available, EU-relevant, smaller datasets
Region	Greece	Local knowledge, EU market alignment
Exposure	Buildings (CSV upload or LitPop)	Simplest to ingest and validate
Calculation	Expected Annual Damage (EAD)	Fundamental metric
Output	GeoJSON FeatureCollection	Web mapping compatible

---

11. Architecture Decision Records

24 ADRs in docs/DECISIONS.md. Phase column maps each ADR to the phase in which it was authored — see docs/plan/ for phase scope.

#	Decision	Status	Phase
001	Backend-first architecture	Accepted	0
002	Two-service architecture (API + Worker)	Accepted	0
003	Model interface abstraction (pluggable engines)	Accepted	0
004	PostgreSQL + PostGIS for primary storage	Accepted	0
005	MinIO for object storage	Accepted	0
006	Celery + Redis for job queue	Accepted	0
007	FastAPI over Flask	Accepted	0
008	Vertical slice MVP scope	Accepted	0
009	Open Core business model	Proposed	0
010	Python 3.12 pin	Accepted	0
011	Repository structure	Accepted	0
012	Git conventions	Accepted	0
013	River Flood as MVP hazard	Accepted	0
014	Greece as MVP region	Accepted	0
015	JRC European Flood data	Accepted	0
016	Clean-room engine reimplementation (Apache 2.0)	Accepted	0
017	Engine as independent package (climate-lama-engine)	Accepted	0
018	Marginal frequencies for EAD computation	Accepted	0
019	Adaptation measures as stored DB entities	Accepted	0
020	Dual engine strategy (climate-lama-engine + CLIMADA fallback)	Deprecated — CLIMADA adapter removed in Phase 0 (#19); climate-lama-engine is sole engine	0
021	Async/sync boundary in Celery workers	Accepted	0
022	SQL unnest for bulk spatial inserts	Accepted	0
023	Impact function sources for TC, WF, WS (Emanuel 2011 + Eberenz 2021 / Lüthi 2021 / Klawa–Ulbrich 2003)	Accepted	1
024	Worker runtime has no CLIMADA dependency	Accepted	2

---

12. Business Model and Strategy

Three-project structure

Project	License	Status
climate-lama-engine	Apache 2.0	Separate repo, fully implemented, publish to PyPI
climate-lama (backbone)	Apache 2.0	This repo. Open-source at Phase 3.
climate-lama-ui	TBD	Own repo (split from backbone in Phase 2). Standalone deployable, consumes backbone API over HTTP. Private until license decided.

Open Core model (ADR 009)

• Engine + backbone: open source (Apache 2.0)
• UI: license TBD (BSL, proprietary, or open -- needs legal review)
• Revenue: managed hosting (SaaS/PAYG), support contracts, premium features (SSO, audit, reports), data packages, training

Pricing tiers

Tier	Price	Includes
Community	Free	Self-hosted, no support
Professional	$500-2,000/mo	Managed hosting, SSO, 10K calcs/mo, support
Enterprise	$2,000-10,000/mo	Dedicated instance, SLA, custom integration
Academic	Free managed	Verified .edu/.ac, limited compute

CLIMADA relationship

• CLIMADA is GPL-3.0, maintained by ETH Zurich
• Climate-Lama does NOT use CLIMADA as a runtime dependency
• The engine is a clean-room reimplementation (Apache 2.0)
• climada_adapter.py was removed in Phase 0 (#19); climate-lama-engine is the sole compute engine
• Position: "CLIMADA as a service" -- same domain, better accessibility (API-first, self-hostable, no Python required)
• Maintain goodwill: cite CLIMADA, contribute upstream, do not position as fork

---

13. Phased Roadmap

Phase 0 -- Stabilize PoC (Weeks 1-4)

Engine: publish 0.1.0 to PyPI, CLEAN_ROOM.md, CHANGELOG, CI/CD. Backbone: climada_adapter.py and [full-worker] removed, pre-commit hooks, GitHub Actions (lint/test/build), integration tests, health check, demo script. UI: confirm control panel works end-to-end.

Phase 1 -- Multi-Hazard + Scenario + Cost-Benefit (Months 2-4)

Closed 2026-04-17 (~3 months ahead of the 2026-07-15 target). Engine 0.2.0 live on PyPI. Backbone and UI shipped all 22 scoped deliverables:

• Ingest: BaseGeoTIFFIngestor (#42), TC/WF/WS ingestors (#47, #43, #48), scenario plumbing (#44)
• Impact functions: sources ADR-023 (#49), TC/WF/WS seeds (#50–#52), per-hazard auto-resolution (#53)
• Measures + cost-benefit: table (#54), CRUD API (#55), Celery task (#56), endpoints (#57)
• Frequency curve: persist (#58), endpoint (#59)
• Insurance: deductible/cover + insured_loss (#60)
• UI: scenario selector (#62), TC/WF/WS activation (#63), freq-curve panel (#64), CB panel (#65)
• Polish (merged same phase): ExposureStep prop cleanup (#83), hazard-grid skeleton (#85), is_default in impf response (#86), centralised exception handlers (#87)

Loose ends (tracked in docs/plan/phase-1-multi-hazard.md): #16 (bbox-scoped centroid assignment — defer to Phase 3), #37 (session-scoped event loop — informational).

Phase 2 -- Production Backend + UI Maturity (Months 4-7)

Auth (JWT + API key), RBAC, multi-tenancy (row-level security), structured logging, report generation (PDF/Excel), portfolio aggregation, UI split to own repo, production Docker Compose.

Phase 3 -- Managed Hosting + First Revenue (Months 7-10)

Open-source backbone, managed hosting infra, billing, landing page, academic tier, pilot engagements (GIZ, World Bank, Greek agency), data packages (JRC flood for all EU).

Phase 4+ -- Expansion (10-18+ months)

Uncertainty quantification in engine 0.3.0, custom impact function builder, data marketplace, SDKs, desktop app, CMIP6 adjustments, Kubernetes scaling.

---

14. Open Points

Topic	When
UI license (BSL vs proprietary vs open)	Before Phase 3
Apache 2.0 legal review for backbone	Before Phase 3
Pluggable engine concept (user-selectable engines)	Phase 4
Desktop app (Electron/Tauri + embedded engine)	Phase 4
Uncertainty quantification in engine	Engine 0.3.0
Pilot customer timing	Phase 2 entry

---

15. Domain Concepts

• Hazard: Probabilistic event set (intensity at centroids + frequency). Types: river_flood, tropical_cyclone, wildfire, storm_europe.
• Exposure: Assets at risk (buildings, population). Points with value and location.
• Vulnerability (Impact Function): Intensity -> damage fraction. MDD x PAA = MDR.
• Impact: Hazard x Exposure x Vulnerability. Key metrics: EAD, AAI, per-event, per-exposure.
• Job: Async computation request. Submitted via API, executed by worker, results in DB.
• Centroid assignment: Links exposures to nearest hazard centroid. Required before impact calculation.
• Marginal frequency: ETL-computed frequency bands for return period maps (not exceedance frequency).

---

16. Usage Metering

Per-tenant usage is tracked so admins can see consumption today and so the future Stripe metered-billing wiring can read from the same schema without a migration. The schema decisions below are deliberately load-bearing — any change after launch means a data migration, so they are stable from this section forward.

What we record

• Billable actions in v1: impact compute and cost_benefit compute. Storage and ingest are explicitly out of scope; they will get their own event types when needed.
• Per-event row, not per-period roll-up at write time. Every billable action writes one row to usage_events; period totals are produced at read time from a materialized view. This keeps the write path cheap and the audit trail full.

Schema (usage_events)

Column	Type	Notes
id	UUID PK	gen_random_uuid()
org_id	UUID FK orgs	RLS-isolated via app.current_org_id
user_id	UUID FK users null	Forensic only; not used in aggregation
job_type	TEXT	'impact' | 'cost_benefit' (CHECK constraint)
job_id	UUID	Links back to jobs.id (or impact_results.id on cache hit)
started_at	TIMESTAMPTZ	Task entry time, not enqueue time — queue wait is not billable
completed_at	TIMESTAMPTZ	Terminal state time
compute_seconds	DOUBLE PRECISION	Wall-clock; completed_at - started_at
cpu_seconds	DOUBLE PRECISION	os.times() user+system delta; nullable (subprocess engines may not report)
result_bytes	BIGINT null	Total artifact size (eai_exp.npz); null when not applicable
cache_hit	BOOL default false	Cache-hit events have compute_seconds≈0 and cpu_seconds=NULL
status	TEXT	'succeeded' | 'failed' (CHECK constraint)
recorded_at	TIMESTAMPTZ default now()	Insert timestamp

Indexes: (org_id, completed_at) for period queries, (job_id) for join-back. RLS is enabled on the table with the same app.current_org_id GUC pattern used by other domain tables.

Period semantics

Months, anchored on first-of-month UTC. The materialized view usage_by_period aggregates (org_id, period) over status='succeeded' rows only and is refreshed nightly at 02:00 UTC by Celery beat (refresh_usage_view) using REFRESH MATERIALIZED VIEW CONCURRENTLY. Consumers should treat the view as up-to-24h stale; the staleness window is acceptable for billing-style aggregates.

Stripe-readiness contract

These fields are stable from this revision forward; future Stripe metered- billing integration will read them as-is:

• compute_seconds, result_bytes, cache_hit, status, period
• The CHECK constraint values ('impact', 'cost_benefit', 'succeeded', 'failed')
• The first-of-month UTC period anchor

New event types (storage, ingest) and additional metrics may be added in later revisions; existing column meanings will not be reinterpreted.

Admin endpoints

• GET /v1/admin/usage?org_id=...&from=YYYY-MM&to=YYYY-MM — JSON envelope of {period, compute_seconds, run_count, result_bytes, cache_hit_count}.
• GET /v1/admin/usage.csv?... — same data, streamed as CSV.

RBAC: Role.ORG_ADMIN is required. The default org_id is the caller's own org; explicit cross-org org_id returns 403 unless the caller's users.is_platform_admin flag is true.

Prometheus

climate_lama_compute_seconds_total{org_id, job_type} is incremented in the same code path that writes a successful usage_events row, so the Grafana fleet view can render the same data live without hitting the DB. The high-cardinality org_id label is acceptable while the platform serves a small-multitenancy tenant base; once the tenant count exceeds ~1000 orgs the label should be bucketed (e.g. by plan tier).

---

17. What NOT To Do (Lessons from RISK WISE)

• No monolithic data loading -- use streaming, chunking, DB queries
• No synchronous compute -- all calculations via job queue
• No tight engine coupling -- always through adapter interface
• No Windows-only assumptions -- everything in Docker/Linux
• No business logic in API endpoints -- extract to core/
• No bare Exception catches -- be specific
• No print statements -- use logging
• No hardcoded config -- use settings
• No skipped migrations -- every schema change needs one