Raster Serving — titiler + Cloud-Optimized GeoTIFF¶

Climate-Lama serves hazard rasters (river flood, wildfire, European windstorm) as Cloud-Optimized GeoTIFFs through titiler, a FastAPI app that translates GET /cog/tiles/{tileMatrixSetId}/{z}/{x}/{y}.png?url=<cog>&bidx=<band> into on-the-fly raster tiles. MapLibre renders them via a plain raster source — no GeoJSON round-trip, no server-side rasterisation of vector pixels.

The browser never talks to titiler directly (issue #355 / #385): MapLibre requests the backbone's own same-origin tile proxy (GET /v1/tiles/cog/{dataset_id}/{z}/{x}/{y}.png), which resolves dataset_id to the dataset's cog_path and fetches from titiler server-side. No titiler host or S3 path is ever visible in an API response or a browser network request.

See ADR-029 for the bucket layout that underpins this pipeline.

Architecture¶

MapLibre GL (browser)
    │  GET /v1/tiles/cog/{dataset_id}/{z}/{x}/{y}.png[?bidx=N]  (same-origin, cookie-authed)
    ▼
Climate-Lama backbone tile proxy (api/v1/tiles.py, ADR-029 read-through cache)
    │  GET /cog/tiles/WebMercatorQuad/{z}/{x}/{y}.png?url=s3://climate-lama/processed/<org>/<ds>/hazard.tif&bidx=1
    ▼
titiler           (stateless, reads COGs from object storage)
    │  range-GETs the COG header + overviews
    ▼
MinIO / Hetzner Object Storage (COG at processed/<org>/<ds>/hazard.tif per ADR-029)

The worker produces the COG up front (see Conversion pipeline below); titiler serves it on demand. Because COGs embed their own overviews, titiler does not need to generate pyramids — it seeks into the pre-built levels via HTTP range requests.

Conversion pipeline¶

Celery task — `convert_to_cog(hazard_dataset_id, raw_raster_key, org_id?)`¶

Source: src/climate_lama/worker/tasks.py

The task:

Downloads the raw raster at raw_raster_key from MinIO (the ADR-029 raw/ layout).
Runs rio-cogeo's cog_translate with the deflate profile (tiled, with overviews, DEFLATE compression — titiler's recommended defaults for web tiles).
Uploads the result to processed/{org_id_or_public}/{dataset_id}/hazard.tif per ADR-029.
Updates hazard_datasets.cog_path with the new key so GET /v1/hazards/{id}/layers starts emitting a same-origin proxy tile URL and GET /v1/tiles/cog/{dataset_id}/... has a COG to resolve.

The task is routed to the existing compute Celery queue; no new worker container is required. The rio-cogeo dependency lives in the worker optional-dependency group (pip install -e .[worker]).

Celery task — `build_dataset_cog(hazard_dataset_id, org_id?)`¶

Source: src/climate_lama/worker/tasks.py

The builder the ingest pipeline actually uses (issue #548). Instead of a raw raster it reads the dataset's committed intensity artifact, hazards/{dataset_id}/intensity.npz, which since #542 carries the grid definition and the per-column pixel indices alongside the CSR matrix. The task:

Reads hazard_datasets.npz_path and fetches the artifact from MinIO.
Densifies the CSR back onto its own grid as a multi-band GeoTIFF — one band per event, in return_periods order, written in row blocks so the dense (height, width) band never materialises. Cells the sparse matrix does not store are written as 0.0 and declared as the raster's nodata sentinel: the sparse reader drops nodata and zero cells identically, so an absent cell is exactly a cell the source declared as "no hazard here", and tiling it as nodata leaves dry land transparent.
Runs the same cog_translate, uploads to the same processed/... key, and stamps cog_path — steps 2–4 above.

Building from the committed artifact makes the COG agree with the dataset by construction: the same grid its centroids are indexed against, one band per return period, and whatever clip/resampling the ingest ran with.

Pre-#542 artifacts have no grid keys and are refused with a pointer to scripts/backfill_grid_geometry.py --apply, which rewrites them to v2.

Auto-trigger on ingest¶

aggregate_and_commit — the ingest chord's terminal callback — queues build_dataset_cog for every dataset it commits (_schedule_cog_build). Every ingest path ends in that callback, so every dataset gets a COG with no per-caller parameter involved. The enqueue is best-effort: a broker hiccup costs the dataset its map layer, never its data, and scripts/backfill_dataset_cogs.py re-queues anything missed.

ingest_hazard still honours a raw_raster_key param by enqueueing convert_to_cog at dispatch time. That path is for a raster pre-staged under the ADR-029 raw/ layout by something other than the chord; no producer of ingest_hazard params sets it, and one usefully cannot — the chord's own staged copy at staging/{ingest_job_id}/... is deleted by aggregate_and_commit on the success path, so a conversion scheduled at dispatch time would race the cleanup. Before #548 this was the only trigger, which is why cog_path was NULL for every dataset ever ingested.

Backfilling existing datasets¶

scripts/backfill_dataset_cogs.py enqueues build_dataset_cog for every dataset with cog_path IS NULL. Dry-run by default; --apply queues the builds. Datasets whose NPZ predates #542 are reported as blocked — run scripts/backfill_grid_geometry.py --apply first, then re-run.

--rebuild (#901) is the same script pointed the other way: datasets that already have a cog_path and whose stack is deep enough that core.cog_layout.cog_interleave would lay them out band-interleaved. On-disk layout is chosen when a COG is written, so #885's band interleaving reaches an existing artifact only by re-rendering it. Depth comes from count(hazard_events); a dataset with a COG and no event rows is reported rather than assumed shallow. Dry-run by default here too, and unlike the backfill lane this one is not self-limiting — re-running re-renders the same datasets.

LayerSpec wiring¶

GET /v1/hazards/{id}/layers returns the UI-facing LayerSpec list. For raster hazards with a populated cog_path, the raster spec's source_url is filled in by layer_specs_for_hazard_dataset() as a same-origin, relative tile-proxy URL template (issue #355 / #385 — _proxy_raster_tile_url()):

{
  "type": "raster",
  "source_url": "/v1/tiles/cog/3f9e2b1a-.../{z}/{x}/{y}.png",
  "style": {"colormap": "Blues", "opacity": 0.7},
  "interactivity_config": {"tooltip_fields": ["intensity"]},
  "legend": {
    "domain": [0.12, 4.8],
    "colormap_name": "blues",
    "unit": "m",
    "source": "titiler:/cog/statistics",
    "bands": [{"band_index": 1, "min": 0.0, "max": 91.4,
               "percentile_2": 0.12, "percentile_98": 4.8}]
  }
}

MapLibre substitutes {z}/{x}/{y} per-tile. dataset_id is threaded straight into the URL — the proxy (GET /v1/tiles/cog/{dataset_id}/{z}/{x}/{y}.png, api/v1/tiles.py::get_cog_tile) resolves it to the dataset's cog_path and builds the titiler ?url=s3://... query itself; no S3 path or titiler host ever appears in an API response.

UI contract: source_url is relative, not absolute. The UI resolves it against its configured API base URL before handing it to MapLibre (the same base URL it already uses for every other /v1/... fetch) — no separate "tile server origin" config is needed anymore. Auth follows the same rule: the proxy is gated by require_role(Role.VIEWER) via get_auth_context (api/dependencies.py), which already accepts the browser's access_token session cookie (via="cookie") as an alternative to a bearer header — so as long as the UI's API requests are same-origin (or its cross-origin cookie/CORS setup already covers its other API calls), MapLibre's tile <img>/fetch requests carry the session automatically. No bearer-header support is needed in MapLibre's tile loader.

Datasets without a cog_path (not yet converted, or not a raster hazard) get source_url: "". The UI treats an empty string as "no layer to render" and falls back to loading states.

What each tile response means (issue #1155)¶

A raster covers a footprint; a map viewport does not. Every view wider than the dataset asks for tiles the COG has no pixels for — panning around a 113 × 67 pixel event snapshot is mostly such tiles — and that is ordinary interaction, not a failure. The proxy's status contract:

titiler answers	The proxy answers	Why
`200`	`200 image/png`, written back to the tile cache	a rendered tile
`404` whose body names an out-of-bounds tile (rio-tiler's `TileOutsideBounds`)	`200 image/png` — a fully transparent 256 × 256 PNG, never cached	the raster has no pixels there
any other `404`	`502 UPSTREAM_TILE_ERROR`	a request that matched no titiler route at all — #831's shape, where a version bump moved the TileMatrixSet segment
any other `>= 400` (a COG titiler cannot read is `RasterioIOError` → `500`)	`502 UPSTREAM_TILE_ERROR`	the tile server is genuinely broken — #993's credentials / endpoint / missing-object class
nothing — connection refused, timeout	`500 E_INTERNAL`	`httpx.ConnectError` is caught nowhere on this path

Before #1155 the second row was a 502 too: _fetch_upstream mapped every upstream >= 400 onto it. A MapLibre raster source treats a 502 as a tile error and retries, so a normal pan of the event map filled the browser console with gateway errors and put routine user behaviour into the API's error rate.

The decision — which upstream response means "no pixels here" — lives in core/tile_upstream.py; the route only translates it into a response.

The 404 has to identify itself. titiler:0.19.2's DEFAULT_STATUS_CODES maps TileOutsideBounds onto 404, but a request matching no route 404s too, and that one must stay loud: answering it with a transparent PNG would turn a completely broken tile chain into a blank map that reports nothing. So the classification requires the body to say why it 404'd, and treats every other 404 as an upstream error. The pattern is outside … bounds, not a contiguous phrase, because rio-tiler 7.x (what titiler 0.19.2 pins) raises the condition two ways: Tile(x=.., y=.., z=..) is outside bounds from Reader.tile — the one the /cog route hits — and Tile z/x/y is outside {input} bounds from ImageReader.tile, which interpolates the dataset path between the words. Everything titiler raises for a COG it cannot read is a 500 upstream, so the #993 triage above is untouched.

It is counted. Out-of-bounds tiles left the API error rate with this change, so they get a signal of their own: climate_lama_tile_out_of_bounds_total{tile_type}, beside the cache hit/miss counters. The failure this preserves visibility for is a COG georeferenced away from its own data — a broken transform, a cog_path pointing at another region — which answers every tile out of bounds and would otherwise render as a blank map with clean logs. A rate that is a small fraction of cache misses is ordinary panning; a rate that tracks them one-for-one is a dataset nothing can render.

Why a transparent PNG rather than 204. The consumer is MapLibre's raster source, and a 204 does not reach it as "no tile": maplibre-gl@4.7.1 resolves the fetch, hands the zero-length ArrayBuffer to createImageBitmap (image_request.ts gates on response.data, and an empty buffer is truthy), the decode rejects, and raster_tile_source.ts marks the tile errored and rethrows — trading a console full of 502s for a console full of decode errors. A transparent PNG of tile dimensions decodes everywhere, draws nothing, and reuses the same texture-pool bucket as a real tile (MapLibre keys that pool on image width, and the UI declares tileSize: 256 on its raster sources).

Why it is not cached. The out-of-bounds key space is the whole world at every zoom, so writing those tiles back would grow tiles/{dataset_id}/ without bound — and every one of those objects would be listed and deleted again by invalidate_dataset — to store ~330 bytes that are cheaper to re-derive. The empty tile escapes TileCache.get_or_fetch before its write-back, which costs one upstream round trip per out-of-bounds tile: exactly what the 502 path cost before.

The smoke walk knows the difference. A transparent tile is a valid, tile-sized PNG that clears assert_png_tile's byte and dimension floors, so the walk (scripts/smoke_e2e.py) rejects it explicitly — otherwise #1002's one raster assertion could pass on a request aimed at nothing. The rejection is structural (core.tile_upstream.is_empty_tile inflates the pixel data and looks for a fully transparent RGBA image), not a comparison against the proxy's exact bytes: the tile is deflated by the api container's interpreter and the walk runs in another, and deflate output differs between zlib builds and levels, so byte equality would fail open — silently ceasing to guard anything.

Legend + rendering parameters (issue #965)¶

The tile route accepts two optional rendering parameters, and legend above publishes the values to pass:

Query param	Legend field	Meaning
`rescale=min,max`	`legend.domain`	Value range the band is stretched over. Absent, titiler stretches each tile to its own pixels, so the same value is a different colour in two adjacent tiles and no single legend can describe the map.
`colormap_name=<ramp>`	`legend.colormap_name`	Colour ramp. Case-insensitive, so `style.colormap` ("Blues") passes straight through. Absent renders titiler's default grayscale.

Both are validated, never forwarded raw (core/tile_render.py): they are parsed into typed values and the upstream query is re-rendered from those, so a malformed range or an unknown ramp is a 422 E_VALIDATION from the backbone rather than a 502 from titiler — the same correction #964 made for bidx — and no caller bytes reach titiler's query string.

Where the numbers come from. legend is measured from the COG itself, on demand, through titiler's /cog/statistics, and memoised per COG URL (core/cog_stats.py, sharing the probe layer in core/cog_probe.py with

964's band-count read). It is deliberately not a column recorded at¶

ingest:

the seeded datasets already exist, so a column would be NULL for all of them until a backfill re-read multi-gigabyte objects;
the COG is reprojected and re-tiled from the source raster, so a min/max captured at ingest can legitimately disagree with what a tile renders — only titiler reading the object cannot;
re-ingesting overwrites a COG in place at a deterministic key, and a stored column would need an invalidation hook the TTL does not.

The cost of that choice is a live dependency on titiler, bounded three ways: the read is decimated (titiler_statistics_max_size), capped in depth (titiler_statistics_max_bands, using the cheap header probe to detect a deep cube), and never on a hot path — GET /v1/hazards/{id}/layers is the only payload that carries a legend, so no tile render and no point lookup ever waits on it. It also fails open: an unreachable titiler yields legend: null and the layer still renders.

The titiler origin and S3 prefix remain configured via two settings, but as of #385 they are internal to the tile proxy (api/v1/tiles.py, core/tile_cache.py) — they are never embedded in a LayerSpec or exposed to the browser:

Setting	Default	Notes
`titiler_base_url`	`http://localhost:7800`	Backbone-reachable titiler origin (server-to-server only).
`titiler_s3_prefix`	`s3://climate-lama`	`s3://` scheme + bucket. Swap for the prod bucket: `s3://climate-lama-storage`.
`titiler_statistics_max_size`	`512`	Longest edge the raster is decimated to before `/cog/statistics` computes. The main cost lever on the legend read (#965).
`titiler_statistics_max_bands`	`16`	Bands one legend read asks for. Above this depth the request names `bidx` explicitly, so a 100-band cube is not 100 band scans.
`titiler_statistics_timeout_seconds`	`6.0`	Budget for the pixel read. With the 3 s `/cog/info` probe it stays inside the tile proxy's own 10 s upstream budget.
`titiler_statistics_cache_ttl_seconds`	`3600.0`	How long a measured domain stays trusted. The self-heal for a COG overwritten in place.

Local dev¶

# 1. Start the full stack.
docker compose up

# 2. Verify titiler is healthy.
curl http://localhost:7800/healthz

# 3. With a COG at s3://climate-lama/processed/public/<ds>/hazard.tif,
#    fetch a tile:
#    The TileMatrixSet segment and &bidx are both mandatory: titiler >= 0.15
#    publishes no TMS-less tile route (it 404s at the route level), and a
#    multi-band COG cannot be PNG-encoded without picking one band (#831).
curl -o /tmp/tile.png \
  "http://localhost:7800/cog/tiles/WebMercatorQuad/5/16/11.png?url=s3://climate-lama/processed/public/<ds>/hazard.tif&bidx=1"
file /tmp/tile.png   # → PNG image data

titiler auto-discovers the MinIO endpoint via the env vars set in docker-compose.yml (AWS_S3_ENDPOINT, AWS_VIRTUAL_HOSTING=FALSE). No config file is involved.

AWS_S3_ENDPOINT is a GDAL config option — titiler resolves s3:// urls through rasterio/GDAL /vsis3/, not through boto3 — so it takes a bare HOST[:PORT] with no scheme. GDAL prefixes the protocol itself from AWS_HTTPS (NO for dev's plain-HTTP MinIO, YES for prod's object store). A scheme only becomes legal at GDAL >= 3.11, which postdates the pinned titiler:0.19.2 image: on this one it is not stripped but doubled, and every COG read resolves to http://http://minio:9000/... (#993). A future titiler bump onto GDAL >= 3.11 can relax the constraint and fold AWS_HTTPS into the endpoint. tests/test_compose_titiler_s3_endpoint.py pins the endpoint in both compose files — and, since the prod one interpolates ${MINIO_ENDPOINT}, the .env.example* values that flow into it.

Automated coverage — the smoke walk renders a tile¶

The curl above is the manual form. The automated one is the last stage of scripts/smoke_e2e.py (#1002), which runs in sdk-smoke.yml — the release gate — against the same stack the workflow brings up (docker compose -f docker-compose.yml -f docker-compose.ci.yml up). After the walk ingests the tracked reference raster it:

Polls GET /v1/hazards/{id}/layers until the raster spec carries a source_url — the COG build is queued after the ingest job completes and is best-effort, so "ingest finished" does not imply "the layer exists".
Reads the dataset's own footprint from GET /v1/hazards/{id}/preview and derives the WebMercatorQuad tile its centre falls in, so the request is over real pixels rather than a hardcoded coordinate.
GETs that tile through the backbone proxy, not titiler directly, and asserts a 200 whose body starts with the PNG magic number, is not the out-of-bounds empty tile, clears a byte-length floor, and decodes (via its IHDR chunk) to tile dimensions — so a 200 carrying an error envelope, an empty body, a transparent placeholder or a spacer image fails.

Going through the proxy is what makes one assertion cover the whole chain: proxy route → TITILER_BASE_URL reachability → titiler's TileMatrixSet route → GDAL endpoint configuration → object-store credentials → the COG itself. Both of #993's defects fail it, on different statuses — worth knowing when reading a failure:

Broken hop	What the proxy answers	Why
titiler reachable, cannot read the COG (`AWS_S3_ENDPOINT`, `AWS_HTTPS`, credentials, bucket path)	`502 UPSTREAM_TILE_ERROR`	that is a `RasterioIOError`, which titiler answers `500`, and `_fetch_upstream` maps every upstream error except an out-of-bounds tile onto `502` (see "What each tile response means")
titiler unreachable (`TITILER_BASE_URL`)	`500 E_INTERNAL`	the `httpx.ConnectError` is caught nowhere on the tile path — `TileCache.get_or_fetch` wraps only its write-back — so it reaches `main.py`'s catch-all

This is the runtime counterpart to the compose parse guard, not a replacement: the parse guard catches config drift for free on every trunk gate, and cannot see a titiler version bump that moves the tile route (which is exactly what

831 was), a COG written with an unreadable profile, or a credential¶

regression. docker-compose.ci.yml therefore deliberately does not override AWS_S3_ENDPOINT, AWS_HTTPS or TITILER_BASE_URL — pinning them for CI would keep the gate green while a dev-compose regression served no tiles locally.

Run the same stage locally by running the whole walk — it ends on the tile assertion. It needs the credentials and database URLs listed in docs/DEPLOYMENT.md, not a bare python scripts/smoke_e2e.py.

Production notes¶

Cache: raster tiles are regenerable from the COG (ADR-029 invariant 4). The backbone exposes a read-through cache at GET /v1/tiles/cog/{dataset_id}/{z}/{x}/{y}.png (issue #232), with an optional ?bidx= band selector defaulting to 1. On cache miss the proxy fetches from titiler, writes the bytes back to tiles/{dataset_id}/b{bidx}/{z}/{x}/{y}.png in the object-store bucket, and returns them — the band is part of the key so two bands of the same tile cannot collide (#831). A request naming rescale/colormap_name gets one more segment, tiles/{dataset_id}/b{bidx}/r{token}/{z}/{x}/{y}.png, for exactly the same reason: those change the rendered pixels too, and without the segment the first caller's stretch would be served to every later caller who asked for a different one (#965). The token is a hash of the same canonical string sent upstream, so two requests share a cached tile if and only if they render identically; a request naming neither reproduces the pre-#965 key byte for byte, so tiles already cached stay hits. Admin flushes the whole tiles/{dataset_id}/ prefix — every band and every rendering — via POST /v1/datasets/{id}/invalidate-tiles. Prometheus counters climate_lama_tile_cache_hits_total{tile_type} and climate_lama_tile_cache_misses_total{tile_type} track hit ratios.
Colormapping: titiler applies colormaps server-side via the &colormap_name=... query parameter, and since #965 the tile route accepts it (with rescale) as a validated pass-through — see "Legend + rendering parameters" above. LayerSpec.style.colormap remains the declaration; the legend echoes it back already lowercased into the form titiler accepts, so a client can render server-side without restating the ramp.
Worker image size: rio-cogeo is optional — it installs only when the worker image is built with pip install -e .[worker]. The core API image does not include it.

Alternatives considered¶

Rasterize to GeoJSON: rejected — payload scales with pixel count, not data complexity, and kills browsers at continental extents.
Pre-baked PNG pyramids: works but doubles object-storage usage and makes colormap iteration expensive. COG + on-the-fly titiler lets us swap colormaps without re-generating pyramids.
Embedded titiler library (import titiler.core into the backbone): smaller network hop, but couples the backbone to GDAL and rasterio at request time. The container path keeps the backbone slim and lets ops scale the tile server independently.