Serving Imagery Tiles to Annotation Tools

An annotation platform wants an image. A geospatial archive holds forty thousand Cloud-Optimized GeoTIFFs in object storage, each one 40 000 pixels square, and copying chips out of them into the platform is the step that quietly turns a 2 TB archive into a 6 TB one and loses the georeferencing on the way. The alternative is to serve the archive in place: a dynamic tile server reads a byte range from the COG, renders the requested XYZ tile, and hands the annotation tool exactly the pixels it asked for — with a manifest recording what those pixels were, so the labels can be put back on the ground afterwards.

This topic covers the serving path end to end: what the source files have to look like, how the tile server is configured so two annotators never see the same scene rendered differently, where caching genuinely helps, and how to keep the georeferencing contract intact through a stack whose middle layer does not know what a coordinate reference system is.

# Prerequisites & Toolchain Alignment

pip install "titiler.application==0.18.6" "rio-tiler==6.6.1" \
            rasterio==1.3.10 rio-cogeo==5.3.0 uvicorn==0.30.1

Beyond the packages:

• Every source must be a valid COG. A dynamic tiler on a striped GeoTIFF reads the whole file for every tile, which turns a 40 ms request into a 40 second one. Validate before serving, as spatial data formats for ML annotation sets out.
• Object storage with byte-range support. S3, GCS, Azure Blob and any HTTP server honouring Range all work. A network filesystem works too and is slower for exactly the reason CVAT deployments prefer local NVMe: the access pattern is small random reads.
• A decision about the tiling scheme. Web Mercator (EPSG:3857) is what every slippy-map client speaks. It is not what your imagery is in, and the reprojection happens per tile inside the server.

# Serving Path

# Step 1 — Confirm the Sources Are Genuinely Cloud-Optimized

Dynamic tiling is only fast because the reader can fetch the exact bytes it needs. That property comes from the file layout, not the file extension.

from rio_cogeo.cogeo import cog_validate

def assert_servable(uri: str) -> None:
    """Refuse to register a scene that will make every tile request read the whole file."""
    valid, errors, warnings = cog_validate(uri)
    if not valid:
        raise ValueError(f"{uri} is not a valid COG: {errors}")
    if warnings:
        print(f"warning for {uri}: {warnings}")

Run this at ingest, not at request time, and store the verdict in the scene registry. A scene that fails is re-encoded once, which is much cheaper than paying for it on every tile of every annotation session.

# Step 2 — Run the Tiler

TiTiler is a FastAPI application; the smallest useful deployment is the packaged app with a mounted COG endpoint.

# tiler.py — a minimal single-collection tile server
from fastapi import FastAPI
from titiler.core.factory import TilerFactory
from titiler.core.errors import DEFAULT_STATUS_CODES, add_exception_handlers

app = FastAPI(title="annotation-tiles")
cog = TilerFactory(router_prefix="/cog")
app.include_router(cog.router, prefix="/cog", tags=["COG"])
add_exception_handlers(app, DEFAULT_STATUS_CODES)

uvicorn tiler:app --host 0.0.0.0 --port 8000 --workers 4

A tile URL then looks like this, with the source and the rendering fixed in the query string:

http://tiles.internal:8000/cog/tiles/WebMercatorQuad/{z}/{x}/{y}.png
  ?url=s3://imagery/2026/alpha/scene_0142.tif
  &bidx=1&bidx=2&bidx=3
  &rescale=0,3000
  &resampling=bilinear

# Step 3 — Pin the Rendering Contract

The single most common annotation complaint — “the imagery looked different yesterday” — is a rendering contract that was never written down. Three parameters decide it, and all three must be per collection rather than per request:

from dataclasses import dataclass

@dataclass(frozen=True)
class RenderProfile:
    """The pixels an annotator sees, fixed for a whole collection."""
    bidx: tuple[int, ...]        # band order, e.g. (1, 2, 3) for true colour
    rescale: tuple[int, int]     # fixed stretch, NOT per-tile statistics
    resampling: str              # "bilinear" for imagery, "nearest" for masks
    nodata: float | None = None

PROFILES: dict[str, RenderProfile] = {
    "pleiades_rgb": RenderProfile(bidx=(1, 2, 3), rescale=(0, 3000), resampling="bilinear"),
    "sentinel2_swir": RenderProfile(bidx=(12, 8, 4), rescale=(0, 4000), resampling="bilinear"),
    "drone_rgb_8bit": RenderProfile(bidx=(1, 2, 3), rescale=(0, 255), resampling="bilinear"),
}

def tile_url(base: str, scene_uri: str, profile: RenderProfile) -> str:
    bands = "".join(f"&bidx={b}" for b in profile.bidx)
    lo, hi = profile.rescale
    return (f"{base}/cog/tiles/WebMercatorQuad///.png"
            f"?url={scene_uri}{bands}&rescale={lo},{hi}&resampling={profile.resampling}")

Auto-stretch — letting the server compute the range from each requested window — is the default in most viewers and is wrong here. It makes a dark roof beside water render differently from the same roof beside a bright field, and annotators calibrate their judgement on brightness whether or not they mean to.

# Step 4 — Cache With the Render Parameters in the Key

Annotation traffic is bursty and narrow: a team works a queue, so a few hundred tiles are requested repeatedly for an hour and then never again. That shape rewards a small cache with a short time to live and punishes a large permanent one.

proxy_cache_path /var/cache/tiles levels=1:2 keys_zone=tiles:64m
                 max_size=8g inactive=2h use_temp_path=off;

server {
  listen 80;
  location /cog/ {
    proxy_pass http://127.0.0.1:8000;
    # the full query string is part of the key: a changed rescale is a different tile
    proxy_cache_key "$scheme$request_method$host$request_uri";
    proxy_cache tiles;
    proxy_cache_valid 200 2h;
    proxy_cache_use_stale error timeout updating;
    add_header X-Cache-Status $upstream_cache_status;
  }
}

Leaving the query string out of the cache key is the failure that produces the worst possible bug: an annotator changes the band combination, the URL changes, the cache does not, and they annotate the previous rendering for an afternoon.

# Step 5 — Keep the Georeferencing Outside the Image

A PNG tile has no CRS, no transform and no scene identity. The annotation platform stores boxes in the pixel space of whatever it displayed, so something outside the image has to remember what that was.

import json
from dataclasses import dataclass, asdict

@dataclass(frozen=True)
class TaskRef:
    """Everything needed to put a pixel coordinate back on the ground."""
    task_id: str
    scene_uri: str
    scene_crs: str          # e.g. "EPSG:32633"
    tile_matrix: str        # "WebMercatorQuad"
    z: int
    x: int
    y: int
    profile: str            # key into PROFILES — the rendering is part of provenance

def write_task_manifest(path: str, refs: list[TaskRef]) -> None:
    with open(path, "w", encoding="utf-8") as fh:
        json.dump([asdict(r) for r in refs], fh, indent=2, sort_keys=True)
        fh.write("\n")

Recording profile alongside the geometry is not bureaucracy. When a batch of labels turns out to be systematically poor, the first question is what the annotator was actually looking at, and a rendering profile that changed mid-batch is a common answer.

# Serving Parameters & Configuration Reference

# Edge Cases & Gotchas

Scenes in different CRS in one collection. The tiler reprojects per request, so mixed source projections work, but they cost a warp on every tile. If a collection is mostly one UTM zone, the tiles over the odd scene will be visibly slower, which annotators experience as the tool stalling on particular tasks.

Nodata rendered as black. Untagged nodata regions come through as valid zeros, so an annotator sees a black field and may label it. Set nodata on the profile and let the tiler render those pixels transparent.

Overviews built with the wrong resampling. A COG whose overviews were built with average shows blurred edges at low zoom, which is fine for imagery and wrong for anything an annotator zooms out to count. Build overviews once with the resampling that suits the content.

Tile requests outside the scene. A client at a zoom level covering a wide area asks for tiles that no scene covers. Return an empty transparent tile with a 204 or a cacheable 404; letting those requests reach storage and fail slowly is a common source of a tiler that feels broken under load.

Authorization forgotten until launch. A tile URL that works in a browser works for anyone who has it. Annotation imagery is frequently under a licence that forbids that, and retrofitting auth means changing every task’s stored URL. Decide it before the first batch, not after.

# Integration & Automation Hooks

Label Studio. Point a task’s image at the tiler URL for a fixed zoom and extent rather than uploading a chip. The platform stores percentage coordinates, which the conversion path turns back into pixels and then, with the manifest from Step 5, into world coordinates.

QGIS. The same tiler serves an XYZ layer, so reviewers can open the identical rendering in a desktop GIS with topology tools — the escalation path described in Label Studio versus QGIS only works if both tools show the same pixels.

Pre-labelling. A model that runs over the same COGs should read them directly with windowed reads rather than through the tiler. The tiler exists to make pixels viewable; the model wants the source values, unstretched and in the source CRS.

# Validation & Testing

import httpx

def test_tile_is_deterministic() -> None:
    """The same URL must return byte-identical pixels twice — otherwise nothing is cacheable."""
    url = ("http://127.0.0.1:8000/cog/tiles/WebMercatorQuad/14/8532/5461.png"
           "?url=s3://imagery/test/scene_0001.tif&bidx=1&bidx=2&bidx=3&rescale=0,3000")
    a = httpx.get(url, timeout=30.0).content
    b = httpx.get(url, timeout=30.0).content
    assert a == b

def test_auto_stretch_is_refused() -> None:
    """A request without an explicit rescale must fail, not silently auto-stretch."""
    url = ("http://127.0.0.1:8000/cog/tiles/WebMercatorQuad/14/8532/5461.png"
           "?url=s3://imagery/test/scene_0001.tif&bidx=1&bidx=2&bidx=3")
    assert httpx.get(url, timeout=30.0).status_code == 400

The second test is the one that earns its place: it feeds the service the request shape that produces inconsistent imagery and asserts the service refuses it. Wiring that refusal in means an annotator cannot construct an unpinned rendering by editing a URL.

# Frequently Asked Questions

# Related

• Integrating Label Studio with Geospatial Workflows — the platform side of the same contract, including how pixel coordinates come back
• Spatial Data Formats for ML Annotation — what makes a GeoTIFF cloud-optimized, and why a tiler is helpless without it
• Step-by-Step CVAT Setup for Drone Imagery Annotation — the chip-extraction alternative, and when its storage cost is worth paying
• QGIS Plugin Ecosystem for Annotation Teams — the desktop client that should be looking at the same rendering as the web queue

Serving imagery is one stage of the broader Labeling Workflows & Toolchain Integration pipeline, which covers everything from ingest through export.