Integrating STAC Catalogs with Versioned Datasets

A versioned training set is only reproducible if you can prove which source scenes it was built from. Pinning each snapshot to its exact acquisition records means storing the STAC item ids plus a catalog hash inside a dataset manifest that DVC versioning tracks. The manifest records the SpatioTemporal Asset Catalog (STAC) items a snapshot draws from — their ids, their self hrefs, and the spatial and temporal query that selected them — then hashes that list so any drift is detectable. With this bridge in place, any training set is reproducible back to its acquisition metadata, and the same spatiotemporal query can be re-run to rebuild or extend the dataset months later.

# Why Acquisition Provenance Belongs in the Version Record

Geospatial training data is derived data. The imagery a model learns from is a filtered slice of a much larger archive, selected by area of interest, date range, sensor, and cloud cover. If a dataset version records only the resulting files, it throws away the query that produced them — and with it, any ability to explain why one scene was included and another dropped, or to reproduce the selection when the archive grows. A directory of GeoTIFFs cannot answer “was this the June 2025 cloud-free Sentinel-2 pass or the July re-processing?”

STAC solves the description problem: it is a JSON specification where each item describes one spatiotemporal asset (a scene) with its geometry, datetime, and links to the underlying files. What STAC does not solve is version binding — nothing in a raw catalog ties a specific training snapshot to a specific set of items at a specific moment. That binding is what the manifest supplies. By recording item ids and a hash of the selection inside a DVC-tracked file, you turn an ephemeral query into a durable, verifiable part of the version record. When an upstream provider re-processes a scene and quietly changes an asset href, the recomputed hash no longer matches and the difference surfaces in dvc status instead of corrupting a downstream model months later.

# Reading the Catalog and Selecting Items

Install the tooling with pinned versions so the manifest is byte-stable across machines:

pip install pystac==1.9.0 shapely==2.0.6 python-dateutil==2.9.0

STAC items carry geometry in EPSG:4326 by convention, so the bounding-box filter below operates in decimal degrees. If your area of interest is expressed in a metric coordinate reference system, reproject it to WGS84 before filtering. The first stage walks the catalog and applies a spatial and temporal predicate:

from __future__ import annotations

from datetime import datetime, timezone

import pystac
from shapely.geometry import box, shape


def select_items(
    catalog_href: str,
    bbox: tuple[float, float, float, float],
    start: datetime,
    end: datetime,
) -> list[pystac.Item]:
    """Return catalog items whose footprint intersects `bbox`
    (minx, miny, maxx, maxy, in EPSG:4326) and whose acquisition
    datetime falls within [start, end]."""
    catalog: pystac.Catalog = pystac.Catalog.from_file(catalog_href)
    aoi = box(*bbox)
    matches: list[pystac.Item] = []

    for item in catalog.get_items(recursive=True):
        dt: datetime | None = item.datetime
        if dt is None:
            continue
        if not (start <= dt <= end):
            continue
        if shape(item.geometry).intersects(aoi):
            matches.append(item)

    # Deterministic ordering is essential for a stable hash.
    matches.sort(key=lambda it: it.id)
    return matches

The matches.sort() call is not cosmetic: STAC iteration order is not guaranteed, and an unordered list would hash differently on each run even for the identical selection. Sorting by item id makes the downstream digest deterministic.

# Recording Item Ids and Self Hrefs into a Manifest

The manifest is the durable artifact. It stores the query that produced the snapshot, the selected item ids, and each item’s self href — the canonical link back to its catalog record — so the selection can be replayed or audited without re-scanning the whole catalog:

import json
from typing import Any


def build_manifest(
    items: list[pystac.Item],
    bbox: tuple[float, float, float, float],
    start: datetime,
    end: datetime,
) -> dict[str, Any]:
    """Assemble a provenance manifest for the selected STAC items."""
    records: list[dict[str, str]] = []
    for item in items:
        self_link = item.get_single_link("self")
        href = self_link.href if self_link is not None else ""
        records.append({"id": item.id, "self_href": href})

    return {
        "query": {
            "bbox": list(bbox),
            "datetime_start": start.isoformat(),
            "datetime_end": end.isoformat(),
        },
        "item_count": len(records),
        "items": records,
    }

Because self_href points at an immutable catalog record and the id uniquely names the scene, these two fields together are enough to rehydrate the imagery later — either by re-fetching from the STAC-hosted remote or by pulling the assets you mirrored into your own DVC remote configuration.

# Hashing the Manifest for Drift Detection

The catalog hash is a SHA-256 digest computed over the manifest in a canonical form. Serialising with sort_keys=True and no incidental whitespace guarantees the same logical content always yields the same digest, regardless of dict insertion order:

import hashlib


def canonical_bytes(manifest: dict[str, Any]) -> bytes:
    """Serialise a manifest to canonical UTF-8 JSON bytes."""
    return json.dumps(
        manifest, sort_keys=True, separators=(",", ":"), ensure_ascii=False
    ).encode("utf-8")


def compute_catalog_hash(manifest: dict[str, Any]) -> str:
    """Return a stable sha256:<hex> digest over the manifest body,
    excluding the hash field itself."""
    body = {k: v for k, v in manifest.items() if k != "catalog_hash"}
    digest = hashlib.sha256(canonical_bytes(body)).hexdigest()
    return f"sha256:{digest}"

Excluding the catalog_hash key from its own input avoids a chicken-and-egg problem and lets you verify a manifest by recomputing the digest and comparing it to the stored value. The content-addressing pattern here mirrors the SHA hashing used elsewhere in the versioning pipeline, applied specifically to acquisition provenance rather than raw pixels.

# Writing the Manifest and Tracking It with DVC

The final stage writes the manifest with its embedded hash to disk and hands the file to DVC. Tracking the manifest — rather than only the imagery — is what makes the STAC-to-version binding survive in git:

from pathlib import Path


def write_manifest(manifest: dict[str, Any], out_path: Path) -> str:
    """Attach the catalog hash, write the manifest, and return the hash."""
    manifest = dict(manifest)  # copy before mutating
    manifest["catalog_hash"] = compute_catalog_hash(manifest)
    out_path.parent.mkdir(parents=True, exist_ok=True)
    out_path.write_bytes(canonical_bytes(manifest))
    return manifest["catalog_hash"]


if __name__ == "__main__":
    start = datetime(2025, 6, 1, tzinfo=timezone.utc)
    end = datetime(2025, 6, 30, tzinfo=timezone.utc)
    selected = select_items(
        "https://example.com/stac/catalog.json",
        bbox=(13.0, 52.3, 13.8, 52.7),  # Berlin AOI, EPSG:4326
        start=start,
        end=end,
    )
    manifest = build_manifest(selected, (13.0, 52.3, 13.8, 52.7), start, end)
    digest = write_manifest(manifest, Path("data/snapshot_manifest.json"))
    print(f"{len(selected)} items, {digest}")

Then bring the manifest under version control. DVC replaces the file with a small .dvc pointer that you commit to git, so the exact STAC-derived composition is recoverable at any revision:

dvc add data/snapshot_manifest.json
git add data/snapshot_manifest.json.dvc data/.gitignore
git commit -m "Snapshot: June 2025 Berlin AOI, sha256:9f3a…"

To reproduce the training set at any later date, check out the revision, run dvc checkout to restore the manifest, recompute the catalog hash, and confirm it matches the stored value before rehydrating assets from the recorded self_href links.

# STAC Fields That Anchor Reproducibility

Not every STAC field matters for version binding. The ones below are the load-bearing pieces — capture them in the manifest or verify them at rehydration time.

# Common Errors and Fixes

Catalog hash changes on every run for the same selection Root cause: STAC item iteration is unordered, so the items list is assembled in a different sequence each time. Fix: sort the selection deterministically (matches.sort(key=lambda it: it.id)) before building the manifest, and serialise with sort_keys=True.

AttributeError: 'NoneType' object has no attribute 'href' when reading self link Root cause: static catalogs generated offline sometimes omit the self link until they are normalised against a root URL. Fix: call catalog.normalize_hrefs(root_href) after loading, or guard with item.get_single_link("self") and fall back to constructing the href from the item id.

dvc add tracks the imagery instead of the manifest Root cause: pointing dvc add at the data directory rather than the single manifest file. Fix: run dvc add data/snapshot_manifest.json on the manifest only; keep multi-gigabyte COGs on their STAC remote and rehydrate from the recorded hrefs.

Recomputed hash mismatches the stored value at checkout Root cause: an upstream provider re-processed a scene and changed an asset href, or the manifest was hand-edited. Fix: this is the system working as intended — diff the current catalog against the manifest to locate the changed item, then decide whether to re-pin the snapshot or restore the archived assets from your DVC remote.

datetime comparison raises TypeError: can't compare offset-naive and offset-aware datetimes Root cause: the query bounds are timezone-naive while STAC item datetimes are UTC-aware. Fix: construct query bounds with tzinfo=timezone.utc so both sides of the comparison are offset-aware.

# Related

• Implementing DVC for Geospatial Training Data — the parent guide covering how DVC pointers, caches, and pipelines version large spatial datasets end to end
• Configuring DVC Remotes for Geospatial Data — set up the S3, GCS, or Azure remote that mirrors the STAC assets your manifest references
• Preserving Metadata Across Dataset Versions — the broader patterns for carrying CRS, geotransform, and acquisition context through every versioned snapshot
• Coordinate Reference Systems in Annotation Pipelines — reproject an area of interest to WGS84 before filtering STAC geometries, and reconcile native asset CRS at rehydration

This guide covers one integration within Implementing DVC for Geospatial Training Data, which is itself part of Dataset Versioning & Spatial Data Sync.