Resolving Sync Conflicts in Distributed Annotation

When two annotation sites edit the same versioned dataset and both push, a naive sync clobbers one side’s work. The durable fix is to key every feature on a stable feature id and perform a three-way merge — comparing a common-ancestor GeoJSON against each site’s version — rather than a two-way file diff. Geometry and attribute changes that touch different features merge automatically; edits to the same feature go to a review queue instead of being guessed at; and a last-writer-wins guard keyed on edit timestamps is the bounded fallback for conflicts that must resolve without a human. This turns “whoever pushed last wins the whole file” into “every non-conflicting edit survives, and real conflicts are surfaced, not lost.”

# Why Silent Clobbering Corrupts Training Data

Distributed annotation teams rarely edit in lockstep. One site refines building footprints in a northern tile while another relabels vehicles in a southern tile of the same GeoJSON collection. Both branch from the same snapshot, both edit, both push. If your sync layer resolves this by keeping the last-uploaded object — the default for object stores and for a careless DVC workflow that tracks the file as one opaque blob — one site’s entire session vanishes with no error. The dataset still validates, still loads, still trains. It is simply missing labels, and the loss surfaces only as unexplained recall degradation weeks later.

A two-way diff does not save you either. Given only ours and theirs, a diff cannot distinguish “they added feature X” from “we deleted feature X” — both look like feature X being present on one side and absent on the other. Without the shared base version as a reference point, every merge decision is a guess. The three-way merge exists precisely to remove that ambiguity: by comparing each side against the common ancestor, an added feature, a removed feature, and a modified feature become three distinct, detectable events.

Two structural preconditions make this work. First, features must carry a stable feature id — a UUID minted at creation, stored in a property, and never regenerated on edit. Array index and geometry hash are both unstable: nudge one vertex and a hash-keyed feature reads as delete-plus-add, silently discarding the annotator’s refinement. Second, features should carry an edit timestamp so the fallback guard has something principled to compare. Getting these two fields onto every feature is upstream work, but everything below depends on them.

# Step-by-Step Implementation

The workflow below loads three FeatureCollections, classifies every feature against the base, auto-merges non-overlapping changes, isolates true conflicts, and writes both a merged file and a conflict report. It assumes each feature stores a feature_id string and an edited_at ISO timestamp in its properties. Coordinates are geographic; the merge is CRS-agnostic as long as all three files share one coordinate reference system — mixing EPSG:4326 on one side with a projected CRS on another would make identical geometries compare as modified, so normalise CRS before merging.

Install the pinned dependencies once:

pip install geopandas==0.14.4 shapely==2.0.6 pandas==2.2.2

# Step 1 — Load Base, Ours, and Theirs Keyed by Feature Id

Read all three versions into geopandas GeoDataFrames indexed by the stable id. Indexing by feature_id — not row order — is what lets the same feature line up across three files that may store features in different orders.

from __future__ import annotations
import geopandas as gpd

def load_keyed(path: str, id_field: str = "feature_id") -> gpd.GeoDataFrame:
    """Load a GeoJSON FeatureCollection indexed by its stable feature id."""
    gdf: gpd.GeoDataFrame = gpd.read_file(path)
    if id_field not in gdf.columns:
        raise ValueError(f"{path}: every feature needs a stable '{id_field}'")
    if gdf[id_field].duplicated().any():
        raise ValueError(f"{path}: duplicate {id_field} values break the merge")
    return gdf.set_index(id_field, drop=False)

base: gpd.GeoDataFrame = load_keyed("base.geojson")
ours: gpd.GeoDataFrame = load_keyed("ours.geojson")
theirs: gpd.GeoDataFrame = load_keyed("theirs.geojson")

# Step 2 — Classify Each Feature Against the Base

For one side, compare every feature id to the base to label it added, removed, or modified. A feature is modified when its geometry or its non-metadata attributes differ from the base version. Comparing on WKB plus a sorted attribute tuple is stable and ignores property ordering.

from shapely.geometry.base import BaseGeometry

IGNORE_PROPS: set[str] = {"edited_at"}  # metadata that should not count as a change

def _fingerprint(row: "gpd.GeoSeries", geom: BaseGeometry) -> tuple[bytes, tuple]:
    props = tuple(
        (k, row[k]) for k in sorted(row.index)
        if k not in IGNORE_PROPS and k != "geometry"
    )
    return (geom.wkb, props)

def classify(side: gpd.GeoDataFrame, base: gpd.GeoDataFrame) -> dict[str, str]:
    """Return {feature_id: 'added' | 'removed' | 'modified'} for one side vs base."""
    changes: dict[str, str] = {}
    base_ids: set[str] = set(base.index)
    side_ids: set[str] = set(side.index)

    for fid in side_ids - base_ids:
        changes[fid] = "added"
    for fid in base_ids - side_ids:
        changes[fid] = "removed"
    for fid in side_ids & base_ids:
        if _fingerprint(side.loc[fid], side.geometry.loc[fid]) != \
           _fingerprint(base.loc[fid], base.geometry.loc[fid]):
            changes[fid] = "modified"
    return changes

# Step 3 — Merge Non-Overlapping Changes and Flag Conflicts

Now walk the union of both changesets. If only one side touched a feature, apply that side’s version. If both sides touched the same feature, it is a conflict unless — the one safe exception — both made the identical edit, in which case either version serves.

from dataclasses import dataclass, field

@dataclass
class Conflict:
    feature_id: str
    ours_change: str | None
    theirs_change: str | None

@dataclass
class MergeResult:
    features: dict[str, "gpd.GeoSeries"] = field(default_factory=dict)
    conflicts: list[Conflict] = field(default_factory=list)

def three_way_merge(base: gpd.GeoDataFrame,
                    ours: gpd.GeoDataFrame,
                    theirs: gpd.GeoDataFrame) -> MergeResult:
    ours_ch = classify(ours, base)
    theirs_ch = classify(theirs, base)
    result = MergeResult()

    # Start from base; unchanged features carry through untouched.
    for fid in base.index:
        result.features[fid] = base.loc[fid]

    touched: set[str] = set(ours_ch) | set(theirs_ch)
    for fid in touched:
        o, t = ours_ch.get(fid), theirs_ch.get(fid)
        if o and not t:                       # only we changed it
            _apply(result, fid, "ours", ours, o)
        elif t and not o:                     # only they changed it
            _apply(result, fid, "theirs", theirs, t)
        elif _same_edit(fid, o, t, ours, theirs):
            _apply(result, fid, "ours", ours, o)   # identical edit, no conflict
        else:                                 # same feature, different edits
            result.conflicts.append(Conflict(fid, o, t))
    return result

def _apply(result: MergeResult, fid: str, side: str,
           gdf: gpd.GeoDataFrame, change: str) -> None:
    if change == "removed":
        result.features.pop(fid, None)
    else:                                     # added or modified
        result.features[fid] = gdf.loc[fid]

def _same_edit(fid: str, o: str | None, t: str | None,
               ours: gpd.GeoDataFrame, theirs: gpd.GeoDataFrame) -> bool:
    if o != t or o == "removed":
        return o == "removed" and t == "removed"
    return _fingerprint(ours.loc[fid], ours.geometry.loc[fid]) == \
           _fingerprint(theirs.loc[fid], theirs.geometry.loc[fid])

# Step 4 — Apply the Last-Writer-Wins Guard

For pipelines that must produce a merged file with no human in the loop, resolve remaining conflicts by keeping the feature with the newer edited_at. Always log the discarded side so the choice stays auditable rather than silent.

from datetime import datetime

def resolve_last_writer_wins(result: MergeResult,
                             ours: gpd.GeoDataFrame,
                             theirs: gpd.GeoDataFrame) -> list[dict[str, str]]:
    """Auto-resolve conflicts by newest timestamp; return an audit log."""
    log: list[dict[str, str]] = []
    for c in list(result.conflicts):
        if c.feature_id not in ours.index or c.feature_id not in theirs.index:
            continue  # delete-vs-modify: leave for human review
        o_ts = datetime.fromisoformat(ours.loc[c.feature_id, "edited_at"])
        t_ts = datetime.fromisoformat(theirs.loc[c.feature_id, "edited_at"])
        winner = "ours" if o_ts >= t_ts else "theirs"
        src = ours if winner == "ours" else theirs
        result.features[c.feature_id] = src.loc[c.feature_id]
        result.conflicts.remove(c)
        log.append({"feature_id": c.feature_id, "kept": winner,
                    "reason": "last_writer_wins"})
    return log

# Step 5 — Write Merged Output and a Conflict Report

Serialize the surviving features back to GeoJSON and emit a JSON report so review tooling and downstream stages know exactly what merged cleanly and what still needs a decision.

import json
import pandas as pd

def write_outputs(result: MergeResult, base: gpd.GeoDataFrame,
                  merged_path: str = "merged.geojson",
                  report_path: str = "conflict_report.json") -> None:
    rows = list(result.features.values())
    merged = gpd.GeoDataFrame(pd.DataFrame(rows), crs=base.crs)
    merged.to_file(merged_path, driver="GeoJSON")

    report = {
        "merged_feature_count": len(result.features),
        "unresolved_conflicts": [
            {"feature_id": c.feature_id, "ours": c.ours_change,
             "theirs": c.theirs_change} for c in result.conflicts
        ],
    }
    with open(report_path, "w", encoding="utf-8") as fh:
        json.dump(report, fh, indent=2)

# End-to-end
res = three_way_merge(base, ours, theirs)
audit = resolve_last_writer_wins(res, ours, theirs)  # optional automatic pass
write_outputs(res, base)

# Change Type to Resolution Reference

The merge’s entire decision surface reduces to how a feature changed on each side. This table is the contract the code above implements.

The two rows that most often get mishandled are delete-versus-modify and same-id collisions. Never let a deletion win automatically over a refinement: the annotator who deleted may have been working from stale context, and dropping a corrected boundary is unrecoverable without digging through history. Same-id collisions usually signal that two sites minted the same UUID or that ids were reused — treat them as conflicts and fix the id generator upstream.

# Common Errors and Fixes

Every feature reports as modified even when nothing changed Root cause: the three files use different CRS or different coordinate precision, so WKB fingerprints differ byte-for-byte. Fix: reproject all sides to one CRS and round coordinates to a fixed precision (for example shapely.set_precision(geom, 1e-7)) before fingerprinting.

Edited features appear as a delete plus an add Root cause: features are keyed on array index or a geometry hash, so any vertex move changes the key. Fix: key strictly on a persistent feature_id UUID assigned at creation and preserved through every edit.

KeyError on edited_at during last-writer-wins Root cause: some features predate the timestamp convention. Fix: backfill edited_at from version history, or skip the guard for those ids and route them to the review queue instead of crashing the merge.

Merged file loses features that neither side changed Root cause: the merge started from the changesets alone and never seeded from base. Fix: initialise the result with every base feature first (Step 3 does this), then apply changes on top so untouched features always survive.

Two sites both push a feature with the same id but different geometry Root cause: non-unique id generation across sites. Fix: mint UUIDs with a site prefix or a UUID4 source, and treat any residual collision as a conflict rather than overwriting — the same discipline that tracking annotation changes with SHA hashing uses to keep identities stable.

# Related

• Debugging Annotation Drift Across Dataset Versions — trace how features changed between snapshots, the diff foundation this merge builds on
• Tracking Annotation Changes with SHA Hashing — content-addressed identities that make base-versus-side comparison deterministic
• Implementing DVC for Geospatial Training Data — version the datasets whose concurrent edits create these conflicts in the first place
• Preserving Metadata Across Dataset Versions — keep the feature ids and edit timestamps the merge depends on attached to every export

This guide is part of the broader Syncing Annotations Across Cloud Remotes topic area within Dataset Versioning & Spatial Data Sync.