Best practices for polygon vs bounding box annotation

Use bounding boxes for rapid, coarse localization when instance separation and throughput matter more than pixel-perfect edges. Use polygons when precise spatial extent, area calculation, or boundary-aware model training is required — for example, building footprint extraction, solar-array delineation, or regulatory land-cover mapping. The optimal strategy depends on your target architecture, annotation budget, and downstream inference constraints. When working within a defined ROI label taxonomy for aerial imagery, assign annotation fidelity per class tier: reserve polygons for high-precision categories and use bounding boxes for auxiliary or rapidly changing objects. For most aerial imagery pipelines, bootstrap large-scale datasets with bounding boxes, then refine high-value classes with polygons where boundary precision directly impacts model performance, regulatory compliance, or geospatial analytics.

Polygon vs Bounding Box annotation decision flow Flowchart showing how object compactness, model architecture, and downstream analytics requirements drive the choice between bounding box and polygon annotation, with a hybrid path for active-learning refinement. New object class Compact / rectangular? Yes No Bounding box Polygon mask YOLO / DETR detector Mask R-CNN / SAM Hybrid: boxes → uncertainty routing → polygon refinement (active learning) Shapely topology + BIoU QA before export

Why geometry choice breaks geospatial pipelines

Annotation geometry determines not just labeling speed but model behaviour, CRS-correctness of area calculations, and the validity of downstream geospatial analytics. Using bounding boxes where polygons are needed floods segmentation heads with background noise and prevents accurate footprint extraction. Conversely, forcing polygon annotation on every class stalls throughput, raises QA failure rates from self-intersecting rings, and inflates GPU memory during mask rasterization — effects that compound across coordinate reference systems in annotation pipelines when geometries are stored in geographic degrees (EPSG:4326) rather than a local metric CRS, causing IoU metrics to distort at mid-latitudes.

Step-by-step implementation

Step 1 — Classify object types by spatial fidelity tier

Group every class in your taxonomy into one of three tiers before annotation begins:

python 
# Python 3.10+ | shapely==2.0.6, geopandas==1.0.1
from dataclasses import dataclass
from enum import Enum

class AnnotationTier(Enum):
    BBOX = "bbox"          # compact, near-rectangular, low boundary importance
    POLYGON = "polygon"    # irregular shape, area analytics, or seg model training
    HYBRID = "hybrid"      # start with bbox; refine to polygon via active learning

@dataclass
class ClassSpec:
    name: str
    tier: AnnotationTier
    rationale: str

taxonomy: list[ClassSpec] = [
    ClassSpec("vehicle",            AnnotationTier.BBOX,    "compact, high density, throughput priority"),
    ClassSpec("building",           AnnotationTier.POLYGON, "footprint area analytics, cadastral compliance"),
    ClassSpec("solar_array",        AnnotationTier.POLYGON, "boundary-precise for area/output estimation"),
    ClassSpec("vegetation_patch",   AnnotationTier.HYBRID,  "box for detection; polygon where area > 500 m²"),
    ClassSpec("shipping_container", AnnotationTier.BBOX,    "rectangular, change-detection use case"),
]

Step 2 — Bootstrap the detector with bounding boxes

Train a lightweight detector on the full dataset using bounding boxes first. This surfaces which classes and spatial regions generate the most uncertainty before expensive polygon work begins.

python 
# Python 3.10+ | ultralytics==8.2.0
from ultralytics import YOLO
from pathlib import Path

def train_bbox_bootstrapper(data_yaml: Path, epochs: int = 50) -> Path:
    """Train a YOLOv8n detector on bounding-box annotations."""
    model = YOLO("yolov8n.pt")
    result = model.train(
        data=str(data_yaml),
        epochs=epochs,
        imgsz=1024,        # typical aerial tile size
        batch=8,
        device="cuda",
    )
    return Path(result.save_dir) / "weights" / "best.pt"

Step 3 — Route high-uncertainty instances to polygon annotators

Use prediction confidence and box IoU variance to identify which instances need polygon refinement. This avoids blanket polygon annotation and focuses expert time on the highest-ambiguity objects. Per-annotation confidence scores are the key signal driving this routing decision.

python 
# Python 3.10+ | ultralytics==8.2.0, numpy==1.26.4
import numpy as np
from ultralytics import YOLO
from pathlib import Path

CONFIDENCE_THRESHOLD = 0.55    # below this → route to polygon annotation queue
IOU_VARIANCE_THRESHOLD = 0.08  # high variance across augmentations → ambiguous boundary

def route_uncertain_predictions(
    model_path: Path,
    image_path: Path,
    n_augment: int = 5,
) -> list[dict]:
    """Return instances that should be refined with polygon masks."""
    model = YOLO(str(model_path))
    all_confs: list[np.ndarray] = []

    for _ in range(n_augment):
        results = model.predict(str(image_path), augment=True, verbose=False)
        confs = results[0].boxes.conf.cpu().numpy()
        all_confs.append(confs)

    # Flag objects where confidence is low OR varies across augmentations
    mean_conf = np.mean(all_confs, axis=0)
    conf_variance = np.var(all_confs, axis=0)

    flagged: list[dict] = []
    results_base = model.predict(str(image_path), verbose=False)[0]
    for i, box in enumerate(results_base.boxes.xyxy.cpu().numpy()):
        if mean_conf[i] < CONFIDENCE_THRESHOLD or conf_variance[i] > IOU_VARIANCE_THRESHOLD:
            flagged.append({
                "box_xyxy": box.tolist(),
                "mean_conf": float(mean_conf[i]),
                "conf_variance": float(conf_variance[i]),
                "action": "polygon_annotation_required",
            })
    return flagged

Step 4 — Validate polygon topology with Shapely before export

Self-intersecting polygons and duplicate vertices corrupt COCO masks and cause silent failures in Mask R-CNN dataloaders. Run this validation as a mandatory pre-export gate; the same check integrates into a DVC pipeline step for automated dataset snapshots.

python 
# Python 3.10+ | shapely==2.0.6, geopandas==1.0.1
import geopandas as gpd
from shapely.validation import explain_validity
from pathlib import Path

def validate_and_repair_polygons(geojson_path: Path) -> gpd.GeoDataFrame:
    """
    Load, validate, and auto-repair a GeoJSON annotation file.
    Raises ValueError if any geometry cannot be repaired.
    """
    gdf: gpd.GeoDataFrame = gpd.read_file(geojson_path)

    invalid_mask = ~gdf.geometry.is_valid
    if invalid_mask.any():
        print(f"Repairing {invalid_mask.sum()} invalid geometries …")
        for idx in gdf[invalid_mask].index:
            reason = explain_validity(gdf.at[idx, "geometry"])
            repaired = gdf.at[idx, "geometry"].buffer(0)
            if not repaired.is_valid:
                raise ValueError(f"Row {idx} cannot be repaired: {reason}")
            gdf.at[idx, "geometry"] = repaired

    # Simplify vertex density: 1 vertex per ~5 px at 30 cm GSD → ~0.15 m tolerance
    gdf["geometry"] = gdf.geometry.simplify(tolerance=0.15, preserve_topology=True)

    return gdf

Step 5 — Measure Boundary IoU per class to confirm quality

Standard IoU is insensitive to jagged edges. Boundary IoU (BIoU) penalises misaligned contours and is the correct QA metric when confidence scores drive active-learning queue prioritisation.

python 
# Python 3.10+ | shapely==2.0.6
from shapely.geometry import Polygon
from shapely.ops import unary_union

def boundary_iou(pred: Polygon, gt: Polygon, dilation_m: float = 0.5) -> float:
    """
    Compute Boundary IoU between predicted and ground-truth polygons.
    dilation_m: boundary band width in metres (assumes metric CRS, e.g. EPSG:32633).
    """
    pred_boundary = pred.boundary.buffer(dilation_m)
    gt_boundary   = gt.boundary.buffer(dilation_m)

    intersection = pred_boundary.intersection(gt_boundary).area
    union        = unary_union([pred_boundary, gt_boundary]).area

    return intersection / union if union > 0 else 0.0

Always reproject geometries to a local metric CRS (e.g. EPSG:32633 for UTM Zone 33N) before calling boundary_iou — computing BIoU in geographic degrees (EPSG:4326) produces meaningless results at mid-latitudes.

Spatial parameters and thresholds reference

Parameter Bounding boxes Polygons Notes
Annotation speed 3–5× faster than polygons Baseline Varies by annotator experience
Storage per instance 4 floats Variable vertex count Polygons grow with shape complexity
Min IoU for QA pass 0.70 (standard) 0.65 BIoU (boundary-sensitive) Lower BIoU threshold accounts for contour difficulty
Vertex simplification tolerance n/a 0.10–0.25 m at 30 cm GSD Douglas-Peucker; target 1 vertex per 5–10 px
Max vertices per polygon n/a 500 Above this, rasterisation stalls COCO dataloader
Min object area (boxes) 32 × 32 px n/a Below this threshold, annotation noise exceeds signal
Recommended CRS for QA EPSG:32633 / local UTM EPSG:32633 / local UTM Never compute IoU in EPSG:4326

Common errors and fixes

TopologicalError: This operation could not be performed : Cause: self-intersecting ring generated by vertex snapping or annotator trace error. : Fix: geom = geom.buffer(0) — Shapely’s zero-buffer trick dissolves self-intersections; verify with geom.is_valid after.

ValueError: A LinearRing requires at least 4 coordinates : Cause: a degenerate polygon with fewer than 4 vertices, often from rushed polygon closure on tiny objects. : Fix: filter gdf[gdf.geometry.apply(lambda g: len(g.exterior.coords) >= 4)] before export; log discarded instances.

IoU scores collapse at mid-latitudes despite correct pixel alignment : Cause: IoU computed in EPSG:4326; degree-based distances distort area calculations. : Fix: gdf = gdf.to_crs("EPSG:32633") (or the appropriate UTM zone) before any metric computation.

COCO mask export produces out-of-memory error during training : Cause: polygon vertex count is too high for dense rasterisation across a full batch. : Fix: apply simplify(tolerance=0.15, preserve_topology=True) and cap at 500 vertices per instance before converting to RLE masks.

This page is one focused how-to within the broader Defining ROI Label Taxonomies for Aerial Imagery workflow, which covers class hierarchy design, confidence scoring, and multi-sensor taxonomy alignment.

Related