Owli-AI Logo Owli-AI
Menu

Owli-AI Research

Open-Vocabulary Annotation for Pedestrian Scene Understanding under Domain Shift

ECCV 2026 Workshops · ACVR & CDEL (2026) - Workshop contribution

Justine Buss; Michael Kreutzer; Seyed Eghbal Ghobadi

Technische Hochschule Mittelhessen – University of Applied Sciences, Gießen, Germany

Notice: this page was machine-translated and is pending editorial review.

Visual

Flowchart of the annotation pipeline: an input image feeds open-vocabulary localization and prompt-based segmentation in parallel. Localization predicts bounding boxes that condition segmentation; the resulting semantic map is stabilized and output as a validated semantic map.
Fig. 1: Foundation-model-based annotation pipeline with open-vocabulary localization, prompt-based segmentation, and downstream annotation stabilization.

Abstract

This work investigates whether open-vocabulary grounding and segmentation models can generate pixel-level annotations under strong domain shift in pedestrian-centered scenes. It combines language-guided object localization with prompt-based segmentation and a conservative filtering strategy. The resulting pseudo-labels are used to train and evaluate several architectures for binary sidewalk segmentation and road–sidewalk segmentation. High-capacity models exceed 80% mIoU for binary sidewalk segmentation. The results indicate that foundation-model-based annotation pipelines can provide a practical starting point for datasets in low-resource assistive-vision domains, while semantic grounding under domain shift remains a key limitation.

Keywords

  • semantic segmentation
  • open-vocabulary annotation
  • foundation models
  • assistive vision
  • pedestrian scene domain shift

Download

Download PDF Back

Figures

2 visuals from the paper.

  1. Flowchart of the annotation pipeline: an input image feeds open-vocabulary localization and prompt-based segmentation in parallel. Localization predicts bounding boxes that condition segmentation; the resulting semantic map is stabilized and output as a validated semantic map.
    Fig. 1: Foundation-model-based annotation pipeline with open-vocabulary localization, prompt-based segmentation, and downstream annotation stabilization.
  2. Comparison of accepted and rejected BGH annotations after postprocessing. Three accepted image/segmentation pairs are shown on the left with plausible masks; three rejected examples on the right contain clearly incomplete or incorrect masks. A color legend identifies classes including road, sidewalk, building, traffic sign, vegetation, person, car, bicycle, and trash can.
    Fig. 2: Comparison of accepted and rejected annotations in the BGH dataset after postprocessing.

Additional notes

Research context

The work grew out of Justine Buss’s bachelor’s thesis and addresses a central prerequisite for assistive computer-vision systems: how domain-specific training data can be created with manageable manual effort.

The contribution was presented at ECCV 2026 in Malmö as a cross-contribution to the workshops ACVR – Assistive Computer Vision and Robotics and CDEL – Curated Data for Efficient Learning. It belongs to Owli-AI’s assistive computer vision and mobility research strand, which also includes the experimental Way-Buddy demonstrator.

THM report on ECCV 2026