Unifying Panoptic Segmentation for Autonomous Driving

Research output: Chapter in Book or Conference ProceedingsConference Proceedings with Poster Presentationpeer-review


This paper aims to improve panoptic segmentation for real-world applications in three ways. First, we present a label policy that unifies four of the most popular panoptic segmentation datasets for autonomous driving. We also clean up label confusion by adding the new vehicle labels pickup and van. Full relabeling information for the popular Mapillary Vistas, IDD, and Cityscapes dataset are provided to add these new labels to existing setups. Second, we introduce Wilddash2 (WD2), a new dataset and public benchmark service for panoptic segmentation. The dataset consists of more than 5000 unique driving scenes from all over the world with a focus on visually challenging scenes, such as diverse weather conditions, lighting situations, and camera characteristics. We showcase experimental visual hazard classifiers which help to pre-filter challenging frames during dataset creation. Finally, to characterize the robustness of algorithms in out-of-distribution situations, we introduce hazard-aware and negative testing for panoptic segmentation as well as statistical significance calculations that increase confidence for both concepts. Additionally, we present a novel technique for visualizing panoptic segmentation errors. Our experiments show the negative impact of visual hazards on panoptic segmentation quality. Additional data from the WD2 dataset improves performance for visually challenging scenes and thus robustness in real-world scenarios.
Original languageEnglish
Title of host publicationProc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Number of pages10
Publication statusPublished - 2022
EventThe IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) 2022 -
Duration: 24 Jun 2022 → …


ConferenceThe IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) 2022
Period24/06/22 → …

Research Field

  • Assistive and Autonomous Systems


  • panoptic segmentation
  • dataset for autonomous driving


Dive into the research topics of 'Unifying Panoptic Segmentation for Autonomous Driving'. Together they form a unique fingerprint.

Cite this