RGB-D structural classification of guardrails via learning from synthetic data

Vision-based environment perception is a key sensing and analysis modality for mobile robotic platforms. Modern learning concepts allow for interpreting a scene in terms of its objects and their spatial relations. This paper presents a specific analysis pipeline targeting the structural classification of guardrail structures within roadside environments from a mobile platform. Classification implies determining the type label of an observed structure, given a catalog of all possible types. To this end, the proposed concept employs semantic segmentation learned fully in the synthetic domain, and stereo depth data analysis for estimating the metric dimensions of key structural elements. The paper introduces a Blender-based procedural data generation pipeline, targeting to accomplish a narrow sim-to-real gap, allowing to use synthetic training image data to train models valid in the real-world domain. The paper evaluates two semantic segmentation schemes for the part segmentation task, and presents a temporal tracking and propagation concept to aggregate single-frame estimates. Results demonstrate that the proposed analysis framework is well applicable to real scenarios and it can be used as a tool for digitally mapping safety-critical roadside assets