Photometric stereo processing in FPGA of a high-speed camera

Description

Image processing for inline inspection and process monitoring is a crucial component of many modern production systems. The reliability and real-time availability of the information obtained are essential for effective operation. To achieve 100% control of manufactured parts, it is often necessary to inspect large areas for small defects, such as cracks, pores, and bumps, within a short timeframe. This need is particularly evident in industries like metal production, including sheet metal and semi-finished products, and in foil production for rechargeable batteries or photovoltaic cells.
In the case of battery electrode foils, the classification of an anomaly as a defect heavily depends on the surface structure information: small holes or flat stains are significantly less hazardous than peaks, which can potentially cause short circuits. This principle applies broadly in industrial quality inspection, where defects are often identified as geometric deviations from a part’s flat surface. In these cases, classic 2D image vision technology does not provide enough information to distinguish, for example, environment-related artifacts such as stains or dust from product quality-related characteristics including defects such as pores. In a conventional 2D image, both artifacts and defects can look very similar. Full 3D vision technology, on the other hand, is often too slow or expensive.
By adding surface-related information to the analysis of conventional 2D image data, which is provided by inline photometric stereo (PS) in terms of the local surface curvature, the subsequent defect classifier can discriminate more easily between environment-related artifacts and product defects. PS works with images acquired under different lighting directions and simultaneously provides texture (albedo) and structural information (2.5D gradients).
This talk presents how the inline PS algorithm sketched above is implemented in real-time in a resource-limited FPGA of a high-speed line-scan camera. The used camera reaches a maximum line frequency of 600 kHz, which allows PS recordings at 300 kHz for two illumination directions, at 200 kHz for three illumination directions, or at 150 kHz for four illumination directions. The PS processing is implemented in the camera’s FPGA. The implementation of the algorithm on this resource-limited hardware, also using high-level synthesis methodology, is presented.
The resulting albedo and gradient images for defect detection have been generated simultaneously and with a delay of a few microseconds. With the presented 2.5D camera, a scanning speed of up to 15 m/s can be achieved (for two illumination directions) with a resolution of 50 μm/px, at which co-registered 2D albedo and 2.5D gradient images can be recorded. This means that roughly every 3 μs the camera delivers 2000 albedo and gradient values each. The energy consumption of the camera is only 15 W. For the total system, the power consumption of the high-performance LED lights and the trigger electronics (~100 W in total) should be taken into account.
This contribution shows how inline photometric stereo processing can be integrated into the FPGA of a camera. Despite rather restricted hardware resources in the FPGA, the implemented algorithm introduces negligible latency and enables better failure discrimination in the complete vision system.

Period	5 Nov 2024
Event title	Bildverarbeitung in der Automation (BVAu) 2024
Event type	Conference
Location	Lemgo, Germany, North Rhine-WestphaliaShow on map
Degree of Recognition	International