The application of 3D vision measurement technology is becoming increasingly widespread. Current main techniques include binocular stereo vision, laser scanning, and structured light scanning. Among these, structured light technology is considered a major direction for future 3D vision development due to its advantages of simple system structure, high speed, and high accuracy. A typical structured light system consists of a projection module and a camera module. Through system calibration, the internal and external parameters of the camera and projector are acquired, and then a precise 3D reconstruction process is achieved based on the principle of triangulation.
The core technology of structured light systems lies in their structured light encoding and decoding methods. Currently, the main structured light encoding strategy employs a sinusoidal phase-shift plus Gray code approach. This involves obtaining local phase encoding through sinusoidal phase-shifted fringes projected by a projection module, and then using Gray code to achieve global phase unwrapping. Most structured light 3D scanners currently use this encoding strategy. However, this method also suffers from significant problems, namely insufficient robustness. Particularly with reflective surfaces, the grayscale information of the projected fringes is affected by surface reflection, causing grayscale distribution distortion and severely impacting the accuracy of phase calculations, even leading to reconstruction failure. In practical applications, it is often necessary to pre-coat reflective surfaces with a developer to eliminate reflection factors, thus greatly limiting the application scope of structured light technology.
To improve the robustness of structured light systems, researchers have proposed encoding methods based on linear displacement fringes, high-frequency displacement fringes, and binary band displacement fringes, which have significantly improved the robustness of the technology. However, for highly reflective surfaces, such as the metal sheet stampings shown in Figure 1, the high reflectivity causes breakage of the projected fringes and local dark areas, making it impossible to extract effective structured light fringe information. This renders existing methods unusable directly. To address this issue, this paper introduces a structured light 3D scanning method based on the principle of high dynamic range imaging (HDR). The basic idea of this method is to apply the traditional HDR imaging method to structured light image processing. By calculating the camera response curve, structured light images under different exposure parameters are fused to obtain a high dynamic range structured light image, enabling better imaging of both overexposed and underexposed areas. This allows for direct 3D scanning of highly reflective metal surfaces without coating treatment.
Figure 1 shows that when a structured light image is projected onto a metal sheet stamping, the surface reflection causes large overexposed and underexposed areas, making it impossible to achieve complete 3D reconstruction.
Camera response curve calculation
High dynamic range (HVR) imaging technology has been widely applied in image processing and photography. By fusing images with different exposure levels, it can significantly improve the dynamic range of an image, allowing both bright and dark areas to be visually rendered effectively. Assuming there are no dynamic targets in the scene, the imaging process of image Iij can be described as follows:
Where f represents the camera's response function, Ei represents the irradiance of pixel i, and represents the camera's exposure time. Assuming the response function f is monotonically invertible, and introducing the function g = lnf-1, then formula (1) can be written as :
In Equation (2), the pixel irradiance Ei and the camera response function g are both unknown. In order to recover the correct irradiance, we need to pre-calculate the camera response function g. Since the gray values of pixels in actual images are finite, that is, distributed in the range of 0~255, theoretically we can obtain the brightness values under different exposure times by adjusting the exposure time, and calculate the camera response curve through the least squares fitting strategy. This calculation method is also often used in practice.
Traditional HDR research mostly deals with natural images, which have a wide range of grayscale distribution. However, for structured light images, especially binary coded structured light images, the grayscale values in structured light images are relatively limited due to the influence of surface reflection. In order to improve computational efficiency, we adopted the method of image sampling and extracted 50, 100, and 200 sampling points to calculate the camera response curve, as shown in the figure below. The results show that even with 50 sampling points, the camera response curve can still be obtained well.
Figure 2 shows the distribution of camera response curves obtained under different numbers of sampling points.
High dynamic structured light image synthesis
After obtaining the camera response curve g, the irradiance value of each pixel can be calculated using the following formula.
After adding the weight function ω(z), the above equation can be rewritten as :
By calculating all image points, we can obtain the irradiance map of the corresponding response curve. However, the range of the irradiance map is much larger than the grayscale range of 0 to 255. We adopted a gradient compression strategy to convert the high dynamic range image to a normal grayscale space without losing local image details.
The absorption function is expressed and described as follows:
By solving the Poisson equation below, we can reconstruct an image with a higher dynamic range.
Since the structured light scanning process involves a series of stripe image projections, it causes changes in lighting conditions. To ensure the consistency of lighting conditions, we used a set of medium-exposure structured light images as a reference and performed post-processing on images with other exposure times. The final HDR structured light image obtained was calculated as follows.
Experimental Results Analysis
The experimental setup, as shown in Figure 3, includes a DLP projector (LGHX300G, 1024×768 pixels resolution), an industrial camera (PointGreyFL3-U3-32S2M-CS , USB3.0 , 60fps, 2080×1552 pixels resolution) , a working distance of approximately 500mm, and employs high-frequency binary displacement stripe coding. The time required to project 30 images is approximately 0.5 seconds. The experimental target was mainly metal sheet stamping parts. The camera exposure time was set to 2ms , 3ms , 4ms , 5ms , 6ms and 7ms respectively. We calculated the 3D reconstruction results under different exposure times, as shown in Figure 4. When the camera exposure time was short, there were large underexposed areas in the image. Since the structured light stripes could not be imaged, a large amount of reconstruction loss was caused. For longer exposure times, the overexposed areas of the image increased, which also caused reconstruction loss. In the HDR image synthesized by the above method, the brightness of the underexposed areas was significantly improved, while the overexposed areas could also be well suppressed, and a complete 3D reconstruction process could be basically achieved.
To evaluate the reconstruction accuracy of the system, we sprayed a piece of stamping and performed 3D scanning on the workpiece before and after spraying. We then compared the accuracy of the two models. The results showed an average error of approximately 0.06 mm, a variance of approximately 0.08 mm, and a maximum error of approximately 0.45 mm . The larger errors mainly occurred at sharp edges, primarily because the edges still interfered with the structured light stripes even at lower exposure times. Only by using even lower exposure times could the structured light stripe image at these points be clearly restored, but this required more exposure parameter settings. Considering that the current reconstruction accuracy is sufficient to meet the detection accuracy requirements of stamping parts, 5-6 exposure combinations are perfectly adequate. Figure 6 shows the 3D scanning results of stamping parts with different shapes.
Figure 3. The developed structured light 3D scanning system consists of a DLP projector and a USB 3.0 industrial camera.
Conclusions and Outlook
Structured light 3D scanning technology has broad application prospects in the field of industrial inspection. Its main technical challenge comes from the uncertainty of the target and scene, which results in poor imaging quality of structured light images and thus affects the reconstruction accuracy. The high dynamic range structured light 3D imaging method we proposed provides an effective solution to this problem. However, multiple exposures also significantly increase the scanning time and data processing volume. Future research can consider how to achieve scene adaptive adjustment of structured light images to improve the robustness of the scanning process without significantly increasing the shooting time, thereby meeting more online 3D inspection needs.
