Stereo matching is a core step in 3D reconstruction. Its input is a pair of corrected color images, and disparity is obtained by matching the projection points of the same 3D object onto the left and right images. Currently, existing stereo matching methods achieve high matching accuracy in unoccluded areas, but suffer from significant matching errors in occluded areas. Furthermore, the computational efficiency of existing methods is generally low. Therefore, this study proposes a segmentation-based stereo matching disparity optimization algorithm to address the problem of fast stereo matching for images containing occluded regions.
The proposed segmentation-based stereo matching disparity optimization method directly optimizes the disparity (WTA) corresponding to the initial matching minimum cost obtained from a color reference image. First, the reference image is segmented into superpixels, and the disparity optimization method processes these superpixels directly. Then, a foreground parallel disparity map is obtained by estimating the average disparity of the superpixels. Finally, a plane is assigned to each superpixel to optimize and obtain a tilted surface disparity map. (Foreground parallel disparity map to tilted surface disparity map)
The transformation of the surface disparity map is achieved through a global optimization layer and a local optimization layer. In the global optimization layer, a Markov random domain optimization algorithm is used to estimate the forward parallel disparity map; in the local optimization layer, a random sampling consensus algorithm and a probability-based disparity plane algorithm are used to optimize the tilted surface disparity map. The proposed method was tested on the Middlebury 2014 and KITTI 2015 databases and compared with state-of-the-art disparity optimization methods. In terms of computational accuracy, the proposed method ranks 6th; in terms of computational efficiency, it is more than 30 times faster than other methods.
The segmentation-based stereo matching parallax optimization method proposed by this institute effectively solves the problem of fast and accurate 3D reconstruction of scenes with occlusion, laying the foundation for real-time 3D scene reconstruction in fields such as industrial robots, as well as real-time 3D reconstruction of tissues and instruments in medical robot applications under special environments.
Figure 1. Overall flowchart of the segmentation-based disparity method
