In recent years, with the rapid development of the 3D sensing industry, machine vision has ushered in the fourth visual revolution. The Bokent Robotics Research Institute team has used a structured light camera in conjunction with a self-developed serial-parallel hybrid six-axis robot to complete 3D vision-related inspection projects, and it is about to be put into practical application.
The fourth visual revolution has witnessed a historic technological explosion, overshadowing the previous three visual revolutions. The most significant feature of this revolution is the direct evolution of machines from a 2D planar "world" to a 3D stereoscopic "world." Artificial intelligence products such as facial recognition payment, Face ID, VR, unmanned convenience stores, and intelligent robots have moved from the laboratory to the general public, frequently appearing as "future products." The key technology behind this is 3D vision technology, which has become a new darling of the artificial intelligence world.
Humans using technology to enhance machine vision and simulate human eyes might seem unremarkable. However, in commercial applications, the impact of 3D vision is comparable to a butterfly effect in the Amazon rainforest. Take Alipay's facial recognition payment system as an example. Orbbec, a leading 3D vision company with the highest shipment volume, uses its 3D structured light facial recognition payment products. These products project tens of thousands of light points onto the user's face using a dot matrix projector. A 3D camera locates the corresponding light points and uses triangulation to calculate the depth information, achieving 3D facial recognition and completing the payment. Compared to 2D facial recognition, 3D vision can easily detect deception from flat photos or videos, revolutionizing the security and convenience of payments. Furthermore, with the help of machine vision technology, facial recognition payment offers a better user experience and significantly reduces security risks.
In the field of industrial robots, there are three main types of sensors for acquiring 3D information about external objects: traditional RGB binocular cameras, structured light cameras, and Time-of-Flight (TOF) cameras. 3D vision enables rapid and accurate measurement of cardboard box dimensions, facilitating the depalletizing and palletizing of randomly sized boxes in the logistics industry. On production lines, various types of workpieces are often haphazardly stacked in crates, exhibiting significant differences in shape and appearance. 3D vision can accurately acquire the 3D contour features of objects within the crates, allowing robots to quickly and accurately sort multiple types of workpieces. In the light industry, acquiring 3D contour information through 3D vision enables robots to perform tasks such as gluing and spraying at high speed and precision.
Comparison of three mainstream 3D information acquisition devices and three technical solutions:
Complementing the stereoscopic imaging capabilities of 3D vision, the hybrid six-axis system comprises a 3P-3R structure, consisting of a parallel mechanism with three degrees of freedom and a serial mechanism with three degrees of freedom. This enables operation across a larger space with six degrees of freedom. While maintaining the characteristics of the original parallel mechanism, it adds features such as random object pickup posture, flexible end-effector placement, and a combination of material handling and sorting processes. After capturing the visual information of three-dimensional materials using a 3D camera, the robot judges the material's position and angle in three-dimensional space, overcoming the limitation of previous robots that could only perform planar grasping. This allows for rapid material handling of stacked materials and also expands the application scenarios for processes such as gluing and injection molding on irregular and uneven materials.
Figure 1: Birkent's independently developed serial-parallel hybrid six-axis robot
Similar to traditional 2D vision, the camera is first calibrated using an Eyetohand mounting method. After calibrating the camera's intrinsic parameters, the extrinsic parameters are calibrated using a checkerboard pattern. Extrinsic parameter calibration determines the camera's translation and rotation relationships with the external environment. Using the classic AX=XB homogeneous coordinate solution method, the robot's end effector moves to multiple positions on the checkerboard, recording its coordinate values, and then the transformation matrix is calculated to complete the camera calibration.
Figure 2: Bokent's independently developed robot controller and structured light 3D camera
Then, the point cloud is preprocessed to remove noise caused by external interference. Planar segmentation and target extraction are then performed on the point cloud to segment the target objects in the scene and extract their features. Furthermore, a specific algorithm is used to estimate and optimize the pose of the target objects, ultimately enabling a serial-parallel hybrid six-axis robot to quickly and accurately sort objects of different shapes under 3D vision.
Figure 3: Raw images captured by the camera
Figure 4: Depth image captured by the camera
Introduction: The Bokent Robotics Research Institute team was established in March 2018. Its members have an average age of 26, with over half holding master's degrees, earning them the reputation of being the "highest IQ" team internally. They have successfully led and participated in the algorithm development for several projects, including the "Intelligent Sorting and Palletizing Project for Irregularly Shaped Cigarettes," the "Parallel 6-Axis Stewart Platform," the "Intelligent Information Management System for Zhenjiang Factory," "Overall Sorting," and the "BeMotion Controller." With the successful implementation of this technology project, it is foreseeable that Bokent will leverage its "eyes" in conjunction with serial-parallel hybrid six-axis robots to continuously explore customers' practical application areas, providing more high-precision solutions and developing a "keen eye" that better meets customer needs.
