Robot perception systems transform various internal state information and environmental information of robots from signals into data and information that can be understood and applied by the robot itself or between robots. In addition to sensing mechanical quantities related to its own working state, such as displacement, velocity, acceleration, force and torque, visual perception technology is an important aspect of industrial robot perception.
Visual servoing systems use visual information as feedback signals to control and adjust the robot's position and posture. Their applications are primarily seen in the semiconductor and electronics industries. Machine vision systems are also widely used in quality inspection, workpiece identification, food sorting, and packaging.
Typically, robot vision servo control is based on position-based vision servoing or image-based vision servoing, also known as 3D vision servoing and 2D vision servoing, respectively. Each of these methods has its own advantages and applicability, but also some drawbacks.
Position-based visual servoing systems utilize camera parameters to establish a mapping between image information and the position/pose information of the robot's end effector, achieving closed-loop control of the end effector's position. The end effector's position and pose errors are estimated by extracting the end effector's position information from real-time captured images and using the geometric model of the target. Then, based on these position and pose errors, new pose parameters for each joint are obtained. Position-based visual servoing requires that the end effector be always observable in the visual scene, and its three-dimensional position and pose information calculated. Eliminating interference and noise in the image is crucial to ensuring accurate position and pose error calculations.
Two-dimensional visual servoing compares the features of images captured by a camera with a given image (not 3D geometric information) to derive an error signal. This signal is then corrected by the joint controller, vision controller, and the robot's current operating state, enabling the robot to perform servo control. Compared to 3D visual servoing, 2D visual servoing is more robust to camera and robot calibration errors. However, in the design of the visual servo controller, issues such as the singularity of the image's Jacobian matrix and local minima are inevitably encountered.
To address the limitations of 3D and 2D visual servoing methods, a 2.5D visual servoing method has been proposed. This method decouples the closed-loop control of camera translational displacement from rotational displacement. Based on image feature points, it reconstructs the object's orientation and imaging depth ratio in 3D space, with the translational portion represented by the coordinates of feature points on the image plane. This method successfully combines image signals and image-derived pose signals, and integrates the resulting error signals for feedback, largely solving problems related to robustness, singularity, and local minima. However, some issues still need to be addressed, such as ensuring the reference object remains within the camera's field of view during servoing, and the non-uniqueness of solutions when decomposing the homography matrix.
When building a vision controller model, it is necessary to find a suitable model to describe the mapping relationship between the robot's end effector and the camera. The image Jacobian matrix method is a widely used approach in the field of robot visual servoing research. Since the image Jacobian matrix is time-varying, it needs to be calculated or estimated online.
