Share this

Neural Network-Based Robot Vision Servo Control

2026-04-06 08:20:01 · · #1
Abstract: Visual servoing can be applied to control systems for robot initial localization, automatic obstacle avoidance, trajectory tracking, and moving target tracking. Traditional visual servoing systems involve two processes during operation: workspace localization and dynamic inverse operation. This requires real-time calculation of the visual Jacobian matrix and the robot inverse Jacobian matrix, resulting in high computational complexity and a complex system structure. This paper analyzes the basic principles of image-based robot visual servoing, using a BP neural network to determine the joint angles required to achieve a specified pose. Visual information is directly integrated into the servoing process, greatly simplifying the control algorithm while maintaining servoing accuracy. Simulation experiments were conducted on a Puma560 industrial robot model, and the results validate the effectiveness of the proposed method. Keywords: visual servo; image Jacobian matrix; inverse Jacobian matrix; BP neural network; visual controller Abstract: Visual servo system can be used in the control systems of robot original orientation guiding, obstacle avoidance, trajectory tracking and moving object tracking, etc. During working, the traditional visual servo system consists of two processes: determination of the workpiece position and inverse kinetic calculation. So real-time computation of visual Jacobian and inverse Jacobian of the robot have been needed. Both the computation and the The structure of the system are complex. In this paper, the basic principle of robot visual servo system is analyzed. A BP neural network is proposed to determine the required joint angles for the set position and orientation. This method can integrate visual data directly into the servo process, so under the condition that the servo precision is ensured, the computation of the control arithmetic is greatly simplified. The simulation experiment for Puma560 robot is made and simulation results proved the effectiveness of the method. Keywords: Visual servo; Image jacobian matrix; Inverse jacobian matrix; BP neural nerwork; Visual controller 1. Introduction Currently, the research on visual servo control is a hot topic in the field of robot research[1]. Shirai and Inouei[2] first introduced visual control into robot operation in 1973. They used a fixed camera to improve the positioning accuracy of the robot by using vision, and realized the tasks of grasping and placing. In 1979, Hill and Park[3] proposed the concept of "visual servo look-and-move mode". Vision was introduced into the closed loop for the first time to overcome the uncertainty of the system model (including the robot, vision system, and environment) and improve the operation accuracy of the robot. Artificial neural network is a simulation and approximation of biological neural network. It is particularly suitable for systems with complex nonlinear relationships that cannot be represented by explicit formulas, and has strong adaptive and learning functions. In recent years, the application of intelligent vision controllers for robots has attracted more and more attention. Many researchers have discussed the feasibility of applying neural networks to robot control. The central issue discussed in this topic is the use of neural networks to learn the characteristics of robot vision systems. Wells used four-point features, Fourier descriptors and geometric moments as inputs to the neural network [4] to conduct localization experiments on a six-degree-of-freedom robot. Its feature is that it uses global image features, which can expand the application range. However, the localization accuracy using global features is low. Sun used two neural networks [5]: one Kohonen network for global control, with visual signals from two cameras fixed in the workspace; the other used a BP network for local control, with visual signals from two cameras installed on the end effector. Stanley used a neural network for feature extraction and inverse Jacobi [6]. Hashimoto used a BP network to learn the relationship between image feature changes and robot joint angle changes [7]. The input of the network is the change in image features, and the output is the desired change in joint angle. However, he only did the design work of the neural network and did not give the servo effect of the designed neural network in the visual servo system. This paper proposes an image-based visual servo control scheme that directly uses visual information for the servo process. To achieve the commanded positioning, an artificial neural network is used to determine the required joint angle changes for robot end effector localization. This method eliminates the need to calculate the visual Jacobian matrix, the robot's inverse Jacobian matrix, and perform hand-eye calibration, simplifying the control algorithm's computation. For details, please click: Neural Network-Based Robot Vision Servo Control
Read next

CATDOLL 136CM Ya

Height: 136cm Weight: 23.3kg Shoulder Width: 31cm Bust/Waist/Hip: 60/54/68cm Oral Depth: 3-5cm Vaginal Depth: 3-15cm An...

Articles 2026-02-22