Robot vision systems have evolved through three generations. The first generation typically processed images according to a prescribed procedure and output results. These systems were generally built with ordinary digital circuits and were primarily used for defect detection in flat materials. The second generation typically consisted of a computer, an image input device, and output hardware. Visual information flowed serially within the system, possessing a certain learning ability to adapt to various new situations. The third generation of robot vision systems is currently under development and in use internationally. It employs high-speed image processing chips and parallel algorithms, exhibiting high intelligence and adaptability, and can simulate advanced human visual functions.
The main problems currently existing in robot vision
1. How to accurately and quickly (in real time) identify targets.
2. How to effectively construct and organize reliable recognition algorithms and implement them smoothly? This requires breakthroughs in high-speed array processing units and algorithms (such as neural network methods, wavelet transforms, etc.), so that the function can be implemented in a highly parallel manner with minimal computation.
3. Real-time performance is a significant and difficult problem to solve. The slow image acquisition speed and the long image processing time introduce significant time lag into the system. Furthermore, the introduction of visual information significantly increases the computational load, such as calculating the image Jacobian matrix and estimating depth information. Image processing speed is one of the main bottlenecks affecting the real-time performance of vision systems.
4. Stability is the primary concern for all control systems. For vision control systems, whether based on position, image, or a hybrid vision servoing method, the following problems are faced: how to ensure system stability when the initial point is far from the target point, i.e., increase the stable region and ensure global convergence; and how to ensure that feature points are always within the field of view to avoid servoing failure.
Issues that should be further studied in robot vision
1. The problem of image feature selection.
The performance of visual servoing is highly dependent on the image features used. Feature selection must consider not only recognition metrics but also control metrics. From a control perspective, using redundant features can suppress the influence of noise and improve visual servoing performance, but it also increases the difficulty of image processing. Therefore, how to select the optimal features, how to process features, and how to evaluate features are all issues that require further research. For tasks that may require switching from one set of features to another, combining global and local features can be considered.
2. Based on the research results of computer vision and image processing, establish a dedicated software library for robot vision systems .
3. Strengthen research on the dynamic performance of the system. Current research focuses on determining the desired robot motion based on image information, while research on the dynamic performance of the entire vision servoing system is lacking.
4. Utilize the achievements of intelligent technology.
5. Utilize the results of active vision.
Active vision is a hot topic in the fields of computer vision and robot vision research today. It emphasizes the ability of a vision system to interact with its environment. Unlike traditional general vision, active vision emphasizes two points: first, it believes that the vision system should have the ability to actively perceive; second, it believes that the vision system should be task-directed or purpose-oriented. Active vision believes that in the process of acquiring visual information, camera parameters, such as orientation, focal length, and aperture, should be adjusted more proactively, and the camera should be able to quickly focus on the object of interest.
More generally, it emphasizes the gaze mechanism, stressing the selective perception of signals distributed across different spatial ranges and time periods using different resolutions. This active perception can be achieved at the hardware level through adjustments to the camera's physical parameters, or, based on a passive camera, at the algorithm and representation levels through selective processing of acquired data. Furthermore, active vision considers visual processes without any purpose meaningless; the visual system must be associated with a specific purpose (such as navigation, recognition, or manipulation) to form a perception/action loop.
6. Multi-sensor fusion issues. Visual sensors have limited applications. However, by effectively combining them with other sensors and leveraging their complementary performance advantages, uncertainties can be eliminated, resulting in more reliable and accurate results.
