Market research indicates that the Chinese industrial robot market was valued at $1.3 billion in 2015 and is projected to maintain a compound annual growth rate (CAGR) of 20%, reaching $3.3 billion by 2020. In 2015, China accounted for 13% of global industrial robot sales revenue, a figure expected to reach 25% by 2020. Midea's significant investment in acquiring KUKA likely reflects its confidence in the strong growth momentum of the industrial robot market.
Industrial robots are a type of intelligent robot. Intelligent robots are developing rapidly. Let's take a look at the three key technologies used in intelligent robots.
I. Multi-sensor information fusion
Multi-sensor information fusion technology has become a very popular research topic in recent years. It combines with control theory, signal processing, artificial intelligence, probability and statistics to provide a technical solution for robots to perform tasks in various complex, dynamic, uncertain and unknown environments.
The key issues in data fusion are model design and fusion algorithms. Data fusion models mainly include functional models, structural models, and mathematical models. The functional model, starting from the fusion process, describes the main functions and databases involved in data fusion, as well as the interaction processes between the various components of the system during data fusion. The structural model, starting from the composition of data fusion, describes the software and hardware components of the data fusion system, related data flows, and the human-computer interface between the system and the external environment. The mathematical model is the algorithm and comprehensive logic of data fusion. Algorithms mainly include distributed detection, spatial fusion, attribute fusion, situation assessment, and threat estimation algorithms, which will be introduced from three aspects below.
1. Functional Model of Information Fusion
Currently, many scholars have proposed general functional models of information fusion systems from different perspectives. The most authoritative one is the functional model proposed by DFS (the Data Fusion Expert Group of the C3I Technical Committee (TPC3) under the Joint Council of the Tri-Service Government-Laboratory (JDL)).
The model divides data fusion into three levels. Level 1 is single-source or multi-source processing, mainly involving data processing, tracking correlations, and associations; Level 2 evaluates the set of estimated targets and their relationships with each other and the background to assess the overall situation; Level 3 uses a system of prior targets to test the assessment.
2. Structural Model of Information Fusion
Data fusion structures can be classified in various ways. One classification criterion is based on the degree of processing sensor data before it is sent to the fusion processing center. Under this criterion, fusion structures are divided into sensor-level data fusion, central-level data fusion, and hybrid fusion. Fusion structures can also be classified according to the resolution of the data processing process. In this case, fusion structures are pixel-level, feature-level, and decision-level fusion.
3. Mathematical Model for Multi-Sensor Information Fusion
Information fusion methods involve many theories and technologies, such as signal processing, estimation theory, uncertainty theory, pattern recognition, optimization techniques, fuzzy mathematics, and neural networks. A great deal of research has been done in these areas abroad.
Currently, these methods can be broadly categorized into two types: randomized methods and artificial intelligence methods.
II. Navigation and Positioning
Autonomous navigation is a core technology in robotic systems and a key and challenging problem in robotics research. There are four commonly used navigation and localization methods for autonomous mobile robots.
1. Visual navigation and positioning
In visual navigation and positioning systems, the most widely used method both domestically and internationally is the navigation approach based on local vision, which involves installing onboard cameras within the robot. In this method, control equipment and sensing devices are mounted on the robot's body, while high-level decisions such as image recognition and path planning are handled by the onboard control computer. A visual navigation and positioning system mainly includes: a camera (or CCD image sensor), video signal digitization equipment, a DSP-based high-speed signal processor, a computer, and its peripherals. Many robot systems now employ CCD image sensors, whose basic element is a row of silicon imaging elements. Photosensitive elements and charge transfer devices are configured on a substrate, and through the sequential transfer of charge, the video signals of multiple pixels are extracted in a time-division and sequential manner. For example, the resolution of images acquired by an area array CCD sensor can range from 32×32 to 1024×1024 pixels. In simple terms, the working principle of a visual navigation and positioning system is to perform optical processing on the robot's surrounding environment. First, a camera is used to collect image information, which is then compressed and fed back to a learning subsystem composed of neural networks and statistical methods. The learning subsystem then links the collected image information with the robot's actual position to complete the robot's autonomous navigation and positioning function.
2. Light reflection navigation and positioning
Typical optical reflection navigation and positioning methods primarily utilize laser or infrared sensors for distance measurement. Both laser and infrared sensors employ optical reflection technology for navigation and positioning.
A laser global positioning system typically consists of a laser rotation mechanism, a reflector, a photoelectric receiver, and a data acquisition and transmission device. During operation, the laser beam is emitted outward through the rotating mirror mechanism. When it scans a cooperative landmark formed by a back reflector, the reflected light is processed by the photoelectric receiver as a detection signal. This triggers a data acquisition program to read the code disk data (the measured angle value of the target) from the rotation mechanism. The data is then transmitted to a host computer for processing. Based on the known landmark position and the detected information, the sensor's current position and orientation in the landmark coordinate system can be calculated, thus achieving further navigation and positioning.
The figure shows a block diagram of an LDSR laser sensor system. Laser ranging has advantages such as narrow beam, good parallelism, low scattering, and high ranging directional resolution. However, it is also greatly affected by environmental factors. Therefore, noise reduction of the acquired signal is a significant challenge when using laser ranging. In addition, laser ranging also has blind zones, making it difficult to achieve navigation and positioning solely based on lasers. In industrial applications, it is generally used for on-site inspection within a specific range, such as detecting cracks in pipelines.
Infrared sensing technology is frequently used in obstacle avoidance systems for multi-joint robots, forming a large-area "sensitive skin" covering the robot arm to detect various objects encountered during its operation. A typical infrared sensor's working principle is shown in the figure. The sensor includes a solid-state light-emitting diode (LED) that emits infrared light and a solid-state photodiode that acts as a receiver. The LED emits a modulated signal, and the photodiode receives the modulated infrared signal reflected from the target object. Ambient infrared interference is eliminated through signal modulation and a dedicated infrared filter. Let the output signal Vo represent the voltage output of the reflected light intensity; then Vo is a function of the distance between the probe and the workpiece.
Vo = f(x, p)
In the formula, p is the workpiece reflection coefficient, which is related to the surface color and roughness of the target object. x is the distance between the probe and the workpiece.
When the workpieces are all similar target objects with the same p value, x and Vo correspond one-to-one. x can be obtained by interpolating the proximity measurement experimental data of various target objects. In this way, the position of the robot from the target object can be measured by the infrared sensor, and then the mobile robot can be navigated and positioned by other information processing methods.
Although infrared sensing positioning also has advantages such as high sensitivity, simple structure and low cost, it is often used as a proximity sensor in mobile robots because of its high angular resolution but low distance resolution. It can detect nearby or sudden moving obstacles, making it easier for the robot to stop in an emergency.
3. GPS Global Positioning System
Currently, pseudorange differential dynamic positioning is commonly used in the navigation and positioning technology of intelligent robots. This method involves using a base receiver and a dynamic receiver to jointly observe four GPS satellites, and then calculating the robot's three-dimensional position coordinates at a specific moment using a specific algorithm. Differential dynamic positioning eliminates satellite clock errors. For users 1000km away from the base station, it can eliminate satellite clock errors and errors caused by the troposphere, thus significantly improving dynamic positioning accuracy. However, in mobile navigation, the positioning accuracy of a mobile GPS receiver is affected by satellite signal conditions and road environment, as well as clock errors, propagation errors, receiver noise, and many other factors. Therefore, relying solely on GPS navigation suffers from relatively low positioning accuracy and reliability. Thus, in robot navigation applications, magnetic compasses, optical encoders, and GPS data are usually supplemented for navigation. Furthermore, GPS navigation systems are not suitable for indoor or underwater robot navigation, or for robot systems requiring high positional accuracy.
4. Ultrasonic navigation and positioning
The working principle of ultrasonic navigation and positioning is similar to that of laser and infrared. Typically, an ultrasonic sensor emits ultrasonic waves from its transmitting probe. These waves encounter obstacles in the medium and return to the receiving device. By receiving the reflected signals of the emitted ultrasonic waves, and based on the time difference between emission and reception, as well as the propagation speed, the propagation distance S can be calculated. This gives the distance from the obstacle to the robot, expressed by the formula: S = Tv/2, where T is the time difference between ultrasonic emission and reception, and v is the wave speed of the ultrasonic wave in the medium.
Of course, many mobile robot navigation and positioning systems use separate transmitting and receiving devices, with multiple receiving devices placed in the environmental map and transmitting probes installed on the mobile robot.
In the navigation and positioning of mobile robots, due to the inherent defects of ultrasonic sensors, such as specular reflection and limited beam angle, it is difficult to obtain sufficient information about the surrounding environment. Therefore, an ultrasonic sensing system composed of multiple sensors is usually used to establish a corresponding environmental model. The information collected by the sensors is transmitted to the control system of the mobile robot through serial communication. The control system then uses certain algorithms to process the data based on the collected signals and the established mathematical model to obtain the robot's position and environmental information.
Due to their advantages such as low cost, fast data acquisition rate, and high distance resolution, ultrasonic sensors have long been widely used in the navigation and localization of mobile robots. Furthermore, they do not require complex image processing techniques when acquiring environmental information, resulting in fast ranging speed and good real-time performance. At the same time, ultrasonic sensors are not easily affected by external environmental conditions such as weather conditions, ambient lighting, obstacle shadows, and surface roughness. Ultrasonic navigation and localization has been widely applied in the perception systems of various mobile robots.
III. Path Planning
Path planning is an important branch of robotics research. Optimal path planning is the process of finding an optimal path from the initial state to the target state in the robot's workspace that avoids obstacles, based on one or more optimization criteria (such as minimizing work cost, shortest travel route, and shortest travel time).
Mobile robot path planning technologies can be broadly categorized into four types: template matching path planning technology, artificial potential field path planning technology, map building path planning technology, and artificial intelligence path planning technology.
1. Template matching path planning technology
Template matching compares the robot's current state with its past experiences to find the closest state. Modifying the path in this closest state yields a new path. First, a template library is built using information used in path planning or already generated. Each template in the library contains environmental and path information from each planning iteration, and these templates can be retrieved through specific indexes. Then, the current planning task and environmental information are matched against the templates in the library to find the optimal matching template. This template is then refined, and this refined template is used as the final result. Template matching technology has good application performance under given environmental conditions. Examples include the case-based autonomous underwater vehicle (AUV) path planning method proposed by Vasudevan et al., and the template matching path planning method for cleaning robots proposed by Liu et al. To improve the adaptability of template matching path planning technology to environmental changes, some scholars have proposed methods that combine template matching with neural network learning. For example, Ram et al. combined case-based online matching with reinforcement learning to improve the adaptive performance of the robot in the template matching planning method, enabling the robot to partially adapt to environmental changes. Arleo et al. also proposed a path planning method that combines environmental templates with neural network learning.
2. Artificial Potential Field Path Planning Technology
The basic idea of artificial potential field path planning technology is to treat the robot's movement in the environment as a kind of movement in a virtual artificial force field. Obstacles exert a repulsive force on the robot, while the target point exerts an attractive force on the robot. The resultant force of the attractive and repulsive forces serves as the robot's control force, thereby controlling the robot to avoid obstacles and reach the target position.
Early research on artificial potential field path planning focused on a static environment, treating obstacles and targets as static and invariant. The robot planned its path solely based on the positions of these objects in the static environment, without considering their speeds. However, real-world environments are often dynamic, with obstacles and targets potentially moving. To address path planning in dynamic environments, Fujimura et al. proposed a relatively dynamic artificial potential field method, treating time as a one-dimensional parameter in the planning model. Moving obstacles are still considered static in the extended model, allowing dynamic path planning to be implemented using static path planning methods. The main problem with this method is the assumption that the robot's trajectory is always known, which is difficult to achieve in the real world. To address this, Ko et al. incorporated the obstacle's velocity parameter into the construction of the repulsive potential function, proposing a path planning strategy for dynamic environments and providing simulation results.
However, two assumptions make this method different from the actual dynamic environment: (1) it only considers the speed of obstacles in the environment and does not consider the speed of the robot; (2) it assumes that the relative speed between the obstacle and the robot is constant, which is not a complete dynamic environment. For dynamic path planning problems, the robot obstacle avoidance is mainly related to the relative position and relative speed between the robot and the obstacle, rather than the absolute position and speed. In this regard, Ge et al. introduced the relative position and relative speed between the robot and the target into the attraction potential function and the relative position and relative speed between the robot and the obstacle into the repulsion potential function, and proposed a robot path planning algorithm in dynamic environment. They applied the algorithm to the path planning of an omnidirectional soccer mobile robot and achieved satisfactory simulation and experimental results.
3. Map building and path planning technology
Map-building and path planning techniques involve dividing the robot's surrounding area into different grid spaces (such as free space and confined space) based on obstacle information detected by the robot's sensors. The obstacle occupancy within each grid space is calculated, and the optimal path is determined according to certain rules. Map building is further divided into landmark methods and grid methods, also known as cell decomposition methods. The former constructs a feasible path map for the robot consisting of marker points and connecting edges, such as the line-of-sight method, tangent graph method, Voronoi diagram method, and probabilistic graph expansion method.
The visual graph method treats the robot as a point, connecting the robot, the target point, and the vertices of polygonal obstacles, ensuring that none of these lines intersect the obstacles, thus forming a graph called the visual graph. Since the vertices of any two lines are visible, all paths from the starting point to the target point along these lines are collision-free paths for the moving object. Path planning is the problem of searching for the shortest distance from the starting point to the target point through these visible lines. The tangent graph method and the Voronoi diagram method modify the visual graph method. The tangent graph method is based on a polygonal obstacle model, replacing obstacles of arbitrary shapes with approximate polygons, constructing a tangent graph in free space. Therefore, from the starting point to the target point, the robot travels along the tangents, meaning the robot must... While the Voronoi diagram method allows for near-obstacle movement and shorter paths, the likelihood of collisions is high if positional errors occur during control. A Voronoi diagram consists of a series of straight and parabolic segments. Straight lines are defined by the vertices or edges of two obstacles, and all points on a straight line segment must be equidistant from the vertices or edges of the obstacles. Parabolic segments are defined by the vertex and edge of one obstacle, and similarly, they must be equidistant from both the vertices and edges of the obstacles. Compared to the tangent method, the path from the starting node to the target node is longer using the Voronoi diagram method. However, even with positional errors, the mobile robot will not collide with obstacles, resulting in higher safety.
The grid method decomposes the space surrounding the robot into interconnected and non-overlapping spatial units; these grids form a connected graph. Based on obstacle occupancy, a collision-free optimal path is searched on this graph from the starting grid to the target grid. Depending on the grid processing method, it is further divided into the precise grid method and the approximate grid method, the latter also known as the probabilistic grid method. The precise grid method decomposes the free space into multiple non-overlapping units, and the combination of these units is exactly equal to the original free space. The figure below shows a commonly used precise grid decomposition method—trapezoidal grid decomposition.
Unlike the exact grid method, all grids in the approximate grid method are of a predetermined shape, usually rectangles. The entire environment is divided into multiple larger rectangles, each of which is continuous. A typical method is the "quadtree" method. If a large rectangle contains obstacles or boundaries, it is divided into four smaller rectangles. This division is performed on all slightly larger grids, and the procedure is repeated between the smaller grids formed within the final boundary of the division until the solution boundary is reached.
Map-building methods are intuitive and straightforward, and they are often integrated with other path planning methods, such as the map-building path planning algorithm based on ART neural networks proposed by Araujo, the map-building path planning algorithm based on Kalman filters proposed by Najjaran, and the complete coverage path planning technology (CCPP) for cleaning robots based on the integration of bio-inspired neural networks and map building proposed by Yang et al.
Currently, map building technology has attracted widespread attention in the field of robotics research and has become one of the research hotspots for mobile robot path planning. However, the limited information resources of robot sensors make it difficult to calculate and process obstacle information in grid maps. At the same time, since robots need to update map data dynamically and quickly, it is difficult to guarantee the real-time performance of path planning when there are many grids and high resolution. Therefore, map building methods must seek a balance between map grid resolution and real-time performance of path planning.
4. Artificial intelligence path planning technology
Artificial intelligence path planning technology applies modern artificial intelligence techniques to path planning in mobile robots, such as artificial neural networks, evolutionary computation, fuzzy logic, and information fusion. Genetic algorithms were among the earliest intelligent optimization algorithms applied to combinatorial optimization problems. These algorithms and their derivatives have been applied in the field of robot path planning research. Building upon the success of ant colony optimization in solving the Traveling Salesman Problem (TSP), many scholars have further introduced ant colony optimization into path planning research for underwater robots (UVs).
Neural networks, as an important component of artificial intelligence, have received widespread attention in mobile robot path planning research. For example, Ghatee et al. applied Hopfield neural networks to optimize path distance; Zhu et al. applied self-organizing SOM neural networks to task allocation and path planning for multi-task, multi-robot applications. In recent years, Canadian scholar Simon proposed a novel bio-inspired dynamic neural network model that maps neural network neurons to discrete coordinates in a two-dimensional planning space. By defining the different excitation and inhibition effects of obstacles and non-obstacles on neuron inputs, the model directly calculates the outputs of relevant neurons, thereby determining the robot's running direction. Since this neural network does not require a learning and training process, path planning exhibits good real-time performance. Furthermore, by utilizing the rapid decay characteristics of the neural network itself, it effectively solves the dead zone problem in robot path planning. The figure shows a bio-inspired neural network structure for local path planning. The figure shows the sensor radius of the robot (at the neuron). Each neuron corresponds to the environmental position coordinates. The outputs of neighboring neurons are dynamically calculated, and the robot determines its next running target based on the neuron output magnitude, thus achieving safe path planning.
Artificial intelligence (AI) technology, when applied to path planning in mobile robots, enhances the robots' "intelligence" and overcomes many shortcomings of traditional planning methods. However, this approach also has limitations. Genetic optimization and ant colony optimization path planning techniques primarily address specific problems in path planning, utilizing evolutionary computation for optimization and often combined with other path planning methods. They are rarely used alone to complete path planning tasks. Information fusion technology is mainly applied to robot sensor signal processing rather than direct path planning strategies. For neural network path planning, most neural networks involve a learning process of planning knowledge, which not only faces difficulties in obtaining learning samples but also suffers from learning lag, thus affecting the real-time performance of neural network path planning. While bio-inspired neural network path planning offers better real-time performance, the setting of its input stimuli and inhibitions is subject to human uncertainty.
In addition, intelligent robots also utilize various technologies such as robot vision, intelligent control, and human-machine interface technology.
