Motion control of complex robots has long been a major obstacle to the development of the robotics industry, and a satisfactory solution has yet to be found. Even Boston Dynamics, representing the pinnacle of robotics, has robots that are still far from practical application. AI, which has seen rapid development in the last two years, seems to be a panacea, being used in various fields, including robot control, and it appears to be achieving good results. Recently, Pieter Abbeel, a reinforcement learning expert at UC Berkeley, founded Embodied Intelligence, whose business directly covers the three hot topics of VR, AI, and robotics.
To understand how new technologies such as VR and AI are applied in the field of robot control, this article provides a brief overview of the applications of VR and AI in robot control based on some relevant papers and public information, including Pieter Abbeel's presentation. It finds that AI and VR do have practical applications in robot control, but there is still a long way to go before they achieve substantial breakthroughs.
Types of robot control
Many robotics research aims to simulate human intelligence, so studying human control systems offers significant insights for robotics. The human nervous system, composed of the brain, cerebellum, brainstem, spinal cord, and neurons, is complex and sophisticated. It includes the central nervous system and the peripheral nervous system. The central nervous system, formed by the brain and spinal cord, is the most important part of the human nervous system. The peripheral nervous system originates from the brain and spinal cord and distributes to all parts of the body. Countless neurons exist throughout the nervous system, forming neural networks.
The central neural network is responsible for motion control and is mainly divided into three layers:
The brain: located at the highest level, responsible for the overall planning of the movement and the issuance of various tasks.
Cerebellum: Located in the middle layer, it is responsible for the coordination, organization, and implementation of movement. Human balance is controlled by the cerebellum.
Brainstem and spinal cord: These are the lowest layers, responsible for executing movement, specifically controlling the movement of muscles and bones, and are accomplished by the brainstem and spinal cord.
The three layers play different roles in regulating movement, from highest to lowest. The lower layers receive and implement downward control commands from the higher layers. The brain can control the spinal motor nerves directly or indirectly through the brainstem.
If we draw an analogy between robots and humans, the robotic arm controller is similar to the human spinal cord, responsible for controlling the specific movements of motors (muscles) and mechanical structures (skeleton). The motion controller of a multi-legged robot is similar to the human cerebellum, responsible for controlling balance and coordination. The robot's operating system layer is similar to the human brain, perceiving and understanding the world and issuing various complex motion goals.
Based on the above analogy and referring to the current situation of various robots, the motion control of robots can be roughly divided into four tasks:
Spinal cord control—the fundamental control system for robotic arm movements. This is the main type of problem faced by industrial robots, various robotic arms, and drones in their underlying motion control.
Cerebellar control—balance and motor coordination control for multi-legged robots. This is currently a difficult area in robot control that has not yet been overcome, and Boston Dynamics is clearly the best at it.
Brainstem control – environmental perception. This mainly refers to the navigation and path planning of robots such as robotic vacuum cleaners and drones, whose underlying motion control is already encapsulated. It requires environmental perception to locate, navigate, and plan movement for itself and its targets.
Spinal cord control—environmental cognition and interaction—is the robot's specific execution of interactive tasks, such as controlling a robotic arm to grasp objects and perform operations. This is a crucial issue that service robots need to overcome.
Several specific AI applications in control
01 Spinal cord control
Two typical applications of spinal cord control are path planning for robotic arms and flight control for drones. These problems belong to traditional automatic control theory, based on mathematical and dynamic modeling. They have been developed over many years and have a very complete theoretical and practical foundation, achieving excellent results. Although deep learning is currently very popular and theoretically could be used for this type of control, it is not currently applied in this fundamental control field. The main reasons may be:
1) High-precision repetitive actions of industrial robots can be mathematically solved based on automatic control theory, and since the principles are understood, they belong to white-box systems. Given the availability of reliable white-box solutions, there is no need to switch to a black-box neural network control system.
2) In applications such as industrial robots, the stability of control algorithms is critical. However, as a black-box solution, the stability of neural network control systems cannot yet be proven through data. Once a problem occurs in a neural network controller, it is difficult to explain and improve it.
3) Neural network algorithms are trained on a large amount of data, while in existing motion control, such as flight control, the cost of obtaining actual experimental data is high and it is very difficult to acquire a large amount of data.
02 Cerebellar control category
A typical problem in cerebellar control is the balance and motion coordination control of humanoid bipedal and multipedal robots. Research in this area has been based on traditional control theory; however, due to the much higher degree of freedom of movement compared to robotic arms or drones, the challenges are significant. Bipedal humanoid robots are often perceived as slow, stiff, and unstable. Boston Dynamics' Atlas and BigDog are among the most advanced in this field. While Boston Dynamics has not disclosed the technology they use, Google engineer Eric Jang stated that, based on information from a presentation, BD's robot control strategy uses a model-based controller and does not involve neural network algorithms.
03 Environmental Perception Category
The main applications are path planning for service robots, target tracking for drones, and visual positioning for industrial robots. By sensing the environment, the system issues target motion commands to the packaged motion control system.
Target recognition
Target recognition in environmental perception processes, such as the identification and tracking of drone targets, can be more accurate with the help of neural networks, and this technology has already been applied to drones such as DJI.
Location, navigation and route planning
Currently, robot localization and navigation are mainly based on popular vSLAM or LiDAR SLAM technologies. Mainstream LiDAR solutions can be roughly divided into three steps, with some deep learning involved in the middle stages, but the majority of the process does not involve deep learning.
Step 1: SLAM, build a scene map, use LiDAR to build a 2D or 3D point cloud of the scene, or reconstruct a 3D scene.
The second step: Constructing a semantic map, which may involve object recognition and segmentation, and labeling objects in the scene. (This step may be skipped in some cases.)
Step 3: Path planning is performed based on the algorithm, and the robot's movement is driven.
04 Environmental Interaction
Typical application scenarios include robotic arms grasping target objects. Interaction with the environment has always been a problem that traditional automatic control has struggled to solve. In recent years, AI-related technologies based on reinforcement learning have been applied to these problems, achieving some research progress. However, whether it will be the mainstream direction in the future remains highly controversial.
reinforcement learning
In a reinforcement learning framework, an agent containing a neural network is responsible for decision-making. The agent takes the environment data collected by the robot's sensors as input and outputs an action command to control the robot. After the robot acts, it observes the new environmental state and the reward resulting from the action, and then decides on the next action. The reward is set according to the control objective and can be positive or negative. For example, if the goal is autonomous driving, a positive reward is reaching the destination, a negative reward is failing to reach the destination, and an even worse reward is causing a car accident. This process is repeated, with the goal of maximizing the reward.
The control process of reinforcement learning is inherently a positive feedback control process, which forms the basis for the application of AI in robot control. Based on this, reinforcement learning has yielded some research results in robot control.
Finding targets in the environment
In 2016, Fei-Fei Li's group released a paper that, based on deep reinforcement learning, could find objects without building a graph, given a target image as input. The general idea was: based on the image the machine sees, it decides how to proceed, then looks at the image again to decide the next step, and so on, until the object is found. The paper showed that the neural network trained using the target image as input had general applicability.
This method of finding objects is closer to human thinking. The trained controller doesn't remember the location of objects, nor does it know the structure of the building. But it does remember how to get to each object from each location.
robot grasping
Traditional robotics research requires a very clear understanding of the three-dimensional geometry of the object to be grasped, analysis of the location and magnitude of forces, and then reverse calculation of how the robotic hand moves step by step to these positions. However, this approach is very difficult for grasping irregularly shaped and flexible objects. For example, a towel might need to be treated as a series of rigid bodies linked together, requiring dynamic modeling and analysis, but this involves a large amount of computation. And for rubber objects like a rubber duck, the degree of elasticity is not apparent from the outside, making it difficult to calculate the correct force to be applied.
Research on robot control by Pieter Abbeel, DeepMind, and OpenAI is all based on deep reinforcement learning. Reinforcement learning-based robot grasping uses images seen from the machine's perspective as input and the final object grasped as the goal. The machine is continuously trained to grasp objects without modeling or force analysis. Pieter Abbeel has already demonstrated robots performing complex actions such as folding towels, opening bottle caps, and assembling toys.
