As application scenarios deepen and expand, higher demands are being placed on robots. Equipping robots with sensors and other technologies allows them to become more intelligent and adapt to more complex scenarios. Simultaneously, the development of artificial intelligence (AI) provides strong support for the development of industrial robots. While traditional AI deployed on-site can execute tasks efficiently, the complexity and variability of industrial environments can lead to the generation of abnormal data that hasn't been trained, resulting in unrecognized outputs. The emergence of large-scale AI modeling technology enables industrial robots to handle complex industrial scenarios more flexibly, further improving detection accuracy and efficiency, assisting enterprises in their digital and intelligent transformation, and driving the transformation and upgrading of the manufacturing industry.
01
introduction
Industrial robots have become one of the most common and indispensable pieces of equipment in industrial production, bringing great convenience to industrial development. Due to the numerous industrial scenarios and diverse needs, to meet the requirements of actual business scenarios, the form of industrial robots on-site has gradually evolved from traditional robotic arms to customized AGVs, gantry robots, and AGVs equipped with robotic arms, among other forms. With the empowerment of new-generation information technology, industrial robots equipped with industrial cameras (area scan cameras, line scan cameras, 3D cameras, etc.), intelligent temperature sensors, and high-sensitivity microphones can solve more complex needs, thereby significantly improving the utilization rate of industrial robots and generating more practical value. The main application of artificial intelligence is to deploy trained inference/detection models, process real-time transmitted image and other data, and execute the next action based on the output results; this method has an absolute advantage in fixed scenarios. During the movement of an industrial robot, if other abnormal data is collected, these anomalies will not be recognized if the model has not been trained, posing a certain risk. The emergence of large-scale artificial intelligence models not only improves the accuracy of detection but also endows industrial robots with more capabilities, greatly promoting the development of the manufacturing industry towards intelligent manufacturing.
02
Planning of large-scale artificial intelligence models in industrial robotics
Enterprises that widely use industrial robots already have a solid digital foundation. They can then conduct overall design and planning for the application of large-scale artificial intelligence models in the field of industrial robots, based on their actual needs.
2.1 Industrial Robot Layer
Industrial robots deployed on-site currently come in various forms, such as multi-joint robotic arms, multi-degree-of-freedom gantry robots, and AGVs equipped with light-duty robotic arms, to meet diverse operational needs. Equipping industrial robots with high-definition cameras to achieve precise positioning, deviation correction, defect detection, and dimensional measurement is a mature and common solution. Alternatively, industrial robots can be equipped with thermal imagers, spectrometers, gas analyzers, and microphones to collect data from different scenarios, fully leveraging the capabilities of industrial robots. Industrial robots have a wide range of applications in my country, covering multiple scenarios in process and discrete manufacturing. Non-standard customized industrial robots have significant advantages in terms of domestic production, offering high levels of customization and flexibility, but also face certain challenges, such as: overall long-term stable operation; improving MTBF (Mean Time Between Failure); domestic replacement of core components such as controllers and high-precision motors; domestic production of industrial software for large-scale scheduling systems; and breakthroughs in the domestic production of high-end multi-joint robots.
2.2 Communication Interface Layer
Communication interfaces are a fundamental requirement in industrial settings, and their stability is paramount. The application design level at this layer varies significantly, necessitating unified planning for industrial and office networks, and the establishment of corresponding security measures to ensure network stability. A common and effective solution involves using industrial-grade switches (such as managed switches from Siemens and Phoenix Contact) to construct a fiber optic ring network for the industrial network, and commercial switches (such as those from H3C and Huawei) to construct an IT network for the office network, with firewalls protecting the two. Industrial robots, cameras, and other sensors employ numerous communication protocols; this layer must ensure the accuracy and timeliness of data transmission. Utilizing 5G for transmitting large data types such as images and videos, and other protocols for exchanging detection results and commands, can be considered. For process industries, using wireless communication technologies such as RTUs and 5G is a practical and reliable solution. The stability of communication interfaces must balance high concurrency, scalability, and compatibility. When data transmission involves cross-company and cross-platform communication, building an industrial internet platform and incorporating technologies such as blockchain and quantum communication can enhance communication security.
2.3 Model Layer
With the development of computer vision, artificial intelligence, and other technologies, a relatively complete theoretical framework has been provided for the model layer, and scholars have conducted extensive theoretical and applied research in this area. The key steps in applying the model layer are data collection, data processing, model training, and model deployment. Its core is to use the trained model to address the "unchanging" needs in the production process (because the production process is relatively stable and repetitive, the inspection needs for products, equipment, etc., are relatively fixed), such as various defect detection models, measurement models, and prediction models. Ultimately, the applied model reflects AI's ability to solve complex problems and generate value. Several commonly used models are briefly introduced below.
(1) CV models are used for image and video detection/recognition in the field, which are all computer vision recognition. Using CV models is a reliable and mature solution. In actual industrial applications, it is necessary to comprehensively consider the accuracy, cycle time, stability and cost-effectiveness of detection to ensure that AI can be successfully implemented and run stably through engineering projects in order to generate value. For example, if large-sized and complex industrial products need to be inspected for appearance defects, the ideal solution is to capture the product in the air and shoot and inspect it from all angles. However, there are two major risks in actual operation: the cycle time is slow, which will affect production; and the moving equipment has high wear and tear, high failure rate and high operating cost. Based on experience, the optimal engineering solution is to use static equipment design to evaluate defects in parts of the product that cannot be photographed. If the probability of the defect is low and the impact is small, it can be disregarded. Otherwise, improvements need to be made to the process, equipment and management to reduce the frequency of defects. For scenarios requiring only defect detection, traditional algorithms such as grayscale recognition, SVM (Support Vector Machine), SIFT (Scale-Invariant Feature Transform), HOG (Histogram of Oriented Gradient), ORB (Oriented Fast and Rotated BRIEF), and LBP (Local Binary Patterns) can be used to save computational resources and investment. For complex tasks requiring the labeling of defect locations, types, and severity levels, CNN neural network algorithms are the optimal solution, such as YOLO, VGG, ResNet, AlexNet, and RevNet. First, samples are collected, then the processed samples (labeled with defects) are trained using an algorithm model. Once the model reaches the preset detection accuracy, it is deployed. Currently, the training process using relevant algorithm models within a framework remains a black box (i.e., difficult to interpret), and the quality and quantity of samples significantly impact the final detection model. In engineering applications, increasing the number of real-world samples (from actual production, not artificially created; with high-quality annotation) can significantly improve the training results of the model. To ensure the algorithm outputs the inference model as expected, researchers have done a great deal of work. Constructing a loss function is a common and relatively easy-to-implement approach in engineering. For example, Berkan Demirel et al. proposed a new meta-tuning loss function, which significantly improved detection results. In engineering applications, designing a loss function tailored to the specific circumstances will yield better results.
(2) The robot prediction model collects and obtains the operation data of the industrial robot, such as action duration, load, current, voltage, running trajectory, battery power and charging and discharging status, etc. The prediction model is trained by Transformer or GNN to predict the maintenance status and faults of the industrial robot, realize the full life cycle management of the industrial robot, rationally plan the use of industrial robots and their spare parts, avoid running with defects, and improve the service life and efficiency of the industrial robot.
(3) The robot scheduling model combines production data such as production scheduling tasks and on-machine tasks to predict the tasks that need to be done. Based on the robot's task execution status, its own status, task priority, action distance and other data, the robot's work tasks and AGV scheduling are pre-arranged, and the industrial robot is matched with the production status in real time, so that the industrial robot's capabilities can be fully utilized, the robot's utilization rate can be improved, and the robot's flexibility and intelligence can be enhanced when it is applied.
2.4 Large Model Layer
The emergence of large-scale AI models, exemplified by ChatGPT, has driven the rapid development of a series of Artificial General Intelligence (AGI) technologies. AGI has sparked a new round of information technology revolution and become an advanced form of productivity. The rapid development of large-scale AI models has also attracted the attention of industry. This paper, based on the theoretical foundations and development trends of large-scale models, proposes their impact on the development of industrial machinery, aiming to contribute to the application of large-scale AI models in industry. The core of large-scale models in industry is solving "change"—due to the complexity of industry, some abnormal situations may occur during the production process, such as sudden failures of complex equipment and product anomalies. Effective means are needed to handle these "variables" in a timely manner to ensure the safe and stable operation of various production activities and achieve cost reduction and efficiency improvement. Large-scale models are continuously trained through data from various scenarios to improve their detection capabilities, build further application scenarios based on data, and realize the value of data. The application exploration of large-scale AI models in the industrial field is in its initial stage and is data-driven. However, it is foreseeable that its future development will begin with leading companies in the industry, fully utilizing their accumulated rich data from various scenarios, starting with large-scale models for individual scenarios as a breakthrough point to achieve the comprehensive development of large-scale models across the industry.
(1) Large-scale CV models: Computer Vision (CV) and large-scale machine learning provide theoretical support for the development of industrial robots towards greater intelligence. Combining AI technology with data collected by sensors on the robot (such as relatively comprehensive scene images and video data collected during robot movement), the application of machine vision in the industrial sector can be explored. For large-scale CV models in the industrial sector, there are two core aspects: data acquisition, data annotation quality, and efficient training. For image classification/recognition tasks of large-scale CV models, relying on manual annotation may not meet the requirements for training data. The proposal of SA (Segment Anything) provides a more reliable technical means for the realization of large-scale CV models.
(2) Industrial knowledge models encompass numerous scenarios in industrial settings, with industrial robots being a key component. These scenarios involve a vast amount of relational and non-relational data, containing significant value. Establishing connections between various dimensions of data from industrial settings to construct industrial knowledge models, fully leveraging the value of data, and ensuring that data originates from and contributes to business operations, is a meaningful and challenging undertaking. The development of knowledge graphs provides theoretical and technical implementation methods for the development of industrial knowledge models. Through knowledge graphs, deep relationships can be established, constructing an enterprise's industrial knowledge system. This integrates and links relevant data and resources within the enterprise, such as personnel, processes, equipment, spare parts, materials, and logistics. A single point can provide directly connected and potentially related information and knowledge, improving the efficiency of enterprise production and operation.
(3) The LLM large-scale model ChatGPT elevates the understanding and generation capabilities of artificial intelligence for general natural language tasks to a new level. Extensive related work has driven the development of the LLM large-scale model, and its theory and applications are relatively well-established. In industrial sectors, with sufficient data foundation and applications, similar scenario functions to ChatGPT can be realized, enabling rapid extraction of key points and completion of routine tasks such as report analysis. When using the LLM large-scale model, to ensure the authenticity of each generated statement, Amos Azaria et al. utilize the activation of the hidden layers in the LLM to determine the authenticity of the statement.
2.5 Data Processing
Industrial environments are characterized by complexity, randomness, and uncertainty, leading to background noise in images and sounds. Preprocessing of the data before training is necessary to ensure optimal results. To avoid the "curse of dimensionality" caused by complex data structures, methods such as Principal Component Analysis (PCA), isometric mapping, and Locally Linear Embedding (LLE) can be used to reduce the dimensionality of the data, ensuring the model can extract effective features. In training large-scale AI models, situations with few or even zero samples may occur in certain scenarios. Research in this area contributes to improving large-scale industrial models.
2.6 IT System Layer
IT systems support the operation of various business processes within an enterprise, such as OA, SCM, SRM, MES, EAM, and information systems integrated with automated equipment. They are planned and constructed based on the enterprise's actual needs. IT system construction is a large and complex undertaking, characterized by long project cycles, low success rates, and low post-launch utilization. Implementation must prioritize meeting business needs, requiring in-depth business analysis, consideration of incremental business development trends, system performance margins, database design capabilities, and system integration and expansion capabilities. A comprehensive approach is needed to ensure successful project implementation. IT systems are closely integrated with the enterprise's relevant business needs and are highly customized. With the development of large-scale models, in the future, by inputting detailed business requirements via prompts, large-scale models will automatically create databases, automatically program functions, and automatically deploy IT systems, significantly reducing the complexity of IT system implementation and better supporting enterprise digital transformation.
2.7 Big Data Layer
Data originates from all aspects of production and operation. Through the application of big data, it is possible to monitor the production and operation situation in real time, and to promptly analyze and guide improvements for any anomalies. Currently, there are significant differences in the level of big data application. Enterprises with well-developed automation, informatization, and digitalization systems have built their own data platforms and applied big data. Data originates from all aspects of production and operation. Through the application of big data, it is possible to monitor the production and operation situation in real time, and to promptly predict, provide feedback, and guide improvements for any anomalies.
03
Summary and Outlook
With the development of industrial digitalization and intelligentization, the application breadth and integration depth of industrial robots and intelligent sensors are increasing, leading to a rise in the dimensions and volume of data in the industrial sector. This lays the foundation for the development of large-scale industrial models. This paper proposes a feasible architecture for the application of large-scale artificial intelligence models in industrial robots. However, due to the numerous technical details involved, its practical application still has a long way to go. Driven by business development needs and technological advancements, large-scale artificial intelligence models will provide crucial technical means for the transformation and upgrading of the manufacturing industry.
The development direction of large-scale artificial intelligence models in industrial robots is to integrate model algorithms, chips, etc., to improve the performance, reliability, and ease of use of industrial robots; to unify the modeling of multiple tasks such as language understanding, visual perception, and control through pre-trained models with hundreds of billions or trillions of parameters, thereby improving the robot's language understanding and autonomous decision-making capabilities; to combine multiple perception methods such as vision, hearing, and touch, and to achieve multimodal understanding and fusion of the working environment through multimodal learning and perception; and to enable robots to continuously learn and optimize during operation through algorithms such as reinforcement learning and transfer learning, so that they can quickly adapt and migrate in different tasks and environments, improve the robot's self-adaptability and intelligence level, and achieve seamless human-robot collaborative safe operation.
The application of large-scale artificial intelligence models in industrial robots will evolve towards greater intelligence, adaptability, multimodality, ease of use, and strong stability, continuously improving and contributing to breakthroughs in domestically produced high-end robots, enabling wider and deeper applications. With the advancement of industrial digitalization, driven by national policies and market demand, it is believed that my country will gradually achieve breakthroughs in both industrial robot hardware and software. Various forms of industrial robots will develop towards higher stability, ease of use, and higher levels of intelligence in the future.
