Since the widespread adoption of 32-bit CPUs to form multi-microprocessor systems in CNC devices in the 1980s, computer software has gradually become more important in CNC equipment. After the 1990s, with the rapid development of computer technology, the software in CNC systems with open architectures, developed using the rich software and hardware resources of PCs, has become more powerful in supporting intelligence and networking, and its scale and functionality have been further enhanced. CNC equipment has become a comprehensive system with highly integrated hardware and software. [b]I. Characteristics of Software in CNC Systems[/b] Most of the software in CNC systems is embedded software, closely related to the hardware and running in a specific hardware environment. Its biggest characteristic is its inseparable relationship with the hardware environment; the performance, intelligence level, and reliability of the entire CNC system are determined by both the hardware environment and the software. However, under current technological conditions, the reliability of software is an order of magnitude lower than that of hardware. According to statistics, 75% of operational failures in embedded systems are caused by software failures. In fact, system failures caused by software failures have become an issue that cannot be ignored in the fault diagnosis of CNC equipment. II. Software Testing and Diagnosis Methods in CNC Systems Compared with hardware, software failures have the following two main characteristics: (1) Software failures do not change over time. A large part of hardware failures are due to wear and tear of equipment and aging of materials, while such problems do not exist in software. Once the software runs correctly, it will not degrade over time. (2) Most software failures are caused by inherent errors in the program code, and for embedded software, interface errors between software and hardware are also an important factor leading to failure. [font=SimSun] Because of the different failure mechanisms, the diagnostic methods for software are not entirely the same as the fault diagnosis methods commonly used for hardware. From the perspective of ensuring equipment reliability, hardware equipment undergoes a series of reliability tests before leaving the factory, with the aim of exposing problems generated during the design and manufacturing process in advance. During the user's use phase, the focus is on monitoring the operating status of the equipment and diagnosing and repairing any faults that occur. Software, on the other hand, is tested before leaving the factory and in the early stages of use, especially the system tests conducted after the software and hardware are integrated. That is, system testing is one of the most effective methods for discovering software problems in embedded systems. In software testing theory, system testing is a type of dynamic black-box testing. This means that testers don't need to delve into the details of the software code; they only need to control the input conditions to drive the actual operation of the software under test. Simply put, dynamic black-box testing aims to simulate the real-world usage of the system under test and discover faults through actual operation. Based on system testing principles, the laboratory independently designed and developed a relatively universal embedded software system testing environment and summarized an effective system testing method. The following section introduces the system testing environment and testing methods using a specific test example. III. Software System Testing in CNC Systems 1. System Testing Environment The first step in system testing embedded software in a CNC machine is to build the system testing environment. The purpose of the system testing environment is to allow the software to run in a real hardware environment, allow testers to apply test cases to the software under test, and collect test result data. The system testing environment is a testing platform composed of both hardware and software. [align=center] Figure 1 shows the system testing environment designed and developed for software testing of a certain type of engine's CNC system. The entire testing environment consists of three main parts: the test case and test script generation environment, the system testing software, and the system testing kit. The test case and test script generation environment and the system testing software constitute the software part of the testing platform, while the system testing kit and its connected I/O devices constitute the hardware part. The test case and test script generation environment is a graphical editing environment where testers can convert designed test cases into test scripts through drawing or writing text. These scripts are the executable form of the test cases. The test scripts contain a series of operations on the system under test; their essential function is to control the test flow to achieve the tester's testing intent. The system testing software consists of four parts: a script interpreter, a model, and a test result collection and display module. The script interpreter is used to interpret the test scripts into executable code, controlling the test flow during testing. The model calculates and generates test data according to the requirements of the test script and sends it to the system under test. The test result collection module collects the status information and output data of the system under test during its operation. The display module is used to display the status information and output data of the system under test during the test according to the requirements of the tester, so that the tester can observe and control the test. The ground testing device (system testing box) is a tool for testing the integrity of the CNC system and the read and write operations of the flash memory in the device before the equipment is running normally, as well as for downloading and analyzing data after the equipment is running. The functions of the ground testing device include simulation of electronic controller and engine model, testing of electronic controller, testing of relay box, testing of sensor and testing of electro-hydraulic servo valve coil, etc. In the test environment, the ground testing device obtains the test data generated by the model according to the test script from the system testing software, converts all the test data into a data form that can be input into the system under test in the device, and sends it to the system under test through the I/O interface to drive the operation of the system under test and complete the test. This system test environment has the following main features: (1) It has a certain degree of universality. This environment can not only test engine CNC software, but also test a similar type of embedded software. (2) It is easy to operate. The test environment only requires the tester to write the test cases into test scripts, and the test can be automated in the test environment without direct operation of the system under test. (3) The test efficiency is high. With the support of system detection software and system detection box, the execution efficiency of this automated test is much higher than that of manual test. (4) Supports repetitive testing. Since the test cases exist in the form of test scripts, the same test can be executed infinitely without difference. Repetitive testing has great application in regression testing after fault modification. 2. System testing process After the test environment is set up, it can be used to conduct system testing. The specific process is as follows: (1) Design test cases. System test cases consist of two parts: test input sequence and expected test results. The test input sequence describes the specific operation process of the device under test in a test, including how to start the device, when to accelerate, when to stop, and when to shut down the device. The expected test results describe what actions or states the device under test should produce under the above operations. The quality of test cases is the key to whether the device software fault can be found. When designing test cases, you can consider several aspects such as normal function, boundary conditions and abnormal operation, depending on the focus of the test. Normal function test is to simulate some representative operations based on the normal use of the engine under test. Boundary condition test refers to testing the boundary values ​​of the analog quantities input to the system under test. Experience shows that when the input quantity is at or near the boundary, it is most likely to produce errors. For example, the engine is prone to failure when running near the boundary point of the normal speed range. Abnormal operation mainly examines the ability of the CNC system software to handle some abnormal situations. For example, when the user makes a mistake or a hardware failure occurs suddenly during use, does the CNC system software have certain protection functions to prevent major accidents? This test mainly examines the reliability and robustness of the CNC system. (2) Run test. The designed test cases can be converted into test scripts that the test environment can recognize by drawing or text in the graphical editing environment provided by the test environment. Each test case is converted into a test script, and the test script can be directly loaded into the test environment for a test. During the test run, the test cases are executed sequentially in the operation sequence, and the system under test is thus run. During the run, the status of the equipment operation can also be observed through the display module provided by the test environment. (3) Analyze the test results and draw test conclusions. After a test is completed, the test environment will automatically collect various operation data of the system under test. Compare these data with the expected results in the test cases to determine whether the equipment under test is operating normally or whether a fault has occurred. 100 test cases were generated from three aspects: normal function, boundary conditions and abnormal operation, and were successfully run in the test environment. Finally, through the analysis of the test result data, several key problems were found. For example, when the flight altitude of the aircraft exceeds a certain value, the interpolation calculation performed by the software will be incorrect. In addition, there are problems with the design logic of the software, which cause the engine to stop when it should not stop. After analysis, it was determined that these faults were caused by defects in the CNC system software. After the software designers made correct modifications to these problems, the reliability and robustness of the system were greatly improved. IV. Conclusion Using the aforementioned system testing environment and methods, we have successfully tested multiple embedded software programs, uncovering many potentially significant defects and playing a crucial role in improving the quality and reliability of our devices.