The BeiDou Navigation Satellite System is a navigation system independently constructed and operated by my country, integrating navigation and positioning, timing, user monitoring, and short message communication [1]. Short message communication is a unique function of the BeiDou system. It is not affected by terrain conditions and environmental climate, and can be used to solve the problem of real-time data communication in remote areas. It is particularly suitable for use in situations where conventional communication methods have many blind spots and require large infrastructure investment, such as valve stations and gate stations in the field of oil and gas, as well as data transmission in factories far from mobile network coverage areas or areas with unstable communication.
The main task of Beidou satellites is positioning and navigation. Communication channel resources are limited, and civilian Beidou short message communication suffers from limitations in single message length and communication frequency. For data in industrial remote monitoring and control and data acquisition (SCADA) projects, the data acquisition frequency is high and the data volume is large. If the data is processed into packets, a large amount of data transmission can be achieved. However, Gu Junxia et al. [2] showed in their 2015 Beidou short message packet loss test that as the number of message packets increases, the message transmission success rate gradually decreases. Reducing the sampling frequency of raw industrial data can reduce the data volume. However, for enterprises, historical data is a valuable asset in the industrial field, providing basic information for subsequent engineering technicians to analyze and troubleshoot; it cannot be arbitrarily reduced or discarded due to transmission limitations. Therefore, a method is needed that can complete the transmission of real-time data without generating a large amount of data.
Short message communication is a unique function of my country's BeiDou Navigation Satellite System, used for data transmission in areas with no public network coverage. However, limitations in communication frequency and message length reduce the efficiency of BeiDou short message communication. Therefore, a compression method for BeiDou communication data is proposed. This compression method consists of two steps: the first step is lossy compression, based on the rotating door algorithm in industrial database compression technology. To achieve adjustable accuracy of lossy compression, an improved BP neural network PID controller is used to adjust the parameters of the rotating door algorithm online. The second step is lossless compression, proposing a lossless compression strategy using a special pre-byte combined with interpolation. Experiments show that after this two-step compression method, high compression is achieved for industrial process data, and the compression accuracy can be accurately adjusted during the compression process.
Among the published papers on BeiDou short message communication, Yu Longhai proposed first establishing a mathematical model of the moving target, and then compressing the BeiDou positioning data through parameter simplification and differential coding [3]. Chen Haisheng proposed a fixed-length index code table for the transmission of fish catch data, which can solve the problem of lossless data compression transmission, but it is not universal [4]. Peng Hao proposed a joint compression algorithm of Chinese intelligent word segmentation and lossless compression coding, thereby realizing the expansion of communication data [5]. However, in industrial process data, real number information is mainly transmitted. At present, there is no practical solution for data compression and transmission of industrial process data for BeiDou short message communication. Therefore, this paper proposes a solution that compresses data step by step from both lossy and lossless compression aspects. Experiments show that this method can effectively compress short message data, thereby improving the efficiency and reliability of industrial process data transmission.
1. The first step in BeiDou communication compression: lossy compression
Lossy compression involves sacrificing certain information during the compression process to achieve a higher compression ratio [6]. To evaluate the lossy compression process later, this paper adopts the evaluation criteria proposed by Qu Yilin [7] in his paper, as follows:
CR is used to measure the compression capability of an algorithm for a set of data, while δ is used to measure the average distortion of a set of data.
The rotating door algorithm is a common process data compression algorithm. The lossy compression stage of the BeiDou communication compression algorithm proposed in this paper is based on the rotating door algorithm. The rotating door algorithm will be introduced first, followed by the lossy compression algorithm based on improved BP neural network PID with self-controlled precision SDT (hereinafter referred to as the self-controlled precision lossy compression algorithm) proposed in this paper.
1.1 Analysis of Standard Revolving Door Algorithm (SDT)
The rotating door compression algorithm was developed by OSI Software Corporation in the United States. This algorithm is mainly designed for floating-point data. SDT, as a simple algorithm for linear fitting, has the advantages of high efficiency, high compression ratio, simple implementation, and controllable error. The basic algorithm principle is shown in Figure 1.
Let ΔE be the compression accuracy parameter of the SDT algorithm. In Figure 1, point A is the starting point, and the two points above and below point A (ΔE away from A) are the pivot points (similar to the hinges of two doors, hence the name "revolving door"). At the start of compression, both doors are closed, and during the algorithm's execution, the doors can only open outwards, not close inwards. As the number of data points increases, the doors will rotate and open. When the sum of the interior angles of the two doors equals 180°, the operation stops, the previous data point is stored, and a new compression segment begins from that point. After compression via the revolving door, the straight line from A to B replaces the data points from A to B.
Decompression process: Based on the starting and ending points of each line segment, the formula for the line segment can be calculated, and then the corresponding ordinate data value can be calculated based on the x-axis coordinate of a certain point.
1.2 Self-controlled precision lossy compression algorithm design
There are two aspects to note regarding the SDT algorithm: (1) ΔE is the key factor in determining the average decompression error δ of the SDT algorithm. If ΔE is chosen to be small, δ will also be small, resulting in a low compression ratio CR; conversely, if ΔE is chosen to be large, CR will be high, but δ will also become large. Even experienced engineers need to adjust ΔE for a long time to find a suitable value, and when the data range and characteristics change, it needs to be reset to meet the requirements of compression ratio and accuracy. (2) δ is usually used as a standard for evaluating the quality of the compression process. If δ does not meet the requirements, a high CR is meaningless.
Based on the above two points, the focus is on the control of δ. Given an ideal δ, an algorithm adaptively adjusts ΔE. This paper introduces the concept of feedback control from engineering, using a PID algorithm to adjust ΔE online, while simultaneously using a neural network algorithm to dynamically adjust the PID parameters. Furthermore, leveraging its powerful learning and network memory capabilities, it achieves rapid compression of similar data.
The algorithm comprises two aspects: an improved BP neural network PID controller (hereinafter referred to as NBP-PID) and a standard SDT compression and decompression algorithm. Data is treated as the controlled object through the SDT compression and decompression process, with the desired average decompression error δs as the given input and the actual average decompression error δ as the system output. The NBP-PID controller continuously adjusts the output ΔE, thereby controlling the δ output, until the deviation value is less than the algorithm's stopping threshold Th, thus completing the compression of the first set of data. The threshold calculation formula is:
After the first set of data is compressed, the controller parameters at this point are retained and used as the initial values for the next set of data compression, thus completing the compression of the second set of data, and so on. Simultaneously, the compressed first set of data is further compressed using the BeiDou lossless compression algorithm. Figure 2 shows the feedback control system model of the first set of data compression algorithm.
1.2.1 NBP-PID Controller Design
In industrial control, adjusting the proportional, integral, and derivative coefficients of a PID controller is generally difficult. According to neural network theory, a three-layer BP neural network can approximate any linear and nonlinear function. Through the self-learning of the neural network, adjusting the weighting coefficients can make its steady state correspond to the PID controller parameters under a certain most regular condition [8]. Therefore, this paper uses a three-layer BP neural network (4-5-3 structure) to adjust the PID parameters.
The input layer has four neurons: input rin, output yout, error, and a constant 1. Five hidden neurons are sufficient. The three output neurons adjust the parameters Kp, Ki, and Kd online. The input to the input layer is:
1.2.2 Implementation Steps of Self-Controlled Precision Lossy Compression Algorithm
(1) Initialize the corresponding parameters during the first compression. Set u, u_1, u_2, u_3, u_4, u_5, i.e., the initial values of the controller output at time zero and at each historical time, to 0. At the same time, initialize the error and the output value you at time zero and at each historical time to 0. Set the initial values of the weighting coefficients of the hidden layer and the weighting coefficients of the output layer to random numbers between -0.5 and 0.5. Select the learning rate η, the inertia coefficients α and β, and the threshold value Th;
(2) Sample to obtain rin(k); input the output u(k-1), i.e. ΔE of the controller at the previous moment into the compression and decompression function representing the controlled object model to calculate the output yout value at this time, and then calculate the system error error(k) = rin(k) - yout(k) at this moment.
(3) Calculate the input and output of each layer of neurons in the neural network. The output of the output layer corresponds to the three adjustable parameters Kp, Ki, and Kd of the PID controller. Based on this, calculate the output u(k) of the controller at the current moment.
(4) Perform neural network learning, adjust the weighting coefficients of the output layer and hidden layer online, and then update each state parameter;
(5) According to formula (3), calculate the Th value of the system at this time. If it is less than the preset threshold value, it indicates that this set of data has reached the given error compression requirement and proceed to step (6). Otherwise, proceed to step (2) and repeat the cycle.
(6) Save the weighted coefficient values of the output layer and hidden layer at this time and the previous 3 times as the initial values of time zero and historical times when compressing the next group of data. Save the values of u and the previous 5 times as the initial values of u at each time when compressing the next group of data. At the same time, the compressed data of this group will be further compressed in the subsequent Beidou lossless compression algorithm.
(7) Use the controller parameters saved in the previous group to start compressing the next group of data, that is, enter step (2) again until all groups of data are compressed and the algorithm ends.
2. The second step in BeiDou communication compression – lossless compression
Compared to lossy compression, lossless compression occupies more space and has a lower compression ratio, but it preserves 100% of the original information without any signal loss and is unaffected by the signal source [9]. General industrial process data have clear physical meanings. Taking the natural gas valve station data provided by an energy company in the project as an example, it includes analog quantities such as temperature, flow rate, pressure, and resistance, as well as status quantities such as valve start/stop and alarm.
This article takes the most common flow/pressure/resistance data in process data as examples to introduce the BeiDou lossless compression algorithm.
2.1 Data Preprocessing
Assume the real number data to be compressed has a precision of two decimal places and a data size of 0.00~999999.00. It is stored in a floatfRevBuf[500] type array. If each element of this array is multiplied by 100, its decimal part is converted into an integer. Then, a type cast is used to obtain a 4B unsigned integer array iPlc[500]. The data ranges that different types of integers can represent are shown in the first three rows of Table 1. Based on the way integers are stored in computers[10], the following four cases can be deduced.
2.2 Lossless Compression Algorithm for BeiDou Short Message Communication
Process data is generally collected in chronological order and contains temporal characteristics; most industrial field data is process data [11]. Therefore, the data are numerically related, so allocating storage space based on the difference between adjacent array elements, rather than the element values themselves, is more reasonable. The specific steps are as follows:
(1) Create a new array intiSub[500] to store the difference between adjacent elements in array iPlc, i.e.: iSub[i+1] = iPlc[i] - iPlc[i+1]; and assign the first element of array iPlc to the first element of array iSub.
(2) Based on the value of each element in the difference array iSub, refer to Table 1 and assign a byte length marker of 1 to 4 to each element, and record it in another array iLethByte. For example, in Table 2, the difference element is 100, and the element with the same index in the iLethByte array is 1.
(3) Each element in the iLethByte array records the minimum byte length of the storage space required for each element of the difference array iSub at the same index position. In order to decompress at the Beidou receiver, in addition to transmitting the iSub array, the length marker array iLethByte also needs to be transmitted, but this requires transmitting too much data. According to Table 1, based on the different numerical ranges that can be represented by different byte lengths, there are 7 types. However, in reality, it is not necessary to divide them into 7 types every time, because the numerical value of the process data has a certain continuity and is not evenly distributed. Therefore, each time compression is performed, 4 types (the selection principle will be supplemented later) are selected as the standard types for this transmission according to the distribution law of the process data itself. The data contained in the other types will eventually be classified into these 4 standard types, and then two binary codes are used to represent the 4 standard types selected this time. For example, the byte lengths of the 4 standard types this time are 1, 2, 2.5, and 3, then their binary codes can be: 00011011. Then, each element is compressed according to the byte length of its standard type, resulting in data of varying storage lengths. These four transformed iSub array elements are then grouped together, with a record byte `bt4Record` added before each group. `bt4Record` is 8 bits, and each two bits represent the code of the standard type for an iSub element. The iSub element appears in the high-order bits of the corresponding `bt4Record` before the iSub element and in the low-order bits of the corresponding `bt4Record` after the iSub element, as shown in Table 2. Therefore, the binary representation of the `bt4Record` bytes preceding these four iSub elements in Table 2 is: 01100011.
(4) The selection and combination of standard types
To allow the receiver to determine the type of combination, the combination number needs to be transmitted additionally. If the number of data items to be compressed in a single batch is not a multiple of 4, the last bt4Record byte will be less than 8 bits. In this case, the missing bytes are padded with zeros, and then the total value of the compressed data in this batch is transmitted additionally, ensuring accurate decompression by the BeiDou receiver. At the beginning of each message, two bytes (16 bits) are taken. Ten bits record the total number of data elements to be compressed (10 bits can represent 2^10 = 1024 numbers, which is sufficient), and the remaining 6 bits (2^6 = 64 > 35) represent the standard type combination number.
At this point, the data packet for a single BeiDou short message service has been packaged.
(5) The processed data is sent to the receiving end via the Beidou device. When the receiving end decompresses the data, it first extracts the first two bytes and determines the four standard types in this group of data based on the classification number of the 6 bits; it determines the total number of original data in this group based on the number of data bits of 10 bits. Then, it takes bt4Record first, and then takes the number of each difference element according to the record length of this byte, saves them as integers respectively, and then takes the next bt4Record byte in turn. For the received data byte length of 1, 2, or 4, it can be directly forced to convert the type back to a 4-byte integer according to Table 1; for other custom types, it is necessary to extract the sign bit, and then convert it into a 4-byte integer according to the principle that the complement of a positive number is the same as the original code, and the complement of a negative number is the inverted original code plus one.
(6) Reduce the integer saved in step (5) by 100 times and save it as a real number. This completes the decompression process of Beidou lossless compression.
Supplement: The type of the maximum value in a set of data must be one of the standard types selected in this case. For example, if the maximum value of the iSub[4] element is 8000000, then the standard type with 3 bytes must be selected, and then the other 3 types that can contain the most element values are selected as the standard types.
3. Experimental Results and Comparative Analysis
The experimental data comes from a riveting test machine, which uses a riveting tool to roll the annular groove on the edge of the bearing's outer ring, causing one side of the groove to deform towards the seat ring hole or bearing, thereby achieving axial fixation. The data comes from a pressure sensor in the machine tool; one set of data represents one clamping process, and the original pressure curve is shown in Figure 3. (There are 18 sets of original data, numbered S1~S18). The downward phase of the curve in the figure is caused by the motor stopping and the pressure gradually being removed. The downtime is manually controlled, so to eliminate human interference, only the first 20 seconds of data (200 data points) of each set are taken.
In subsequent experiments, rin(k) was set to 1, the learning rate η to 1.4, the inertia coefficient α to 0.3, β to 0.5, and the threshold value Th to 0.1. Furthermore, in the following paragraphs, "the first stage" refers to the lossy compression stage in BeiDou communication compression, and "the second stage" refers to the lossless compression stage.
3.1 Performance Comparison of NBP-PID and Standard BP Neural Network PID Controller
Figure 4 shows a comparison of the number of compression training steps required for the NBP-PID algorithm to reach the threshold when each of the six sets of data is compressed individually, compared to the standard BP neural network PID algorithm (referred to as BP-PID) in the first stage.
As shown in Figure 4, when modifying the network weights, adding an inertia term at time k-2 and a "step inertia term" coefficient to the NBP-PID algorithm can reduce the number of repeated compressions, thus reaching the threshold value faster.
Figure 5 shows the decompression error and training times of the first group (S1) data using the two algorithms. The NBP-PID algorithm reaches the threshold after 11 training iterations, meaning it reaches over 90% of the given value and terminates training. In contrast, the BP-PID algorithm cannot reach 90% of the given value within the maximum training iterations (20). In summary, the NBP-PID algorithm effectively accelerates the training process, reduces the number of repeated compression iterations, and thus improves the algorithm's practicality.
3.2 Compressing multiple sets of data using the NBP-PID algorithm
Table 3 shows the test data on the number of compression cycles required to reach the threshold after the first group of data has been compressed.
As shown in Table 3, the initial data compression takes about 10 iterations. Subsequent data groups, except for a few reaching their maximum training iterations, can generally be compressed within one or two iterations. Therefore, by utilizing the NBP-PID network's weight memory function, when similar data appears, the compression of subsequent groups can be accelerated to some extent, improving the overall compression efficiency.
3.3 Overall Compression Analysis of BeiDou Communication Compression Algorithm
Table 4 describes the specific compression information during the two-stage compression process. To eliminate randomness, the data used here differs from the data in the previous experiments; five sets of data from S10 to S14 are used as the experimental data.
The average decompression error δ of these five data sets is around 1.0, meeting the requirement of fluctuation within 10% of the given value. The average compression ratio in the lossy compression stage is 4.452, achieving a good compression ratio while allowing users to customize the compression error. In the lossless compression stage, the compressed storage space is the sum of the space occupied by the compressed array elements, the sum of the space occupied by the record bytes, and the sum of the first two bytes of the message.
The overall compression ratio is the ratio of the storage space occupied after lossless compression to the storage space occupied by the original data (800 = 4 × 200). As can be seen, after two stages of compression, the data is compressed to approximately 10.5% of the original data. This demonstrates the good compression effect of the BeiDou compression algorithm.
4. Conclusion
The lossy compression algorithm proposed in this paper has been tested in a laboratory environment, and the lossless compression algorithm has been successfully deployed in a natural gas pipeline monitoring project. The industrial data compression algorithm based on BeiDou communication solves problems such as uncontrollable SDT compression accuracy and difficulty in parameter setting. Furthermore, the data achieves a high compression ratio through two stages of compression. In addition, this algorithm can be used not only for BeiDou short message communication but also in other situations where communication frequency and message length are limited, thus possessing practical engineering value.
References
[1] Yang Yuanxi. Progress, contributions and challenges of the BeiDou Navigation Satellite System [J]. Acta Geodaetica et Cartographica Sinica, 2010, 39(1): 1-6.
[2] Gu Junxia, Wang Chunfang, Song Zhiguang. Performance test and statistical analysis of Beidou short message communication channel [J]. Meteorological Science and Technology, 2015, 43(3): 458-463.
[3] Yu Longyang, Wang Xin, Li Shujian, et al. Positioning data compression and reliable transmission based on Beidou short message [J]. Electronic Technology Application, 2012, 38(11): 108-111.
[4] Chen Haisheng, Guo Xiaoyun, Wang Feng, et al. A method for compressed transmission of fish catch information based on Beidou short message service [J]. Transactions of the Chinese Society of Agricultural Engineering, 2015, 31(22): 155-160.
[5] Peng Hao. Research on Data Expansion Technology for Beidou System User Communication [D]. Xi'an: Xi'an University of Electronic Science and Technology, 2013.
[6] Xu Hui. Research on data compression algorithms in real-time databases [D]. Hangzhou: Zhejiang University, 2006.
[7] Qu Yilin, Wang Wenhai. Automatic control precision SDT algorithm for process data compression [J]. Computer Engineering, 2010, 36(22): 40-42.
[8] Tan Yonghong. Adaptive control based on BP neural network [J]. Control Theory and Applications, 1994, 11(1): 84-87.
[9] Zheng Cuifang. Research on several commonly used lossless data compression algorithms [J]. Computer Technology and Development, 2011, 21(9): 73-76.
[10] Zhang Dewei, Shen Peifeng, Zhang Dezhen, et al. Analysis of the concept of computer complement [J]. Microcomputer Information, 2005, 21(20): 177-178.
[11] Shen Chunfeng, Huang Songxin, Zhang Dong, et al. A general online compression method for industrial data based on parameter estimation [J]. Control Engineering, 2011(s1): 142-145.
