Abstract: Industrial control networks have high real-time requirements, requiring not only fast transmission speed but also deterministic data transmission. Ethernet communication is uncertain and cannot meet real-time requirements, becoming a major obstacle to the application of Ethernet in the field of industrial control. The real-time Ethernet [2] implemented in this design strictly limits the communication of each node in the network to a specified time, ensuring that only one node is sending data in the same network segment at the same time.
Keywords: Industrial Ethernet; determinism; scheduling; real-time
1. Introduction
Industrial control networks are a type of network for specific applications. Compared with commercial information networks, they have their own requirements and characteristics. One of the most important aspects is the high real-time requirement. Not only is fast transmission speed required, but also fast response and deterministic data transmission. Ethernet's communication scheduling method—Carrier Sense Multiple Access with Collision Detection (CSMA/CD)—is a non-deterministic communication scheduling method. Each node in the network must compete for the right to send information: the node listens to the channel and can only send information when it finds the channel to be idle. After the information starts to be sent, it is also necessary to check whether a collision has occurred. If a collision occurs, it is necessary to wait. The waiting time depends on the random delay obtained by the "binary exponential backoff algorithm" [1][4]. This random delay adds unavoidable uncertainty to the data transmission of industrial communication.
2. System Model
This design adds a deterministic scheduling layer above the Ethernet MAC layer. The upper layer adopts the data encapsulation format of the EPA[3] (Ethernet of Plant Automation) protocol, thus making it possible to determine the type of data packets and obtain the scheduling time. EPA is a fieldbus standard based on industrial Ethernet , which is a distributed digital control system for communication between the control system and field measurement and control devices, as well as between field measurement and actuators. In this way, all upper-layer data packets are uniformly scheduled and sent within the entire real-time network segment, thereby avoiding collisions and ensuring the deterministic requirements of data transmission. The communication model is shown in Figure 1:
Figure 1 Real-time Ethernet communication model
3. System Design
This design is implemented on an embedded platform, using a hardware platform built with an ARM7 processor and a CS8900 network interface chip. In the same network segment, each running device is equivalent to a network node.
The upper layer uses the EPA protocol for encapsulation, separating real-time and non-real-time data transmission and reception. Upon arrival of real-time data, it is submitted downwards through the real-time communication service interface, buffered, and awaits scheduled transmission. When the next transmission cycle arrives, the corresponding buffer is updated with new real-time data. Non-real-time communication services are provided through the BSD interface in the OS. However, to achieve deterministic scheduling, when non-real-time data packets traverse the TCP/IP protocol stack to reach the data link layer, direct calls to the hardware driver interface for data packet transmission and reception are not allowed. Instead, a virtual driver interface places the data packets into the corresponding transmission and reception queues. The original E-NIC driver interface is modified to become a virtual driver interface, preventing direct data delivery to the hardware driver and instead placing data into a non-periodic queue. Transmission and reception operations for the hardware network card are implemented in the hardware driver interface. The design diagram is shown in Figure 2.
Figure 2. Ethernet Deterministic Scheduling Design Diagram
The deterministic scheduling layer is responsible for determining when to invoke the hardware driver interface to send and receive data frames. When the deterministic scheduling layer receives a data frame, it distinguishes between real-time and non-real-time data frames based on the type field in the Ethernet frame header. If it is a real-time data frame, the corresponding buffer is updated with the received real-time data; if it is non-real-time data, it is placed in the appropriate receive queue and the upper-layer protocol stack is notified of the data frame arrival event through the task communication mechanism. The scheduling table of the deterministic scheduling layer is generated by a scheduling algorithm.
4. Description of main modules
The real-time communication service interface, virtual driver interface, clock synchronization service, and real-time scheduling algorithm are the four main modules of this design.
4.1 Real-time Communication Service Interface
The real-time communication service interface is responsible for providing periodic real-time communication services for upper-layer applications (protocols).
The sending and receiving of periodic real-time messages employs a buffering mechanism. Specifically, a corresponding buffer is defined in memory for each periodic real-time message that the node will send during operation. Whenever an upper-layer application sends a new message through the real-time communication service interface, the old message value in the corresponding buffer is updated with the new message value. Similarly, a corresponding buffer is defined in memory for each periodic real-time message that the node will receive during operation. Whenever an upper-layer application receives a new message through the real-time communication service interface, the message value in the corresponding buffer is returned. Each periodic real-time message that the node will send during operation must have a corresponding known target address (including the target MAC address – used to identify the node, and the port number – used to identify the buffer element corresponding to the real-time message).
The real-time communication service interface should provide the following interface functions:
(1) RtAppInterface() creates a real-time interface function for sending and receiving real-time periodic messages.
(2) RtMapping() maps the interface process created by the RtAppInterface() function to a specific address (MAC, port number (i.e., the index of the buffer element)).
(3) RtSend() uses the real-time interface process that has been created and mapped to send real-time messages.
(4) RtRecv() uses the created and bound real-time interface process to receive real-time messages.
4.2 Virtual Driver Interface
The virtual driver interface provides a virtual driver interface for the upper-layer TCP/UDP/IP protocol stack. This interface must be consistent with the original hardware driver interface used by the upper-layer protocol stack, but the specific function implementations within the interface must be modified so that the specific functions corresponding to the virtual driver interface cannot directly control the hardware (such as opening and closing the device, sending and receiving data packets, etc.). It must be ensured that, from the perspective of the TCP/IP protocol stack using the driver interface, replacing the original hardware driver interface with the virtual driver interface is the same as using the original hardware interface; that is, the modification of replacing the original hardware driver interface with the virtual driver interface must be transparent to the upper-layer protocol stack. Data packets sent from the upper-layer protocol stack are buffered in FIFO order in a message queue. When to call the hardware driver interface to send these data packets is determined by the deterministic scheduling layer.
The main functions provided by the virtual driver interface are:
(1) RtVirtualIn() receives data packets sent by the IP layer and determines whether to transfer them to a queue or submit them downwards based on the data packet type.
(2) RtVirtualOut() calls the send function of the driver layer to send data and receives the send return information.
(3) RtVirtualPacketType() retrieves the corresponding field and determines the type of the incoming packet (EPA type field, Ethernet type field).
4.3 Clock Synchronization
In real-time network segments, clock synchronization is crucial because task execution and message sending on each node in a real-time network segment are based on a time base. In particular, message scheduling on the bus requires sufficient gaps between messages to accommodate synchronization errors between nodes in order to avoid collisions. To improve clock synchronization accuracy, the clock synchronization function should be placed as close to the hardware as possible in the protocol stack. In this system, the clock synchronization function is provided by the deterministic scheduling layer. The clock synchronization protocol adopts the IEEE 1588 protocol [6].
Within the real-time network segment, a node is selected as the master clock. The master clock periodically publishes clock synchronization messages, which serve as periodic messages on the master clock node. Since the clock synchronization messages are sent within a scheduled synchronization window, it can be guaranteed that the transmission of clock messages is independent of the current network load and is collision-free. Therefore, the delay of the clock message from the publishing node to each receiving node is a constant for each receiving node.
4.4 Real-time Scheduling Algorithm
The algorithm used in this design is an improvement on the RM and EDF algorithms [7]. At the beginning of each basic cycle, the scheduling schedule of this basic scheduling unit (ESU) is dynamically generated. The entire link time is composed of continuous basic scheduling units (ESU) of equal length. Usually, the ESU is the greatest common divisor (HCF) of the periods of all periodic real-time messages in the real-time network segment. Each ESU is divided into periodic real-time message window (PW) and aperiodic message window (AW), as shown in Figure 3. The scheduling in PW is based on the scheduling table generated in advance by the periodic real-time message scheduling algorithm. The length of the scheduling table is the least common multiple (LCM) of the periods of all periodic real-time messages. The part of the scheduling table related to each node is stored in each node. Each node executes the sending action of periodic real-time messages repeatedly in a macro cycle according to the time specified in the scheduling table. In AW, the data is sent in a priority queue node polling manner. According to the real-time requirements of aperiodic data, the aperiodic data is divided into i priorities. Each priority maintains an aperiodic priority queue in the node. Therefore, there are i non-real-time priority queues in each node. When sending data, a node polling method is used to send data in descending order of priority. Each polled node sends all the data in the specified priority queue.
Figure 3 Data transmission window within the basic cycle
5. Conclusion
The real-time Ethernet implemented in this system can schedule and transmit data within a network segment. Based on this design, the real-time Ethernet can be fully applied to general process control industrial fields. By improving the clock synchronization mechanism of the prototype system, the system's real-time performance and clock accuracy can be further enhanced. The goal of improving the real-time Ethernet based on this design is to increase clock accuracy to the microsecond level, thereby enabling its application in industrial fields with high time accuracy requirements, such as motion control.
References
[1] Xu Aidong, Wang Hong, Yang Zhijia. Industrial control network based on Ethernet. Information and Control, 2000, 29(2).
[2] Xu Aidong, Wang Hong, Xing Zhihao, Industrial Ethernet Real-time Communication Technology. Information and Control, 2005, 34(1)
[3] Feng Dongqin, Jin Jianxiang, Wang Weimin, Chu Jian. Distributed network control system architecture based on EPA. Automation Instrumentation, 2003, 24(9)
[4] IEEE Std 802.3, 2000 Edition, Part 3: Carrier Sense Multiple Access with Collision Detection [CSMA/CD] Access Method and Physical Layer Specifications, ISO/EEC 8802-3:2000 (E), 2000
[5] Y.Rajendra,P.Prashant,F.Raphael. A reservation-based CSMA Protocol for integrated manufacturing networks. IEEE Trans. on System, Man and Cybernetics, Vol 24 no 8, Aug 1994, pp1247-1258
[6] IEEE Standard 1588-2002, Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems, November 2002, New York, USA.
[7] E.Tover, F.Vasques, A.Burns. Communication response time in P-Net Network: worst-case analysis considering the actual token utilization. JSA Report YCS 312 University of York, 1999
