High availability achieved through redundancy and fault tolerance is a critical component of many industrial network deployments. While a few minutes' downtime in a corporate network can cause significant inconvenience, an outage in an industrial network could have catastrophic consequences. Enhancing the capabilities of the standard Spanning Tree Protocol (STP) can add resilience to ring-structured LANs and improve the rate of error recovery without sacrificing the benefits of standards-based redundancy solutions.
When deploying redundant industrial Ethernet applications, the company primarily relies on custom-developed systems adapted to individual applications. There has been relatively little practical experience gained from putting operational equipment into production over the past three years. Industries that have used costly serial connection protocols and proprietary solutions for years are attempting to adapt standard Ethernet to fault-tolerant industrial networks. The benefits of standard flexibility and interoperability are obvious. However, adapting software standards designed for more relaxed office environments to harsh industrial environments presents a real challenge.
Standard software
The IEEE 802.1d standard Spanning Tree Protocol (STP) has been available for use with managed switches and bridges for several years. This software provides a mechanism to maintain the operation of a standard Ethernet LAN with redundant physical connections, preventing packets from having more than one path at a time. STP is included in the software of Ethernet managed switches, which is offered by all major vendors of Ethernet managed switches and is widely available on the market.
Furthermore, through years of widespread use, STP has proven to be interoperable, and commercial systems using products from multiple manufacturers can routinely implement it. Standard STP supports any form of redundancy configuration: mesh, ring, or combined.
Recently, the Fast Spanning Tree Protocol (RSTP) has emerged and begun to be used in some redundant LAN systems. It is designed for small or medium-sized meshes and some small rings, offering faster error recovery times than STP. Generally speaking, there is no readily available experience with RSTP interoperability, as it is relatively new, and its emergence has led to some proprietary modifications by supporting vendors.
Manufacturers sometimes discourage the use of STP in order to encourage the selection of proprietary technologies. This is because STP does not receive manufacturer sponsorship, the data available for its use is not widely available, and training materials and application experience are relatively limited.
Ring structures in industry
Ring topology is the most practical and widely used redundant LAN configuration in industrial applications. Mesh topology is impractical and very expensive for large-area distributed applications. There are three options for managing redundant rings:
1. Use with current standards, even if they are not optimized for ring applications.
2. Using the manufacturer's proprietary solution for ring structures offers faster error correction time, but it is more expensive and has relatively less flexibility in implementation.
3. Choose a standards-based implementation with special ring extensions, which speeds up error handling and allows the use of any standard Ethernet product in the topology.
Select 1, use STP
STP has been used in redundant LAN applications for over 10 years. It was initially designed to support redundancy using bridge technology in 10Mbps bus topologies. Currently, STP has evolved for use in multi-port switches with star topologies in high-speed LANs.
Ring topologies are a simple subset of mesh topologies, which is where STP excels. However, in some high-end applications, the latency of standard STP in performing repairs is unacceptably long. STP is used with most (if not all) Ethernet-managed switches that can be mixed and matched in a single configuration. STP is designed to support various LAN topologies and works well with hubs and switches.
STP's weaknesses lie within its strengths: it is not inherently ring-oriented, and its complexity allows it to support various topologies, meaning its performance cannot be fully utilized in a relatively simple redundant ring. When a fault occurs in the ring, the obvious solution is to treat the interrupted ring as two separate strings until the fault is repaired. Because there is only one fault-repairing method, the time required for standard STP to collect data and perform analysis is unacceptable.
Other weaknesses of STP include its inability to easily scale up to control larger rings. STP works well when relaying information between redundant and conflicting switches, as long as all switches are between a pair of hops of the root switch making the decision. However, when a ring contains ten switches, each transmitting messages along the line, the ring becomes difficult for STP to operate and its message processing becomes inefficient.
For a decision-maker, a simple ring structure is the easiest to handle; only the two ends of the ring need to be taken care of, and the ring and its nodes must conform to the standard Ethernet packet processing protocol. Generally speaking, the STP structure is too complex, making the protocol overkill for a simple ring.
Option 2 – Dedicated Solution
They can quickly and effectively handle ring Ethernet failures, but the proprietary nature of network equipment limits the use of equipment from only a single manufacturer, including bearing the associated risks.
Option 3 – STP with enhanced ring functionality
Developing a faster, ring-based fault-correction process and a secure, fast-running solution built on the STP standard always requires further innovation to ensure better future performance. The recommended solution allows for multi-vendor implementations while maintaining selectability. One option is provided by GarrettCom's S-Ring software. It currently requires a Magnum6K managed switch but determines the occurrence of a fault using standard STP stateful multicast packets (called Bridge Protocol Data Units, or BPDUs). Without interfering with standard STP operation, the S-Ring software can be selected to run on a pair of ports supporting a ring architecture, reducing ring fault correction time (from 1.5 to 5 minutes for standard STP) from minutes to seconds (less than 2 seconds, within the buffer time allowed by the ring switch). Because it uses a proprietary ring correction method, the S-Ring solution is competitively fast while being built on standard STP, allowing for the use of multiple vendors, industry-standard Ethernet hubs, and switches.
Ring counting problem
Ethernet switches remember MAC addresses, allowing them to switch data packets to their destination port. Once activated, these addresses are stored in memory. If a MAC address becomes inactive, it is evicted from the switch's memory after a few minutes. This delayed evicting of the stored address can cause problems when the LAN urgently needs reconfiguration. When repeaters (hubs) lack memory buffers and are unable to establish a repair bottleneck, the stored address prevents data packets from being transmitted through the new repair path until the address is cleared from the switch's memory.
If the switches in the ring can prevent Ethernet communication packets from being moved to the repair communication path, then there is little benefit to fast ring repair techniques. Different switch manufacturers implement different address buffer slack times. In multi-vendor implementations, the slowest slack time in the repair path will control the ring repair time. Establishing the slack time for the switch address buffer is crucial when calculating repair time in a redundant ring.
In the company's mP62 edge switches, a proprietary feature called Link-Loss-Learn (LLL) can be activated, which immediately aligns the address buffer and re-acquires the MAC address for routing around the point of failure. This process is similar to switch initialization and occurs in milliseconds, resulting in rapid ring repair. An S-Ring implementation monitors for Link-Loss and STP BPDU packet failures and responds to the first failure to occur. In most cases, Link-Loss (link loss) is detected in less than two seconds, within which time the BPDU packet can successfully travel throughout the ring. Typical ring repair time (using S-Ring software and Mp62 edge switches with the LLL feature embedded in the ring port) is less than 250 milliseconds, even with 50 or more Mp62 switches in the ring structure. Without activated LLL, the address buffer expiration time of the mP62 switches (default 3 minutes) can be a threshold factor for ring repair time.
Multi-ring for dual redundancy
In industrial networks, dual-port PLCs require two independent channels on the LAN with no common point of failure to maximize uptime. Although the installation and operating costs are very high, this redundancy justifies the expense of downtime.
The ring structure has a common point of failure, which is the switch at the top of the ring. Due to the dual-system requirements for maximum redundancy and uptime, multiple points of failure, and rapid fault recovery time, two rings operated by Magnum 6K switches running on S-Ring software provide excellent performance and security.
Determining the optimal dual redundancy solution requires an analysis of cost-effective redundant media costs, fast-responding products, and an optimal balance of technology. However, it is highly useful to know that industry-standard interoperable platforms such as STP Ethernet can support dual redundancy ring structures, and that they can coordinate sockets and switches from many manufacturers. S-Ring technology-supported dual redundancy ring structures can achieve fault recovery times down to the second level, making them suitable for any fault-tolerant redundant LAN design.
What constitutes a truly standards-based solution?
Currently, many solutions are available for achieving rapid fault recovery in redundant Ethernet rings. Hirschmann was the first to propose ring topology technology with its Hiper Ring concept. Although this system requires all components in the ring to be proprietary, the LAN typically recovers in 50 milliseconds or less.
A few weeks ago, Moxa announced its ED6008 Ethernet redundant switch, which can be used to establish redundant Ethernet with a recovery time of 300 milliseconds. While specific details were not provided, the company's implementation of the Turbo ring is not based on the traditional Spanning Tree Protocol (STP). It is also unclear whether switches from other manufacturers can be used in Moxa rings. Establishing the ring requires connecting port 8 of each switch to port 7 of the adjacent switch.
While GarrettCom's S-Ring represents a major improvement in standard spanning tree repair time, it currently only runs on the company's 6K series management switches—somewhat inconsistent with the concept of a truly standards-based redundancy implementation. However, company president Frank Madren is willing to license his software to other industrial Ethernet manufacturers for use in their products, creating a truly open market for redundant switches.
Doug Thompson, GarrettCom's UK manager, told us: "Currently, the S-Ring software only runs on GarrettCom 6K series management switches. The question is, 'Is there anyone else who can run this software?' Anyone who wants a software license and contacts us is, in principle, allowed to do so. In the current situation, we are in a transitional phase where redundant ring solutions tend to become proprietary. Sometime in the not-too-distant future, most people will not want to take this approach and will prefer a complete, open solution. What we just presented technically allows this—and we are not against software licensing."
"We made it clear to all our clients 18 months ago that we would not consider endorsing anyone else's proprietary solutions," he said.
Finally, GarrettCom President Frank Madren said, “True standardization of redundant Ethernet technology is very possible. I don’t expect it to happen, but we’ve adopted it as a company strategy rather than rejecting it or introducing conflicting proprietary technologies. It’s about S-Ring. Adopting standards and doing them better in the spirit of Ethernet. If others do the same, users will be the winners, and we will all gain a larger market share accordingly.”
