To ensure this process is successful, operating systems like Windows are not suitable because they contain internal mechanisms that can introduce infinite latency, leading to unpredictability in the interaction process. For example, in applications such as high-speed surface mounting, precision assembly based on visual servoing technology, and the increasingly popular stacking of irregularly shaped batteries, Windows-based visual guidance faces increasing challenges in terms of execution stability, yield control, and cycle time optimization.
To meet the requirements of high-performance vision-guided systems, advanced automation systems necessitate the introduction of a real-time operating system (RTOS). Unlike operating systems like Windows, RTOS allows designers to accurately predict system jitter—the time it takes to start and complete tasks such as image acquisition, processing, and I/O control. While RTOS can be used for such tasks, they are typically used in conjunction with general-purpose operating systems like Windows. This allows developers to leverage readily available development kits to implement features such as graphical user interfaces, while simultaneously utilizing the RTOS to handle time-critical tasks.
ProV Real-Time (ProV RT) is a real-time vision guidance system developed by the ProU team, based on an RTOS and Microsoft Windows. We migrated the GigE protocol used for image acquisition to the real-time system. Simultaneously, image algorithms and alignment calculations are also performed on the real-time system. Through real-time interaction with the motion control module on the ProU WinPLC, which is also based on an RTOS, we achieved an excitingly fast and stable vision guidance application. The system composition and functions of the ProU WinPLC are shown in the following figure:
From the perspective of the entire visual guidance process, Windows intervention is required at every stage, from the initial capture command, camera driver invocation, image transmission, query completion, image processing, communication, and motion control. This is the root cause of system latency. Through extensive experiments and data analysis, we have documented the latency issues that may arise due to Windows intervention. Compared to ProV RT, the overall execution time is longer, and the uncertainty is increased.
However, the fixed time overhead of image transmission and processing remains. The overall process analysis is shown in the following diagram:
Specific case studies provide a clearer view of the differences between ProV RT and traditional solutions based on the Microsoft Windows operating system.
The WinPLC controller controls three Leadshine closed-loop steppers via EtherCAT bus, and a Daheng 500MP/GigE interface industrial camera is connected to the ProV RT module on the WinPLC.
WinPLC simultaneously performs visual alignment and motion control, and is applied to common image tools such as calipers and blobs. Image processing and transmission are both completed in a real-time operating system (RTOS). Users can perform image processing and alignment calculations by dragging and dropping block diagrams on the ProV RT platform, and then pass the calculation results to the motion control module on the ProU platform.
For comparison, we used the H image algorithm library and the G motion control board to implement the same image algorithm and motion control. The data comparison is as follows:
With approximately 2000 continuous image transmissions and processing cycles (as shown in the figure above), the average execution time of ProV RT is reduced by about 10%. The more significant difference lies in the stability of the execution cycle. As can be clearly seen in the figure above, there are several peaks in the execution cycle based on Windows vision, with the longest execution cycle reaching 540ms.
The complete test involves 2000 consecutive visually guided alignments (as shown in the figure above).
ProV RT improves execution efficiency by about 20%, but the biggest difference lies in the stability of the execution cycle. Windows-based motion control and vision can cause a delay of up to 1080ms in some cycles.
For machines that prioritize high speed and stable execution cycles, a delay of up to 1 second can lead to real problems such as machine downtime, decreased yield, and process verification failures.
It's worth noting that ProV RT is an open real-time vision platform. Image algorithm companies can run their own image algorithms within ProV RT and directly call the motion control module in ProU. Currently, ProV RT has demonstrated its unique value in several scenarios with extreme performance and stability requirements, such as high-speed surface mounting and precision assembly based on visual servoing technology.
For most automated equipment, the performance and value offered by ProV RT are not immediately apparent. However, amidst the turbulent world around us, the ProU team's goal is to calmly and relentlessly pursue ultimate performance.
