Machine vision , as a specific field of optoelectronic technology application, has developed into a promising and dynamic industry with an average annual growth rate exceeding 20%. Machine vision is widely used in numerous industries, including microelectronics, electronic products, automotive, medical, printing, packaging, scientific research, and the military. While the underlying technologies are similar, the applications vary significantly, a common characteristic of various machine vision application systems.
Machine vision system integration involves multiple technologies. Even the most basic system requires lighting, imaging, image digitization, image processing algorithms, and computer software and hardware. More complex systems may also utilize mechanical design, sensors, electronic circuits, PLCs, motion control, databases, SPC, and more. Integrating so many different technologies and knowledge into a system, ensuring their perfect coordination and stable operation, places high demands on system integrators. Based on years of experience, the author explains the various technologies involved in machine vision system integration, the factors that need to be considered comprehensively, and methods for evaluating the likelihood of success for machine vision system projects.
After decades of rapid development, China's economy has achieved leaps in many fields, from nothing to something, and from inability to capability. Now, it has reached a stage of increasing production efficiency and product quality, and fierce competition. Many previously manual processes are gradually being replaced by machines, creating a growing demand for machine vision systems. Machine vision technology was originally developed to solve various problems in production. In human production activities, the human eye performs many important tasks, such as placing and fixing workpieces, observing and estimating positions, inspecting dimensions, determining product consistency, and verifying product quality. These tasks are increasingly being replaced by machine vision systems. This is partly due to the increasing demands for production speed and product quality, which exceed the capabilities of the human eye; and partly due to advancements in imaging, computer, and image processing technologies, enabling machine vision systems to perform increasingly complex tasks at a lower cost. However, machine vision technology, as an emerging technology, has only been introduced to China for a short time, and there are relatively few truly experienced system integrators. Machine vision systems involve various aspects such as lighting, imaging, electronics, automatic control, computer software and hardware, mechanical design, sensors, and optics. Integrating these different technologies into a single system and ensuring their seamless cooperation is inherently a challenging task. This article attempts to offer some suggestions on machine vision system integration technology based on the author's many years of experience, hoping to help users of machine vision systems evaluate their systems and provide reference for making informed decisions, while also offering successful experiences to system developers and companies. This article will cover aspects such as requirements analysis, resource allocation, and system integration.
A. Requirements Analysis
Accurately describing the functions and operating environment of a machine vision system is crucial for its successful integration. Describing the requirements essentially defines the scenario in which the vision system operates. Our goal is to design a system around this scenario to acquire suitable images and extract useful information or control the production process. This step seems so simple that it is often overlooked. Sometimes, users generate certain needs during production, but due to a lack of knowledge and technical expertise, they don't know what kind of vision system they need or cannot accurately describe their requirements. If the system integrator lacks experience or doesn't give it enough attention, they cannot help the user clarify the functional details of the system. This inevitably leads to a circuitous system integration development process, or even failure.
Therefore, the first step in system integration is to clarify user needs! Generally, the table below can be used to help us achieve this goal.
B. Resource Allocation
Machine vision system integration involves multiple technologies. Even the most basic system requires lighting, imaging, image digitization, image processing algorithms, and computer software and hardware. More complex systems may also utilize mechanical design, sensors, electronic circuits, PLCs, motion control, databases, SPC, and so on. Clearly, combining so many different technologies and knowledge into a single system, ensuring their perfect coordination and stable operation, places high demands on system integrators. They must determine the necessary resources and strategies based on specific requirements.
B.1 Mechanical Design
Due to varying needs, the requirements for the mechanical components differ significantly. For fully automated machine vision systems, the mechanical components typically need to perform functions such as material handling, conveying, positioning, rejection, and unloading. In contrast, some machine vision systems only require suitable mounting brackets to secure the camera, lens, and computer. For example, a vial light inspection machine used on a pharmaceutical production line needs to perform multiple actions, including bottle handling, conveying, rotation, emergency braking, camera synchronization, rapid repositioning, unloading, and sorting/rejection. It also requires up to 15 inspection stations to detect various indicators such as suspended solids in the liquid, glass fragments, bottle damage, bacterial colonies, bottle sealing, and bottle bottom. This involves thousands of mechanical parts, making the system extremely complex. In contrast, the mechanical components of a license plate recognition system used in parking lots are very simple, requiring only camera mounting and protection accessories. System integrators are generally reluctant to get involved in particularly complex mechanical designs. This is partly because users perceive mechanical design as lacking in technical sophistication, and partly because machining cycles are long, and even a small design error can lead to rework, further delaying the process. Therefore, although mechanical design often constitutes a significant portion of machine vision systems and is a critical component, relatively little investment is made in it, leading to difficulties in the overall system development process. Sometimes, collaborating with companies that manufacture mechanical equipment is the best option. It should be noted that for fully automated systems requiring complex mechanical systems, since most systems can only operate for specific products, semi-automatic systems are generally sufficient unless production volume reaches a certain level or the product will be produced for an extended period. Automated systems have long development cycles, high costs, and are difficult to changeover during production, while semi-automatic systems only require changing one fixture and resetting the inspection indicators and pass/fail criteria.
B.2 Lighting Source
This seemingly simple lighting system is the most critical part of a machine vision system, directly affecting its success or failure; its importance cannot be overstated. Good lighting allows us to obtain a good image, thereby improving the overall system resolution and simplifying software computation, while inappropriate lighting can cause many problems. For example, speckles and overexposure can hide much important information; shadows can cause false edge detection; and reduced signal-to-noise ratio and uneven lighting can make it difficult to select image processing thresholds. In practical applications, obtaining an image of the scene is easy, but obtaining an image suitable for the requirements of machine vision algorithms is difficult because the targets being measured are diverse, with different shapes and sizes, varying detection metrics, and different reflective properties and colors of various materials. Highlighting the features that need to be detected while suppressing unwanted features is not a simple task. This is mainly solved by designing or selecting appropriate lighting sources. Designing/selecting a low-cost, easy-to-install, and effective lighting system for a specific object and scene is the first task we need to accomplish when building a machine vision system. So what kind of image is a good image? Generally speaking, a good image needs to meet the following conditions:
1. Make full use of the field of view to fill the field of view with the features of the detected object, thereby maximizing the use of the system's resolution;
2. With appropriate contrast, the maximum grayscale value is close to 255, while the minimum grayscale value is close to 0;
3. The image is sharp due to depth of field or motion, and the focus is accurate.
4. Ensure even lighting and avoid glare;
5. Minimal image distortion;
6. Features of interest are easily detected and recognized, while other features are either not displayed or suppressed. If the chosen light source enables the image to meet these basic requirements, the first step in integrating a machine vision system is complete.
B.3 Imaging System
From a system integration perspective, here are some simple principles for selecting these hardware components.
Since camera lenses and circuit boards have become quite mature after years of development, and there are not many choices in the domestic market, the general principle is to use what is sufficient for the needs of the user.
When choosing a camera, my advice is to use one with a USB or 1394 interface if possible. This saves the cost of a circuit board and reduces the workload for future hardware or software upgrades. For high-end applications, there seems to be no other choice besides CameraLink. The newly developed GigE interface cameras, due to the characteristics of TCP/IP packet transmission, cannot guarantee real-time data transmission in some situations. However, its biggest advantage is that the data cable distance can be very long, and the main unit can be installed in the office, so on-site, only proper installation and protection of the camera and lens are required. Lens selection is even less; besides a few manufacturers such as Computar, Nikon, Tamron, Navitar, and Moritex, there aren't many options on the market, although in most cases, they are sufficient. If better image quality is required, SLR camera lenses can be used, which generally requires an adapter. Fujinon manufactures in Xiamen, but it is not sold in the domestic market. German Carl Zeiss lenses are excellent, but unfortunately, they are too expensive and not suitable for the Chinese market.
B.4 Computer System
The optimal configuration for a machine vision system—whether to use an industrial PC or an embedded system—is a matter of opinion, with each having its advantages and disadvantages. It mainly depends on the application and system requirements. The main characteristics of the two systems are shown in the table below:
These comparisons show that, if installation space allows, choosing an industrial computer-based vision system still has more advantages. However, if a small embedded system must be used on-site, then we have no other choice. Mature products from various companies generally use embedded host systems, such as HKeyence, Omron, Siemens, and Cognex. This is largely due to commercial sales considerations, as it ensures product uniqueness, providing an excuse for differentiated sales; on the other hand, it prevents users from replacing parts themselves, thus guaranteeing profits from after-sales service.
B.5 Image Processing
Image processing, as an indispensable part of machine vision systems, is so important that many beginners mistakenly believe it is the same as machine vision. In fact, image processing encompasses two parts: image enhancement and image analysis. Image enhancement refers to altering an image through processing to improve contrast, sharpness, and feature prominence. Image analysis, on the other hand, involves extracting useful information, such as presence, quality, and location, through computation for judgment or control. For example, restoring a blurry image to clarity is a typical example of image enhancement; obtaining a clear image achieves the goal. Conversely, capturing the faces of passengers at a busy airport to identify suspected terrorists is a typical example of image analysis.
After years of development, image enhancement algorithms have become largely mature, including techniques such as grayscale stretching to improve contrast, false coloring, edge detection, filtering, Fourier transform, and wavelet transform. In machine vision system integration, these are generally performed as image preprocessing before image analysis, while image analysis algorithms are the real problem that machine vision needs to solve.
Designing an algorithm to extract useful information from images of a real-world scene relies entirely on the experience and skills of technical personnel. Companies invest significant time and money in research, and the resulting algorithms, often of economic value, are strictly confidential. For example, if there's only one passenger in an airport waiting area, designing an algorithm to locate them is relatively easy; many companies or individuals can implement it, using similar methods and computation times. However, finding a passenger with specific characteristics among thousands of travelers is far more challenging. Algorithms designed will vary greatly, and the one that finds the target most reliably in the shortest time is the most valuable. In this case, in addition to experience, inspiration and creativity are also crucial. In other words, image processing algorithms required for system integration necessitate extensive practical experience, and most of these algorithms cannot be learned from books or articles.
B.6 Automatic Control
As a subsystem perfectly integrated with a machine vision system, it naturally utilizes industrial control components such as photoelectric sensors, digital I/O interfaces, PLCs, and motion control systems. Knowledge in these areas is essential. However, these are mature technologies; those with basic knowledge can be used directly without redesigning, and component suppliers typically provide technical support. Generally, this aspect requires the least time and effort in the overall system development process.
C. System Integration
Having clarified our needs, estimated the resources we require, and understood the tasks each subsystem needs to perform, it's time to integrate this knowledge into a complete system. Simply combining components from different fields and expecting them to work perfectly upon powering on is unrealistic. This process will inevitably encounter many unexpected problems. Generally, we need to follow certain principles to minimize these issues. Here are some suggestions based on our experience.
1. Use readily available parts whenever possible: Whenever possible, use readily available spare parts, such as lenses, cameras, and circuit boards. Don't make things yourself if you can buy them. I've seen many companies and research labs design and manufacture easily obtainable parts to save costs, resulting in wasted time and many detours. For example, one company consulted me about poor image quality. When I visited their site, I found they were using a 20mm diameter convex lens, bought for only 15 RMB to save money. You can imagine the poor image quality! Even Sony has to purchase lenses from Carl Zeiss!
2. Divide and Conquer: Divide the system into multiple modules, integrate each part to ensure it works correctly, and then combine these modules into a larger system. This is similar to a common software development approach; only by breaking down a large system into smaller, easily manageable pieces can it be effectively completed. This also requires experienced project managers to accomplish this task.
3. Fully Consider On-Site Conditions: Systems developed in the laboratory often encounter problems after installation at the field. This is mainly due to factors such as ambient light, electromagnetic interference, and vibration. Machine vision systems, after all, use optical systems, and the lighting conditions at the installation site, or the influence of natural light, can sometimes prevent our designed algorithms from working, leading to system failure. Factory power supplies generally carry various electromagnetic interferences, which can easily enter the computer system from power lines or other sources, causing system instability. If the system works normally at times and crashes at others, this is generally the cause. Furthermore, in industrial settings, various machines operate simultaneously, and vibration can sometimes cause problems. For example, if the algorithm uses the method of subtracting two images, it will certainly not work properly because vibration causes image displacement, making the images inaccurate. In this case, dimensional measurements will also be inaccurate. Therefore, during system integration, it is best to simulate on-site conditions; otherwise, it is difficult to achieve success on the first attempt.
4. Multiply the Difficulties by 2: Unexpected problems often arise during new product development. For example, sometimes a single faulty wire can cause the entire system to malfunction, and finding this problem is no easy task; sometimes a minor software issue might only take a few minutes to correct, but it could take two weeks to locate the problem. Therefore, the time required to solve the problems encountered during system integration is often longer than anticipated. This is especially true for complex and large-scale systems. Generally, we need to multiply the total number of known problems that need to be solved by 2 or 3 to create a truly realistic work plan.
