(Zhong Shenghong, Shenzhen Vision Dragon Technology Co., Ltd., Shenzhen, Guangdong 518000)
Abstract: This paper proposes a novel, non-programmable machine vision system. Addressing the challenges of existing vision systems on the market, such as high difficulty in use and high technical requirements for engineers, this system leverages the stability and portability of embedded systems and utilizes a configurable software development framework to design a compact yet feature-rich standard vision product. The product has been launched on the market and can be applied to various scenarios including object detection, localization guidance, and measurement, receiving positive market feedback. This paper will provide a detailed description of the system's hardware framework and software design principles.
Keywords: Embedded systems; DSP; Machine vision; Smart camera
Chinese Library Classification Number: Document Identification Code:
Overview
Machine vision is the use of machines to replace human eyes for measurement and judgment. It can quickly acquire a large amount of information, is easy to process automatically, and is easy to integrate with design information and processing control information. Therefore, in modern automated production processes, people widely use machine vision systems in the fields of working condition monitoring, finished product inspection and quality control[1]. Due to its high precision, non-contact and stable characteristics, it has been widely used in domestic and foreign industrial fields, which has greatly improved product quality and the degree of automation of production lines[2].
Overall, there is still a certain gap between China and foreign countries in the research and development of machine vision products. At present, China's research and development level of machine vision products is still at a relatively low level [3]. Foreign products are generally used. At present, the applications are mainly concentrated in several fields such as detection, positioning, motion control, and robotics. In the past two years, the application demand for machine vision has grown the fastest in several fields such as motion control, production lines, diagnosis, testing, and CNC equipment. Research on machine vision software is also deepening. As industry applications increase and requirements become higher, the current machine vision systems on the market are difficult to meet industry needs and generally have drawbacks such as difficulty in development, difficulty in use, and difficulty in sales. These drawbacks are mainly reflected in the fact that the standardization of the system is not high enough, a lot of development manpower needs to be invested for each customer, and the reusability of the developed system is weak, the user experience is not good, and the later maintenance is difficult and costly. Therefore, it is necessary to develop a machine vision system that does not require programming, has simple configuration, provides hardware and software configuration, and is easy to sell.
This article introduces a novel, non-programmable machine vision system. Addressing the current market situation, it offers the following advantages: 1. Configurable and programmable, as simple and easy to use as a household appliance; 2. Provides a universal application platform encompassing numerous unit machine vision tools; 3. Not based on a specific smart camera or general vision system, but integrating underlying and application development; 4. Vertically integrated, combining the core system with application processes; 5. Modular design, providing aftermarket support for equipment manufacturers and end-users; 6. Simplified system sales model; 7. Optimized system cost structure, resulting in a product with a significant price advantage. The detailed design concept of this system is described below from both hardware and software perspectives.
1 Hardware Framework
This system monitors and measures equipment operation based on image analysis technology. It uses feature extraction and template comparison to compare real-time images acquired on-site with standard images from normal operating conditions. This enables it to detect anomalies, analyze and judge their effects, and trigger alarms and shutdowns. The standard template is based on the analysis of images from normal operating conditions, extracting features. The comparison only compares the features (combinations) of the template with corresponding features in the real-time on-site images to determine if any anomalies have occurred. This monitoring method cannot replace the self-diagnostic function of automated equipment, but it is highly effective in preventing operational deviations, preventing defective products, and even preventing equipment damage. The product consists of the following main parts:
1. DSP/ARM-based core processor
2. Optical lens, image sensor, front-end processing and interface (i.e., camera part)
3. FPGA acts as an accelerator for image processing and a peripheral interface logic.
4. GPIO and serial communication
5. Ethernet communication
Figure 1 System Hardware Framework
As shown in Figure 1, the DSP/ARM processor uses TIOMAP4460CPU[4], whose ARM Cortex-A9 main frequency reaches 1.8GHz and has a built-in DSP, which is higher than the performance level of most smart camera hardware platforms to date; the camera resolution is mainly 1280x1024, and it also supports cameras with 5 million pixels or more; the storage supports NORFLASH and NANDFLASH, and can also expand SD cards; the FPGA is used as a logic expansion device for peripherals, and has two main functions: responsible for buffering and preprocessing image data, and then sending it to the DSP processor for corresponding algorithm processing; in terms of communication, it supports Ethernet and serial ports, which can be easily connected to devices such as robotic arms and PLCs; the I/O settings are equipped with GPIO (general purpose input/output), which connects to the trigger signals sent from the field (input, triggering to take pictures) and the control signals sent to the device (output).
In addition, the system can be provided in two forms:
Visual flat panel. A hardware platform with various functional interfaces required for visual applications, featuring the following characteristics: LCD display and touchscreen functionality (large screen); built-in high-performance processor (ARM architecture); camera access; I/O interfaces: RS-232, USB, 10/100/1000 Ethernet; network communication function: WiFi; encryption protection.
Figure 2 Visual tablet Vpad
Smart camera. Features include: compact size and light weight; industrial-grade product; on-board camera; I/O: GPIO; network connectivity: Ethernet, WiFi; encryption protection, etc. (See Figure 3).
Figure 3 Smart Camera VDSR
2. Software Technology
The ARM processor in the system is mainly responsible for resource management and logic control. It is equipped with the AngstromLinux system and integrates the drivers of the functional modules required for machine vision. The specific content of system software development includes: driver implementation of USB camera, GPIO, LCD touch screen, RS-232, etc., and porting of the AngstromLinux system. AngstromLinux is a desktop distribution system that runs on embedded platforms. It integrates most of the system tools and library files, including QT and opencv[5][6] libraries used for software development and algorithm development. It can select the tool libraries it needs through online compilation, and has a high degree of customization. It is very suitable for use in embedded products with scarce resources. The drivers of the underlying modules are implemented by modifying and porting the official Linux kernel.
The application-layer software development is based on the Ubuntu system, using QT for both interface and logic design. Ubuntu is a free and open-source desktop PC operating system based on Linux, which can easily configure environments similar to those on embedded platforms, such as QT and OpenCV. The system's software design goals are no programming required, simple configuration, and modularity. Therefore, all potentially used vision tools are integrated into a single integrated software platform, vdStudio, including commonly used tools for positioning, measurement, detection, and calibration. Combined with a graphical user interface, users can easily select and configure each tool, and the interface is designed based on user habits, making it simple, easy to learn, and easy to use. Furthermore, the system supports offline operation, achieving one-time configuration and permanent operation. (See Figure 4.)
Figure 4 User Graphical Interface
3. Algorithm Research
Algorithms are the core of machine vision systems, contributing the most to visual functionality and proving most critically compared to other parts of the system. In terms of market share, standard machine vision systems based on core software algorithms account for almost half of the entire machine vision industry, highlighting the importance of algorithms and software. If the algorithm is poor, visual functions cannot be achieved, or while they may be achieved, their performance will be uncompetitive. Even with a good algorithm, poor software development can directly impact the optimization and performance of the vision system. Algorithms and software systems are the most important assets for a vision company that produces standard systems. In market competition, the competitiveness of algorithms often plays a crucial role. Besides existing resources, acquiring more algorithmic software resources is also a key factor in the success of this project. In addition to developing our own algorithms in specific niche areas, collaborating and sharing revenue with internationally renowned software and algorithms is an important path to improving our products.
The system software design employs several excellent design patterns, allowing for flexible selection of algorithms during practical use. These include the independently developed VD300SDK (Software Development Kit), which includes function libraries for CG algorithms, datum plane correction, image stitching, and scanning control; an OpenCV-based algorithm SDK, including tools for template matching, measurement, and blob detection; and the general-purpose machine vision function library IPT (Image Processing Toolset).
4. Conclusion
This system allows for direct use of visual appliances without ASP integration or programming. Like household appliances, it's easy to operate by simply reading the instruction manual. There's no technical integration layer between the manufacturer and the user, saving costs that can be passed on to the user, lowering the product price. Users receive the functionality they need without needing extensive technical details. Salespeople only need to identify their target audience, explain the product's features, and highlight its competitive advantages; even ordinary salespeople without strong technical backgrounds can handle the job.
