Industrial robots, as a crucial pillar of my country's industrial development, are increasingly widely used in manufacturing and logistics. 3D vision, serving as the "eyes" of industrial robots, has become a key factor influencing their application and performance. How to improve the accuracy, speed, and reliability of industrial robots through 3D vision control technology has become a focus of attention both domestically and internationally. This article will introduce 3D vision technology for industrial robots and its applications through a search of relevant domestic and international patent literature.
Distribution of 3D technology abroad
A search of patent documents published up to February 2, 2018, using the Derwent World Patent Database and the World Patent Abstracts Database, revealed that since General Motors filed its US4254433A patent application in 1979 for a "visual motion tracking system," the total number of patent applications worldwide related to 3D vision control technology for industrial robots has reached 2,774. Common types of this technology include: passive 3D vision technology (divided into monocular 3D, binocular 3D, and multi-view 3D, which use one, two, or more cameras to form a stereo vision system, respectively); laser 3D scanning technology that achieves stereo measurement by capturing projected laser images with a camera; structured light 3D scanning technology that uses a projector and structured light encoding technology to reconstruct the 3D viewpoint; and Time-of-Flight (TOF) camera technology that uses LEDs to emit high-frequency light signals and measures distance based on the time difference between signal emission and return. Monocular 3D vision technology is simple in structure but has low accuracy; binocular and multi-view 3D vision technologies have higher accuracy but relatively complex algorithms and structures; laser 3D technology is fast, but is limited by laser speckle defects, making it difficult to achieve high accuracy. While structured light 3D technology boasts high precision and speed, it cannot be used on transparent, black, or highly reflective surfaces. Time-of-flight (TOF) camera technology offers high speed and accuracy, but its cost is relatively high. Currently, passive 3D, laser 3D, and structured light 3D vision technologies are relatively mature and widely used.
Accuracy is an important indicator for measuring the visual performance of industrial robots. Currently, many basic technologies and core patents for improving visual accuracy are held by large companies in Japan and the United States, such as FANUC Corporation of Japan and Cognex Corporation of the United States.
FANUC's main products in this field are industrial robots deployed in industrial settings. The machine vision systems they use include stereo vision, structured light, and laser scanning. Their representative 3D vision system technology is exemplified by patent application US6490369B1, which stores a box template with ID parameters and lens position information. It transforms the distorted image in the stored information into a real image, determines the parameters of the real image, and compares them with the template parameters, enabling the computer to identify the box for the robot's initial operations. Furthermore, their patent application EP1584426A1 relates to a tool center point calibration system. Building on this, FANUC developed the FANUC RVision3DL vision system in 2006. This system uses a 3D laser sensor mounted on the ground to acquire visual data. This technology solves the problem of changing the position of the loading positioning hole after the robot grasps a blank part, automatically completing the position change and achieving high-precision loading.
Cognex focuses primarily on improving system accuracy through image processing. Its PatMax technology, a high-accuracy, high-speed, and high-processing-capacity object localization technology in machine vision, was first patented globally in 1997. Subsequent patent applications, such as US6856698B1 "Fast and High-Precision Multidimensional Image Localization," US6850646B1 "Fast and High-Precision Multidimensional Image Detection," and US6658145B1 "Fast and High-Precision Multidimensional Image Detection," all relate to PatMax technology. This technology uses a series of boundary curves independent of pixel grids to obtain the geometry of an object, then searches for similar images within the overall image. The most significant feature of this technology is that it is not limited by specific grayscale levels. In addition, between 2006 and 2008, the company filed patent applications US2007081714A1 ("Method and Apparatus for a Practical 3D Vision System"), US2008298672A1 ("System and Method for Locating 3D Objects by Machine Vision"), and US2010166294A1 ("System and Method for 3D Calibration of Targets Using Machine Vision"). Meanwhile, in May 2011, the company launched the VisionPro3D software system, based on these patents, which provides accurate, real-time 3D positioning information and can be applied to loading, unloading, packaging, and assembly verification in precision manufacturing industries such as automotive.
Besides improving technical precision, safety is also a crucial indicator in industrial robot applications. SICK, a German company, is dedicated to enhancing the safety of industrial robots through 3D vision technology. For example, their patent application with publication number EP2380709A2 utilizes structured light 3D imaging technology. When a robot and operator coexist in an area, 3D sensors are used to determine the operator's movement trajectory, preventing accidents and providing intelligent safety assurance.
3D technology domestic patent layout
A search of the China Patent Abstracts Database revealed that, as of February 2, 2018, domestic applicants had filed 594 patent applications related to 3D vision in industrial robots. These applications primarily involve the application of 3D vision in industrial robots, with approximately one-third originating from universities and research institutions. Some domestic companies specializing in industrial robot R&D have already begun patent filings in this area. For example, Siasun Robot & Automation Co., Ltd. has filed over a dozen patent applications in this field, most of which were filed within the last five years. One application, CN104511905A, utilizes structured light vision based on color stripes to guide robot arm movement, resulting in more accurate robotic arm operation. This technology optimizes robot movement speed and accuracy. Another application, CN106595511A, employs an image matching method combining a heritage algorithm and Hausdorff distance to locate the position of the object in the camera image. The robot is then moved to illuminate the object with a laser, and the laser information in the camera image is read. Finally, the principle of triangulation is used to measure the object's three-dimensional information. The above technologies demonstrate the efforts Chinese industrial robot companies have made in 3D vision technology, particularly in the application of 3D technology towards technological innovation. Meanwhile, in recent years, in addition to innovations in relatively mature fields such as laser 3D and structured light 3D technologies, domestic companies have also made improvements to Time-of-Flight (TOF) technology, which has fewer patent applications. For example, the patent application CN106772414A, entitled "A Method for Improving the Ranging Accuracy of TOF Phase-Measuring Radar," can improve ranging accuracy without changing the hardware technology.
In summary, 3D vision control technology applied to industrial robots has gradually matured. Many key technologies originate from countries and regions such as Japan, the United States, and Europe. For example, Japan's FANUC has integrated 3D vision products as add-ons into its own products. American vision technology companies, such as Cognex, focus more on the vision technology itself, while European vision technology companies concentrate on applications in specific fields. Currently, my country's 3D vision control technology mainly involves theoretical research in universities and specific product applications, and is gradually moving towards the development and production of vision technology products. Chinese industrial robot companies can catch up and develop products with independent intellectual property rights in specific application areas, giving industrial robots a pair of "eagle eyes."
