By fusing multiple sensors, autonomous driving systems can obtain a more accurate model, thereby improving their safety and reliability. For example, millimeter-wave radar can compensate for the limitations of cameras, which are affected by cloudy or rainy weather, and can identify obstacles at a relatively long distance, but cannot identify the specific shape of the obstacles; lidar can compensate for the limitation of millimeter-wave radar in identifying the specific shape of obstacles. Therefore, in order to integrate the external data collected by different sensors to provide a basis for the controller to make decisions, multi-sensor fusion algorithms are needed to process the data to form a panoramic perception.
The following introduces three key sensors for achieving high-level autonomous driving: 4D millimeter-wave radar, lidar, and infrared thermal imaging.
4D millimeter-wave radar
Millimeter-wave radar was arguably the first sensor to be applied to mass-produced autonomous driving. While its accuracy isn't as high as lidar, it remains among the highest-level sensors. It possesses exceptional penetration capabilities through fog, smoke, and dust, and performs better overall in adverse weather conditions. It primarily functions as a ranging and speed measurement sensor. Currently, the number of millimeter-wave radar sensors installed per vehicle remains relatively low. From January to August 2022, only 0.86 millimeter-wave radar sensors were installed per new passenger vehicle delivered.
This isn't to say that traditional millimeter-wave radar performance is inferior. For L2+ level vehicles, the stable point cloud collection enabled by the high resolution of millimeter-wave radar is crucial for achieving 360° environmental perception. However, this isn't enough. For L3, L4, and higher-level vehicles, the perception accuracy and fusion effect are significantly reduced. With 4D millimeter-wave radar gradually being installed in vehicles this year, 2023 will be a banner year for large-scale pre-installed mass production. According to Yelo's forecast, the global 4D millimeter-wave radar market will reach $3.5 billion by 2027.
Currently, the application of 4D imaging radar in the market mainly focuses on two directions. The first is to replace traditional low-resolution forward-facing radar to meet the needs of improving the multi-sensory fusion performance of advanced intelligent driving. The second major application scenario is integrated 4D surround high-resolution radar (divided into point cloud enhancement and imaging), whose performance is slightly lower than that of forward-facing radar.
LiDAR
This year, "LiDAR in vehicles" has become the latest "label" for automotive intelligence. At the Guangzhou Auto Show, an increasing number of models, including the XPeng G9, Weltmeister M7, Nezha S, and Salon Mech Dragon, were equipped with LiDAR. Compared to ordinary radar, LiDAR has advantages such as high resolution, good concealment, and strong anti-interference capabilities. It is likened to the "eyes" of autonomous vehicles, determining the evolution level of the autonomous driving industry and playing a crucial role in the "last mile" of realizing autonomous driving.
LiDAR possesses irreplaceable advantages in high-level autonomous driving, where information accuracy is paramount. Currently, emerging electric vehicle manufacturers, traditional OEMs, and internet companies are all investing in LiDAR, driving a surge in demand for its production capacity. According to Zoson Automotive Research, 24,700 LiDAR units were installed in new passenger vehicles in China during the first half of 2022. In the second half of 2022, more than ten new vehicles with LiDAR systems are scheduled for delivery in China, including the XPeng G9 and WM M7, significantly increasing the number of LiDAR units installed. The total installation volume for the year is expected to exceed 80,000 units.
Infrared thermal imaging
Compared to traditional CIS and LiDAR, infrared thermal imaging offers significant advantages in various scenarios, including high dynamic range, rain, fog, low light, and sandstorms, making its introduction into advanced autonomous driving solutions an inevitable trend. Infrared thermal imaging devices integrating infrared detectors are particularly suitable for distinguishing pedestrians from other inanimate obstacles due to their ability to detect heat, offering advantages not found in other sensors. Furthermore, they are unaffected by rain, fog, haze, or lighting conditions, and can observe distances up to hundreds of meters, ensuring their future place in the autonomous driving field.
Previously, the main reason why infrared thermal imaging failed to be widely adopted in vehicles was its high price. In recent years, with the localization of key raw materials such as infrared thermal imaging chips, costs have decreased, leading to its widespread application in the civilian sector. Autonomous driving will rapidly expand the market size of infrared detectors. Data from the China Research Institute of Industry shows that the Chinese infrared thermal imager market reached US$6.68 billion in 2020, and is projected to continue growing at a compound annual growth rate of 10.8% in 2021, reaching US$12.34 billion by 2025.
In conclusion, multi-sensor fusion-based autonomous driving solutions are an inevitable trend in the future development of automobiles. Fusing information from multiple sensors can compensate for the limitations of a single sensor, improving the safety redundancy and data reliability of autonomous driving systems. However, the coordinate systems, data formats, and even acquisition frequencies of different sensors vary, making the design of fusion algorithms a complex task.
