To achieve SAEL4/L5 fully autonomous driving capabilities in the 2021/2022 model year vehicles, a redundant multi-sensor system is required. Current semi-autonomous driving systems utilize a wide variety of radar and camera systems in varying numbers and designs. However, the development of high-performance, cost-effective laser detection and ranging systems capable of detecting information within a 300-meter radius is still in the preliminary research stage. Most automakers believe that cameras, radar, and lidar—the three major sensor systems—are indispensable for achieving fully autonomous driving.
Currently, ultrasonic radar, millimeter-wave radar, and multi-camera systems are already being used in high-end vehicles. With the rapid development of intelligent driving, environmental perception technology will advance rapidly, further enhancing its synergistic effects. Although sensors are only one part of autonomous vehicles, their market prospects are vast. Therefore, relevant organizations predict that by around 2020, the global market for automotive cameras, millimeter-wave radar, and night vision systems will enter a period of rapid growth.
Camera
The Insight of Intelligent Driving
In-vehicle cameras are fundamental to many ADAS functions, including warning and recognition features. Among these functions, visual image processing systems are relatively basic and more intuitive for drivers, and cameras are the foundation of visual image processing systems. Therefore, in-vehicle cameras are indispensable for intelligent driving.
Many functions, such as Lane Departure Warning (LDW), Forward Collision Warning (FCW), Traffic Sign Recognition (TSR), Lane Keeping Assist (LKA), Pedestrian Collision Warning (PCW), Surround View Parking (SVP), and Driver Fatigue Warning, can be achieved with the help of cameras, and some functions can even only be achieved through cameras.
ADAS functions achievable by cameras
The price of automotive cameras continues to decline, and multiple cameras per vehicle will become a future trend. They are significantly cheaper and easier to implement than sensors such as automotive radar. Tesla's Autopilot 2.0 hardware system includes eight cameras, and multiple cameras per vehicle will likely be a future standard.
Depending on the requirements of different ADAS functions, the installation location of cameras varies. Based on their installation location, cameras can be divided into four categories: front-view, side-view, rear-view, and built-in. To achieve full ADAS functionality in the future, a single vehicle will need to be equipped with at least five cameras. According to estimates from the Gaogong Intelligent Industry Research Institute (GGII), with the increasing penetration rate of ADAS, the camera market size grew from 2 billion yuan in 2016 to 5.8 billion yuan in 2020, representing a compound annual growth rate of 30%.
The price of car cameras continues to decline
The forward-facing camera is the most frequently used, and a single camera can perform multiple functions. Through algorithm development and optimization, a single forward-facing camera can achieve multiple functions, such as driving recording, lane departure warning, forward collision warning, and pedestrian recognition. In the future, it is also expected that more ADAS functions can be achieved through algorithm integration.
Front-view cameras are typically wide-angle lenses and are mounted high on the rearview mirror or windshield to achieve a longer effective distance.
Side-view cameras will increasingly replace rearview mirrors. Because rearview mirrors have a limited range, a vehicle diagonally behind it becomes invisible; this area is called a blind spot. Blind spots significantly increase the likelihood of traffic accidents. Installing side-view cameras on both sides of the vehicle can largely cover these blind spots. When a vehicle enters the blind spot, the system automatically alerts the driver – this is a blind spot monitoring system.
A new trend has emerged: replacing rearview mirrors with side-view wide-angle cameras. This reduces wind resistance and provides a wider field of view, preventing accidents in dangerous blind spots. The BMW i8 Mirrorless concept car uses this design.
blind spots in car rearview mirrors
The panoramic parking system utilizes multiple cameras around the vehicle to provide a "God's-eye view" for parking. By simultaneously capturing images from all sides of the vehicle using multiple ultra-wide-angle cameras installed around the vehicle, the system processes and stitches these images together to create a panoramic overhead view of the vehicle's surroundings, which is then transmitted in real-time to the display on the center console.
The driver can have a "God's-eye view" from inside the car to see the vehicle's location and obstacles around it, allowing for easy maneuvering when parking or navigating complex road surfaces, effectively reducing the occurrence of accidents such as scratches and collisions.
Image stitching technology for panoramic parking systems
Vehicle cameras are widely used and relatively inexpensive, making them the most basic and common type of sensor. Their future market potential is expected to exceed 10 billion RMB.
Cameras are essential for many ADAS functions, and their unit price is expected to continue to decline in the future, driving rapid growth in the automotive camera market. According to estimates, global shipments of automotive cameras will grow from 28 million units in 2014 to 83 million units in 2020, representing a compound annual growth rate of 20%.
Based on this estimate, the global automotive camera market size will grow from RMB 6.2 billion in 2015 to RMB 13.3 billion in 2020, with a compound annual growth rate of 16%. The main consumer regions are the Americas, Europe, and the Asia-Pacific region, with the Asia-Pacific region expected to be the fastest-growing market.
2020 Domestic Demand for Automotive Cameras
The camera industry chain mainly consists of lens assembly, CMOS (Complementary Metal-Oxide Semiconductor), DSP (Digital Signal Processor), module packaging and other links.
In recent years, the rapid growth of smartphones has driven the booming development of the camera market. However, since 2014, the growth rate of smartphones has slowed down, and the future growth rate of mobile phone cameras will inevitably slow down as well. With the rise of the automotive camera market, the production capacity of various links in the mobile phone camera industry chain will shift to the automotive camera industry. It is expected that the CMOS, lens, module packaging and other links in the industry chain will continue to maintain high growth in the future.
radar
An essential sensor for ranging and speed measurement
Radar illuminates a target object by emitting sound or electromagnetic waves and receives its echo, thereby obtaining information such as the target object's distance, rate of change of distance (radial velocity), size, and azimuth. Radar was first used in the military and has since been gradually applied to civilian use.
With the development trend of automotive intelligence, radar has begun to appear in automobiles, mainly for functions such as distance and speed measurement. Automotive radar can be divided into ultrasonic radar, millimeter-wave radar, lidar, etc. Different radars have different principles and performance characteristics, and can be used to achieve different functions.
Basic architecture of radar sensors (only collecting raw data)
(1) Ultrasonic radar
Ultrasonic radar uses an ultrasonic generator within a sensor to produce 40kHz ultrasonic waves, which are then received by a receiving probe after being reflected back from obstacles. The distance to the obstacle is calculated based on the time difference between the reflected and received ultrasonic waves. Ultrasonic radar is relatively inexpensive, has a short detection range, high accuracy, and is unaffected by lighting conditions, making it commonly used in parking systems.
Automatic parking relies heavily on ultrasonic sensors. BMW's latest i-series and 7-series vehicles already support remote automatic parking via the car key. During operation, the user only needs to give two commands: forward or reverse. The car will then continuously use ultrasonic sensors to detect parking spaces and obstacles, automatically operating the steering wheel and brakes to achieve automatic parking.
(2) Millimeter-wave radar: a core sensor for ADAS
Millimeter waves are electromagnetic waves with wavelengths between 1 mm and 10 mm. Converted to frequency, millimeter waves have frequencies between 30 GHz and 300 GHz. Since their wavelengths fall between centimeter waves and light waves, millimeter waves combine the advantages of both microwave and photoelectric guidance.
Millimeter-wave radar has wide applications in missile guidance, target surveillance and acquisition, artillery fire control and tracking, high-speed communication, and satellite remote sensing. In recent years, with the improvement of millimeter-wave radar technology and the reduction of its cost, millimeter-wave radar has begun to be applied in the automotive field.
The key technologies for millimeter-wave radar are mainly controlled by foreign electronics companies. A millimeter-wave radar system mainly includes an antenna, a transceiver module, and a signal processing module, while the MMIC (Monolithic Microwave Integrated Circuit) chip and the antenna PCB board (Printed Circuit Board) are the core hardware components of millimeter-wave radar.
Currently, key technologies for millimeter-wave radar are mainly monopolized by component giants such as Bosch, Continental, Denso, and Autoliv. In particular, the 77GHz product technology is mastered by only a few companies such as Bosch, Continental, Denso, and Delphi.
LiDAR
Powerful features and significantly reduced costs are expected.
LiDAR (Light Detection and Ranging) is a high-precision radar technology that has been adapted for civilian use. Initially, lidar applications were primarily in the military field, attracting significant attention from military departments worldwide. Compared to conventional radar, lidar can provide high-resolution geometric images of radiation intensity, distance, and velocity. In the civilian sector, lidar is also widely used due to its superior performance in range and velocity measurement, 3D modeling, and other areas.
LiDAR boasts superior performance and is considered the optimal technology for autonomous driving. LiDAR offers significantly better performance compared to other autonomous driving sensors:
1) High resolution. LiDAR can achieve extremely high angular, range, and velocity resolution. Typically, the angular resolution of a LiDAR is no less than 0.1 mad, meaning it can distinguish two targets 0.3m apart at a distance of 3km and can track multiple targets simultaneously; the range resolution can reach 0.1m; and the velocity resolution can reach within 10m/s. Such high range and velocity resolution means that LiDAR can use Doppler imaging technology to obtain very clear images.
2) High precision. Lasers propagate in a straight line, have good directionality, and a very narrow beam with very low dispersion, thus lidar has very high precision.
3) Strong resistance to active interference. Unlike microwave and millimeter-wave radars, which are susceptible to the influence of electromagnetic waves that are widely present in nature, there are not many signal sources in nature that can interfere with lidar. Therefore, lidar has a strong resistance to active interference.
LiDAR can be categorized into one-dimensional LiDAR, two-dimensional LiDAR, three-dimensional laser scanners, and 3D LiDAR. One-dimensional LiDAR is mainly used for ranging and velocity measurement, two-dimensional LiDAR is mainly used for contour measurement, object recognition, and area monitoring, while three-dimensional LiDAR can achieve real-time three-dimensional spatial modeling.
Vehicle-mounted 3D LiDAR is typically installed on the roof of the vehicle. It can rotate at high speed to obtain point cloud data of the surrounding space, thereby creating a 3D spatial map of the vehicle's surroundings in real time. At the same time, LiDAR can also measure the distance, speed, acceleration, angular velocity, and other information of other vehicles in the surrounding area in three directions. Combined with GPS maps, this information is used to calculate the vehicle's position. This vast amount of data is transmitted to the ECU for analysis and processing, allowing the vehicle to make quick decisions.
