The number of cameras installed in passenger vehicles has surged, with some luxury models even equipped with more than a dozen. Automakers face challenges in adding more sensors to improve safety while also considering the economic cost and space required for each camera. As a result, automakers have begun searching for solutions that use a single camera to capture images optimized for both human and machine vision. However, the image quality required for human and machine vision differs, necessitating trade-offs, making this approach equally difficult to implement.
Human vision
The human visual system perceives differences in brightness between pixels differently than machine vision algorithms. Human brightness perception is non-linear; that is, if the number of photons in the environment doubles, the perceived brightness only doubles. This necessitates adjusting the camera image used for human vision to correspond to its dynamic range, thus fully amplifying the details in both bright and dark areas perceived by the human eye. Furthermore, we are highly sensitive to general colors and the flicker of LED light sources (a problem that is becoming increasingly prevalent). Therefore, if the camera causes color distortion, even if the image is clear and otherwise of high quality, it will affect the human visual experience. For passive safety systems like rearview cameras, drivers also have an advantage over machine vision systems because they can automatically detect image defects without relying on the camera. While this won't cause a major safety incident, the camera's inconvenience is unavoidable; therefore, drivers rely more on proactive judgment rather than camera images.
Machine vision
Unlike human vision, automated systems using machine vision examine the digital value of each pixel in an image, thus responding linearly to the number of photons. Unlike images used for human vision, these must be adjusted to output an image corresponding to the measured pixel values. Furthermore, machine vision systems must be programmed or employ specialized error detection hardware to detect image defects. Systems lacking this hardware may malfunction and fail to inform the driver that their functionality is impaired or inoperable. For active safety systems like automatic emergency braking, a false alarm could cause the system to brake even when there is no collision risk, while a missed alarm could cause the system to completely fail in the event of a collision, resulting in serious consequences. If a driver uses such an assistance system, a message needs to be displayed to indicate its malfunction, but a warning of impaired functionality may not be possible. Some systems will alert the driver to impaired or "unavailable" functionality; these typically rely on specialized hardware to detect errors or malfunctions in sensors. Such functionality must comply with relevant industry standards such as Automotive Safety Integrity Level (ASIL). ASIL-compliant sensors will have the ability to detect and report malfunctions, improving safety. These are the two reasons why sensors used for machine vision and sensors used for human vision need to be configured differently.
Sensor solutions that enable observation and perception using a single camera
The good news is that some sensors already possess excellent capabilities for both human and machine vision, and can be optimized to output two simultaneous data streams. This helps engineers design camera systems that can be used for both human and machine vision functions. As a result, automakers only need to deploy a single camera in a specific location within the vehicle, minimizing space requirements and reducing system costs, while obtaining images optimized for both workloads.
