Robot vision solutions represent one of the biggest challenges in achieving robot vision. Even as they become increasingly user-friendly, some thorny issues remain. Many factors influence a robot's vision within its environment, task setup, and workplace. Here are nine summarized challenges in robot vision:
illumination
Anyone with experience shooting digital photos in low light knows that lighting is crucial. Poor lighting can ruin everything. Imaging sensors are not as adaptable or sensitive as the human eye. If the type of lighting is wrong, the visual sensor will not be able to reliably detect objects.
There are various ways to overcome lighting challenges. One approach is to incorporate active lighting into the vision sensor itself. Other solutions include using infrared illumination, fixed lighting in the environment, or techniques that use other forms of light, such as lasers.
Deformation or hinge
A sphere is a simple object detected using computer vision. You might just detect its circular outline, perhaps using a template matching algorithm. However, if the sphere is flattened, it changes shape, and the same method will no longer work. This is deformation. It can cause considerable problems for some robotic vision technologies.
Similar to articulation, this refers to deformation caused by movable joints. For example, when you bend your arm at the elbow, the shape of your arm changes. The individual links (bones) maintain the same shape, but the outline is deformed. Because many vision algorithms use shape outlines, sharpness makes object recognition more difficult.
Job and direction
The most common function of robot vision systems is to detect the position and orientation of known objects. Therefore, most integrated vision solutions typically overcome the challenges of both.
Detecting an object's position is usually straightforward, provided the entire object is visible within the camera's image. Many systems are also robust to changes in object orientation. However, not all orientations are equal. While detecting an object rotating along a single axis is simple enough, detecting when an object is 3D rotated is far more complex.
background
The background of an image has a significant impact on the ease of object detection. Imagine an extreme example where an object is placed on a piece of paper, and an image of the same object is printed on that paper. In this case, the robot's vision setup may not be able to determine which is the real object.
A perfect background is blank and provides good contrast with the detected object. Its exact properties will depend on the visual detection algorithm being used. If an edge detector is used, the background should not contain sharp lines. The color and brightness of the background should also differ from the color and brightness of the object.
Blockage
Occlusion means that part of an object is hidden. In the previous four challenges, the entire object appeared in the camera image. Occlusion is different because part of the object is missing. The visual system obviously cannot detect something that is not present in the image.
Various things can cause occlusion, including other objects, parts of a robot, or poor camera positioning. Overcoming occlusion typically involves matching the visible parts of an object to its known model and assuming the existence of hidden parts of the object.
Proportion
In some situations, the human eye can be easily fooled by differences in scale. Robotic vision systems can also be confused by them. Imagine you have two identical objects, one larger than the other. Imagine you're using a fixed 2D vision setup where the object's size determines its distance from the robot. If you train the system to recognize the smaller object, it might incorrectly detect that the two objects are identical and that the larger object is closer to the camera.
Another issue with scale, perhaps less obvious, is the pixel count. If the robot's camera is placed far away, objects in the image will be represented by fewer pixels. Image processing algorithms generally work better when more pixels represent objects, but there are exceptions.
Camera placement
Incorrect camera positioning can cause any problems that have occurred before, so it's important to use it correctly. Try to place the camera in a well-lit area so that you can see the object as clearly as possible without distortion, and get as close to the object as possible without obstructing it. There should be no interfering background or other objects between the camera and the viewing surface.
sports
Movement can sometimes cause problems in computer vision setups, especially when there is blur in the image. For example, this might happen with an object on a fast-moving conveyor belt. Digital imaging sensors capture images for short periods, but not instantly. If an object moves too quickly during the capture process, it will cause the image to blur. Our eyes might not notice the blur in the video, but the algorithm will. Robot vision works best when there is a clear, static image.
expect
Compared to the technical aspects of vision algorithms, the final challenge involves more about your vision setup methodology. One of the biggest challenges in robotic vision is the unrealistic expectations that staff have of what the vision system can provide. You will get the most out of the technology by ensuring that expectations match its capabilities. You can achieve this by ensuring that staff are educated about vision systems.
