According to the 2021 Pitney Bowes Parcel Shipping Index, 131 billion parcels were shipped globally in 2020. The global pandemic and the booming e-commerce industry have accelerated this trend, and this number is expected to more than double by 2026. With the dramatic increase in online retail, the need for automated logistics, warehousing, and transportation processes has become a top priority.
Packaging measurement, quality inspection, barcode reading, optical character recognition/optical character verification (OCR/OCV), and material handling optimization (currently performed manually by many companies) are key parts of the transportation value chain and could be automated.
“Logistics, warehousing, and shipping organizations are striving to accelerate operations. But speed means accuracy and precision are paramount, as there’s no time to deal with errors. There’s also the issue of staffing,” said Mark Wheeler, Director of Supply Chain Solutions at Zebra Technologies. “When you put these three things together, you get a very open market where you can try new things by combining existing and new technologies in innovative ways.” Much of this innovation revolves around machine vision.
01
Visual guidance robot
In warehouses or distribution centers, pallet loading typically marks the beginning and end of the warehousing process. Once inside the facility, palletized goods are either unloaded from the pallet and placed into individual boxes, or stored as complete pallets. Pallet unloading has shifted from being primarily manual to relying on vision-guided robots. As the robot places an item onto a conveyor, machine vision locates the next package to be picked up, thus accelerating the process.
“Most packages arrive at and leave the warehouse in palletized form, which is central to most machine vision applications in modern warehouses,” said Garrett Place , Business Development, Robotics Perceptions at ifm.
Ben Carey, Senior Manager of Logistics Vision Products at Cognex, said, “In logistics applications, machine vision involves four stages: measurement, inspection, guidance, and identification. These four stages are involved from receiving and sorting to leaving the checkpoint.”
02
Autonomous mobile robots
Ask a machine vision solution developer how to achieve a repeatable approach for a use case, and they might mention issues related to limiting the number of variables. However, most warehousing and logistics operations involve moving packages that can be of any color, size, shape, and material. This variability makes technology selection and solution creation extremely difficult.
Place explained, " Amazon 's Pick Challenge is a perfect example of this over the past few years. This is also the main reason why most machine vision applications in the logistics field use multiple cameras and multiple modes. One camera and one technology are not enough to manage the variability of this type of application."
John Leonard, Zivid's Product Marketing Manager, agreed. He explained, "The primary application is depalletizing and palletizing boxes entering and leaving the facility. Between these in/out operations are mainly single-item picking and order picking to fulfill orders. There are many different ways to accomplish these tasks."
These methods include autonomous mobile robots (AMRs) guided by onboard 3D vision. For example, an AMR can autonomously move to cabinets to locate and select items. Robots can also pick up items transported by conveyors. Other mobile robots can transport items to vision stations for inspection of the type and quantity of goods.
03
Automated Guided Vehicles
Alternatively, for storage of fully loaded pallets, many warehouses deploy Automated Guided Vehicles (AGVs) to pick up and store pallets for retrieval. During operation, the AGVs rely on machine vision for pallet orientation and obstacle detection . Throughout the process, machine vision code reads and tracks the load on the pallets and boxes.
As fully loaded pallets prepare to leave the facility, AGVs manage the movement while robotic arms convert boxed goods into fully loaded pallets. Weighing and measuring these pallets before they enter the trucks makes pallet sizing another powerful use case for logistics machine vision.
Daniel Howe, Americas Regional Development Manager at LMI Technologies , stated, “The industry has shifted from strictly weight-based freight rates to size-weight-based rates, making accurate dimensional measurements more important than ever. Smart 3D sensors are a key driver of automation in packaging and logistics processes, including volumetric dimensions, specifications, sorting, and surface defect detection.”
Many AMRs and AGVs rely on the ifm O3R platform for robot perception. It consists of a small camera head (VGA camera and runtime sensor) and a vision processing unit (VPU) with an NVIDIA Jetson TX2 processor for data evaluation. Up to six cameras can be connected to Linux -based devices, including sensors from other companies.
04
The need to increase speed and throughput
While numerous challenges exist in logistics and warehousing applications, the need for higher speeds and throughput remains constant. Challenges include items packaged in transparent plastic bags, which present imaging challenges due to their reflective properties. Other workpiece pick-up operations may require color to be integrated into the material inspection process, potentially necessitating 3D vision support for recognizing color information in images.
Calibrating all 3D cameras is a huge challenge because they are designed to operate within the micrometer range, but the knocking noises, temperature fluctuations, and vibrations common in industrial environments can easily affect calibration, thus impacting the accuracy of the 3D camera. Leonard stated, "Some cameras (such as the Zivid 3D camera) are specifically designed and manufactured for industrial environments, with an IP65 protection rating and automatic calibration. This means that if there is a 5-degree temperature change due to the opening and closing of a large roller shutter door (which is common in logistics warehouses), the camera will adjust to maintain perfect calibration."
05
Box volume and void filling
LMI has developed the ultra-wide field-of-view (FOV) Gocator 2490 sensor, designed to provide fast and accurate package size measurements for transport. For example, it can be used to measure the box to provide precise volume for determining its size and weight. The box may be moving on a conveyor at a speed of 2 m/s. According to Howe, a single wide-field-of-view Gocator 2490 smart sensor can scan and measure the full box size of 1 m x 1 m within the scanned area at a rate of 800 Hz, providing 2.5 mm resolution in all three dimensions (X, Y, Z).
“Competitor-based camera systems typically offer only 3 to 5 millimeters of resolution along the X, Y, and Z axes. However, each sensor has a different measurement range and resolution, so you must choose the right sensor for your application,” Howe said. The Gocator 2490 has high enough resolution to measure not only dimensions of various sizes but also detect minute defects in packaging. If a defective package is detected, this online inspection function can trigger a pass/fail decision.
The Gocator 2490 also offers opportunities for more advanced packaging applications, such as gap filling, which involves scanning the items inside an opened package and determining how much packaging material is needed to fill empty spaces. For this application, the dual-camera configuration helps avoid obstructions from inside boxes or suitcases.
06
Edge deep learning
Machine vision faces challenges in the logistics field due to the increasing complexity of applications. For example, attempting to randomly detect objects of different types and sizes on a high-speed conveyor. In such cases, traditional rule-based machine vision inspection/inspection will struggle.
However, easy-to-use machine learning (ML) and deep learning (DL) technologies are emerging in embedded platforms to help users solve previously challenging applications. For example, Cognex recently launched the In-Sight 2800 vision system with edge learning capabilities, which can be easily set up without programming . According to Carey, the In-Sight 2800 can quickly and accurately classify all kinds of items, from boxes and handbags to plastic bags, all done on a smart camera.
By incorporating advanced technologies such as edge learning, the In-Sight 2800 improves package inspection rates, reduces manual rework, and processes orders more accurately through more advanced material handling automation. “Our customers are benefiting from increased processing speed and less human interaction, enabling these companies to cope with labor shortages and manage fluctuating demand without changing their headcount,” Carey said.
07
Democratization of Machine Vision
Most of the technologies deployed in modern warehouses, including 2D and 3D cameras and enhanced computing power, are iterations of previous approaches. Even where there are differences, they all involve applying these technologies to a multi-frame, multi-modal strategy with powerful processing capabilities, combined with machine learning to manage the application.
“In the past, we often saw single-vendor solutions in warehouses,” Place explained. “Now, it’s a combination of multiple vendors and technologies, leveraging their respective strengths to jointly address this challenge. This approach will continue to unlock use cases previously untouched by machine vision. You can think of it as the democratization of machine vision in warehousing and logistics.”
It's difficult to categorize any single technological advancement that has opened up new use cases for machine vision in warehousing and logistics. While cameras are providing better, more repeatable data and computation is faster, they haven't been game-changers. The biggest advancement may be that, within a multi-technology approach, components are easier to use in solving warehouse problems.
“Logistics is moving towards robotics as a primary method for managing the industry’s massive growth,” Place concluded. “Robotics is an integration problem. Machine vision and all its complexities are moving from a single camera to reducing the component integration issues required in modern warehouses. This approach is taking us to the next stage of our journey.”
Key concepts:
Manufacturers are ramping up automation to meet surging demand and cope with a growing labor shortage.
■ AMR and AGV use machine vision and sensing technologies to move freely.
■ Machine learning and deep learning in embedded platforms are being used to solve applications that were once very challenging.
