Workpiece sorting is an important part of industrial production. Its purpose is to classify and place different types of materials or workpieces in their corresponding positions to avoid mixing of different materials or workpieces. The process is mainly divided into four stages: positioning, identification, grabbing, and placement.
Traditional sorting is generally done manually. However, workers are prone to fatigue, and the efficiency is obviously not high enough for long-term operations.
Even when using industrial robots for sorting, traditional industrial robots typically work by teaching or offline programming, so the robot's position at the target point is fixed, and each action is repeated, meaning it can only repeat pre-defined actions.
The limited ability to perceive the processing objects and working environment, as well as the uncertain direction of incoming materials or workpieces, restricts the use of robots. However, with the development of machine vision and artificial intelligence, robots can adapt to the needs of the sorting environment and change the work objects and sorting processes at any time.
Machine vision sorting systems are essential for determining the position and type of workpieces or materials. Compared with traditional mechanical sorting operations, industrial robots using integrated machine vision motion control sorting systems with forward motion technology are not only highly efficient and accurate but also highly stable, giving them a significant advantage in industrial sorting systems.
Case study video of machine vision-based integrated sorting system
In the last lesson, we introduced how to implement multi-contour matching function in the VPLC series machine vision motion control all-in-one machine. In this lesson, we will share with you how to implement the part sorting function.
Instructional Videos
Introduction to the ladder diagram of the detection principle
(I) Testing Requirements
There are several circular parts, slightly larger hexagonal nut parts, and slightly smaller hexagonal nut parts at the same workstation. The task is to detect the quantity of parts of different shapes and send the corresponding part positions to the robotic arm for grasping.
(II) Software Algorithm
First, the image is binarized. Then, the black BLOB connected regions are calculated. After storing the BLOB region results in a list, the BLOB result data is analyzed and compared.
First, sort the area results in descending order and display the corresponding numbers in the center of the BLOB results. Then, classify the BLOBs according to the area range of different shaped parts and count the corresponding quantities. Finally, print the center position of the BLOB in the command and output window.
Introduction to Software Implementation of Ladder Diagrams
(I) Software Implementation
1. Open ZDevelop software: Create a new project named "sorting.zpj" → Create a new "HMI" file → Create a new "main.bas" file to write the interface response function → Create a new "global_variable.bas" file to store global variables and enable HMI automatic task execution → Create a new "detectParam.bas" file to initialize measurement parameters → Create a new "camera.bas" file to implement camera acquisition function → Add the files to the project.
2. Design the HMI interface.
3. Define global variables in the "global_variable.bas" file, and then run the "Hmi.hmi" file after the definition is complete.
'Global variables mostly use array structures.'''''''Note: In BASIC programming, many functions use TABLE (system data structure) as parameters.''table Description '0 ' Automatic binarization threshold 4 Area of dark regions''2 Area of bright regions 6 Number of connected components''20~24 Image information'Main task status'0 - Uninitialized'1 - Stopped'2 - Running'3 - Stopping GLOBAL DIM main_task_state main_task_state = 1
'Acquisition Switch' 0 - Stop acquisition '1' - Request acquisition of GLOBAL DIM grab_switchgrab_switch = 0
Number of cameras: GLOBAL cam_num cam_num = 0
'Camera type,' zmotion; mvision; basler; mindvision; huaray" GLOBAL DIM CAMERA_TYPE(16) CAMERA_TYPE = "mvision"
Define the main task ID - 10GLOBAL DIM main_task_id main_task_id = 10
Define the continuous data acquisition task ID - 9GLOBAL DIM grab_task_id grab_task_id = 9
'Define global image variable GLOBAL ZVOBJECT grabImg 'Capture image GLOBAL ZVOBJECT binImg 'Capture image GLOBAL ZVOBJECT disImg 'Display image
Define commonly used color variables: GLOBAL C_RED, C_GREEN, C_BLUE, C_YELLOW C_RED = RGB(255, 0, 0) C_GREEN = RGB( 0,255, 0) C_BLUE = RGB( 0, 0,255) C_YELLOW = RGB(255,255, 0)
'Detection parameters: threshold mode (automatic threshold or manual threshold), low threshold, high threshold, minimum area, maximum area GLOBAL DIM d_detect_param(5) 'The 'd' prefix indicates the data structure
Display the printed characters GLOBAL ShowString(64)
Define a result variable array, including the number of washers, the number of large nuts, and the number of small nuts. GLOBAL DIM d_num_rst(3)
'Define the detection time GLOBAL DIM d_detect_time
'***********Defines variables related to the local file reading function**************''Note: This function is only valid when using the emulator.'Defines whether to use the local image flag GLOBAL DIM d_use_imgfile
Define the local image index GLOBAL DIM d_index
Define the path to read the image: GLOBAL DIM File_Name(100)
'***********End of definition of variables related to reading local files**********'
Run the HMI file "Hmi.hmi", 1
4. Initialize the measurement parameters in the "detectParam.bas" file.
end
GLOBAL SUB init_detect_param() 'Initialize measurement parameters' Initialize detection parameters: threshold mode (automatic threshold = 1 or manual threshold = 0), low threshold, high threshold, polarity (black or white), maximum, minimum, inverse (i.e., invert the result, success becomes failure, failure becomes success) d_detect_param(0) = 0 'Manual threshold d_detect_param(1) = 200 'Low threshold d_detect_param(2) = 255 'High threshold d_detect_param(3) = 20000 'Minimum area, i.e., number of pixels d_detect_param(4) = 50000 'Maximum area for i=0 to 2 d_num_rst(i)=0 next d_use_imgfile = 1 'Use local image by default d_index = 0 END SUB
5. Associate HMI interface control variables.
6. Add an HMI interface initialization function to the "main.bas" file and associate the initialization function in the HMI system settings.
end
Note: 'When using Region-related operators, the ZV_RESETCLIPSIZE(width, height) operator must be called during system initialization to set the image size to match the camera resolution, as the default size is 640*480.'
'HMI interface initialization function GLOBAL SUB hmi_init()
ZV_RESETCLIPSIZE(2448, 2048) 'Sets the cropping size of the image region according to the image resolution. Here, the image resolution is 2448x2048. ZV_LATCHSETSIZE(0, HMI_CONTROLSIZEX(10, 7), HMI_CONTROLSIZEY(10, 7)) 'Sets the latch size. init_detect_param() 'Initializes the measurement parameters. ZV_SETSYSDBL("CamGetTimeout", 1000) 'Sets the acquisition timeout. ZV_LATCHCLEAR(0) 'Clears latch channel 0. END SUB
7. Add functions to the "camera.bas" file to capture the responses of relevant buttons in the HMI interface and associate them with action functions. (Note: The specific implementation of these functions has already been demonstrated in the previous two lessons and will not be repeated here.)
8. Add a function to test the button response in the HMI interface and associate it with the action function in the "main.bas" file.
The function GLOBAL SUB btn_test() that responds when the test button is pressed on the HMI interface.
TICKS=0 for i=0 to 2 d_num_rst(i)=0 next ZVOBJECT regionWhite, regionMask, regionBlack,re_connecte,circle_connect dim loop_num,bigNut_num,smlNut_num loop_num=0 bigNut_num=0 smlNut_num=0 ZV_REGENFULLIMG(grabImg,regionMask)'Generate the region covering the entire image into the re region variable ZV_DILATE (grabImg,binImg,15,15) 'Dilute the image to separate slightly stuck parts' Binarize if d_detect_param(0) = 0 then 'If manual thresholding mode is selected' Generate white pixel image regionWhite based on low threshold and high threshold parameters ZV_RETHRESH(binImg, regionMask, regionWhite, d_detect_param(1), d_detect_param(2)) 'Perform a 1*1 closing operation on the white pixel region ZV_RECLOSING(regionWhite,regionWhite,7,7) else 'If automatic thresholding mode is selected Dim autoThresh 'Define the binarization threshold in automatic thresholding mode'Perform automatic binarization processing on the specified region in the grabImg image and output the binarized region regionWhite ZV_REAUTOTHRESH(binImg, regionMask, regionWhite, 0) 'Perform a 1*1 closing operation on the white pixel region ZV_RECLOSING(regionWhite,regionWhite,7,7) autoThresh = TABLE(0) ? "autoThresh = " autoThresh 'Print the prompt message, the current binarization threshold endif ZV_REDIFF (regionMask, regionWhite, regionBlack) 'Perform a difference operation on the regionMask region (detection region) and regionWhite (white pixel region), the result is regionBlack (black pixel region) ZV_REAREA(regionBlack, 4) 'Calculate the area (i.e., number of pixels) of the black pixel region and store it in table(4) if(TABLE(4) >0) then 'If the number of black pixels obtained is greater than 0 ZV_RECONNECT(regionBlack,re_connecte) 'Calculate the connected regions of the region and store them in the re_connecte list zv_refilter(re_connecte,0,d_detect_param(3),d_detect_param(4),0)'Filter the regions in the region list zv_refilter(re_connecte,20,0.8,1.2,0) ZV_LISTCOUNT(re_connecte,6) 'Get the number of connected regions in the list and store them in table(6) else TABLE(6)=0 endif 'Number of products' TABLE(6) 'Print prompt information' Draw the effect diagram Dim width, height ZV_IMGINFO (binImg, 20)'Get the image information of grabImg width = TABLE(20) height = TABLE(21) ZV_GRAYTORGB(binImg,disImg) 'Converts the grayscale image to an RGB image for drawing the detection result image. ZV_REGION(disImg, regionMask, 0, ZV_COLOR(0,0,0)) 'Draws the black region regionMask in disImg. ZV_REGION(disImg, regionWhite, 0, ZV_COLOR(255,255,255)) 'Draws the white region regionWhite in disImg. ZV_RESORT(re_connecte,0,0) 'Sorts the BLOB results in descending order of area. for i=0 to TABLE(6)-1 ZV_LISTGET(re_connecte,circle_connect,i) 'Gets the element with index i in the list, i.e., sequentially gets the connected regions of the parts in the list. ZV_REAREACENTER(circle_connect,40) 'Calculate the area and center position of each part and put the data into TABLE(40) ShowString=TOSTR(i+1,1,0) 'Convert the number of BLOBs into a string variable if (TABLE(40)>47000) AND (TABLE(40)<50000) then 'Filter out washers loop_num=loop_num+1 ?"Serial number "ShowString" is a ring: position is "TABLE(41),TABLE(42) endif if (TABLE(40)>33000) AND (TABLE(40)<36000) then 'Filter out large nuts bigNut_num=bigNut_num+1 ?"Serial number "ShowString" is a large nut: position is "TABLE(41),TABLE(42) endif if (TABLE(40)>27000) AND (TABLE(40)<29000) then 'Filter out small nuts smlNut_num=smlNut_num+1 ?"Serial number"ShowString" is a small nut: position is "TABLE(41),TABLE(42) endif ZV_TEXT(disImg,ShowString,TABLE(41),TABLE(42),140,ZV_COLOR(255,131,6)) 'Display result text'?i+1,"Area"TABLE(40) next d_num_rst(0)=loop_num d_num_rst(1)=bigNut_num d_num_rst(2)=smlNut_num d_detect_time=ABS(TICKS) ZV_LATCH(disImg, 0) 'Display the result image in latch channel 0END SUB
9. Add the function that responds to the buttons in the HMI interface and associate it with the action function in the "main.bas" file.
'When the run button is pressed on the HMI interface, the function GLOBAL SUB btn_run() if(2 = main_task_state) then 'If the main task is running, print the message "Continuous running task has been started, please do not repeat the operation!" return endif if (1 = main_task_state) then 'If the main task is stopped if (0 = PROC_STATUS(main_task_id)) then 'If the task is not started main_task_state = 2 'The main task status is set to 2, indicating that continuous task RUNTASK main_task_id, main_task 'Start the main task endif endifEND SUB
'Main task implementation function main_task: while(1) if (3 = main_task_state) then 'If the main task state is 3, i.e., when the stop button is pressed, main_task_state = 1 'Set the main task state to 1 exit while 'Exit the loop endif delay (500) 'Repeatedly execute the acquisition and detection function if (d_use_imgfile=1) then if(d_index=3) then d_index=0 endif File_Name="sorting/"+TOSTR(d_index,1,0)+".bmp" 'When reading local images, the path name of the image is ZV_IMGREAD(grabImg,File_Name,1) d_index=d_index+1 else if cam_num = 0 then 'Please scan the camera first!' return endif CAM_SETPARAM("TriggerSoftware", 0) 'Send the trigger command CAM_GET(grabImg, 0) 'Get a frame image and store it in the grabImg variable endif btn_test() wendEND
10. Add a function to stop the HMI interface button from responding in the "main.bas" file and associate it with an action function.
The function GLOBAL SUB btn_stop() that responds when the stop button is pressed on the HMI interface.
if (2 = main_task_state) then 'If the main task state is 3, i.e., continuously executing tasks, main_task_state = 3 'Set the main task state to 3 and exit the loop endif END SUB
Three-operation demonstration ladder diagram introduction
(I) Operating Procedures
To check the running effect: Copy the "sorting" image used in this course to the flash folder project in the ZDevelop software directory → then download the project to the simulator → use local image → single acquisition → select manual threshold mode → set the threshold range → set the area range of the black pixel region to be filtered → click test to check the single run effect → click run to check the continuous run effect → end.
(II) Effect Demonstration
This concludes our presentation on the application routine (8) of the Zhengdong Technology Vision Motion Control All-in-One Machine: Parts Sorting System.
This article is original content from Zheng Motion Technology. We welcome everyone to reprint it for mutual learning and to jointly improve China's intelligent manufacturing level. Copyright belongs to Zheng Motion Technology. Please indicate the source if you reprint this article.
