1. Overview of N-point calibration
When a mechanism platform is used with a camera, whether the camera is fixed outside the mechanism platform or installed at the end of the mechanism, hand-eye calibration must be performed first. The most common hand-eye calibration is N-point calibration, which aims to obtain the camera extrinsic parameter matrix. By recording N sets of pixel coordinate points and physical coordinate points, the transformation matrix T from the image coordinate system A to the physical coordinate system B of the actuator is calculated. This matrix is the calibration file generated by N-point calibration.
The essence of N-point calibration is to unify the coordinate system, transforming points in the image coordinate system to the physical coordinate system, and then performing numerical processing in the physical coordinate system to guide the actuator to perform operations. It is generally used in XYZ platforms, XYθ platforms, Scara robots, six-degree-of-freedom robots, etc.
2. N-point calibration principle
From a mathematical perspective, a two-dimensional coordinate system xoy can be transformed into another coordinate system x'o'y' through translation, rotation, and scaling.
The transformation relationship is expressed as follows:
The matrix above describes the relationship between two coordinate systems, where fx and fy are the scaling factors in the x and y directions, respectively, a and b are the translation factors in the x and y directions, respectively, and θ is the rotation angle of the coordinate system. The matrix has 6 degrees of freedom, requiring at least 6 equations, or 3 sets of points, to solve. If more than 3 sets of points are needed, nonlinear least squares optimization is used, with 9 sets of points being the most common approach.
LeadVision's vision platform N-point calibration includes 9-point calibration and 16-point calibration (where the last 7 points are rotation points). 16-point calibration is a combination of 9-point translation calibration and 7-point rotation calibration, and is mostly used to handle situations where the gripping point is not at the center of rotation.
The essence of 7-point rotation calibration is to solve for the coordinates of the rotation center, providing two methods: the fitted circle method and the trigonometric function method. The fitted circle method involves rotating the point seven times, recording the same Mark point at seven different positions, and then finding a circle passing through them, the center of which is the rotation center.
The trigonometric function method involves rotating the image seven times, recording the same Mark point at three different positions and the same Mark point at four repeated positions, and calculating the rotation center corresponding to the three Mark points. The last four Mark points are used for verification, as shown in the figure below. Generally, the fitted circle method is used for scenes with a large field of view, while the trigonometric function method is used for scenes with a small field of view.
Note: When rotating the 7 o'clock position, keep the XY axis stationary, only move the rotation axis, and record the rotation angle.
3. N-point calibration process
N-point calibration, combined with communication functions, enables one-click automatic calibration, which involves four steps:
(1) Move to the calibration start position: The mechanism moves to the photo position, as shown in Figure 1 above (the same as position 10 in the following figure), and picks up the calibration object (which can be the product or the calibration block).
(2) XY translation calibration: The mechanism moves to the 9 positions shown in the figure above in sequence according to the set offset value, and then sends the photo command, point number and corresponding mechanism coordinates to the LeadVision vision platform. The format can be customized as "calibrate Cal, point number is 1~9, X coordinate, Y coordinate, angle", such as "Cal,1,10,10,2", to obtain 9 sets of mechanism platform coordinates and pixel coordinates of calibration objects.
(3) Rotation calibration: The mechanism moves to the 7 positions shown in the figure above in sequence (at this time, only the R axis is rotated), and then sends the photo command, point number and corresponding mechanism coordinates to the LeadVision vision platform in sequence. The format is the same as step (2), and the point number is 10~16. The 7 sets of mechanism platform coordinates and the pixel coordinates of the calibration object are obtained.
(4) Generate calibration file: Click the "Calculation Results" button in the LeadVision vision platform calibration tool to generate calibration file.
4. Implementation on the LeadVision platform
01. Objective
The absolute coordinate transformation relationship from the camera coordinate system to the Scara robot end effector coordinate system is achieved through 16-point calibration.
02. Creating a Project
(1) New construction projects
Open the LeadVision software, select "Project" → "New Project" in the menu bar, and a New Project dialog box will pop up. Enter the number of cameras, the number of task flows, the project name, and the save path, as shown in the figure below.
(2) Camera configuration
In the menu bar, select "Camera" → "Camera Configuration" to open the camera settings dialog box. In this example, the image source is the local image folder (if a camera is actually connected, check the camera here); enter the relative path (./samples/images/calibration) in the image source path field. Of course, you can also open the file dialog box and select the absolute path where the images are stored; leave the other parameters at their default settings, and finally click "Save Settings", as shown in the figure below.
Note: When the image source path is a relative path, you need to press Enter after entering the path to apply the changes.
(3) Set the reference image
The baseline image is used to configure the parameters of subsequent tools, and a baseline image needs to be set for each new project. Select "Image" → "Baseline Image" from the menu bar to open the baseline image settings dialog box. Click "Open" to select a baseline image, and then modify the baseline image's name (e.g., Baseline) for easy personal identification.
03. Setup Process
The calibration process requires setting up the process in the calibration interface. Click "Enter Coordinate System Calibration Interface" in the toolbar.
The N-point calibration process built in the calibration interface is shown in the following figure. The following text focuses on explaining the TCP communication in the process and the configuration of the N-point calibration tool.
(1) TCP communication reception
"TCP Communication Receive" can retrieve data from a communication device. Before using this tool, you need to add a communication device. You can add a communication device in the menu bar under "Communication" → "Communication Settings". Here we add a TCP server and configure it as shown below.
After adding the communication device, begin configuring the "TCP Communication Reception" parameters. In this example, the data format is configured as "Cal, Id, X coordinate, Y coordinate, Angle". When the command "Cal" is matched, the subsequent process is executed. Detailed parameters are as follows:
(2) Image acquisition
Use the tool with the default parameters.
(3) Contour matching
Contour matching parameters were explained in detail in the first lesson of LeadVision, and will not be elaborated upon here. Select the center ROI to outline the central region of the reference image, create a template, and the result is as follows:
(4) N-point calibration
In this example, the camera is fixed outside the Scara robotic arm platform. Therefore, in the Basic Settings tab, configure the platform type as "XYθ", the camera mounting method as "fixed", the rotation center calibration method as "trigonometric function method to find the rotation center", the calibration file name as "Calib106", the coordinate point number as "ID" received by TCP communication, the image coordinates X/Y as "cx/cy" of the contour matching tool, and the platform coordinates X/Y/T as "x/y/theta" received by TCP communication, as shown below.
04. Results
The coordinates of the 16 platforms and the data sent in this example are as follows, where the translation distance in the X direction is 15mm, the translation distance in the Y direction is 15mm, and the end rotation is 15 degrees.
Click the ① running button to run the calibration program. Use the TCP/UDP testing tool to connect to the LeadVision TCP server 127.0.0.1, port number 60000. After a successful connection, send platform coordinate data of 16 points in sequence to simulate platform operation, such as "Cal,2,221.6,-305.8,-97.8", to trigger the photo taking.
After all 16 platform coordinates have been completed, click the "running" button again to stop the calibration program. Double-click the "N-point calibration" tool, select the coordinate point list tab, and view the calibration data. Click ① Update image coordinate points → ② Display coordinate points to observe the distribution of the 9-point data.
Switch to the Processing Results tab, click ① "Calculate Results" → ② "Save Results" to obtain the final calibration matrix file Calib106. At this point, the N-point calibration is complete.
END
