A vision-sensing-based cricket system uses a camera to capture images of a ball moving on the paddle and identifies the ball's center of mass. The control system calculates the ball's true coordinates on the paddle through coordinate transformation and controls the ball's position. Previously, calibrating the camera's intrinsic and extrinsic parameters was crucial for coordinate transformation. This paper studies the coordinate transformation method in a cricket system.
1 Introduction
The cricket-ball system is a multivariable, nonlinear controlled object, a two-dimensional extension of the ball-and-stick system. As shown in Figure 1, as a two-dimensional mechanical system, the cricket-ball system is commonly used in the study of dynamic systems and in laboratory research on control processes based on classical and modern control theories. Its controlled object is a board with two mutually perpendicular axes of rotation, the purpose of which is to allow a freely rolling ball to maintain balance on the board at a specific position or to roll along a certain trajectory. The rotation of the board around the x-axis and y-axis is driven by two motors. Sensors obtain the ball's position on the board and feed it back to the control system. The control system employs a specific control strategy to control the rotation angle of the board in the x-axis and y-axis directions, thereby controlling the ball's equilibrium position and trajectory.
Figure 1 Figure 2Currently, laboratory cricket systems typically employ two sensing methods to detect the ball's position on the paddle: visual sensing and touchscreen sensing. This project utilizes a combined visual feedback and encoder feedback approach to obtain system feedback and drive control, as shown in Figure 1. A camera is suspended directly above the paddle, continuously capturing images of the ball and the paddle. The control system then processes and analyzes these images to determine the ball's position on the paddle. Real-time image acquisition and processing are crucial in this process, as they affect the system's response speed and control accuracy.
A camera acts as a ball position sensor, capturing images of the ball. The control system's image recognition module identifies the ball's coordinates within the image matrix. To obtain the ball's true position on the board, a coordinate transformation is then performed, incorporating the rotation angle information from the two motors fed back by the encoder. Prior to this, calibrating the camera's intrinsic and extrinsic parameters is crucial for performing the coordinate transformation and obtaining the ball's true position on the board.
2. Camera Calibration
2.1 Camera Model
A vision system should be able to calculate the position, shape, and other geometric information of objects in a 3D environment based on image information acquired by a camera, and thereby identify objects in the environment. The brightness of each point on the image reflects the intensity of light reflected from a point on a spatial object, and the position of that point on the image is related to the geometric position of the corresponding point on the spatial object. The relationships between these positions are determined by the camera's imaging geometry model.
The goal of the vision module in this control system is to obtain the true position of the ball on the ball disk. This paper calibrates the camera's intrinsic and extrinsic parameters by treating the camera as a linear camera model (i.e., a pinhole model).
2.2 Calibration Procedure
(u, v) represent the coordinates in the image coordinate system in pixels, and (x, y) represent the coordinates in the image coordinate system in millimeters, as shown in Figure 1. (Xc, Yc, Zc, 1) are the homogeneous coordinates of the object point in the camera coordinate system, and (Xw, Yw, Zw, 1) are the homogeneous coordinates of the object point in the world coordinate system. The camera coordinate system and the world coordinate system are shown in Figure 2. The specific process is as follows:
(1) Identify the position (ui, vi) of each calibration point in the image matrix based on the collected images. The coordinates (Xwi, Ywi, Zwi) of each calibration point in the world coordinate system are known.
(2) Based on the projection or rotation/translation relationships between the coordinate systems, the coordinate mapping relationship between each calibration point and its homogeneous coordinates (ui, vi, 1) in the image matrix and its corresponding homogeneous coordinates (Xwi, Ywi, Zwi, 1) in the world coordinate system is as follows:
(2-1)
(3) In Equation 2-1, mij is the element of the camera parameter matrix M to be solved. After eliminating Zci in Equation 2-1, the following two linear equations about mij are obtained:
(2-2)
(4) Camera calibration requires n calibration points (n≥6), which can yield 2n linear equations about the M matrix.
(2-3)
As shown in Equation 2-3, the M matrix can be obtained by solving the linear equation system using the least squares method with the coordinates of more than 6 known points in space and their image points (in general calibration work, dozens of known calibration points are identified, so that the number of equations greatly exceeds the number of unknowns, and the least squares method is used to solve the equations to reduce the impact of errors).
Figure 33. Three-dimensional information recovery
After calculating the elements of the M matrix according to Equation 2-3, if the camera's intrinsic and extrinsic parameters are not required, the M matrix does not need to be decomposed. At this point, the camera calibration is complete. As shown in Figure 2, for any point P on the surface of a spatial object, we can easily obtain its coordinates p in the image coordinate system from the M matrix; however, we cannot obtain the actual coordinates of point P on the surface of the spatial object from the image point p through the reverse process. This is because any point P' on the line connecting OP (O is the optical center of the camera) has the same image point P. Therefore, from the position of point p, we can only know that the spatial point P is located at a certain position on the line connecting OP, but we cannot know the depth of point P, thus failing to recover the three-dimensional spatial information from the two-dimensional image information.
In other words, we need additional information to calculate the depth of point P.
3.1 Plane Equation of the Sphere
The ball rolls on the plane of the disk, so its depth information is related to the disk's orientation. Since the ball rolls on the disk, we can approximate it as a point mass, making it a point on the disk's plane. Based on the encoder readings, we can calculate the disk's rotation angles around the x-axis and y-axis at any given time, thus obtaining the disk's rotation matrix relative to its horizontal position, and consequently, the disk's planar equation. The spatial coordinates of the ball's center of mass are the intersection of the line connecting points OP and the disk's planar equation.
Suppose that at any given moment, the rotation angle of the disk about the x-axis is α, the rotation angle about the y-axis is β, and the rotation angle about the z-axis is γ (all angles mentioned here are relative to the horizontal position). Then the rotation matrix of the disk relative to the horizontal position is:
(3-1) If the rotation angle γ of the sphere about the z-axis is 0, then the rotation matrix of the sphere is:
(3-2)
When the disk is in a horizontal position, the equation of the plane is zw=0. According to the rotation matrix of the disk shown in Equation 3-2, the equation of the plane in which the disk rotates by an angle α around the x-axis and by an angle β around the y-axis at any given time is:
(3-3)
3.2 Equation for connecting OP lines
Assuming the current coordinates of the ball's centroid in the image matrix are (u, v), based on the M matrix calculated during the camera calibration process, the equation for the OP connection can be obtained as follows:
(3-4)
Combining equations 3-3 and 3-4 yields a system of linear equations.
(3-5)
Solving the linear equations in Equation 3-5 yields the actual coordinates of the ball in the world coordinate system.
4. Conclusion
This article introduces the steps of camera calibration in a cricket system, and how the cricket system obtains the camera M matrix through the camera calibration process; it also describes how the vision module analyzes the images of the ball captured by the camera to obtain the ball's centroid, calculates the M matrix, and obtains the ball's position in space through coordinate transformation.
