Abstract: This paper proposes a target tracking algorithm based on planar projection invariants. The algorithm extracts straight line edges from the image and calculates projection invariants for target modeling and tracking. To extract straight line edges, an improved sequence thinning algorithm is used to thin the edges to a single pixel width. Then, a fast curvature estimation method is used to estimate the curvature of the edge points, and points with very small (approximately zero) estimated values ​​are used to fit straight lines. Lines are selected from the resulting family of lines according to the nearest neighbor rule or the window rule to calculate projection invariants. Image processing experiments show the effect of obtaining straight line edges using the proposed image preprocessing algorithm, and the stability and viewpoint invariance of the obtained invariants are measured by using the values ​​of the invariants calculated from the obtained lines. Tracking experiments verify the robustness and practicality of the tracking algorithm. Keywords: Projection invariants; Target tracking; Visual servoing 1 Introduction Target tracking in real-time continuous image sequences is a very important problem in machine vision, especially in the field of visual servoing. In fact, a robust and real-time visual feature extraction and tracking process is key to the success of visual servoing tasks. Currently, tracking algorithms can be broadly categorized into three types: 2D geometric primitive-based, 2D template (region)-based, and model-based (2D, 3D) target tracking algorithms. The first type uses 2D geometric primitives, such as dots, feature points, straight line edges, and object outlines. These algorithms primarily use grayscale gradients to extract features. This type of algorithm is most widely used in visual servoing because the tracked feature parameters can be directly used for visual servoing control, and it is simple and fast. However, if the target's shape and texture are too complex to be described by geometric primitives, or if the background is extremely complex, this algorithm is not suitable. The second type, 2D template-based methods, do not use gradients and do not require feature extraction; they directly use the image's brightness information for 2D template matching. The goal of this algorithm is to estimate a set of parameters through correlation matching to describe the deformation and displacement of the template in the image sequence; therefore, this type of technique is also called image feature motion estimation. The image feature motion model may be an affine model or a perspective projection model. This type of algorithm can track complex targets and adapt to complex backgrounds, but it has high algorithm complexity and requires manual initialization of the tracking process. The third type of method attempts to establish a 2D or 3D model of the target, thereby estimating the motion of any camera or rigid target. This algorithm can overcome partial occlusion of the target in the image and can be applied to any visual servoing algorithm (2D, 3D, and 2.5D). All three methods have successful application cases. While the latter two algorithms can be used in complex environments, neither effectively solves the problem of automatic initialization; the initial position needs to be manually set. Requiring manual assistance to specify the initial position for each task execution is unreasonable for autonomous intelligent robots. Given that the surface features of man-made objects can often be described by planar straight lines and quadratic curves, this paper attempts to apply planar projection invariants to achieve feature tracking of planar targets, and more importantly, to automatically complete the initialization and tracking process. Projection invariants can be generated using points, straight lines, quadratic curves, or combinations thereof. The positions of feature points extracted from images are often inaccurate, resulting in low accuracy of the invariants calculated using them; the calculation error using quadratic curve invariants is also relatively large. Straight lines have many excellent properties: their perspective projection remains a straight line under all circumstances, they can be expressed by simple equations, and the fitting accuracy of straight lines is easy to measure, etc. This paper uses straight lines to construct invariants. [Click for details]