Changes in joint angles can effectively reflect the main characteristics of human movement. This paper proposes a gait recognition method based on joint angle information. First, the legs of the moving human are modeled, and the boundary is fitted using the least squares method to obtain the temporal information of the thigh and calf joint angles. Based on the quasi-periodicity of gait movement, the temporal information of joint angles is expanded in the form of a Fourier series. A genetic algorithm is used to search for the coefficients of each harmonic and perform scaling transformation to generate feature vectors. Finally, a KNN classifier is used for classification and recognition. Experiments were conducted on the CMU database, yielding satisfactory recognition results. Moreover, the algorithm shows significant advantages when gait occlusion or self-occlusion occurs.
1 Introduction
Gait recognition technology is an emerging biometric measurement technology.
It is pointed out that gait characteristics are unique and can be used for identity recognition. Identification based on human gait behavior characteristics can overcome the shortcomings of traditional recognition technologies to some extent and has gradually gained widespread attention in recent years.
Gait recognition methods can be divided into two categories: statistical feature-based and model-based. Statistical-based methods directly extract the corresponding statistical parameters from the image sequence as feature indicators for object classification and recognition, such as the principal component analysis method and linear decision method hybrid transformation proposed by Huang et al. [2] based on optical flow features, and the contour-based unwinding recognition method proposed by Wang et al. [3]. Model-based methods focus on human motion information, pre-establish human body models, obtain model parameters by matching the model with the image sequence, and then use these parameters as gait features for classification. Lee et al. [4] used 7 ellipses to match different body parts of the binary contour of the moving human body, and then extracted 29 moment features of the ellipses for analysis. Yoo et al. [5] based on anatomical knowledge, performed topological analysis of the body, and simplified the moving outer contour of the human body into a 2D stick model.
At present, both types of methods have some drawbacks: statistical gait recognition methods are sensitive to changes in background and lighting signals, and it is difficult to avoid the serious impact of occlusion in motion scenes on recognition ability; while model-based gait recognition methods treat each frame of the entire gait sequence separately, losing the organic wholeness of the gait cycle. Reference [6] points out that the dynamic information of the body is mainly manifested in the swinging pattern of the thigh and calf, so some simplifications can be made when building the human body model.
To address the aforementioned issues, this paper proposes a gait recognition algorithm based on joint angle information: the leg contour boundary is approximated using a line segment model and fitted using the least squares method to calculate the joint angles of the thigh and calf; considering the quasi-periodicity of gait motion, the temporal information of joint angles is expanded in the form of a Fourier series, and the coefficients of each harmonic are searched using a genetic algorithm. The amplitude values of each harmonic are scaled and standardized to form a feature vector for gait classification and recognition.
2. Extraction of leg joint angle information
2.1 Extraction of moving human body contours
Background subtraction is used to extract moving targets. First, the difference between the background image and the current image is calculated. Then, the difference image is binarized to obtain binary images, as shown in Figures 1(a), (b), and (c). Further post-processing is then performed on these binary images. The final extracted human contour is shown in Figure 1(d). Since background subtraction is a common method for moving target detection, it will not be elaborated upon here due to space limitations.
2.2 Modeling of the Legs of a Moving Human
The Stick model in reference [5] treats the human body as a mechanism composed of several rigid components, and under this assumption, the skeleton of the moving human body is transformed into a set of line segments that follow a specific connection order, and each line segment has a certain rotational flexibility. Gait recognition research based on kinematic analysis uses the temporal changes of joint angles to describe human gait behavior. In the Stick model, the joint angle can be defined as the angle between the corresponding line segment and the given axis. Considering that leg movement is the main component of gait, this paper simplifies the Stick model and only performs local modeling analysis on the thigh and calf of the moving human body. That is, leg line segments are generated from the contour image to obtain the joint angle information of the thigh and calf.
2.3 Leg joint angle extraction
Boundary search is used to detect the thigh (or calf) boundary in the contour image. Linear least squares is used to fit line segments to the thigh (or calf) boundary in the human contour image, and the fitting result is used as the corresponding line segment of the thigh (or calf) in the model of this paper. Let the line containing the fitted line segment be:
Then the angle of the corresponding joint of the thigh (or calf)
for:
The above processing was performed sequentially on the contour images to extract the temporal information of the thigh and calf joint angles.
3. Leg feature signal extraction
3.1 Gait signal periodicity
Murray[1]'s gait study shows that gait motion has quasi-periodic characteristics. Reference [7] uses a self-similar graph to determine the gait cycle, while this paper determines the image frame corresponding to the maximum contour width as the starting point of the gait cycle. Since a single cycle contains all the information of the periodic signal, the algorithm in this paper selects any gait cycle in the gait sequence to participate in the gait recognition process.
3.2 Genetic Algorithm for Searching Fourier Series Coefficients
According to Fourier theory, periodic continuous signals can be expanded into Fourier series. For example, a continuous signal with a period of can be expanded as follows:
In the formula,
The DC component of the signal.
The coefficients of each harmonic of the corresponding signal are the maximum harmonic order. For gait signals, the research of C. Angeloni et al. [8] shows that the maximum frequency component of normal gait does not exceed 5 Hz, and the duration of a single cycle of normal gait is about 1 s, that is, the fundamental frequency of normal gait is 1 Hz. In view of this, when performing Fourier series expansion on the temporal information of the thigh and calf joint angles, we take 5. Due to the occlusion problem and other interference factors during human movement, there will always be a situation where the joint angle information of some frames is difficult to extract, so it is impossible to directly solve for each harmonic coefficient.
Genetic algorithms are stochastic global search and optimization methods developed by mimicking the biological evolution mechanism in nature. They automatically acquire and accumulate knowledge about the search space during the search process and adaptively control the search process to obtain the optimal (or near-optimal) solution. This paper uses a genetic algorithm to search for the coefficients of each harmonic, essentially estimating the continuous variation of joint angles within a gait cycle based on the extracted joint angles of a finite number of frames. The objective function selected for optimization in this paper is:
Multiple searches are performed, and the set of coefficients that achieves the minimum value is selected as the harmonic coefficients of each gait cycle. Here, N is the total number of image frames in one gait cycle, and z(t) represents the thigh (or calf) joint angle value obtained from the t-th frame image. The coefficients obtained by the genetic algorithm are then substituted into equation (4) to calculate the result. Figure 2 shows the continuous temporal variation process of the thigh and calf joint angle values within a certain gait cycle estimated using a genetic algorithm based on known joint angle information. It can be seen that the estimated value has high accuracy when the joint angle value is known.
Experiments have shown that the estimated curves of the temporal changes in thigh (and lower leg) joint angles in different gait cycles of the same person are quite similar, while there are certain differences between the estimated curves of different people. Figure 3 shows the estimated curves of the temporal changes in thigh joint angles of different people.
3.4 Generating Gait Feature Vectors
The duration of the gait cycle varies among individuals during normal walking, and even for the same person, the gait cycle differs depending on the gait pattern (e.g., brisk walking versus slow walking). Clearly, scaling transformation of the harmonic coefficients searched by the genetic algorithm is necessary for accurate gait recognition. This paper utilizes the scaling property of Fourier transform to normalize the frequencies corresponding to each harmonic coefficient and then normalizes the amplitude values of the transformed harmonic coefficients.
The vectors formed by these features are used as gait feature vectors for gait classification and recognition.
4. Experiment and Analysis
4.1 Gait Database
This paper uses the Carnegie Mellon University (CMU) gait database for experimental validation. The database contains 25 individuals, each with four different gait patterns: slow walking, slouching, fast walking, and ball-carrying walking, as shown in Figure 4. Each gait pattern has six viewpoints, resulting in 24 gait video sequences for each individual. Each video sequence is approximately 11 seconds long, with a frame rate of 30 frames per second and an image resolution of 640×480.
4.2 Experimental Results and Analysis
This paper randomly selected video sequences of 9 people from a single perspective from the CMU database for gait recognition (each person has 4 different gait patterns, for a total of 36 sequences).
The experiment selected two pattern classification methods: nearest neighbor classifier and K-nearest neighbor classifier. Leave-one-out verification was used to estimate the unbiased recognition rate of the proposed method. One sample was left out at a time, and all remaining samples were then trained. Finally, the left-out sample was classified based on its similarity to the remaining samples. Table 1 shows the correct recognition rates using the NN and 3NN classifiers, respectively.
As shown in Table 1, fusing feature vectors in a cascaded manner and using a 3NN classifier for classification can achieve a higher recognition rate. Feature vector cascading, to some extent, avoids misclassification caused by errors in individual joint feature data. When using the NN classifier, for the two walking states of slow walking and walking with a ball, the cascaded recognition rate did not significantly improve compared to the lower leg joint recognition rate, which may be related to the limited number of samples.
4.3 Algorithm Comparison
Table 2 lists the recognition results of literature-related algorithms that use databases of the same or similar size as those used in this paper, which are comparable to each other. The statistical results show that the algorithm in this paper has a high recognition rate.
5 Conclusion
This paper proposes a gait recognition algorithm based on joint angle information, overcoming the challenge of recognition failure due to occlusion in common cases. The method was experimentally validated on the CMU database, achieving satisfactory recognition results. This paper only considers the case of a single viewpoint (where the main axis of the camera lens is perpendicular to the direction of human walking).
Further research includes: improving the real-time performance and robustness of the algorithm, testing the recognition performance of the proposed method on a larger-scale database, analyzing the sensitivity of feature extraction to changes in viewpoint, and extracting gait features that are insensitive to viewpoint and walking speed.
References
[1]MurrayM.P,DroughtAB,KoryRC.Walkingpatternsofnormalmen,J.BoneJointSurg,6-A(2)(1964)335–360.
[2]HuangP,HarrisC,NixonM.Humangaitrecognitionincanonicalspaceusingtemporaltemplates.VisionImageandSignalProcessing,1999,146(2):93-100
[3]WangL,HuWMandTanTN.Silhouetteanalysisbasedgaitrecognitionforhumanidentification.IEEETrans.PatternAnalysisandMachineIntelligence,2003,Vol.25,No.12,pp.1505-151
