Artificial intelligence (AI) is currently revolutionizing all aspects of society. For example, by combining the strengths of data mining and deep learning, AI can now be used to analyze massive amounts of data from various sources, identify patterns, provide interactive understanding, and make intelligent predictions.
One example of this innovative development is the application of artificial intelligence to data generated by sensors , particularly data collected through smartphones and other consumer devices. Motion sensor data, along with other information such as GPS addresses, provides a vast amount of diverse datasets. Therefore, the question is: "How can artificial intelligence be used to fully leverage these synergies?"
Sports data analysis
A real-world application can determine a user's activity over time by analyzing usage data, whether they are sitting, walking, running, or sleeping.
In this context, the benefits of smart products are self-evident:
1. Increase customer lifetime value
Increasing user engagement can reduce customer churn.
2. A more competitive product positioning
The next generation of smart products will meet consumers’ growing expectations.
3. Create real value for end users
Accurate detection and analysis of indoor motion enables sensitive navigation, health risk monitoring, and improved device efficiency. A deep understanding of real-world usage scenarios across various smartphones and wearable platforms greatly helps product designers understand repetitive user habits and behaviors, such as determining the correct battery size or the right timing for push notifications.
Smartphone manufacturers are showing great interest in artificial intelligence capabilities, highlighting the importance of recognizing simple daily activities, such as step counts, which will inevitably evolve into more in-depth analytics, such as for sports activities. For popular sports like football, product designers will not only focus on athletes but will also provide convenience for a wider range of people, including coaches, fans, and even large companies such as broadcasters and sportswear design firms. These companies will benefit from in-depth data analysis, enabling them to accurately quantify, improve, and predict athletic performance.
Data acquisition and preprocessing
After identifying this business opportunity, the next logical step is to consider how to effectively collect these massive datasets.
For example, in activity tracking, raw data is collected through axial motion sensors , such as accelerometers and gyroscopes in smartphones, wearable devices, and other portable devices. These devices acquire motion data on three axes (x, y, z) in a completely covert manner, enabling continuous tracking and evaluation of activities in a way that is convenient for user applications.
Training Model
Supervised learning in artificial intelligence requires training a "model" with labeled data so that a classification engine can use this model to categorize real-world user behavior. For example, we collect exercise data from test users who are running or walking and feed this information to the model to help it learn.
Since this is essentially a one-off method, simple applications and photographic systems can accomplish the task of "labeling" users. Our experience shows that as the sample size increases, the human error rate in classification decreases. Therefore, it makes more sense to obtain a larger sample set from a limited number of users than a smaller sample set from a large number of users.
Simply acquiring raw sensor data is insufficient. We observed that achieving highly accurate classification requires careful identification of certain features; that is, the system needs to be informed of the features or activities important for distinguishing different sequences. Human learning is iterative, and during the preprocessing stage, it is not yet clear which features are most important. Therefore, the device must make some guesses based on expertise that may affect classification accuracy.
To perform activity identification, indicative features may include "filtered signals," such as body acceleration (raw acceleration data from sensors) or "derived signals," such as high-speed Fourier transform (FFT) values or standard deviation calculations.
For example, the Machine Learning Database (UCI) at the University of California, Irvine, created a dataset that defines 561 features based on six basic activities of 30 volunteers: standing, sitting, lying down, walking, going down stairs, and going up stairs.
Pattern recognition and classification
After collecting the raw motion data, we need to apply machine learning techniques to classify and analyze it. The machine learning techniques available to us range from logistic regression to neural networks, among others.
Support Vector Machines (SVMs) are such learning models applied to artificial intelligence. Physical activities, such as walking, involve sequences of various movements, and because SVMs excel at sequence classification, they are a reasonable choice for activity classification.
Support vector machines (SVMs) are very simple to use, train, scale, and predict, so multiple sample collection experiments can be easily set up side-by-side for non-linear classification of complex real-world datasets. SVMs also allow for a variety of different sizes and performance optimizations.
Once a technology is chosen, we must select a software library for Support Vector Machines. The open-source library LibSVM is a good choice; it is very stable and well-documented, supports multi-class classification, and offers extensions for all major developer platforms from MATLAB to Android.
The Challenge of Continuous Classification
In practice, as users move around, the devices they use need to be classified in real time for activity identification. To minimize product costs, we need to balance the costs of transmission, storage, and processing without compromising the quality of the information.
Assuming we can afford the data transfer costs, all data can be stored and processed in the cloud. In reality, this would incur significant data costs for users, whose devices would naturally need internet connectivity; the costs of wireless networks, Bluetooth, or 4G modules would inevitably further increase device costs.
Worse still, 3G network access is often unsatisfactory in non-urban areas, such as when hiking, cycling, or swimming. This reliance on large amounts of data transfer to the cloud slows down updates and requires regular synchronization, significantly offsetting the practical benefits of AI-powered motion analytics. Conversely, processing these operations solely on the device's main processor leads to a significant increase in power consumption and reduces the execution cycle of other applications. Similarly, storing all data on the device increases storage costs.
Turn the circle into a square
To resolve these conflicting issues, we can follow four principles:
1. Splitting – Separating feature processing from the execution of the classification engine.
2. Reduce – Intelligently select the features needed for accurate activity identification to reduce storage and processing requirements.
3. Usage – The sensors used must be able to acquire data with low power consumption, perform sensor fusion (combining data from multiple sensors), and be able to perform feature preprocessing for continuous execution.
4. Retention – Retain the model of system support data that can determine user activities.
By separating feature processing from the classification engine execution, the processor connected to the accelerometer and gyroscope sensors can be significantly smaller. This effectively avoids the need to continuously transfer blocks of real-time data to a more powerful processor. Feature processing, such as high-speed Fourier transforms for converting time-domain signals to frequency-domain signals, would require a low-power Phi processor to perform floating-point operations.
Furthermore, in the real world, individual sensors have physical limitations, and their outputs deviate over time, due to offsets and nonlinear scaling caused by welding and temperature. To compensate for this irregularity, sensor fusion, as well as rapid, inline, and automated calibration, is required.
Figure 1: Functional flow of activity classification (Source: Bosch Sensortec)
Furthermore, the chosen data acquisition rate can significantly impact the required computation and data transfer. Generally, a 50Hz sampling rate is sufficient for normal human activity. However, when analyzing fast-moving activities or motion, a 200Hz sampling rate is needed. Similarly, for faster response times, a separate 2kHz accelerometer can be installed depending on the user's purpose.
To meet these challenges, low-power or specialized sensor hubs can significantly reduce the CPU cycles required by the classification engine. For example, Bosch Sensortec's BHI160 and BNO055 are such sensor hubs. The associated software can directly generate the fused sensor output at different sensor data rates.
The initial selection of features to be processed subsequently has a significant impact on the size of the trained model, the amount of data, and the computational power required to train and perform inline predictions. Therefore, the selection of features needed to classify and distinguish specific activities is a critical decision and can also be a significant business advantage.
Recalling the UCI machine learning database mentioned above, its complete dataset of 561 features achieved a test accuracy of 91.84% for activity classification using a model trained with the default LibSVM kernel. However, after training and feature ranking, selecting the 19 most important features was sufficient to achieve a test accuracy of 85.38% for activity classification. A careful examination of the ranking revealed that the most relevant features were the frequency domain transform and the mean, maximum, and minimum values of the original sliding window acceleration data. Interestingly, these features cannot be achieved solely through preprocessing; sensor fusion is essential to ensure sufficient data reliability and is therefore particularly useful for classification.
in conclusion
In summary, technological advancements have now reached the point where advanced artificial intelligence can be run on portable devices to analyze motion sensor data. These modern sensors operate with low power consumption, while sensor fusion and software partitioning significantly improve the efficiency and feasibility of the entire system, while also greatly simplifying application development.
To complement the sensor infrastructure, we leverage open-source libraries and best practices to optimize feature extraction and classification.
Providing truly personalized experiences for users has become a reality. Through artificial intelligence, systems can leverage data collected by sensors in smartphones, wearables, and other portable devices to offer people more advanced functionality. In the coming years, a range of devices and solutions that are currently unimaginable will see further development. Artificial intelligence and sensors have opened up a new world full of exciting opportunities for designers and users.
