The combination of artificial intelligence and microprocessors has opened up new markets for microcontrollers. It enables a growing number of new applications and use cases to leverage simple microcontrollers and AI acceleration to facilitate intelligent control. These AI-powered microcontrollers offer a unique combination of DSP computing and machine learning (ML) capabilities for inference and are currently being used in a wide variety of applications such as keyword recognition, sensor fusion, vibration analysis, and speech recognition. High-performance microcontrollers can provide more complex applications in vision and imaging, such as facial recognition, fingerprint analysis, and autonomous robotics.
Artificial intelligence technology
Here are some artificial intelligence technologies that can be used with IoT devices:
Machine learning: Machine learning algorithms build models based on representative data, enabling devices to automatically recognize patterns without human intervention. ML vendors provide the algorithms, APIs, and tools needed to train these models, which can then be built into embedded systems. These embedded systems then use the pre-trained models to drive inferences or predictions based on new input data. Examples include sensor centralization, keyword recognition, predictive maintenance, and classification.
Deep learning is a type of machine learning that trains a system by using many layers of neural networks to extract higher-level features and insights from complex input data. Deep learning works with very large, diverse, and complex input data, enabling the system to learn iteratively, improving the results with each step. Examples of applications using deep learning include image processing, chatbots for customer service, and facial recognition.
Natural Language Processing (NLP): NLP is a branch of artificial intelligence that processes interactions between systems and humans using natural language. NLP helps systems understand and interpret human language (text or speech) and make decisions based on this. Examples include speech recognition systems, machine translation, and predictive typing.
Computer Vision: Machine/computer vision is a field of artificial intelligence that trains machines to collect, interpret, and understand image data, and to take action based on that data. Machines collect digital images/videos from cameras, use deep learning models and image analysis tools to accurately identify and classify objects, and take action based on what they "see." Examples include fault detection on manufacturing assembly lines, medical diagnosis, facial recognition in retail stores, and testing of self-driving cars.
Maternal and Infant Health Assessment Program
In the past, artificial intelligence was dominated by MPS and GPS, featuring powerful CPU cores, large memory resources, and cloud connectivity for analytics. However, in recent years, with increasing levels of intelligence, we have begun to see the use of microcontrollers in embedded AOT applications. Moving to the edge is driven by latency and cost factors, involving moving computation closer to the data. AI on microcontroller-based IoT devices can make real-time decisions and respond to events faster, offering advantages such as lower bandwidth requirements, lower power consumption, lower latency, lower cost, and higher security. This is further facilitated by the high computing power of recent microcontrollers and the availability of thin neural network (NN) frameworks, which are well-suited to the resource-constrained microcontrollers used in these end devices.
A neural network is a collection of nodes arranged in a layer, receiving input from the previous layer and generating an output calculated from the weighted sum of the inputs and biases. This output is passed along all output connections to the next layer. During training, training data is fed into the network's first or input layer, and the output of each layer is passed to the next. The final or output layer generates the model's predictions, which are compared to known expected values that evaluate the model's errors. The training process involves refining or adjusting the weights and biases of each layer in each iteration using a process called backpropagation until the network's output closely correlates with the expected values. In other words, the network repeatedly "learns" from the input dataset and progressively improves the accuracy of its output predictions.
Training neural networks requires very high computational performance and memory, and is typically done in the cloud. After training, the pre-trained NN model is embedded in a microcontroller and serves as an inference engine based on new input data from the training.
This inference generation requires significantly less computational power than model training, making it suitable for microcontrollers. The weights of this pre-trained neural network model are fixed and can be stored in flash memory, reducing SRAM requirements and making it suitable for more resource-constrained microcontrollers.
Implementation of maternal and child health care units
Implementing AOOT on a microcontroller involves several steps. The most common approach is to use one of the existing neural network (NN) framework models, such as Kafi or Tensioner, which is suitable for microcontroller-based end-device solutions. Training the NN model for machine learning is conducted in the cloud by AI experts using tools provided by the AI vendor. The NN model is then optimized and integrated into the microcontroller using tools from both the AI vendor and the microcontroller manufacturer. Inference is then performed on the microcontroller using the pre-trained NN model.
The first step in the process is done entirely offline, capturing large amounts of data from the end device or application and then using it to train the neural network model. The model's topology is defined by the AI developers to fully utilize the available data and provide the output required by the application. Training the neural network model is done iteratively by feeding the dataset into the model, with the aim of continuously reducing errors in the model's output. Several tools within the National Network Framework can assist in this process.
In the second step, these trained models are converted into a format suitable for microcontrollers. These models are optimized for certain functions, such as keyword recognition or speech recognition. The first step in this process is to convert it into a flat buffer file using an AI conversion tool. This can be selectively run through a quantizer to reduce its size and optimize it for the microcontroller. The flat buffer file is then converted into C code and transferred to the target microcontroller as a runtime executable file.
This microcontroller, equipped with a pre-trained embedded AI model, can now be deployed on end devices. When new data is input, it will run through the model and generate inference based on the training. When new data classes appear, the neural network model can be sent back to the cloud for retraining. The new retrained model can be programmed on the microcontroller, possibly via OTA (over-the-air) firmware upgrades.
There are two different ways to architect MCU-based AI solutions. For the sake of discussion, we assume that an ARM cortex MCU is used in the target microcontroller.
In the first approach, the transformed neural network model is executed on a cortical-MCPU core and accelerated using the CMS-NN library. This is a simple configuration that can handle tasks without any additional hardware acceleration, making it suitable for simpler AI applications such as keyword localization, vibration analysis, and sensor hubs.
A more sophisticated and higher-performance option involves a neural network accelerator (NN) or micro-neural processing unit (U-NSP) hardware on a microcontroller. These U-NSPs accelerate machine learning in resource-constrained IoT end devices and may support compression, thereby reducing the power and size of the model. They support operators of most public NN networks capable of fully performing audio processing, speech recognition, image classification, and object detection. Networks not supported by UNN can be returned to the main CPU core and accelerated by the CMS-NN library. In this approach, the NN model is executed on the UNP.
These methods only demonstrate a few approaches to integrating AI into microcontroller-based devices. As microcontrollers push the performance boundaries to higher levels, closer to expectations from MPPS, we expect to begin seeing full AI capabilities, including lightweight learning algorithms and inference, built directly on microcontrollers.
Artificial intelligence at the edge is the future.
In the future, the implementation of artificial intelligence for resource-constrained microcontroller systems will grow exponentially. As microcontroller systems push the performance limits, the line between microcontrollers and MPS becomes blurred, and more and more "thin" neural network models suitable for resource-constrained devices begin to emerge. We will continue to see new applications and use cases emerge.
In the future, as microcontroller performance improves, we may see the implementation of lightweight learning algorithms, in addition to inference, running directly on microcontrollers. This will open up new markets and applications for microcontroller manufacturers and become a major investment area for them.
