According to Eirik Midttun, artificial intelligence (AI) and machine learning (ML) are useful tools for interpreting sensor data, especially when the input is complex, such as vibration, voice, and vision. The main challenges of using machine learning on microcontrollers are the constraints in computing power available and cost-related requirements that come with microcontroller-based designs, as he mentioned in his talk at NDC TechTown:
Memory use impacts costs and CPU use will impact energy consumption, which is an issue for battery-operated devices. Such constraints come on top of the common challenges with AI design.
A variety of tools and skills are needed to develop a solution, Midttun mentioned. Data scientists have tools for data analysis, ML engineering is mostly done in Python using ML toolkits, and embedded software engineers work with C/C++ code, as he explained:
In our industry of embedded development, the C/C++ coding is familiar territory, but the other fields are typically not. More is needed for efficient collaboration between the different fields of expertise.
Midttun showed an application of ML using a table fan which is being monitored by an accelerometer. The goal is to detect anomalies in its operation from the sensor data. You could do this by analysing the sensor input and inventing a classic algorithm to do this, or use sensor data to train a neural network to identify normal operation and report deviation from the correct operation, as Midttun explained:
Neural networks and machine learning really perform well at these sorts of tasks, and you can develop a solution faster and better than signal processing. Anomaly detection and condition monitoring are the main use cases for machine learning using vibration data.
Since the data will vary depending on the type of machine, and even the environment, it is important to have a solution that can quickly generalise and adapt to the signals that the sensor is monitoring, Midttun said.
TensorFlow is an open source deep-learning framework for models intended to run on powerful computers, Midttun mentioned. RTLite, formerly known as TensorFlow Lite, can convert the trained ML models into implementations suitable for microcontrollers.
Generative AI is at the peak of its hype and energy consumption is becoming a big concern. The System-on-Chips (SoCs) we design are mainly for battery-operated devices, so we are very concerned about power consumption, Midttun said. Training small ML models is not as energy-consuming as ChatGPT for example, and running inference costs very little energy, he added.
Two main contributors of power consumption in our domain are the radio and the CPU. ML, being compute intensive, is something that affects CPU utilisation. What we see is that the energy required to run inference is usually less than the energy needed for wireless data transfer, as Midttun explained.
The energy needed varies a lot depending on the wireless technology, but even for Bluetooth LE, which consumes the least energy, there is an interesting trade-off in making the device smarter and reducing the need for data transfer.
There are products on the market today that use ML on microcontrollers to enable new and innovative functions, Midttun said. The solutions are mature and microcontrollers are efficient enough to run the algorithms on battery. The use of this approach is far though from its full potential. As microcontrollers are getting even more efficient, targeting energy harvesting as a power source, it is a good time to explore AI and ML, he concluded.
InfoQ interviewed Eirik Midttun about applying artificial intelligence on microcontrollers.
InfoQ: How did you use artificial intelligence and machine learning?
Eirik Midttun: The device collects sensor data from an accelerometer at a given rate, 100 Hz in the demo of the table fan, until there is a large enough sample to analyse. The sensor data is then processed by spectral analysis before an edge optimised neural network classifies the input data.
To detect anomalies, the classification results are analysed by a k-means algorithm to determine the deviation from known good operation. The table fan has a relay attached to the setup so the system on chip can cut the power if an anomaly is detected.
InfoQ: What have you learned?
Midttun: The introduction to AI and ML was a steep learning curve, and it involved an understanding of neural networks, training process, tools and integration in applications.
One important thing I learned is the importance of data in providing a good solution. Very often, in order to improve inference results, the answer lies in the quality of the training data. The field of data science is a very important one to achieve success, so this is work I have learned to appreciate.
InfoQ: What will the future bring?
Midttun: The CPU is the part that benefits the most from technology advances, in terms of energy use. For example, the use of modern process nodes for semiconductors will increase the clock speed in the processors, reducing execution time, while also reducing the electric energy needed for the processor. AI can also benefit from dedicated accelerator hardware, similar to the use of GPUs in computers, but adapted for the needs of constrained devices.
