The next Windows 10 update opens the way for the integration of artificial intelligence within Windows applications, directly impacting hundreds of millions of devices from Windows PCs and tablets to IoT Edge devices.
The new version of the Windows ML platform allows developers to integrate pre-trained deep-learning models within their applications directly in Visual Studio. The models must be converted into the Open Neural Network Exchange (ONNX) format before importing into VS tools.
ONNX is an open-source machine-learning framework launched by Microsoft and Facebook in September 2017, later joined by AWS. ONNX enables portability between neural-network frameworks, making it possible for models trained with tools like Pytorch, Apache MxNet, caffe2 or Microsoft Cognitive Toolkit (CNTK) to be translated to ONNX and later implemented in Windows applications. Several Windows hardware vendors including NVIDIA, Intel, Qualcomm and AMD are supporting the format and have released ONNX optimized hardware.
Artificial intelligence is already widely used across many Microsoft applications and services, with image and facial processing, search-results relevance, speech recognition and data security. These AI functionalities rely on models trained in the cloud and require intense computing resources. However, relying on cloud APIs for product features creates latency, raises privacy concerns and increases power usage by the devices.
With the Windows ML and ONNX combination, the computation-hungry model-training phase still takes place in the cloud, but the inference calculations are carried out directly in the application. This enables offline usage, lowers power consumption, avoids private data transfers and allows for real-time processing by reducing latency. During the model-building phase, developers and data scientists can still work with the most efficient framework for the task at hand, then convert the trained models to ONNX, and finally integrate the ONNX-formatted model into the Windows application.
According to Gartner, potential use cases for artificial intelligence in the device may focus on personal assistants, fraud detection, device resources optimization or virtual reality. Embedding artificial intelligence also facilitates applications that directly exploit private user data such as health diagnostics, personalized writing tools and biometric authentication.
Windows ML is not restricted to deep-learning models, and can also import classic machine-learning models from other frameworks like Core ML, Scikit-Learn, XGBoost and LibSVM.
Apple is following a similar path with its CoreML model format. CoreML is a Python package that converts a variety of model types into the CoreML model format for integration in OS X and IoS applications. CoreML leans towards classic machine-learning models such as scikit-learn, LIBSVM and XGBoost and only supports Caffe V1 and Keras 1.2 deep-learning frameworks. Meanwhile, Google is also bringing machine learning to mobile devices running Android with its TensorFlow Lite framework.
Although Tensorflow and Core ML are not ONNX compatible, the ONNX-Tensorflow and the ONNX-CoreML libraries are community-based projects that allow respective model conversion to the ONNX format.
This artificial intelligence acceleration is part of a similar trend to embed some level of machine learning in IoT devices for the same efficiency-improvement goals. By pushing artificial intelligence onto the devices and applications and closer to the end user, Microsoft, Google and Apple are trying to conquer the last mile of artificial intelligence.