Google have recently introduced ML Kit, a machine-learning module fully integrated in its Firebase mobile development platform that is available for both iOS and Android. With this new Firebase module, Google simplifies the creation of machine-learning powered applications on mobile phones, and solves some of the challenges of implementing computationally intense features on mobile devices.
ML Kit allows mobile developers to create machine-learning features based on several of the models available in its deep-learning Vision API, such as image labeling, OCR and face detection. ML Kit is available both for Android and iOS applications directly within the Firebase platform alongside other Google Cloud based modules such as authentication and storage.
ML Kit aims at solving several of the challenges specific to mobile devices which are raised by the computationally-intensive operations required for artificial intelligence. The goal is to reach a satisfactory trade-off between offering good enough model accuracy with reduced size models, while preserving battery life and refreshing the models with local data on an environment that has very limited computational resources.
Optimizing machine learning for mobile devices is a layered approach. At the hardware level, the Android Neural Net API is an Android C API designed for running computationally intensive operations by distributing the computation workload across available on-device processors.
Moving up to the model layer, optimization is obtained by reducing model complexity and size. This led Google to release Tensorflow Lite a year ago, while Apple had previously launched Core ML and Facebook Coffee2Go. These lightweight formats facilitate downloading pre-trained models on the device to carry out inferences on local data. But the conversion from, for instance Tensorflow to Tensorflow Lite, remains complex according to Eric Hsiao, a mobile Machine Learning engineer.
As Brahim Elbouchikhi, ML Kit product manager at Google, emphasized during the Google I/O 2018 ML Kit presentation, ML Kit is the third layer needed to bring machine learning to mobile phones by putting deep learning models in the hand of mobile developers, right in the Firebase mobile app development platform.
ML Kit allows both on-device and cloud-based inference. On-device inference works with smaller models and offers reduced accuracy, but is free. Inference on cloud-based models is only free for the first one thousand API calls, but brings higher accuracy. As an example, the on-device image labeling feature taps into a set of 400+ labels while the cloud-based model has access to over 10k labels. However, on-device inference drives a better user-experience with real-time interaction and addresses privacy concerns as the user's data remains on the mobile phone.
ML Kit also allows dynamic models download via the Firebase Remote Config. The model is uploaded to the Firebase platform and is served dynamically to users removing the need to bundle the model into the Android Development Kit. This flexible model deployment enables simple A/B testing and deployment of models tailored to user segmentation.
ML Kit comes with pre-trained models and access to some of Google Machine Learning API services. It also allows integration of custom models trained on proprietary datasets. At this point, available features in the base APIs include text recognition (OCR), image labeling, barcode scanning, face detection and landmark recognition. Face contour and smart reply are expected to be released in the near future. A new Tensorflow to Tensorflow Lite conversion service based on Learn2Compress is also expected to be launched soon. In the meantime, developers can sign up for the conversion service beta release.
Keeping in mind that ML Kit has only been available for a few weeks, some of the issues raised by developers in forums revolve for the most part around undetected barcodes and troubles with text recognition, but less about image classification.