Google has announced a new mobile ML stack, dubbed Android ML Platform and built around TensorFlow Lite, which aims to solve a number of problems that developers find when using on-device machine learning.
The real announcement behind the Android ML Platform is that its foundation, TensorFlow Lite, will become available on all Android devices supporting Google Play Services. This means it will become part of the backbone that powers the Android platform. Goggle's and third-party apps will thus not need to bundle it anymore in their packages and developers can take the availability of its API for granted.
This will reduce overall device storage usage, since TensorFlow Lite will be shared by all apps. Storage usage can be a significant concern, Google says, for many apps that are size-constrained, even more so given the fact TensorFlow Lite is not exactly a small library.
In addition, this will make it possible to automatically update TensorFlow Lite, just as happens with any other Android component installed through Google Play Services. Automatic update is beneficial, according to Google, in that often developers stick with some older versions of TensorFlow Lite in order to maximize API availability.
With the Android ML Platform, though, Google is heading to a much more ambitious goal than simply making TensorFlow Lite available by default. In fact, one of the major issues that it is attempting to tackle is device heterogeneity, which can be a factor of staggering complexity in the Android world. This shows in relation to performance as well as testing.
In addition to its basic capabilities, TensorFlow Lite will provide Automatic Acceleration to enable models to automatically leverage hardware acceleration when available on device. This feature, which will become available later this year says Google, is based on the creation of allow-lists for specific devices which take performance, accuracy and stability into account. Allow-lists are created when testing a model and can be used at runtime to decide when to use hardware acceleration. Automatic Acceleration will require developers to provide additional metadata to verify correctness.
As a consequence of supporting TensorFlow Lite automatic update through Google Play Services, Google is also stabilizing the Neural Networks API across Android versions. This will prevent a non-backward compatible update from breaking existing apps. Additionally, to make it easier for developers to test their apps, Google says it is working with chipset vendors to make it possible to reduce testing from thousands of devices to a handful of configurations.
The Android ML Platform is still in preview and early access can be requested by interested developers.