After two years in beta, Google has announced the general availability of ML Kit for iOS and Android along with improvements to the Pose Detection API. Furthermore, Selfie Segmentation is now available in public beta.
We launched ML Kit back in 2018 in order to make it easy for developers on Android and iOS to use machine learning within their apps. Over the last two years we have rapidly expanded our set of APIs to help with both vision and natural language processing based use cases.
ML Kit vision-related stable APIs cover text recognition, face detection, barcode scanning, image labelling, oject detection and tracking, and digital ink recognition. Natural-language capabilities include language identification, text translation, and smart replies.
Among the latest features added to ML Kit are Selfie Segmentation, Pose Detection, and Entity Extraction, which are still considered beta-quality, though.
Selfie Segmentation, which was previously available only in closed beta, enables separating the background of a selfie from the people in the foreground, which can be useful to apply visual effects or replace the background altogether. Selfie segmentation can be applied to both still photos and video footage.
The Selfie Segmentation API takes an input image and produces an output mask. Each pixel of the mask is assigned a float number that has a range between [0.0, 1.0]. The closer the number is to 1.0, the higher the confidence that the pixel represents a person, and vice versa.
Since its initial announcement, ML Kit Pose Detection API, which is able to provide a 33-point skeletal match of a user's body, has been improved by making it capable of recognizing more poses, including typical fitness and yoga poses. Additionally, the new API uses smaller models, which are half the original size, and adds Z-coordinate support to make it possible to determine whether parts of the user's body are in front or behind their hips. One noteworthy use-case using pose detection is classifying specific pose, for which Google is providing a complete tutorial and sample app that also shows how to count repetitions using the classifier.
To get started with ML Kit, head to the samples section of Google's ML Kit developer portal.