At the recent re:Invent conference, Amazon announced a preview release of AWS IoT Analytics. The solution was presented during a few sessions at the event.
AWS IoT Analytics is the latest fully managed service offering of AWS IoT for performing advanced analytics on data ingested from IoT devices and customers can take advantage of typical cloud computing benefits such as automatic scaling and pay-as-you-go pricing models.
The solution uses the concept of channels and a pipeline that takes unprocessed messages and allows processing and activities to be performed on them. Messages are stored in data stores and data sets are retrieved from the stores for further analysis, visualization and action.
Image source: https://aws.amazon.com/iot-analytics/
You work with AWS IoT Analytics through a console that allows you to create the artefacts needed as part of your IoT message processing.
The first task is to create a channel which acts as the data ingestion point. The channel can filter messages from MQTT topics in AWS IoT Core or route messages from the Rules Engine, and supports both binary and JSON data.
A pipeline can then enrich, transform or filter messages. Enrichment can use external sources to add information such as add attributes from the AWS IoT device registry, transformation can perform simple calculations such as unit conversion, and filtering can use AWS Lambda functions, for example to estimate values if data is missing.
Raw and processed data are stored within the service to allow for future processing, and in a time-series data store for analysis. The data stores are not databases but are abstractions above several database technologies within the platform.
Once the data is stored it can be analyzed by querying the data stores and retrieving data sets. AWS IoT Analytics supports running ad hoc SQL-like queries, time-series analysis such as checking the performance of a device over time or looking for trends, and statistical and machine learning analysis. To perform machine learning and statistical analysis, AWS IoT Analytics supports fully hosted Jupyter notebooks. There are several templates that can be chosen as a starting point to speed up delivery of results from the stored data such as AWS-authored models for predicting device failure or K-means clustering algorithms for device segmentation.
After analysis, the data can be visualized using Amazon Quicksight or through the embedded Jupyter notebooks in the AWS IoT Analytics console.
AWS IoT Analytics adds a capability that can take large volumes of unstructured data and deliver insights and actions through the use of a single service, as Ron Miller recently noted in TechCrunch:
Because of the sheer volume of data involved, Amazon created a dedicated service instead of letting customers deal with IoT data in a more general tool like QuickSight. That could be because it requires a predictive element, rather than one that looks back at what happened.
The service has been in private preview with a few customers including Valmet who are a global company that works across the pulp, paper and energy industries, and iDevices who create smart home solutions. Eric Ferguson, chief software architect at iDevices said of AWS IoT Analytics:
The tools provided by AWS IoT Analytics to ingest, filter, transform, and analyze our data sources cut out a lot of the undifferentiated heavy lifting for our team, enabling them to focus on the enrichment activities in the pipeline and the downstream machine learning models, rather than the mechanics of the pipeline itself.
Pricing for the service is based on pay-as-you-go, and there is a free tier for customers that is valid for the first 12 months.
AWS IoT Analytics is available today in limited preview in US West (Portland), US East (N Virginia), and S East (Ohio). You can sign up for the preview through the products preview page.