Designing IoT Solutions with Microsoft Azure

An IoT-based solution is capable of monitoring multiple devices, sending/receiving telemetry data, performing hot and cold path analytics, sending event-based notifications wherever a particular event occurs, and storing and/or displaying the data using insights.

Any IoT device requires the following key features:

•  Sensors
•  Network connectivity
• Data storage, processing, and analysis

With Azure, one could design IoT-based solutions while also providing features like scalability, more effortless and cost-effective device maintenance, and monitoring devices at a single place, Azure IoT Hub. In addition, it offers a variety of services to create, manage and perform several operations on the IoT devices registered on the cloud. For instance, the following operations could be performed on Azure:

• Use Azure IoT Hub to create, connect, monitor, and manage all the IoT devices in one place.
• Use the Azure Device Provisioning Service (DPS) to provide all the IoT devices inside an IoT Hub.
• Create and manage certificate-based registration using Symmetric Key and X.509 certificates.
• Consume Azure Event Hub and Azure Digital Twin services to handle multiple event types and define the required actions.
• Simulate a registered IoT device to send telemetry data across the IoT Hub.
• Implement Azure's Time Series Insights (TSI) to view the telemetry data in a graphical form and export all or part of the data in another form like Excel reports.
•  Store the telemetry data inside Azure's blob or hierarchical storage to perform Hot and Cold Path Analytics.
• Make use of the Logic App to define operations and actions to handle any event type occurrence inside an IoT Hub.

In this article, we will learn how the IoT solutions can work with Microsoft Azure and what all services are available to perform different operations across multiple domains. Furthermore, it covers a few case studies to gain hands-on experience on Azure IoT that are common and provide a good starting point for utilizing cloud-based IoT services.

Before proceeding with Azure IoT, let us begin with the key components that Microsoft Azure provides and set up a foundation to further advance on the topic. 

Following are a few essential Azure IoT services:

Azure IoT Hub

An Azure IoT Hub acts like a single place where one could create, register, monitor, and manage all the IoT devices connected on the cloud while also handling two-way communication between devices, sending and receiving telemetry data by the IoT devices, handling event-based notifications, and monitoring the connected devices in the hub.

One could also define the access management using the "Access Control (IAM)" inside the hub, maintain tags, diagnose, and solve issues, determine the shared access policies, manage certificates, define endpoints, automate the device management using IoT Edge, managing logs and metrics, etc.

Device Provisioning Service

Azure's IoT Device Provisioning Service (DPS) helps to configure zero-touch device provisioning to a specified IoT hub. With the DPS, one could provide multiple devices secure and scalable. The DPS automates the following mentioned steps required to provide any IoT device:

• Establishing the connection between the device and the IoT solution by registering the device.
• Applying configuration to the device based on the solution requirements.

The DPS also provides specific certificate-based registrations:

•  X.509 certificates based on the standard X.509 certificate authentication flow.
• Trusted Platform Module (TPM) to provide and use a signed Shared Access Signature (SAS) token.
• Symmetric Key based on the Shared Access Signature (SAS) security tokens include a hashed signature and an embedded expiration.

Azure Stream Analytics

Azure Stream Analytics is an engine that processes events to analyze streaming data received from edge devices, sensors, applications, etc. It uses SQL-like queries to process such data streams.

Azure Digital Twin

Azure Digital Twin (ADT) allows the creation and interaction of live models of real-world environments. First, each element is modeled as a digital twin, and then, these models are connected into a knowledge graph that can respond to live events and be queried for information.

Azure Logic App

Used to define operations or events that might occur inside an IoT Hub and further defines the actions that are required to be executed if that event occurs. This is then referenced inside an IoT Hub's event feature to directly associate a Logic App within the hub and execute the actions defined inside the Logic App.

Azure Stack

This acts as an extension over Azure to provide users with both online and offline experiences. For example, one could run their cloud apps in an on-premises environment using Azure migration tools.

Azure IoT Edge

It is a fully managed service built on top of the Azure IoT Hub that provides edge computing which is when data is analyzed on devices (at the edge of the network) rather than in the cloud itself.

It is composed of the following components:

• IoT Edge modules – containers that can run our services. Modules are deployed to IoT Edge devices and execute locally on those devices.
• IoT Edge runtime – provides a runtime environment to execute containers on each IoT Edge device and manages the modules deployed to each device.
• Cloud-based interface – this enables to remotely monitor and manage IoT Edge devices.

Time Series Insights (TSI)

The TSI can visualize and monitor the telemetry data received from any IoT device registered inside an IoT Hub and store or export the data into other forms like Excel reports. This provides a graphical view of how the data changes inside an IoT Hub and cloud further be used to perform stream analytics, debugging, troubleshooting, or maybe, even maintaining logs about the data information.

Hot, Warm, and Cold Data

• Hot Data - For processing or displaying data in real-time. It might consume an Azure Function App, SignalR, and web app hosted on an Azure App Service.
• Warm Data - For storing or displaying only a recent subset of data. Time-consuming analytics and batch processing are performed on this data. For instance, a web app hosted on an Azure App Service.
• Cold Data - For archival data or long-term storage of data. Time-consuming analytics and batch processing are performed on this data. Azure Data Explorer is used for a longer duration (presently defaults to 100 years), an easy-to-use analytic engine built using Kusto Query Language (KQL).

Key Elements of Azure IoT

Following are a few key elements often observed on Azure while designing IoT-based solutions:

However, this may vary from case to case; the picture below presents a more general overview of all services consumed while designing IoT solutions using Azure.

The following components have been illustrated in the diagram:

• IoT/Streaming data sources
• Stream ingestion (using IoT Hub, Event Hub, and Kafka)
• Stream processing (Stream Analytics and Azure functions)
• Operational transactions, store and change feeds (web apps with databases and storage systems like Azure Cosmos DB and Azure data lake storage)
• Analytical store (Event Hub)
• Digital modeling and visualization (Azure Digital Twin)
• Insights (TSI, LogicApp, and Power BI)
• Advanced Analytics (Databricks and Azure ML)

In the following paragraphs, we will overview a few case studies that involve the usage of multiple Azure IoT services consumed in different scenarios.

Case Study 1 – Sending Telemetry Data and Using TSI

The above diagram depicts the exact requirement for the case study. We'd need to simulate a registered IoT device inside an IoT Hub to send telemetry data while using the DPS enrollments and send the events across the Event Hub to generate insights under Azure's TSI. Further, we could perform stream analytics and store the results in either blob or hierarchical storage.

Create Azure IoT Hub and Register an IoT Device

We can either create an Azure IoT Hub and devices using the Azure portal or use the Azure CLI commands to perform similar actions. After creating the IoT Hub, we could execute the following command to get the connection string for the newly created IoT Hub:

az iot hub connection-string  show --hub-name <IoTHubName>

Next, we could create a new device inside the IoT Hub and get its connection string using the following commands:

az iot hub device-identity create --device-id <IoTHubDeviceID> --hub-name <IoTHubName>

az iot hub device-identity connection-string show --device-id <IoTHubDeviceID> --hub-name <IoTHubName>

We could also use the Azure IoT Explorer to connect and view devices inside the newly created IoT Hub and use the connection string we received.

Sending and analyzing telemetry data

We could also simulate a virtually created device using the following command and send telemetry data like the Azure SDK samples located at Azure IoT SDK Samples.

az iot device simulates -d <IoTHubDeviceID> -n <IoTHubName>

Here, we can send the telemetry data using the Azure SDK, maybe using a console or web application while also using the connection string received for the device. Please note that we'd require the device ID and not just the IoT Hub name, as the data is sent only through a registered device.

Creating TSI and viewing telemetry data

We can now create a Time Series Insights service. We can provide the property as "deviceId" in the service to view the insights for a particular device ID sending the telemetry data across the hub. We can also perform the following with the data:

• Export data into logs or Excel form.
• Perform Stream analytics to analyze data (using Hot and Cold Path).

Case Study 2 – Stream Analytics and Storage Containers

Now that we've received the insights on the telemetry data being sent to the IoT Hub, we could perform the stream analytics, process the information and use the storage containers to save the data into blobs or hierarchical storage.

Create an Azure Storage Container

We could create a blob storage and add containers to it to use a storage container. This will ensure that any data received on a specific fulfilled criterion will be stored on the blob.

Create and Configure Azure Stream Analytics

Now, we could create a new Azure Stream Analytics job with the help of which we can manage to store data inside the newly created blob storage container. To do this, we need to create a "Stream" input inside the job and then add a "Blob" output to the same job.

We may try to add the following query, for instance, that could determine and filter what data we need to store inside the blob container whenever the simulation and analytics job run together:

SELECT *
INTO streamOutput
FROM streamInput
HAVING Temperature > 21

Finally, we could make use of the online simulator available, Raspberry Pi Azure IoT Online Simulator, to connect to the IoT Hub and IoT device while sending telemetry data to the hub and also triggering the stream analytics job to store the data into blob container storage whenever the "Temperature" values go beyond 21.

Case Study 3 – Creating Logic App and Monitor Events

Azure also provides the facility to receive alerts and notifications based on specific events occurring using the Logic App and associating the respective events inside the IoT Hub.

Creating Operations and Actions Using Logic App

One could create appropriate operations (like HTTP request received) and define relevant actions when the event occurs (like sending an email to someone if the device is connected/disconnected or maybe alerting the system if the temperature goes beyond a defined threshold value, etc.).

Creating Events Inside Azure IoT Hub

Please note that before creating or associating any events, we'd require an Event Subscription first, and then we can add or associate the required events.

Here, we may associate a webhook using Microsoft Event Grid and define the states when to invoke this event – maybe whenever a device is connected or disconnected.

Note: Please note that if we're selecting any event type as the Microsoft Event Grid and receive an error, it might be since, under our subscription, this event is unregistered. We'd need to launch the subscription page to register the same.

Conclusion

To conclude, Microsoft Azure provides various robust and scalable services if one wishes to design an IoT-based solution on the cloud. This also means that we only pay for the services that are actually in use and save a lot of effort, time, and cost while consuming the readily available services from Microsoft Azure.

Above all, we could create, register, manage, monitor, scale, analyze and measure the IoT devices and the associated properties on the run by either using the CLI or Azure Portal seamlessly and without worrying about the actual devices. Moreover, the Simulation feature also facilitates the ease of using virtual IoT devices and performing a plethora of operations to save any investment costs before going for the actual device altogether.