The Perfect Pair: Digital Twins and Predictive Maintenance

Why Predictive Maintenance Now?

Every business needs their essential equipment to operate at peak efficiency and utilization to realize their return on capital investments.

This equipment could range from aircraft engines, turbines, elevators, or industrial chillers that cost millions to purchase.

For maintenance of this equipment, businesses follow the approaches below:

1. Most of the businesses depend on Corrective Maintenance that ensures parts are used completely, but this costs business in downtime, labour, and unscheduled maintenance requirements, etc.
2. Some of the businesses have gone next level by practicing Preventive Maintenance, where they determine the lifetime of each part and perform maintenance before the actual failure happens. This approach avoids unscheduled and catastrophic failures but the high costs of scheduled downtime, under-utilization of the component during its useful lifetime, and labour still remains.
3. Now with advent of IoT & Digital Twins, businesses are moving towards building a Predictive maintenance model that optimizes maintenance cycle and balances between a corrective and preventative maintenance approach. Predictive maintenance uses just in time replacement of components. This approach not only replaces those components that are close to a failure but also extends component lifetime by reducing unscheduled maintenance and labour costs. By following this approach, businesses can achieve cost savings and competitive advantages.

Introducing Digital Twins

Digital Twins technology brings the exact replica in digital format of a process, a product, or a service.

Basically, it takes real-world data about a physical object or system as inputs, and produces outputs in the form of predications or simulations of how that physical object or system will be affected by those inputs.

Some of the most common use case across the industry is given below:

1. Visualization of products in use, by real users, in real-time
2. Troubleshooting far away equipment
3. Managing complexities and linkage within systems-of-systems
4. Connecting disparate systems and promoting traceability

Figure 1: Most common use-cases of Digital Twins

According to a recent IoT implementation survey by Gartner, Inc Digital Twins are entering mainstream use:

13% of organizations implementing Internet of Things (IoT) projects already use digital twins, while 62% are either in the process of establishing digital twin use or plan to do so.

Figure 2: Forbes view on Digital Twins | Source

Figure 3: Gartner Top 10 Strategic Trends for 2019 Features Digital Twins | Source

Figure 4: GE Insight | Source

Insights from different types of Digital Twins

There are different types of Digital Twins, to choose the right digital twin, you would need to understand the needs and its benefits that you can get out of it.

Figure 5: Insights from different types of Digital Twins

What makes up Digital Twins?

Sensors are the heart of any measurement, control, and diagnostic devices. Device telemetry is collected using the smart sensors available on the hardware / software environment and then used to create the digital twin model of the physical equipment.

All of the data is then aggregated and compiled to generate actionable information. The digital twin model is then continuously updated to mirror the current state of the physical thing. It can then be used to effectively model, monitor, and manage devices from a remote location. It also enables continuous intelligence & estimated time for the next needed maintenance, which the maintenance system can use to schedule at the optimal time.  

Figure 6: Elements of Digital Twin

1. Physical equipment is the actual equipment that we are interested in creating a twin for.
2. Twin Model - Comprises of hierarchy of systems, sub-assemblies, and components that describe the twin and its characteristics enriched by asset, operational, historical, and context data.  
3. Knowledge - Data sources that feed the twin with operational settings, domain expertise, historical data, and industry best practices.
4. Analytics – Model gets empowered by physics-based models, statistical models, and machine learning/AI models to help describe, predict and prescribe the behavior (current and future) of the asset, system, or process.  

We can see in detail on how to build digital twins in the sections below.

How to qualify use cases for predictive maintenance using Digital Twin?

Not all use cases or business problems can be effectively solved by predictive maintenance using a digital twin. Here are important qualifying criteria that needs to be considered during use case qualification:

1. The problem must be predictive in nature, meaning there should be a target or an outcome to predict.
2. The problem should have a record of the operational history of the equipment that contains both good and bad outcomes.
3. The recorded history should be relevant and be sufficient quality to support the use case.
4. Finally, the business should have domain experts who have a clear understanding of the problem.

Now that we have see how to qualify use cases for predictive maintenance using the digital twin approach, the next step is to explore the various options to build/deploy them.

Options to build/deploy Digital Twins

Figure 7: Options to build/deploy Digital Twins

#1. Vendor Agnostic Frameworks

This section summarizes the different frameworks available for building digital twins.

1.    Eclipse Ditto - Digital twins can be built by leveraging pre-built capabilities, for example, routing requests to hardware, applying policies, etc.

• a. Positives:
  • Based on Docker, Client SDK’s are available for Java/Python.
  • API abstraction for the hardware.
  • Routes requests between hardware and customer apps.
• b. Negatives:
  • Persists only last reported state of hardware.

2.    Swim OS - Integrated solution for building scalable, end-to-end streaming applications.

• a. Positives:
  • Distributed OS.
  • Built in stateful process execution scheduler.
  • Right choice for building stateful data-driven applications.
• b. Negatives:
  • Steep learning curve in understanding concepts/capabilities.

3.    iModel JS - Platform for creating, accessing, leveraging and integrating infrastructure digital twins.

• a. Positives:
  • Provides framework for creating, accessing, leveraging and integrating infrastructure.
• b. Negatives:
  • Effort Intensive

#2. Cloud Vendors

This section lists the various public cloud vendors offering digital twins.

1.    Azure Digital Twins

• a. Azure Device twin is automatically created when a device is connected to the IoT Hub.
• b. Azure Device twin is implemented via JSON file that stores the device state information that can be used to synchronize device information with back-end processes.

Key concepts:

• a. Spatial intelligence graph, or spatial graph, is a virtual representation of the physical environment. Same can be used to model the relationships between people, places, and devices.
• b. Digital twin object models are predefined device protocols and data schema. Same can be aligned to solution's domain-specific needs.
• c. Solutions can scale securely and be reused for multiple tenants.
• d. Azure Digital Twins instance can be connected to Azure services such as Azure Stream Analytics, Azure AI, Azure Storage, Azure Maps, Microsoft Mixed Reality, Dynamics 365, or Office 365.

2.    AWS Digital Twins

• a. Implemented via device shadow JSON file that contains the state information, meta-data, timestamp, unique client token, and version of a device connected to the device shadow service.
• b. There are three basic REST APIs that can be used to interact with the device shadow: GET, UPDATE, DELETE. You can also interact with device shadows using MQTT messages.

3.    IBM Digital Twins

• a. Watson IoT has data management for definition, managing Device Twin.
• b. Recent announcements of new lab services for Maximo that brings Augmented Reality (AR) into asset management.
• c. Digital Twins has been enabled on the following IBM products
  • i. IBM Maximo Enterprise Asset Management
  • ii. IBM Maximo Asset Performance Management
• d. Through IBM Digital Twin Exchange, manufacturers, OEMs and third-party providers can share digital resources as digital twins.

#3. Industrial Vendors

This section lists the various industrial vendors offering digital twins.

1.    GE Predix concentrates mostly on asset-centric digital twins.

• a. Detailed tutorial on how to create a digital twin for analytics.
• b. Offers Asset Service that allows modelling of assets that are essential to any digital twin solution.

2.    Bosch’s digital twin solution Bosch IoT Things has detailed technical documentation, including developer guides, demo applications, and hosted dashboard.

3.    Siemens MindSphere platform for developing new digital business models for industrial companies.

Walkthrough: Example of how to build Digital Twin

In this walkthrough section, we are going to build a digital twin model of a Intel NUC kit so that we can:

• a. Predict whether it may fail in the near future.
• b. Estimate the remaining useful life.

Figure 8: Digital Twin of Intel NUC Kit

Some of the parameters that determine health of the Intel NUC Kit are (this forms the basis for building digital twins):

1.  State of the CPU can be monitored via analysing:
   • a. CPU Core Sensors
   • b. Mainboard Sensors
2.  System-wide CPU utilization
3.  Hard Drives Temperature
4.  SSD wear level, host reads/writes etc., by SSD Hard Drive Sensors
5.  Disk usage statistics
6.  Disk I/O statistics
7.  Fan controllers
8.  Total physical memory (exclusive swap).
9.  Connection States

For building a digital twin of the Intel NUC kit, we are going to leverage the Eclipse Ditto framework that enables us to work with, and manage, the state of digital twins.

Figure 9: Ditto Framework to build Digital Twin

Following are the key capabilities of the Ditto Framework:

1. Provides capabilities (APIs) to interact with digital twins.
   • Live Channel architecture – routes a command/message towards an actual device.
   • Twin channel connects to the digital representation of a Thing and its state and properties can be read and updated.
2. Ensures that access to twins can only be done by authorized parties.
3. Allows to not only interact with single twins but also with populations of many of them.
4. Integrates into other back-end infrastructure (like messaging systems, brokers)

Step #1. Definition of Twin model – Things and Features

The first step is to define things and features. Things are generic entities and can be used to depict multiple features belonging to thing. For example, physical devices like lawn mower, a sensor. In the below example, we are going to treat the entire Intel NUC Kit as a Thing.

Feature is used to manage all data and functionality of a thing that can also be grouped based on technical context. In the below example, we have CPU, Memory, etc.

Figure 10: Things & Features

Step #2. Use the client SDK to pull the sensor values and publish it to eclipse ditto

Once we have defined Things/Features, we can now use any of the client SDK (ex. Python) to pull the respective sensor values and publish it to Eclipse Ditto.

Figure 11: CPU Core Sensors - Sample values

Figure 12: HDD Temperature Sensors – Sample values

Figure 13: SSD S.M.A.R.T Sensors – Sample values

Figure 14: Intel NUC Board from Sensors - Detect

Below is a sample JSON format that is being used to represent the thing (here its Intel NUC Kit)

Each thing has unique thingId and set of features that we discussed in the earlier section. We can also have attributes that describe the thing in more detail.

Also, we can find access control lists on who can perform read/write or use administer permissions.
 

{
    "thingId": "org.eclipse.ditto.example:demothing",
    "acl": {
        "ditto": {
            "READ": true,
            "WRITE": true,
            "ADMINISTRATE": true
        }
    },
    "attributes": {
        "manufacturer": "Demo Manfacturer",
        "hostname": "Demohost"
    },
    "features": {
        "TemperatureSensor": {
            "properties": {
                "temperatureValue": 20.5,
                "lastUpdate": "2019-10-16 15:07:31.436733",
                "samplingRate": 1
            }
        }, …..    
        }        
    }
}

The features section holds the current temperature value of the Temperature Sensor.

Figure 15: Things/Attributes/Features

Step#3. Use prebuilt Ditto APIs to Retrieve & Modify the state of the Thing/Feature/Attribute

Once we have the data about features/attribute from the sensors we can use Ditto API’s to retrieve or modify the state of Thing/Feature/Attribute. All the state and properties can be read, updated and collated.

There is also an option to route a command/message towards an actual device.

Figure 16: Example Web Application that is built to view/modify sensors values

Step #4. Build ML model based on Digital Twin to predict failure

Now we have the timeseries data available and stored in our Digital Twin Server, our next step is to build an ML Model based on the data collected to predict failure based on core attributes such as CPU, Memory, Disk Space, connection state or the performance of external interfacing systems.

Data for predictive maintenance model is time series data collected from the digital twin model.

Classification approach - predicts whether there is a possibility of failure in next n-steps. Suited for Greater accuracy with less data.

Sample Data Set for Intel NUC core attributes:

Figure 17: Sample Data set

This classification model is based on a decision tree classification approach where it offers a flowchart-like tree structure where an internal node represents feature (or attribute), the branch represents a decision rule, and each leaf node represents the outcome.

Figure 18: Decision Tree classification approach

After many iterations, below is the model’s confusion matrix & precise score. A highlight of the decision tree classification approach is it learns to partition based on the attribute value & also partitions the tree in recursively.

Figure 19: Classification model confusion matrix, accuracy

From the confusion matrix and our precision score, we are able to achieve ~96% accuracy in determination of the possibility of failure.

Regression approach - predicts how much time (RUL) is left before the next failure. We need more data but it provides more information about when the failure will happen.

• Input: Static and historical data are available, and every event is labelled. Several events of each type of failure are present in the dataset.
• Sample dataset with 3 operational settings, 21 sensor measurements and cycles to depict. System under observation is operating normally at the start of each time series and develops a fault at some point during the series.
• Output: How many days/cycles are left before the system fails?

Figure 20: Sample dataset

In order to predict the RUL for each engine, we are classifying like below.

• 0 (fault) to 15 remaining cycles as 2
• from 16 to 45 cycles as 1 and
• the rest (>46) as 0

Category labeled as 2 is the most economically valuable. If we predict this class with good performance, it will permit us to operate an adequate program of maintenance, avoiding future faults and saving money.

Next step is to transform timeseries data to Recurrence Plots and run the model.

Figure 21: Timeseries data to Recurrence plots

From the below confusion matrix, we can see that our model can well discriminate when the system is close to failure (2 labels: <16 cycles remaining) or when it works normally (0 label: >45 cycles).

Figure 22: Model Recall score, precision

Figure 23: Model Confusion matrix

We are satisfied to achieve result with 2D CNN model for the prediction of class 2 -- i.e., near to failure.

Congrats! we now have built Digital Twin and ran predictive maintenance model to predict the failures.

Key challenges while building Digital Twins

This section lists some of the challenges that should be handled while building digital twins.

1. Establishing Real time communication of data and latency – Real time communication is key requirement for building digital twins, it should always reflect the status of the physical equipment.
2. Sourcing Quality Data (Data Management, Data quality, etc.), Realistic Model and future projections & Data driven modelling – All of these are inter-related because when you have quality data then only you can build model and future projects/predictions would be accurate. Data for ex. operational settings, domain expertise, best practices, etc must be sourced with greater quality because this becomes knowledge base for your digital twin. All the predictions are directly based on the quality data you source.
3. Data Security, Privacy & Ethical Issues – How to ensure the privacy of data communications? How can the collected data remain private? How much data should a particular user see? Access control is some of the key challenges that should be handled effectively.
4. Large data volume, data generation rate, variety of data – Sensors on the field  (physical equipment) is going to send large volume of data and each of the device could generate variety of data in different formats. IoT Infrastructure that is collecting the data and the big data processing should be equipped enough to handle these kinds of volume and variety.
5. Consider using AR/VR visual Interaction with Physical asset – Some kinds of Twin for ex. inspection of an equipment where visual interaction is required consider using AR/VR type of interaction. Interaction with Physical asset need not be only with web application it can also differ based on scenario.

Conclusion

Businesses are moving towards developing a predictive maintenance model using digital twins that optimizes the maintenance cycle with the advances in IoT space, extending the life of the part by reducing unplanned maintenance and labor costs. By using digital twins and the predictive maintenance strategy, companies gain cost savings and strategic advantages in the industry.

About the Author

Karthikeyan Shanmugam (Karthik) is an experienced Solutions Architect professional with about 20+ years of experience in design & development of enterprise applications across Banking, Financial Services, Healthcare and Aviation domains. Currently engaged in Technical consulting & providing solutions in the Application Transformation space. Karthik regularly publishes his technology point of view. His articles on emerging technologies (includes Cloud, Microservices etc.) can be read on his blog here.