InfoQ Homepage Articles Automating Data Protection Across the Enterprise

Automating Data Protection Across the Enterprise

Bookmarks

Feb 07, 2013 21 min read

Write for InfoQ

Feed your curiosity. Help 550k+ global
senior developers
each month stay ahead.Get in touch

Introduction

With the unabated growth of the internet, companies are being challenged to protect unprecedented quantities of data from determined and targeted attacks. While security regulations mandating the protection of sensitive financial, health-care or personally identifiable information (PII) serve as the primary impetus for data-protection, breached companies have learned the hard way that brands can be damaged, leading to significant financial costs when even unregulated data, such as e-mail and home-addresses, are breached¹.

Although encryption technology has been available for more than forty years², its use was initially relegated to military and banking applications where breached information has extremely high consequences. Twenty years ago, the world-wide web ushered in the Secure Socket Layer (SSL) protocol with its use of encryption to protect information as it traversed the internet. However, it was perceived as an enhancement to network protocols, and consequently, encryption technology continued to remain outside the domain of business applications.

Encryption started receiving attention from the general-purpose information technology (IT) industry - e-commerce, retail, etc. - only in the last five-six years, thanks to the credit-card payment sector³. There are several reasons why the general IT industry has been slow to adopt encryption technology:

Cryptography is a complex subject, and integrating it into applications has required specialized knowledge that is expensive and/or difficult to obtain;
Cryptography has tended to slow down business transactions without the addition of expensive accelerators – that are difficult to integrate and manage;
Companies have needed cryptography only in specific business applications, thereby relegating it to a niche. Consequently, this has also prevented the flow of knowledge of cryptography to general IT professionals perpetuating its use to niches;

As a result, companies have lacked a universal architecture for the protection of sensitive data across the enterprise. Global companies are becoming aware of this handicap as countries start enacting legislation for data-protection within their borders.

Cloud Computing

With the advent of cloud-computing, businesses can now come to market at lightning speed with massively scalable infrastructure to address their just-in-time computing needs. While the business benefits are indisputable, security concerns remain the single largest road-block to the wholesale adoption of cloud computing.

In 2011, an application architecture called Regulatory Compliant Cloud Computing (RC3)⁴, provided a solution to address the security problem. While RC3 addresses data-security needs on an application-basis, what remains missing is an all-encompassing infrastructure that allows an enterprise to address its data-security problems regardless of:

Where that data is generated, stored and used;
The type or size of data; or
Which country's data-security regulation governs protection of that data.

Data Encryption Infrastructure (DEI)

This paper defines a Data Encryption Infrastructure (DEI) - also sometimes referred to as a Document Encryption Infrastructure - as a collection of technology components and application architecture governing the protection of sensitive data within an enterprise.

A DEI is defined to have the following characteristics:

Any data-type - It must protect any type of data: structured data-elements such as credit-card numbers, passport numbers, etc., as well as unstructured binary large objects (blobs) such as audio and video files, documents, images, etc.;

Any data-size - It must protect data of any size: elements as small as the date-of-birth of an individual to objects as large as a database backup of an online transaction system;

Ubiquitous - It must be ubiquitous and easy to use from a software developer's point-of-view without awareness of the infrastructure's mechanics. An analogy is the Domain Name Services (DNS) name-service look-up library. Software developers have been using this single library and application programming interface (API) for more than thirty years to find the address of any computer on the internet without knowledge of how DNS is structured or how it works;

Independent - It must be independent – much like DNS – without being tied to a specific application, platform, programming language, protocol or hardware component;

Centralized - It must be manageable centrally, regardless of the number of components in the infrastructure and how distributed they may be across the globe;

Location-transparent - It must protect information regardless of whether the application and its data is resident within an enterprise's closed perimeter or in a Cloud Service Provider's (CSP) cloud;
Regulation-transparent - It must comply with data-security regulations anywhere in the world, regardless of where the data-owning company, its customers, partners, suppliers and/or service-providers maintain their headquarters and operations;

Auto-scaling - It must dynamically scale up to handle transaction surges or scale down to conserve resources, while maintaining uniform response-times to applications, without the need for human intervention;

Secure - It must support industry best-practices for cryptographic key-management: industry-standard algorithms, split-key knowledge amongst key-custodians, cryptographic hardware modules, etc.;

Free and Open-Source - Ideally, the infrastructure should be capable of being built with free and open-source software (FOSS) to ensure it is within the reach of even small companies; however, this is not mandatory.

DEI Reference Implementation

While a DEI can be built in many different ways, this paper defines a DEI Reference Implementation (DEIRI - pronounced “Dairy”) that has proven to be successful in multiple industries and companies of different sizes and locations.

The DEIRI is based on the following diagram and depicts a minimal Production implementation that addresses a fixed level of business requirements. Depending on a company's software development and quality-assurance practices, it may choose to deploy similar DEIs for Development and Quality Assurance environments. It does not matter whether the DEIRI is implemented within the company's perimeter or is hosted by a service-provider within a virtual perimeter dedicated to the company⁵.

The following components make up a DEIRI:

Redundant locations;
Front-end Processors;
Cryptographic Engines;
Key Management Systems;
Identity and Access Management System;
Plaintext/Ciphertext Storage Locations;
Optional Encrypted Volumes.

Redundant locations

The DEIRI has a minimum of two geographically separated deployment locations where components of the DEI are duplicated for redundancy; replication between the separated DEI keeps them synchronized.

Depending on their risk-aversion, legal and performance requirements, companies may deploy a DEI in as many locations as necessary. Global companies are likely to benefit from a DEI on each continent they operate in, for optimizing responsiveness to business transactions, addressing component and location-specific disasters, and/or complying with country-specific legal obligations

(Click on the image to enlarge it)

Front-end Processor

A DEIRI has at least one Front-end Processor (FEP) per location serving as the primary interface to applications requiring document-encryption and decryption services. The FEP is a stateful machine with a local database and provides web-services to applications.

The FEP is responsible for the following functions:

Ensuring the correctness of parameters in a request;
Authenticating and authorizing the request against an Identity and Access Management (IAM) system;
Scheduling the request on a Cryptographic Engine (CE);
Storing the output of the cryptographic operation in the appropriate storage location;
Replicating objects to other FEP within the DEI;
Taking over the processing of incomplete requests from a failed FEP in the DEI; and
Managing CE so they may be started or shutdown depending on the current processing load.

Cryptographic Engine

A DEIRI has a private cloud of Cryptographic Engines (CE) per location serving as the auto-scaling processing engine for encryption and decryption requests. They are stateless machines without any local storage and are responsible for the following functions (for encryption requests):

Receiving the requests from an FEP;
Authenticating and authorizing the request against an IAM;
Generating a cryptographic key in accordance with defined policies;
Escrowing the cryptographic key within a Key Management System (KMS) in the DEI;
Encrypting the supplied plaintext with the generated and escrowed cryptographic key;
Storing the ciphertext (encrypted data) on the configured/specified storage location; and
Handing the results of the request back to the FEP.

During a decryption request, the CE is responsible for:

Receiving the request from an FEP;
Authenticating and authorizing the request against an IAM;
Retrieving the ciphertext from the specified location and parsing it;
Recovering the escrowed cryptographic key from the KMS system in the DEI;
Decrypting the ciphertext;
Storing the plaintext on the configured/specified storage location; and
Handing the results of the request back to the FEP.

Key Management System

A DEIRI has at least one Key Management System (KMS) per location serving as the centralized repository and manager of all cryptographic keys. It is a stateful machine with a local database, and provides web-services to applications for the encryption and decryption of small, structured data-elements. It can be used by applications to request the encryption of data such as credit-card numbers, drivers license numbers, etc., without the overhead of going through the FEP.

The KMS can also be called upon by a CE and optional Encrypted Volumes (EV) to encrypt and store cryptographic keys, serving as a secure key-vault to the enterprise.

The KMS is responsible for the following functions in an encryption request:

Receiving the request from an application, CE or EV;
Ensuring the correctness of parameters in a request;
Authenticating and authorizing the request against an IAM;
Generating a cryptographic key in accordance with defined policies;
Encrypting and escrowing the cryptographic key within its internal database;
Encrypting the supplied plaintext⁶ with the generated and escrowed encryption key;
Storing the result of the cryptographic operation in its internal database; and
Replicating objects to other KMS within the DEI.

During a decryption request, the KMS is responsible for:

Receiving the request from an application, CE or EV;
Ensuring the correctness of parameters in a request;
Authenticating and authorizing the request against an IAM;
Retrieving the ciphertext and associated meta-data from its internal database;
Recovering the escrowed cryptographic key from its internal database;
Decrypting the recovered ciphertext with the recovered key; and
Handing the results of the request back to the calling application.

Identity and Access Management System

A DEIRI has an Identity and Access Management (IAM) system serving the DEI. The IAM is based on a Lightweight Directory Access Protocol (LDAP)-based Directory Server, and may be an existing enterprise IAM or one dedicated to the DEI. It is responsible for authenticating and authorizing requests to all components of the DEI and can be managed independently or through administrative functions on the DEI's FEP. The IAM is redundant and accessible from all DEI locations for business continuity.

Storage Locations

A DEIRI has one or more Storage Locations (SL) serving the DEI. They may be Direct Attached Storage (DAS), Network Attached Storage (NAS), Storage Area Networks (SAN) or Cloud Storage from commercial service providers or within the enterprise. The FEP and/or CE are capable of communicating with the SL and managing a large number of files on them.

When passing plaintext blobs between components in the DEI, the plaintext blobs are stored in Temporary Storage Locations (TSL), which are cleaned up by the FEP upon successful encryptions and by the calling applications upon successful decryptions. TSLs are always configured on DAS, NAS or SAN devices within the control of the data-owning company, and always inaccessible outside the company's perimeter.

Ciphertext files are stored on Permanent Storage Locations (PSL) which may be on DAS, NAS, SAN or Cloud Storage - anywhere the company chooses, since the data is encrypted on the CE (or the KMS in the case of small, structured data-elements) and the cryptographic key is within the KMS in the controlled part of the DEI.

Since the DEI is an implementation of the RC3 architecture, when the CEs and KMS are within a perimeter under the control of the data-owning company, the ciphertext comes back to the controlled environment before it is decrypted. This allows companies to comply with data-security regulations by simply keeping the CE and KMS in the jurisdiction controlled by the regulation regardless of where the ciphertext itself resides.

For example, if the CE and KMS were deployed within a member nation of the European Union (EU) in accordance with the security requirements of the EU Directive, companies would be in compliance with the Directives' security provisions⁷ even if the ciphertext were stored in a public cloud outside the EU. This is because the cryptographic keys never leave the safe-harbor of the DEI and, consequently, the ciphertext must come back into the EU's safe-harbor to be decrypted.

Encrypted Volumes

An Encrypted Volume (EV) is one in which the underlying storage volume or file-system is encrypted by a kernel driver or hardware component in the storage device. An EV does not require applications to be aware of cryptographic services since the EV assumes responsibility for encrypting/decrypting data, transparently, as it is written to/read from the volume.

However, since an EV does not distinguish between different applications and/or users, once an EV is accessible to a system, all software – including malware – may gain access to decrypted data. On the other hand, a cryptographically-aware application would make decrypted content available only to authorized roles/users within the application. Unless authorized users and/or applications themselves are compromised, breaching sensitive data outside the cryptographically-aware applications remains very difficult.

A DEIRI does not necessarily need EVs; however, they are useful as a transition-technology until the business application can become cryptographically-aware. In such an implementation, EVs will store their cryptographic keys within the KMS of the DEI and recover them only when necessary; this eliminates the risk of the key becoming compromised if the EV is stolen or lost.

Typical Transaction in the DEIRI

A DEIRI application can be written in any programming language. When a business transaction involves sensitive data (Class-1 and Class-2 data in the RC3 architecture) the application, typically, follows the following steps:

The application makes a web-service request of the FEP with the required web-service operations' parameters;

The FEP receives the web-service request and validates the request's input parameters;

The FEP authenticates and authorizes the request against the configured IAM;

Upon accepting the request, the FEP stores it in a Request queue and replicates it to other FEPs in the DEI. This ensures that the request is processed even if the receiving FEP becomes inoperable before processing it. Note that received – but non-replicated requests – are not likely to be processed by any FEP if the receiving FEP becomes inoperable before replicating the request. For the sake of this illustration, we will assume the receiving FEP remains operable for the entire transaction;

Depending on the current processing load of all CE in the private cloud, the FEP chooses a CE and schedules the request on a specific CE. If the FEP realizes that all CE are operating at or above a configured threshold, the FEP automatically starts up a new CE in the private cloud (to the limit of virtual machines available to it in the private cloud) and schedules the request on the new CE;

The CE receives the request from the FEP, and authenticates & authorizes the request against the configured IAM⁸;

If the transaction is for an encryption:

Data Encryption Key

DEK

⁹

Key Encrypting Key

KEK

Token

Permanent Storage Location (PSL)

RequestID

pathname/URI

If the transaction is for a decryption:

Temporary Storage Location (TSL)

Where the plaintext is a small, structured data-element, as in a credit-card number, a bank-account number, etc., applications may call the KMS directly for the direct encryption and Tokenization of the data-element. This not only allows for faster processing, but better optimizes the different components of the DEI within a company;

It is even possible for applications to make direct requests to dedicated CE of a DEI for direct control of the request scheduling and meta-data management, if desired.

Performance

How does a DEI hold up in a real-world environment? The complexity of the infrastructure, and communication between different components of the infrastructure, seem to imply that while it might address one set of problems related to data-management, control and security, it might introduce another set of problems with respect to business transaction performance and usability.

An implementation of a DEI for an e-commerce company in the US recently provides some insight into the performance of the DEI. While the business use-case described here might not apply to others, some information may be extrapolated from this implementation. In order to provide context for the performance metrics, the following technical details of the DEI are provided:

The FEPs are industry-standard six-core, 64-bit x86 processors operating at 3.2 Ghz with 16-GB of DDR3 DRAM operating at 1600Mhz. They use 64-bit CentOS Linux 6.x for an operating system. FEP services are provided by a Java Enterprise Edition 6 (JEE6) application running within an application server. Service requesters are authenticated and authorized against an LDAP Directory. The FEP's persistent data is stored in a local relational database management system, while bi-directional replication between FEPs is managed in the application layer;
The CE virtual machines operate on an industry-standard single-core, 64-bit x86 processor operating at 3.2Ghz with 8GB of DDR3 DRAM operating at 1600Mhz. They use 64-bit CentOS Linux 6.x for an operating system. CE services are provided by a JEE6 application running within an application server. Service requesters are authenticated and authorized against an LDAP Directory. There is no persistent data on a CE, and as such, there is no local database or replication services on this component;
The KMs are industry-standard six-core, 64-bit x86 processors operating at 3.2 Ghz with 8-GB of DDR3 DRAM operating at 1600Mhz. They use 64-bit CentOS Linux 6.x for an operating system. KM services are provided by a JEE5 application running within an application server. Service requesters are authenticated and authorized against an LDAP Directory. The KM's persistent data is stored in a local relational database management system, while replication between KMs occur at the database layer. Cryptographic operations of the KM use a Trusted Platform Module (TPM) with a true random number generator (TRNG) within the TPM;
Work-in-progress and completed files are stored on a NAS device using the Network File System (NFS) protocol;
The encryption algorithm uses the Advanced Encryption Standard (AES) algorithm with 256-bit keys; the hashing algorithm uses the Secure Hashing Algorithm (SHA-256) message digests while public-key cryptographic algorithm use 2048-bit RSA key-pairs;
Communications between all layers of the DEI, with the exception of the NAS (which uses network-segmentation for access control), are over SSL/TLS;
The network consists of gigabit Ethernet speeds between all components of the DEI, with the exception of the wide-area network between data-centers;
Every component of the DEI authenticates and authorizes requests - no assumptions are made about up-stream service requesters at down-stream layers;
Messages are logged at each component of the DEI, in a manner consistent with audit-requirements;
All software components of this DEI - with the exception of the CE private cloud's hypervisor - consists of open-source software.

Given the configuration above, the DEI implementation for this e-commerce company provided the following metrics:

The application using this infrastructure migrated 6+ million Portable Document Format (PDF) files of sizes ranging from 50K-200K each from its inventory to the DEI;
Daily transaction loads range between 20,000-25,000 PDF files steadily increasing the inventory of encrypted files managed within the DEI;
Following the typical transaction described in an earlier section of this document, the DEI executed web-service operations for document encryption in an average of 300ms each;
Similarly, the DEI executed web-service operations to decrypt documents in an average of 200ms each;
A second application, encrypting and decrypting Base64-encoded images of sizes ranging between 2-3K each, saw the DEI deliver cryptographic web-services at an average rate of 100 transactions per second;

During internal testing, the author's company was able to benchmark (on equipment less capable than that described above) the encryption of large files at the rate of one (1) gigabyte per minute.

While the example cited above is unlikely to match other business use-cases, given the configurability of the DEI and its components, companies can expect to tailor a DEI's performance and throughput through the use of faster components, more components (to distribute the workload), cryptographic accelerators and other performance-enhancing techniques. Just as network and storage technologies evolved from simpler configurations addressing local needs to highly complex infrastructures handling global requirements, the DEI is a similar evolution from simpler cryptographic tools to a sophisticated infrastructure designed to address scalable, highly-available and secure cryptographic services for the enterprise.

Summary

Cryptography components have been available for over forty years to protect sensitive information within business applications; sophisticated cryptographic key-management has been available for nearly a decade to address some of the complexities involved in handling cryptographic keys. The DEI is the logical evolution of these technologies to make data-protection an ubiquitous service on the network, accessible to systems and applications through a uniform interface, with the ability to address diverse data-security regulations while leveraging the cloud for business benefits.

About the author

Arshad Noor is the CTO of StrongAuth, Inc., a Silicon Valley company focused on enterprise key-management infrastructure for 11+ years. Arshad has been in the IT industry for 26+ years building applications and complex IT infrastructure for some of the largest companies in the world. He is the creator of the industry's first open-source symmetric key-management system and many cryptographic tools to simplify the use of this technology within business applications. He is the author of many white-papers – including the Regulatory Compliant Cloud Computing (RC3) architecture. He can be reached at arshad.noor@strongauth.com.

¹ As Sony Counts Hacking Costs, Analysts See Billion-Dollar Bill – The Wall Street Journal, May 09, 2011

² Data Encryption Standard (DES) - Wikipedia

³ Payment Card Industry Security Standards Council

⁴ Build a regulatory compliant web application – IBM, March 2012
Rethinking web application architectures for the cloud – InfoQ, December 2011

⁵ While it is our opinion that a data-owner company can outsource the operations of a DEI to anyone, it remains their responsibility to enforce the policies under which the DEI is used and managed. This author recommends consulting with legal counsel within their jurisdiction to determine their legal obligations before considering outsourcing a DEI.

⁶ The reader should be careful to recognize that when the CE sends an encryption request to the KMS, it is sending a cryptographic key as the plaintext to be escrowed. The KMS treats all plaintext as opaque objects and generates its own internal cryptographic keys to protect the supplied plaintext. Consequently, the cryptographic key generated in the KMS can function as a Data Encryption Key (DEK) when encrypting non-cryptographic plaintext, or as a Key Encrypting Key (KEK) when encrypting cryptographic keys. The cryptographic keys generated within the KMS are themselves, protected by a chain of KEKs, terminating in the Master KEK inside a cryptographic hardware module.

⁷ It is, nonetheless, prudent for companies to consult with their counsel for definitive legal advice on this implementation architecture.

⁸ The DEI operates on the premise that the network cannot be trusted. As such, it authenticates and authorizes requests in every component of the DEI providing for the highest level of security in the application. This lends itself to the possibility that DEI components can be geographically separated across the internet and yet operate securely using Secure Socket Layer (SSL) or Transport layer Security (TLS).

⁹ It is also possible for the KMS generate the cryptographic key, escrow it within its internal database and send the cryptographic key to the CE. However, a DEI is likely to have far more CE than KMS devices. Consequently, the DEI will experience greater throughput with more CE generating symmetric cryptographic keys than with fewer KMS doing so. From a security point-of-view, since the CE has visibility to plaintext sensitive data and the cryptographic key, its risk is not elevated by having it generate keys locally and sending it to the KMS for escrow.

InfoQ Software Architects' Newsletter

Login with:

Don't have an InfoQ account?