Database security, and the lack thereof, has been constantly in the news. It seems like at least once a month we hear about a major breach in a large corporation or government agency. Some of this can be mitigated by applying encryption and hashing techniques, but doing so is tedious so developers often only do it for the most sensitive of data such as passwords.
SQL Server 2016 seeks to make encryption easier via its new Always Encrypted feature. This feature offers a way to ensure that the database never sees unencrypted values of sensitive columns without the need to rewrite the application. In order to maintain reasonable performance, non-sensitve columns such as primary keys are left unencrypted.
The actual process of encrypting and decrypting data is handled in the database driver level. While the database only sees encrypted values, the application code works exclusively with unencrypted data. When a query is executed, the driver automatically looks up the master key in the Windows Certificate Store (or other OS-dependent location). The master key is then used to decrypt a column specific key, which in turn is used for encrypting and decrypting fields and parameters.
Scenarios
Microsoft offers three use-cases for Always Encrypted.
Client and Data On-Premises
A customer has a client application and SQL Server both running on-premises, at their business location. The customer wants to hire an external vendor to administer SQL Server. In order to protect sensitive data stored in SQL Server, the customer uses Always Encrypted to ensure the separation of duties between database administrators and application administrators. The customer stores plaintext values of Always Encrypted keys in a trusted key store which the client application can access. SQL Server administrators have no access to the keys and, therefore, are unable to decrypt sensitive data stored in SQL Server.
Client On-Premises with Data in Azure
A customer has an on-premises client application at their business location. The application operates on sensitive data stored in a database hosted in Azure (for example in SQL Server running in a virtual machine on Microsoft Azure). The customer uses Always Encrypted and stores Always Encrypted keys in a trusted key store hosted on-premises, to ensure Microsoft cloud administrators have no access to sensitive data.
Client and Data in Azure
A customer has a client application, hosted in Microsoft Azure (e.g. in a worker role or a web role), which operates on sensitive data stored also stored in Microsoft Azure. The customer uses Always Encrypted to reduce security attack surface area (the data is always encrypted in the database and on the machine hosting the database).
Types of Encryption
SQL Server offers two encryption modes: deterministic and random. Deterministic encryption ensures that a given value always has the same encrypted representation. This allows you to use the column for equality comparisons, joins, and grouping.
The downside of deterministic encryption is that is can “allow unauthorized users to guess information about encrypted values by examining patterns in the encrypted column”. This is especially true when there are a small number of possible values.
For more security, you can use random encryption. Like salting a password, this prevents guessing by ensuring that a given value’s encrypted representation is never the same twice. Microsoft continues,
Use deterministic encryption for columns that will be used as search or grouping parameters, for example a government ID number. Use randomized encryption, for data such as confidential investigation comments, which are not grouped with other records, or used to join tables.
General Limitations
If a column is encrypted, then all range-like operations such as greater/less than, pattern matching using LIKE, etc. are disallowed. Furthermore, you can’t pass encrypted values to functions, user-defined or otherwise, because the database doesn’t have access to the unencrypted values.
Equality comparisons can only be performed on columns that use deterministic encryption.
Indexes can only be applied to deterministically encrypted columns.
If joining between two columns, both columns need to use the same column encryption key.
Constants expressions that refer to encrypted columns are not allowed. For example, you cannot write WHERE SSN = '111-11-1111', but you can write WHERE SSN = @SSN. This is necessary because the driver works with the SqlParameter class to handle encryption requirements.
Unsupported data types include: xml, rowversion, image, ntext, text, sql_variant, hierarchyid, geography, geometry, and user defined-types.
Currently the only driver that supports this feature is .NET 4.6.