ptg 344 Encrypting Columns Using a Passphrase As our first example, let’s keep things simple and look at how to encrypt a column using a passphrase. To do so, let’s look at the Sales.CreditCard table, which currently stores card numbers in cleartext: select top 5 * from Sales.CreditCard go CreditCardID CardType CardNumber ExpMonth ExpYear ModifiedDate 1 SuperiorCard 33332664695310 11 2006 2007-08-30 2 Distinguish 55552127249722 8 2005 2008-01-06 3 ColonialVoice 77778344838353 7 2005 2008-02-15 4 ColonialVoice 77774915718248 7 2006 2007-06-21 5 Vista 11114404600042 4 2005 2007-03-05 Credit card numbers really should not be stored in their cleartext form in the database, so to fix this, first create a copy of the Sales.CreditCard table and define the CardNumber_encrypt column as a varbinary(256) so you can store the encrypted credit card numbers in the column (encrypted columns in SQL Server 2008 can be stored only as varbinary values): USE AdventureWorks2008R2; GO select CreditCardID, CardType, CardNumber_encrypt = CONVERT(varbinary(256), CardNumber), ExpMonth, ExpYear, ModifiedDate into Sales.CreditCard_encrypt from Sales.CreditCard where 1 = 2 Now, you can populate the CreditCard_encrypt table with rows from the original CreditCard table using the EncryptByPassPhrase function to encrypt the credit card numbers as the rows are copied over: declare @passphrase varchar(128) set @passphrase = ‘unencrypted credit card numbers are bad, um-kay’ insert Sales.CreditCard_encrypt ( CardType, CardNumber_encrypt, ExpMonth, ExpYear, CHAPTER 12 Data Encryption Download from www.wowebook.com ptg 345 Column-Level Encryption 12 ModifiedDate ) select top 5 CardType, CardNumber_encrypt = EncryptByPassPhrase(@passphrase, CardNumber), ExpMonth, ExpYear, ModifiedDate from Sales.CreditCard Now, try a query against the CreditCard_encrypt table without decrypting the data and see what it returns (note, for display purposes, the values in the CardNumber_encrypt column have been truncated): select * from Sales.CreditCard_encrypt go CreditCardID CardType CardNumber_encrypt ExpMonth ExpYear ModifiedDate 1 SuperiorCard 0x010000007C65089E 11 2006 2007-08-30 2 Distinguish 0x010000000C624987 8 2005 2008-01-06 3 ColonialVoice 0x01000000AA8761A0 7 2005 2008-02-15 4 ColonialVoice 0x010000002C2857CC 7 2006 2007-06-21 5 Vista 0x0100000095F6730D 4 2005 2007-03-05 In the preceding results, you can see that the credit card numbers have been encrypted as a varbinary value, and no meaningful information can be obtained from this. To view the data in its unencrypted form, you need to use the DecryptByPassPhrase function and convert the value back to an nvarchar(25): declare @passphrase varchar(128) set @passphrase = ‘unencrypted credit card numbers are bad, um-kay’ select CreditCardID, CardType, CardNumber = convert(nvarchar(25), DecryptByPassPhrase(@passphrase, CardNumber_encrypt)), ExpMonth, ExpYear, ModifiedDate from Sales.CreditCard_encrypt GO Download from www.wowebook.com ptg 346 CreditCardID CardType CardNumber ExpMonth ExpYear ModifiedDate 1 SuperiorCard 33332664695310 11 2006 2007-08-30 2 Distinguish 55552127249722 8 2005 2008-01-06 3 ColonialVoice 77778344838353 7 2005 2008-02-15 4 ColonialVoice 77774915718248 7 2006 2007-06-21 5 Vista 11114404600042 4 2005 2007-03-05 So that’s a simple example showing how to encrypt a column. You may be thinking, however, using a passphrase like this probably isn’t very secure. The passphrase used to encrypt the column would have to be shared with all users and applications that need to store or retrieve data in the CreditCard_encrypt table. A shared passphrase like this can be easily compromised, and then the data is visible to anyone who can gain access to the database. It is usually more secure to encrypt data using a symmetric key or certificate. Encrypting Columns Using a Certificate One solution to the problem of encrypting using a shared passphrase is to encrypt the data using a certificate. A primary benefit of certificates is that they relieve hosts of the need to maintain a set of passwords for individual subjects. Instead, the host merely estab- lishes trust in a certificate issuer, which may then sign an unlimited number of certificates. Certificates can be created within SQL Server 2008 using the CREATE CERTIFICATE command. The certificate created is a database-level securable that follows the X.509 stan- dard and supports X.509 V1 fields. The CREATE CERTIFICATE command can load a certifi- cate from a file or assembly, or it can also generate a key pair and create a self-signed certificate. The ENCRYPTION BY PASSWORD option is not required; the private key of the certificate is encrypted using the database master key. When the private key is encrypted using the database master key, you do not have to specify a decryption password when retrieving the data using the certificate. The first step is to create the certificate with the CREATE CERTIFICATE command: USE AdventureWorks2008R2; CREATE CERTIFICATE BillingDept01 WITH SUBJECT = ‘Credit Card Billing’ GO After you create the certificate, the next step is to create a symmetric key that will be encrypted by the certificate. You can use many different algorithms for encrypting keys. The supported encryption algorithms for the symmetric key are DES, TRIPLE_DES, RC2, RC4, RC4_128, DESX, AES_128, AES_192, and AES_256. The following code creates a symmetric key using the AES_256 encryption algorithm and encrypts it using the BillingDept01 certificate: USE AdventureWorks2008R2; CREATE SYMMETRIC KEY BillingKey2010 WITH ALGORITHM = AES_256 ENCRYPTION BY CERTIFICATE BillingDept01; GO CHAPTER 12 Data Encryption Download from www.wowebook.com ptg 347 Column-Level Encryption 12 Now you empty out the rows inserted previously in the CreditCard_encrypt table using the PassPhrase encryption method by truncating it: USE AdventureWorks2008R2; Truncate table Sales.CreditCard_encrypt Next reinsert rows from the CreditCard table, this time using the symmetric key associated with the certificate to encrypt the data using the EncryptByKey function. The EncryptByKey function requires the GUID of the symmetric key as the first parameter. You can look up this identifier by running a query against the sys.symmetric_keys table or simply use the KEY_GUID() function, as in this example: USE AdventureWorks2008R2; First, decrypt the key using the BillingDept01 certificate OPEN SYMMETRIC KEY BillingKey2010 DECRYPTION BY CERTIFICATE BillingDept01 Now, insert the rows using the symmetric key encrypted by the certificate insert Sales.CreditCard_encrypt ( CardType, CardNumber_encrypt, ExpMonth, ExpYear, ModifiedDate ) select top 5 CardType, CardNumber_encrypt = EncryptByKey(KEY_GUID(‘BillingKey2010’), CardNumber), ExpMonth, ExpYear, ModifiedDate from Sales.CreditCard If you examine the contents of the CreditCard_encrypt table, you can see that they have been encrypted: select * from Sales.CreditCard_encrypt go CreditCardID CardType CardNumber_encrypt ExpMonth ExpYear ModifiedDate 1 SuperiorCard 0x0046C380E7A27749 11 2006 2007-08-30 2 Distinguish 0x0046C380E7A27749 8 2005 2008-01-06 Download from www.wowebook.com ptg 348 3 ColonialVoice 0x0046C380E7A27749 7 2005 2008-02-15 4 ColonialVoice 0x0046C380E7A27749 7 2006 2007-06-21 5 Vista 0x0046C380E7A27749 4 2005 2007-03-05 Now, an authorized user that specifies the appropriate certificate can retrieve the data by using DecryptByKey function: USE AdventureWorks2008R2; OPEN SYMMETRIC KEY BillingKey2010 DECRYPTION BY CERTIFICATE BillingDept01 select CardType, CardNumber = convert(nvarchar(25), DecryptByKey(CardNumber_encrypt)), ExpMonth, ExpYear, ModifiedDate from Sales.CreditCard_encrypt go CreditCardID CardType CardNumber ExpMonth ExpYear ModifiedDate 1 SuperiorCard 33332664695310 11 2006 2007-08-30 2 Distinguish 55552127249722 8 2005 2008-01-06 3 ColonialVoice 77778344838353 7 2005 2008-02-15 4 ColonialVoice 77774915718248 7 2006 2007-06-21 5 Vista 11114404600042 4 2005 2007-03-05 When you are done using a key, it is good practice to close the key using the CLOSE SYMMETRIC KEY statement: CLOSE SYMMETRIC KEY BillingKey2010 The keys defined in a database can be viewed through the system catalog table, sys.symmetric_keys: select name, pvt_key_encryption_type, issuer_name, subject, expiry_date = CAST(expiry_date as DATE), start_date = CAST(start_date as DATE) from sys.certificates go name pvt_key_encryption_type issuer_name subject expiry_date start_date CHAPTER 12 Data Encryption Download from www.wowebook.com ptg 349 Column-Level Encryption 12 BillingDept01 MK Credit Card Billing Credit Card Billing 2011-05-01 2010-05-01 The certificates defined in a database can be viewed through the system catalog tables, sys.certificates: select name, key_length, key_algorithm, algorithm_desc, create_date = CAST(create_date as DATE), modify_date = CAST(create_date as DATE), key_guid from sys.symmetric_keys go name key_length key_algorithm algorithm_desc create_date modify_date key_guid ##MS_DatabaseMasterKey## 128 D3 TRIPLE_DES 2010-04-30 2010-04-30 A3550B00-6BAE-41E2-A1BC-D784DC35779E BillingKey2010 256 A3 AES_256 2010-04-30 2010-04-30 10C5C800-0B4C-44C2-9F71-5415007C2E81 If the usage of the key and certificate are no longer needed, they should be dropped from the database: DROP SYMMETRIC KEY BillingKey2010 DROP CERTIFICATE BillingDept01 NOTE There is a lot more information about column-level encryption and key management that could be discussed at this point, but such discussion would be beyond the scope of this chapter; our intent here is to merely introduce the concepts of data encryption. For more information on column-level encryption, refer to SQL Server 2008 Books Online. Download from www.wowebook.com ptg 350 Transparent Data Encryption As mentioned previously, transparent data encryption (TDE) is a new feature introduced in SQL Server 2008 that allows an entire database to be encrypted. Unlike column-level encryption, in TDE the encryption and decryption of data is performed automatically by the Database Engine, and this is fully transparent to the end user and applications. No changes to the database or applications are needed. Consequently, TDE is the simpler choice when bulk encryption of data is required to meet regulatory compliance or corpo- rate data security standards. The encryption of a database using TDE helps prevent the unauthorized access of data in the scenario in which physical media or backups have been lost or stolen. Transparent data encryption uses a database encryption key (DEK) for encrypting the database. The DEK is stored in the database boot record and is secured by a certificate stored in the master data- base. The database master key is protected by the service master key, which is in turn protected by the Data Protection API. When TDE is enabled on a database, attaching data files to another SQL Server instance or the restoring of a backup to another SQL Server instance is not permitted until the certificate that was used to secure the DEK is available. NOTE Optionally, the DEK can be secured by an asymmetric key that resides in a Hardware Security Module with the support of Extensible Key Management. The private key of the certificate is encrypted with the database master key that is a symmetric key, which is usually protected with a strong password. For example, if a hard drive that contains database files is stolen, without TDE, those data- base files can be attached in another SQL Server instance, thus allowing access to the nonencrypted data in those files. With TDE, the data and log files are automatically encrypted, and the data within these files cannot be accessed without an encryption key. Additionally, backups of databases that have TDE enabled are also encrypted automati- cally. We’re all familiar with stories about how backup tapes containing sensitive informa- tion have been lost or stolen. With TDE, the data in the backup files is completely useless without also having access to the key used to encrypt that data. The encryption and decryption of data with TDE are performed at the page level as data moves between the buffer pool and disk. Data residing in the buffer pools is not encrypted. TDE’s specific purpose is to protect data at rest by encrypting the physical files of the database, rather than the data itself. These physical files include the database file ( .mdf), transaction log file (.ldf), and backup files (.bak). Data pages are encrypted as they are written from the buffer pool to the database files on disk. Conversely, the data is decrypted at the page level when the data is read in from the files on disk into the buffer pool. The encryption and decryption are done using a background process transparent to the database user. Although additional CPU resources are required to implement TDE, overall, this approach offers much better performance than column-level encryption. According to Microsoft, the performance hit averages only about 3–5%. CHAPTER 12 Data Encryption Download from www.wowebook.com ptg 351 Transparent Data Encr yption 12 TDE supports several different encryption options, such as AES with 128-bit, 192-bit, or 256-bit keys or 3 Key Triple DES. You make your choice when implementing TDE. Implementing Transparent Data Encryption Like many encryption scenarios, TDE is dependent on an encryption key. The TDE data- base encryption key is a symmetric key that secures the encrypted database. The DEK is protected using a certificate stored in the master database of the SQL Server instance where the encrypted database is installed. Implementing TDE for a specific database is accomplished by following these steps: . Create a master key. . Create or obtain a certificate protected by the master key. . Create a database encryption key and protect it by the certificate. . Configure the database to use encryption. Listing 12.1 demonstrates the commands needed to encrypt the AdventureWorks2008R2 database, including the creation of a master key, certificate, and DEK protected by the certificate. LISTING 12.1 Encrypting the AdventureWorks2008R2 Database USE master; GO Create the master key which is stored in the master database CREATE MASTER KEY ENCRYPTION BY PASSWORD = ‘mystrongpassword$$’; GO Create a certificate that is also stored in the master database. This certificate will be used to encrypt a user database CREATE CERTIFICATE MyCertificate with SUBJECT = ‘Certificate stored in Master Db’ GO Create a Database Encryption Key (DEK) that is based on the previously created certificate The DEK is stored in the user database USE AdventureWorks2008R2 GO CREATE DATABASE ENCRYPTION KEY WITH ALGORITHM = AES_256 ENCRYPTION BY SERVER CERTIFICATE MyCertificate GO Download from www.wowebook.com ptg 352 Turn the encryption on for the AdventureWorks2008R2 ALTER DATABASE AdventureWorks2008R2 SET ENCRYPTION ON GO After you enable TDE, you might want to monitor the progress of the encryption. This can be done by running the following query: SELECT DBName = DB_NAME(database_id), encryption_state FROM sys.dm_database_encryption_keys ; GO DBName encryption_state tempdb 3 AdventureWorks2008R2 3 This query returns the database encryption state. A database encryption state of 2 means that encryption has begun, and an encryption state of 3 indicates that encryption has completed. When the tempdb database and user database you are encrypting reach a state of 3, the entire user database and tempdb database are encrypted. When TDE is enabled for a given database, encryption is applied to a variety of files related to the database, including the following: . Database Data Files—All data files that contain the database data are encrypted. These files typically have the extension .mdf or .ndf. . Database Log Files—The transaction log files are encrypted so that no clear text is visible in the files. These files typically have the extension .ldf. . Database Backups—All database backups, including full, differential, and log, are encrypted. . Tempdb—If any databases on a server are encrypted with TDE, the tempdb database is also encrypted. In addition to these files, you can also manually enable TDE on the distribution and subscriber database involved in replication. This encrypts a portion of data involved in replication, but there are still some unencrypted files. Snapshots from snapshot replica- tion as well as the initial distribution of data for transactional and merge replication are not encrypted. Managing TDE in SSMS You can also view and manage transparent data encryption in SQL Server Management Studio. To do so, right-click on the database in the Object Explorer for which you want to configure TDE and select Tasks; then select Manage Database Encryption. If you are setting up the initial configuration for TDE in a database, you see a dialog like that shown in Figure 12.2. CHAPTER 12 Data Encryption Download from www.wowebook.com ptg 353 FIGURE 12.2 Enabling TDE in SSMS. Transparent Data Encr yption 12 The options available in this dialog correspond to commands shown in Listing 12.1. You specify the encryption algorithm to be used and the server certificate used to protect the database encryption key. When you are ready to enable TDE for the database, put a check mark in the Set Database Encryption On check box. If TDE is already enabled for a database, the dialog changes to provide you with options to re-encrypt the database encryption key and to regenerate the DEK using a different encryption algorithm. You can also enable/disable database encryption (see Figure 12.3). A second page displays the current TDE properties and encryption state of the database (see Figure 12.4). Backing Up TDE Certificates and Keys The most important issue to consider when using TDE is that you must back up and retain the certificate and private key associated with the encryption. If these things are lost or unavailable, you are not able to restore or attach the encrypted database files on another server. The following warning message displayed after creating a certificate drives home this point: Warning: The certificate used for encrypting the database encryption key has not been backed up. You should immediately back up the certificate and the private key associated with the certificate. If the certificate ever becomes unavailable or if you must restore or attach the database on another server, you must have backups of both the certificate and the private key or you will not be able to open the database. Backup up the certificate and private key Download from www.wowebook.com . credit card numbers in the column (encrypted columns in SQL Server 2008 can be stored only as varbinary values): USE AdventureWorks200 8R2; GO select CreditCardID, CardType, CardNumber_encrypt. to SQL Server 2008 Books Online. Download from www.wowebook.com ptg 350 Transparent Data Encryption As mentioned previously, transparent data encryption (TDE) is a new feature introduced in SQL. TDE is enabled on a database, attaching data files to another SQL Server instance or the restoring of a backup to another SQL Server instance is not permitted until the certificate that was