34 Part I: Oracle Database Security New Features TDE Setup: Oracle 10g TDE does require a small amount of configuration. The setup basically centers on creating the facilities for the key management. TDE uses the Oracle Wallet Manager to maintain encryption keys. You can think of the Oracle Wallet as secure container used to store authentication and signing credentials. These credentials may take the form of certificates needed for Secure Sockets Layer (SSL), Public Key Infrastructure (PKI) private keys, or a TDE Master Key. When the wallet is used to store the TDE Master Key, it’s called an encryption wallet. A user with the ALTER SYSTEM privilege must create this wallet. Then you will find it as a file (ewallet.p12) in one of three locations, as specified by the ENCRYPTION_WALLET_LOCATION in the SQLNET.ora file. If this is not specified, the wallet creating process will look next to the WALLET_LOCATION parameter in SQLNET.ora, and lastly to the default location, <ORACLEBASE>/admin/<SID>/wallet). When first configuring TDE, you must specify a value for the ENCRYPTION_WALLET_ LOCATION parameter, which lives in the SQLNET.ora file—here’s an example: ENCRYPTION_WALLET_LOCATION= (SOURCE=(METHOD=FILE)(METHOD_DATA= (DIRECTORY=c:\<ORACLEBASE\admin\SID\wallet))) Later, we will discuss the options regarding the security, functionality, and location of the wallet; for now, assume the simple case: it exists on the filesystem of our database server in the directory specified earlier. After including the ENCRYPTION_WALLET_LOCATION in the SQLNET.ora file, you can create the wallet and populate it with a generated Master key. You will need the ALTER SYSTEM privilege to issue the following: security_manager@AOS> alter system set key identified by "AppliedOracleSecurity"; This command does two things. It physically creates the file that holds the Oracle Wallet used for TDE, and it generates a Master Key that is then stored in the wallet. The wallet itself can be in one of two states, open or closed. The open state means the wallet is accessible to the database and encryption and decryption can be performed. Closed means that the credentials are still locked in the wallet. This is a security measure; to protect the data, the wallet also needs to be protected. After the creation of a wallet, it is left in an open state until it is closed or the database is restarted. The wallet state can be managed with ALTER SYSTEM commands. The following command closes the wallet: security_manager@AOS> ALTER SYSTEM SET WALLET CLOSE; System altered We can predict that the wallet has been closed by the database and will no longer make the Master Key available to TDE. To check the state of the wallet, check the gv$encryption_wallet view: security_manager@AOS> COL WRL_TYPE format a7 security_manager@AOS> COL WRL_PARAMETER format a55 security_manager@AOS> SELECT WRL_TYPE, WRL_PARAMETER, STATUS FROM gv$encryption_wallet; WRL_TYP WRL_PARAMETER STATUS file /home/oracle/product/11.1.0/admin/aos/encryption_wallet CLOSED security_manager@AOS> Chapter 2: Transparent Data Encryption 35 Note that the process of closing the wallet required only the ALTER SYSTEM privilege and did not require using the wallet password. This should alert you to the fact that ALTER SYSTEM privileges should be granted only to those requiring the permission (see the least privilege discussion in Chapter 1). If a person closes the wallet intentionally or unintentionally, all encryption and decryption halts. Opening the wallet is accomplished in a similar fashion and requires both the ALTER SYSTEM privilege and the wallet password. Note that the use of keyword ENCRYPTION is optional: security_manager@AOS> ALTER SYSTEM SET ENCRYPTION WALLET OPEN IDENTIFIED BY "AppliedOracleSecurity"; System altered. A quick look back at the wallet status in gv$encryption_wallet provides an updated wallet status: security_manager@AOS> SELECT WRL_TYPE, WRL_PARAMETER, STATUS FROM gv$encryption_wallet; WRL_TYP WRL_PARAMETER STATUS file /home/oracle/product/11.1.0/admin/aos/encryption_wallet OPEN As expected, the wallet is in a open state, making the wallet available for the encryption and decryption of data provided by TDE. The Oracle Wallet The Oracle Wallet exists on the filesystem as a PKCS #12, which is a standard key-storage type under Public Key Cryptography Standards. The default filename is ewallet.p12. The file itself is encrypted and must be protected by operating system permissions. The wallet looks like this on the filesystem: oracle@aosdb encryption_wallet] (aos-db)$ pwd /home/oracle/product/11.1.0/admin/aos/encryption_wallet oracle@aosdb encryption_wallet] (aos-db)$ ls -lt -rw 1 oracle dba 1694 Aug 2 16:09 ewallet.p12 NOTE Since the encryption wallet stores the Master Key for all encryption carried out by TDE, it is critical that you back it up! It should be backed up often and should not be stored with datafile backups. The wallet is stored in an encrypted format. To decrypt, you must supply the proper password that releases the encryption key and opens the wallet. Note that the password effectively protects not just the wallet, but all the other credentials inside, such as the other encryption keys and possibly passwords. If the wallet is not open, queries that rely on keys stored in the wallet will fail. Since TDE stores the Master Key in the Oracle Wallet and encrypts table keys using the Master Key, if the wallet is closed, the Master Key is unavailable and no data may be decrypted. As a result, each time the database is restarted, a security administrator must open the wallet before the database can encrypt and decrypt data using TDE. 36 Part I: Oracle Database Security New Features Auto Login With the major emphasis on security and protecting the wallet file and password, you may be surprised to learn that an Auto Login feature is supported. This allows the wallet to remain open— that is, after a reboot of the database or host operating system, the wallet is automatically accessible to the database. There are good reasons for using Auto Login. Having a security administrator and the separation of duties provided by a separate wallet password is often not a requirement in development, testing, and other nonproduction environments. In these situations, an easy way to mirror the production security environment, but still allow for easy restarting of the database and not requiring password entry for the encryption wallet at startup, is to get up the wallet to auto Login. An Auto Login wallet is always open and can be recognized by a second file in the wallet’s directory, cwallet.sso. When a wallet has been set to Auto Login, the directory appears as follows: [oracle@aosdb encryption_wallet] (aos-db)$ ls -lt total 8 -rw 1 oracle dba 1722 Aug 2 16:21 cwallet.sso -rw 1 oracle dba 1694 Aug 2 16:21 ewallet.p12 [oracle@aosdb encryption_wallet] (aos-db)$ Configuring a wallet to perform the Auto Login function uses Oracle Wallet Manager. To start Wallet Manager, type owm at the command line or from Configuration and Administration in the Oracle group from the Windows Start menu. In Wallet Manager, first choose Wallet | Open to open the wallet. After navigating to the directory that stores the wallet you want, the Auto Login will appear with a checkbox on the Wallet menu, as shown in Figure 2-6. Select the Auto Login checkbox and save the wallet. At this point, the wallet will remain open through database and host restarts. FIGURE 2-6 Setting Auto Login using the Wallet Manager Chapter 2: Transparent Data Encryption 37 CAUTION Auto Login is a relatively insecure method of wallet management and its use should be reserved for nonproduction environments. Auto Login creates a wallet that requires no password to open (since it’s always open); you should take extra care when using this feature. TDE’s Key Management The Oracle Wallet stores the Master Key for the database. Rather than using one key to encrypt every table, column-level TDE generates a key for each table with an encrypted column. After generation, this table key is used to encrypt column values. TDE uses symmetric key technology to encrypt and decrypt the data. Symmetric key is the standard approach to performing bulk encryption because of its optimal performance characteristics. As such, the key used to encrypt the data will be the same key used to decrypt the data. Therefore, the key must be stored securely and be available when needed. For tablespace key storage, TDE uses the data dictionary, but it first encrypts the table key using the database Master Key. Figure 2-7 shows an overview. TDE uses two cryptographic keys and one password (the wallet password) to encrypt data. Encrypted data stored in an encrypted column in your table, like the credit card numbers in the earlier example, are encrypted using a table key. These table keys are stored in the Oracle data dictionary after first being encrypted using the encryption Master Key. The Master Key is stored in the Oracle Wallet. Access to the Master Key is then controlled by the wallet’s password. This two-key mechanism for storing keys used in TDE provides the additional benefit of allowing for rekeying of data (as is often specified in security guidelines and mandates) without having to decrypt/re-encrypt each column in each row first. By rekeying only the Master Key, only FIGURE 2-7 Encryption wallet overview 38 Part I: Oracle Database Security New Features the table keys must be decrypted and then encrypted with the new Master Key and not the actual data elements. This helps operationally by dramatically reducing the potential number of reads and writes. When a large number of rows are stored in a table, this amounts to a considerable time savings. NOTE With one very large customer, we calculated that decrypting and then reencrypting a credit card number stored in a table with about 7 billion records would take 17 hours when performed on high-end hardware. Using TDE, rekeying the Master Key required that only the table keys be decrypted and reencrypted, with the new Master Key taking only seconds. Creating an Encrypted Column in a New Table Let’s start with basic column-level encryption. This capability was introduced in the 10g R2 release and simply encrypts the contents of a specific column. The general form of SQL to create a new table with an encrypted column is as follows: SQL> CREATE TABLE <table_name> (<column_name> <date_type> ENCRYPT [algorithm] [nomac] [no salt]); Three elements relating to TDE are relevant to this statement. First, encrypt [algorithm] tells the database that you will be encrypting the column using the specified algorithm. The following algorithms in 10g R2 are available: 3DES168 The 168-bit key length implementation of the Digital Encryption Standard AES128 The 128-bit key length implementation of the Advanced Encryption Standard AES192 The 192-bit key length implementation of the Advanced Encryption Standard (the default if nothing is specified) AES256 The 256-bit key length implementation of the Advanced Encryption Standard The NOMAC directive/parameter was introduced in 10.2.0.4 and is not available in the original 10g R2 (10.2.0.3) release. A message authentication code (MAC) is generated by default when encrypting a column requires an additional 20 bytes of storage for each encrypted value stored. Its purpose is to provide an integrity check for the encrypted data. Certainly, a performance penalty is paid when building the integrity check, as it requires CPU cycles to build and is created for each column inserted into a table with an encrypted column. An obvious storage penalty also results, and while 20 bytes is not much, depending on the volume of data stored in the encrypted table, it could become significant. As is often seen in matters of security, an obvious trade-off exists in performance and the security and integrity of stored data. Salt A salt is a cryptographic tool that effectively makes any encryption algorithm stronger by introducing a random value—in TDE, 16 bytes—that is concatenated with the plaintext to stop “known text” attacks on encrypted data. Although valuable for additional security, adding a salt in your definition also somewhat limits the ability to create meaningful indexes on encrypted columns by essentially removing the relationship between values. The use of a salt is required when data is sufficiently sensitive to ■ ■ ■ ■ Chapter 2: Transparent Data Encryption 39 necessitate the strongest protections possible; it can require that data architects build surrogate keys on tables with naturally occurring keys, making it time-consuming, costly, and detrimental to project schedules. The DDL for creating a new table with an encrypted column in its most basic form is performed by adding the ENCRYPT directive to a CREATE TABLE statement: CREATE TABLE foo (columnA datatype, columnB datatype ENCRYPT); This statement defaults to the Advanced Encryption Standard (AES) encryption algorithm, with a key length of 192 bits, a salt, and a MAC of 20 bits. Any column that you might consider a candidate for an index should use the no salt directive: CREATE TABLE foo (columnA datatype, columnB datatype ENCRYPT no salt); Applied Example In 10g, you can create a table with an encrypted column by adding the word encrypt to the table’s CREATE DDL. This generates a new table key that’s stored in the data dictionary after being encrypted by the database Master Key, and encrypts any data written to that column. Here is an example of the syntax used to encrypt the credit card number from the earlier example using the 192-bit implementation of the AES: system@AOS> CREATE TABLE sh.customer_enc ( 2 cust_id NUMBER(6) PRIMARY KEY, 3 cust_firstname VARCHAR2(20), 4 cust_lastname VARCHAR2(20), 5 cust_address VARCHAR2(40), 6 cust_city VARCHAR2(15), 7 cust_state VARCHAR2(2), 8 cust_zip VARCHAR2(10), 9 cust_credit_card_no VARCHAR2(19) encrypt using 'AES192', 10 cust_credit_card_exp VARCHAR2(4) 11 ) 12 TABLESPACE customer_info_protected 13 / At this point, by reinserting the data into the newly created table, customer_enc, you can validate that the credit card numbers are protected. If you again use the UNIX command strings on the new customer_info_protected datafile, you’ll see that no credit card numbers appear in the clear: oracle@aosdb aos]$ strings customer_info_protected.dbf | more }|{z CUSTOMER_INFO_PROTECTED BEGINCREATE . . %oEM 1209, David Knox 202 Peachtree Rd. Reston 40 Part I: Oracle Database Security New Features 20190 76210D 1008< Richard . . More In the unencrypted example, you were able to see that the credit card number stored for this record in the tablespace’s datafile was clearly visible. By encrypting the column used to store this data, as in this example, data stored using the TDE’s protection is unreadable from a file system perspective and measurably less vulnerable than it was when stored in its original form. TDE vs. DBMS_CRYPTO Package As often happens with technology, there is more to implementing encryption than what appears at first view. Let’s examine the details by comparing TDE to DBMS_CRYPTO. The DBMS_CRYPTO package is used to encrypt values programmatically in PL/SQL that may then be written to the database. To do this, you must give DBMS_CRYPTO the data, an encryption algorithm, and a key to perform the encryption. When the same data is read from the database, it must then be decrypted using the same algorithm and key. Subtle differences can be made by changing padding, modifiers, initialization vectors, and cipher modifiers, but those details are covered sufficiently in the Effective Oracle Database Security 10g by Design text. DBMS_CRYPTO provides one function and two procedures used to encrypt data. The function is built to handle the RAW data type, while the two procedures are large object, or LOB (binary lob object [BLOB] and character large object [CLOB]), centered. Often, the data type in an existing application is a NUMBER, CHAR, or VARCHAR2, which means that another RAW or LOB column must be added, which is not always possible. The second issue that arises when implementing or planning to implement a strategy with DBMS_ CRYPTO is key storage. If the goal is to protect the data, you must protect the keys using very secure methods. Where do you store the keys? Arguably, by storing them in the database with no additional protection, you don’t really gain much over storing the data in plaintext as you rely of the Discretionary Access Controls of Oracle Database, so why couldn’t you rely on those controls to begin with? Among the storage options are the following: Write the keys to a file system using UTL_FILE or another method; however, the procedure or function that reads/writes this data can be protected only using the same methods of access control used by the database. Encrypt the keys with DBMS_CRYPTO (as they are then protected), but then you’re left with another key to protect. Use a key management appliance, called a hardware security module (HSM), that can then make keys available securely. Write a custom solution that manages these keys for you programmatically and provides a secure storage mechanism. The last option basically describes the basic functionality of TDE, but TDE goes beyond this with its tight integration to the database and the SQL engine of Oracle. The following table provides a side-by-side comparison of the attributes of DBMS_CRYPT and TDE. ■ ■ ■ ■ Chapter 2: Transparent Data Encryption 41 Option DBMS_CRYPTO TDE Key management None, must be developed Oracle Wallet / HSM Key storage None, must be developed Dual-key Wallet + data dictionary Implementation Programmatic Declarative (DDL) Algorithms DES, 3DES, AES, RC4, 3DES_2KEY 3DES168, AES (128, 192, and 256 bit) Hardware support None HSM (via PKCS #11) Hashing available Yes (SHA-1) MAC Viewing Encrypted Columns The database dictionary allows administrators and schema owners to see what columns within a particular database or single schema are being stored using encryption and to provide information on the encryption algorithm used in each case. This data is stored in the DBA_ENCRYPTED_ COLUMNS view (for administrators) to view details of all encrypted columns with a database. Similarly, the USER_ENCRYPTED_COLUMNS view allows a schema owner access to the details of encrypted column in the objects they own. security_manager@AOS> COL OWNER format a10 security_manager@AOS> COL TABLE_NAME format a25 security_manager@AOS> COL COLUMN_NAME format a25 security_manager@AOS> COL ENCRYPTION_ALG format a18 security_manager@AOS> COL SALT format a5 security_manager@AOS> SELECT owner, table_name, column_name, encryption_alg, salt FROM dba_encrypted_columns; OWNER TABLE_NAME COLUMN_NAME ENCRYPTION_ALG SALT SH CUSTOMER_ENC TABLE_TYPE AES 192 bits key YES Encrypting an Existing Column In many organizations, applications have been running for many years, having been built on an Oracle database or perhaps built on another database subsystem and later migrated to Oracle Database. It certainly makes sense to look at these legacy applications with a critical eye in regard to security and compliance initiatives. It is quite easy to make some minor changes to the underlying schemas of these historical applications to make them more secure in their handling and use of sensitive data. TDE allows for the modification of existing tables to encrypt columns at rest. The general form for changing an existing column in a table from plaintext to encrypted storage is as follows: SQL> ALTER TABLE <table_name> MODIFY (<column_name> ENCRYPT ['nomac'] [no salt]); Alter table foo modify (columnA datatype encrypt); You should consider both storage and performance when encrypting an existing column. Earlier, we mentioned that adding a MAC to the encrypted data adds 20 bytes; including a salt will add a few more bytes, and the encryption algorithm itself adds a few bytes per column as it “pads” data to create evenly sized blocks of data to encrypt. As a result of this extra space being added to your plaintext data as it is encrypted, it will no longer fit into the blocks it occupied before encryption. In other words, you encrypted “foo” and now have “@#SK^X”, which will no longer fit in the space previously occupied by “foo”. 42 Part I: Oracle Database Security New Features The database handles this additional requirement for more space by chaining the column and storing it out of line in the tablespace’s free list. This chaining will not be noticed in many situations, especially those with tables that have a small number of rows (less than 10,000). When the tables you want to encrypt have greater data volumes, the chaining created by the encryption process will increase the load on the database’s I/O subsystem by potentially doubling the number of read operations as it first reads the row data and then reads the encrypted column in a separate disk read. Knowing that the encryption operation can lead to a slightly greater load on the CPU and storage subsystem gives you some specific areas for preproduction testing. Often, a simple reorganization of the table can be the easiest solution to chaining. In very highly available environments with mission-critical applications, online redefinition should be considered because it takes the effected table offline for only an instant. DBAs will often reorganize a table by first exporting the data (exp or datapump) to pull data out of the existing table, dropping the old cleartext table, and then creating a new table with the sensitive columns encrypted (usually in a newly created tablespace). Reloading the contents of the export into the newly created secure table resolves the row-chaining problem. This technique also takes care of any unencrypted “artifacts” (unencrypted data left in their original data block while newly encrypted blocks are stored in free space) that may remain in datafiles. The caveat to this approach is that it can require quite a bit of time for large tables, and the table will be unavailable while the process is being carried out. NOTE It is definitely a best-practice to reorganize any table that has been modified to include an encrypted column—including tables not originally defined as having an encrypted column and tables with encrypted columns added. Another technique to use is online redefinition. First introduced with 9i, online redefinition allows for reorganization to take place. But instead of the database administrator doing the work of creating the new structure, loading data from the old table into the new, and dropping the old and renaming the new, it instructs the database to handle the whole process and doesn’t require a manual process. To make an in-place, online redefinition of our example table, customer, you would follow these steps to create the table shown in the example: 1. EXECUTE dbms_redefinition.can_redef_table ('SH', 'CUSTOMER'); 2. CREATE TABLE sh.customer_stage as SELECT * FROM sh.customer; 3. ALTER TABLE sh.customer_stage MODIFY (cust_credit_card_no encrypt); 4. EXECUTE dbms_redefinition.start_redef_table ('SH', 'CUSTOMER', 'CUSTOMER_ STAGE'); 5. EXECUTE dbms_redefinition.finish_redef_table ('SH', 'CUSTOMER', 'CUSTOMER_ STAGE'); Example: system@AOS> desc sh.customer; Name Null? Type CUST_ID NOT NULL NUMBER(6) CUST_FIRSTNAME VARCHAR2(20) Chapter 2: Transparent Data Encryption 43 CUST_LASTNAME VARCHAR2(20) CUST_ADDRESS VARCHAR2(40) CUST_CITY VARCHAR2(15) CUST_STATE VARCHAR2(2) CUST_ZIP VARCHAR2(10) CUST_CREDIT_CARD_NO VARCHAR2(19) ENCRYPT CUST_CREDIT_CARD_EXP VARCHAR2(4) system@AOS> desc sh.customer; Name Null? Type CUST_ID NOT NULL NUMBER(6) CUST_FIRSTNAME VARCHAR2(20) CUST_LASTNAME VARCHAR2(20) CUST_ADDRESS VARCHAR2(40) CUST_CITY VARCHAR2(15) CUST_STATE VARCHAR2(2) CUST_ZIP VARCHAR2(10) CUST_CREDIT_CARD_NO VARCHAR2(19) CUST_CREDIT_CARD_EXP VARCHAR2(4) system@AOS> EXECUTE dbms_redefinition.can_redef_table ('SH', 'CUSTOMER'); PL/SQL procedure successfully completed. system@AOS> desc sh.customer_stage; Name Null? Type CUST_ID NOT NULL NUMBER(6) CUST_FIRSTNAME VARCHAR2(20) CUST_LASTNAME VARCHAR2(20) CUST_ADDRESS VARCHAR2(40) CUST_CITY VARCHAR2(15) CUST_STATE VARCHAR2(2) CUST_ZIP VARCHAR2(10) CUST_CREDIT_CARD_NO VARCHAR2(19) ENCRYPT CUST_CREDIT_CARD_EXP VARCHAR2(4) system@AOS> EXECUTE dbms_redefinition.start_redef_table ('SH', 'CUSTOMER', 'CUSTOMER_STAGE'); PL/SQL procedure successfully completed. system@AOS> EXECUTE dbms_redefinition.finish_redef_table ('SH', 'CUSTOMER', 'CUSTOMER_STAGE'); PL/SQL procedure successfully completed. system@AOS> desc sh.customer; Name Null? Type CUST_ID NOT NULL NUMBER(6) CUST_FIRSTNAME VARCHAR2(20) CUST_LASTNAME VARCHAR2(20) CUST_ADDRESS VARCHAR2(40) CUST_CITY VARCHAR2(15) CUST_STATE VARCHAR2(2) CUST_ZIP VARCHAR2(10) CUST_CREDIT_CARD_NO VARCHAR2(19) ENCRYPT CUST_CREDIT_CARD_EXP VARCHAR2(4) . running for many years, having been built on an Oracle database or perhaps built on another database subsystem and later migrated to Oracle Database. It certainly makes sense to look at these. set key identified by "AppliedOracleSecurity"; This command does two things. It physically creates the file that holds the Oracle Wallet used for TDE, and it generates a Master Key. you encrypted “foo” and now have “@#SK^X”, which will no longer fit in the space previously occupied by “foo”. 42 Part I: Oracle Database Security New Features The database handles this additional