Nielsen c60.tex V4 - 07/21/2009 4:00pm Page 1292 Part VIII Monitoring and Auditing Result: StartLSN Operation DepartmentID Name GroupName 2009-03-07 19:49:26.383 insert 21 CDC New Row SQL Rocks 2009-03-07 19:49:26.383 insert 22 Test Two CDC Rocks 2009-03-07 19:49:26.390 update/deleted 21 CDC New Row SQL Rocks 2009-03-07 19:49:26.390 update/inserted 21 Changed Name SQL Rocks 2009-03-07 19:49:26.393 insert 23 Row Three PBM Rocks 2009-03-07 19:49:26.393 insert 24 Row Four TVP Rock 2009-03-07 19:49:26.400 update/deleted 22 Test Two CDC Rocks 2009-03-07 19:49:26.400 update/inserted 22 Test Two T-SQL Rocks 2009-03-07 19:49:26.403 delete 24 Row Four TVP Rock Querying net changes All the previous queries returned all the changes within the requested time frame. But for many ETL operations or synchronizations, only the final net values are needed. Change data capture can automat- ically determine the net, or final, values. Use the cdc.fn_cdc_get_net_changes_schema_table function to return the net changes: Querying Net Changes - ‘all’ option SELECT sys.fn_cdc_map_lsn_to_time( $start_lsn) as StartLSN, Operation.Description as ‘Operation’, DepartmentID, Name, GroupName FROM cdc.fn_cdc_get_net_changes_HumanResources_Department net changes (sys.fn_cdc_map_time_to_lsn(’smallest greater than or equal’, ‘20090101’), sys.fn_cdc_map_time_to_lsn(’largest less than or equal’, ‘20091231’), ‘all’) as CDC JOIN (VALUES (1, ‘delete’), (2, ‘insert’), (3, ‘update/deleted’), ‘all update old’ option to view (4, ‘update/inserted’) ) as Operation(OperationID, Description) ON CDC. $operation = Operation.OperationID ORDER BY $start_lsn Result: StartLSN Operation DepartmentID Name GroupName 2009-03-07 19:49:26.390 insert 21 Changed Name SQL Rocks 2009-03-07 19:49:26.393 insert 23 Row Three PBM Rocks 2009-03-07 19:49:26.400 insert 22 Test Two T-SQL Rocks 1292 www.getcoolebook.com Nielsen c60.tex V4 - 07/21/2009 4:00pm Page 1293 Change Data Capture 60 When querying net changes using Change Data Capture, it’s also possible to work with a column mask to determine whether a given column has changed. In the following query, the all with mask option and sys.fn_cdc_has_column_changed function are used together to test for changes in the GroupName column: update the GroupName column UPDATE HumanResources.Department SET GroupName = ‘Updated 2’ WHERE Name = ‘Test Two’; Querying Net Changes - ‘all with mask’ option SELECT Operation.Description as ‘Operation’, DepartmentID AS DeptID, GroupName, sys.fn_cdc_is_bit_set (sys.fn_cdc_get_column_ordinal (’HumanResources_Department’, ‘GroupName’) , $update_mask ) as GroupNameUpdated, sys.fn_cdc_has_column_changed (’HumanResources_Department’, wrong in BOL ‘GroupName’, $update_mask) as GroupNameHasChanged FROM cdc.fn_cdc_get_net_changes_HumanResources_Department net changes (sys.fn_cdc_map_time_to_lsn(’smallest greater than or equal’, ‘20090307 8:40pm’), change datetime to pick up update as net change sys.fn_cdc_map_time_to_lsn(’largest less than or equal’, ‘20091231’), ‘all with mask’) as CDC JOIN (VALUES (1, ‘delete’), (2, ‘insert’), (3, ‘update/deleted’), ‘all update old’ option to view (4, ‘update/inserted’) ) as Operation(OperationID, Description) ON CDC. $operation = Operation.OperationID ORDER BY $start_lsn Result: Operation DeptID GroupName GroupNameUpdated GroupNameHasChanged update/inserted 22 Updated 2 1 1 1293 www.getcoolebook.com Nielsen c60.tex V4 - 07/21/2009 4:00pm Page 1294 Part VIII Monitoring and Auditing Walking through the change tables For most ETL and synchronization operations, selecting the data as a set is the best practice, but change data capture also supports walking through the change table data iteratively. Think of these functions as CDC cursors. The following script uses the sys.fn_cdc_get_min_lsn() function to identify a starting point in the change table and then iterates through the entries sequentially using the sys.fn_cdc_ increment_lsn() function, which finds the next entry following the one passed in as a parameter: DECLARE @BeginLSN VARBINARY(10) = sys.fn_cdc_get_min_lsn(’HumanResources_Department’); SELECT @BeginLSN; SET @BeginLSN = sys.fn_cdc_increment_lsn(@BeginLSN); SELECT @BeginLSN; SET @BeginLSN = sys.fn_cdc_increment_lsn(@BeginLSN); SELECT @BeginLSN; Result (obviously, your result will be different): 0x000000420000136A003D 0x000000420000136A003E 0x000000420000136A003F Likewise, CDC can move backward through the entries: SET @BeginLSN = sys.fn_cdc_decrement_lsn(@BeginLSN); SELECT @BeginLSN; Result: 0x000000420000136A003E Removing Change Data Capture Removing change data capture is a flexible and simple process. CDC can be disabled table by table, or for the whole database. When CDC is disabled for the database, it automatically disables all tables, removing the SQL Agent jobs, and dropping the custom tracked table functions. There’s no need to remove CDC from each table individually before disabling CDC from the database: EXEC sys.sp_cdc_disable_db; 1294 www.getcoolebook.com Nielsen c60.tex V4 - 07/21/2009 4:00pm Page 1295 Change Data Capture 60 To remove CDC from a specific table, use the following system stored procedure: EXEC sys.sp_cdc_disable_table @source_schema = ‘HumanResources’, @source_name = ‘Department’, @capture_instance = ‘all’; Summary Change Data Capture, Change Tracking’s big brother, is Microsoft’s high-end feature intended for heavy transaction OLTP systems to capture changes for ETL to the data warehouse. ■ CDC uses the transaction log asynchronously to reduce the impact on OLTP transactions, but there will be some impact. ■ Working with CDC means working with transaction log sequence numbers, or LSNs. ■ Using CDC, you can query for all changes or net changes. The next chapter continues the trend of examining new auditing technologies in SQL Server 2008 with a look at yet another all-new technology: SQL Audit. Based on eExtended Events, SQL Audit can audit any action in SQL Server. 1295 www.getcoolebook.com Nielsen c60.tex V4 - 07/21/2009 4:00pm Page 1296 www.getcoolebook.com Nielsen c61.tex V4 - 07/21/2009 4:02pm Page 1297 SQL Audit IN THIS CHAPTER Configuring SQL Audit Tracking server events A t one of the pre-Katmai (the code name for SQL Server 2008 while it was being developed) NDA (non-disclosure agreement — that is, secret) ses- sions for MVPs, the SQL Server team asked how many of us would like an easy way to audit selects. Nearly every MVP’s hand went up. The SQL Server community has wanted a more powerful auditing mechanism for a long time. SQL Audit is the answer. Based on the new Extended Events technology, SQL Audit is both lightweight and powerful. While it’s possible to ‘‘roll your own’’ auditing solution from Extended Events, SQL Audit is an out-of-the-box solution to leverage Extended Events and collect server and database events. It’s blazingly fast, easy to configure, and cool. While Extended Events is available for all editions of SQL Server, SQL Audit is available only for Enterprise (and Developer) Edition. SQL Audit Technology Overview It takes several SQL Audit components working together to create a functioning Audit. A SQL Server Audit object is a bucket that collects the audit events defined by a Server Audit Specification and the Database Audit Specification, and sends the audited events to a target. Here are the facts: ■ A SQL Server Audit object can be written to by one Server Audit Specification and one Database Audit Specification per database. ■ A SQL Server Audit can belong to only one SQL Server instance, but there may be several SQL Server Audits within an instance. 1297 www.getcoolebook.com Nielsen c61.tex V4 - 07/21/2009 4:02pm Page 1298 Part VIII Monitoring and Auditing ■ A Server Audit Specification defines which server-level events will be captured and passed to the SQL Audit. ■ A Database Audit Specification defines which database-level events are captured and passed to the SQL Audit. ■ Both Server Audit Specifications and Database Audit Specifications can define sets of events or groups to be captured. Event groups encapsulate a number of related events. Database actions include select, insert, update, and delete, and they capture the user context and the entire DML query. ■ The audited data includes user context information. ■ The SQL Server Audit sends all the captured events to a single target: a file, the Win- dows Security event log (not in Windows XP), or the Windows Application event log. The Management Studio SQL Audit UI includes a tool for browsing the audit logs. ■ SQL Server Audits, Server Audit Specifications, and Database Audit Specifications can all be created and managed either with Object Explorer or by using T-SQL. ■ SQL Server Audits, Server Audit Specifications, and Database Audit Specifications can all be enabled or disabled. They may be modified only while disabled. All are disabled by default when they are first created, because that’s how Extended Events works. ■ SQL Server Audits, Server Audit Specifications, and Database Audit Specifications can all be managed by Policy-Based Management. ■ SQL Audits are serious. The SQL Server Audit object can be configured to shut down the server if the audit doesn’t function properly. Creating an Audit The first step to working with SQL Audit is to create a SQL Server Audit object. In Object Explorer, SQL Server Audit objects are listed under the server ➪ Security ➪ Audits node. The New Audit command in the Audits node context menu opens the Create Audit dialog shown in Figure 61-1. The queue delay, which determines how long SQL Server can wait before processing the Extended Event, ranges from 1 second (1,000 milliseconds) to almost 25 days (2,147,483,647 milliseconds). The default (1 second) is reasonable for most situations. If the server is hit with very heavy traffic, increasing the queue delay gives SQL Audit more flexibility. Selecting true for ‘‘Shut down server on auditing failure’’ ensures that the target file or log receiving the events can be written to. If SQL Audit can’t write to the target, then it will write a MSG_AUDIT_ FORCED_SHUTDOWN event to the error log and shut down the server. Fortunately, except for the name, all of the SQL Server Audit attributes may be changed after the object is created. 1298 www.getcoolebook.com Nielsen c61.tex V4 - 07/21/2009 4:02pm Page 1299 SQL Audit 61 FIGURE 61-1 The Create Audit dialog is used to define SLQ Server Audit objects, which collect events defined by the Server Audit Specification or the Database Audit Specification. If ‘‘Shut down on auditing failure’’ is set to true, and SQL Audit does indeed shut down the server, here’s what to do: Start SQL Server with the minimal configuration option using the -f flag. This will start SQL Server in single-user mode, and put SQL Audit into Auditing failure=continue mode. Defining the target The events can be sent to either a file, the Windows Security event log (not available in Windows XP), or the Windows Application event log. If the target is the log, then there are no other options. If the target is a file, then the receiving directory, the size of the file, and the number of rollover files may be defined. The minimum file size is 1024 KB. SQL Server will automatically name the files and place them in the specified directory. I recommend using a dedicated local directory and limiting the file size to a few MB. If the target is the Windows Security Log, then there are special security permissions and configurations required. See http://msdn.microsoft.com/en-us/library/cc645889.aspx for detailed information. 1299 www.getcoolebook.com Nielsen c61.tex V4 - 07/21/2009 4:02pm Page 1300 Part VIII Monitoring and Auditing Using T-SQL Of course, the SQL Server Audit object can be created using the CREATE SERVER AUDIT command. The following example creates the same SQL Server Audit object shown in Figure 61-1: CREATE SERVER AUDIT [SQL Server 2008 Bible Audit] TO FILE ( FILEPATH = N’C:\SQLData’, MAXSIZE = 64 MB, MAX_ROLLOVER_FILES = 2147483647, RESERVE_DISK_SPACE = OFF ) WITH ( QUEUE_DELAY = 1000, ON_FAILURE = CONTINUE ) The SQL Server Audit object can also be modified using an ALTER command. Enabling/disabling the audit Object Explorer’s SQL Server Audit node visually indicates whether the Audit is enabled or disabled witharedmarkonthenodeiftheitemiscurrentlyturned off. The context menu includes commands to enable or disable the Audit. Using T-SQL, the ALTER command has an additional parameter that enables or disables the SQL Server Audit. The following command enables the SQL Server 2008 Bible Audit: ALTER SERVER AUDIT [SQL Server 2008 Bible] WITH (State = ON) Server Audit Specifications A new Server Audit Specification may be created from Object Explorer using the Security ➪ Server Audit Specifications’ context menu ➪ New Server Audit Specification command, which opens the Create Server Audit Specification dialog, shown in Figure 61-2. Each SQL Server Audit object may have only one Server Audit Specification, but there may be multiple Server Audits running, and each may have a Server Audit Specification. The new Server Audit Specification can’t be created unless it points to an existing SQL Server Audit object and that SQL Server Audit object currently does not have a Server Audit Specification connected to it. 1300 www.getcoolebook.com Nielsen c61.tex V4 - 07/21/2009 4:02pm Page 1301 SQL Audit 61 FIGURE 61-2 Creating a new Server Audit Specification using Management Studio Adding actions Without a doubt, the most important part of defining the Server Audit Specification is adding actions to the specification. Unfortunately, these actions aren’t in a hierarchy like the DDL Triggers events and groups; each action group must be added individually. The server-related events that can be audited are organized into 35 action groups (most are shown in the drop-down list in Figure 61-2). Potentially, a Server Audit Specification could have all 35 action groups. The Server Audit State Change Audit group, which audits whether SQL Audit is enabled or disabled, is automatically audited. Creating with T-SQL Using T-SQL’s CREATE command, it’s easy to create a new Server Audit Specification. The principal parameter is the ADD(ACTION GROUP) option, which configures the Server Audit Specification with 1301 www.getcoolebook.com . Audit. Using T -SQL, the ALTER command has an additional parameter that enables or disables the SQL Server Audit. The following command enables the SQL Server 2008 Bible Audit: ALTER SERVER AUDIT [SQL Server. the SQL Server Audit object can be created using the CREATE SERVER AUDIT command. The following example creates the same SQL Server Audit object shown in Figure 61-1: CREATE SERVER AUDIT [SQL Server. logs. ■ SQL Server Audits, Server Audit Specifications, and Database Audit Specifications can all be created and managed either with Object Explorer or by using T -SQL. ■ SQL Server Audits, Server