Nielsen c58.tex V4 - 07/21/2009 3:57pm Page 1262 Part VIII Monitoring and Auditing SQL Server MVP and all-around smart guy, Jonathan Kehayias, has developed ‘‘Extended Events Manager,’’ a UI for Extended Events that you can download from www.codeplex .com/ExtendedEventManager . Jonathan is also writing a white paper on Extended Events for Microsoft. Having read a draft, if you’re interested in implementing an Extended Events solution, I highly recommend it. See www.sqlserverbible.com for a link. Packages At the top level, Extended Events organizes its components into three Microsoft-developed packages (it’s really four, but the fourth package, SecAudit, is a private package dedicated to SQL Audit): ■ Package0: Provides base functionality used by the other packages ■ sqlos: Works with the SQL/OS events ■ sqlserver: Works with the rest of SQL Server Each of these packages contains several objects. To see the full list of objects within packages, query the following DMVs: SELECT p.NAME AS Package, o.object_type, o.NAME AS Object, o.description FROM sys.dm_xe_objects o JOIN sys.dm_xe_packages p ON o.type_package_guid = p.guid ORDER BY p.NAME, o.object_type Result (abbreviated): Package object_type Object description package0 action tsql_stack Collect Transact-SQL stack package0 action sql_text Collect SQL text package0 action plan_handle Collect plan handle package0 action create_dump_single_thread Create mini dump for the current thread package0 action create_dump_all_threads Create mini dump including all threads The object_type column reveals seven types of objects: actions, events, maps, pred_compares, pred_sources, targets, and types. Building a crosstab query from the previous query analyzes the distribution of object types within the packages: SELECT Package, action, event, map, pred_compare, pred_source, target, type FROM (SELECT o.Name AS Object, p.Name AS Package, o.object_type FROM sys.dm_xe_objects o JOIN sys.dm_xe_packages p 1262 www.getcoolebook.com Nielsen c58.tex V4 - 07/21/2009 3:57pm Page 1263 Extended Events 58 ON o.type_package_guid = p.guid) sq PIVOT (COUNT(Object) FOR object_type IN (action, event, map, pred_compare, pred_source, target, type) )ASpt; Result: Package action event map pred_compare pred_source target type package0 35 254 61 111 29 13 27 sqlos 1 0 0 0 0 0 1 sqlserver 1 0 0 0 0 0 1 Objects Within each package is a host of objects. Creating an Extended Event is basically a process of evoking the right objects. To dig into these object types: ■ Events: Similar to SQL Trace events, XE has more events and their columns are listed in sys.dm_xe_object_columns. Extended Events borrows the concept of event channels from Event Tracing for Windows (ETW). In the Channel read-only column, every event is in one of four predetermined channels: admin, analytic, debug, or operational. ■ Actions: XE actions fire as the result of an event and can gather additional data about the context of the event. ■ Types and Maps: A high-performance optimization, types and maps both serve as pointers to data. Instead of having to gather every data point synchronously, a type or map can point to the data in memory. ■ Pred_Compares and Pred_Sources: Predicates are essentially filters, similar to SQL Trace filters. ■ Targets: The target object defines the destination for the event data and the synchronous or asynchronous nature of the target. XE Sessions The Extended Events equivalent of starting a SQL Trace is a session. Sessions can be launched, queried, and stopped using T-SQL code. The following example launches a simple event that tracks when SQL Server performs checkpoints. Using the ring_buffer as the target captures the data in memory: CREATE EVENT SESSION [CheckPoint] ON SERVER ADD EVENT sqlserver.checkpoint_end ADD TARGET package0.ring_buffer; 1263 www.getcoolebook.com Nielsen c58.tex V4 - 07/21/2009 3:57pm Page 1264 Part VIII Monitoring and Auditing The session is now created, but it’s not running until an ALTER command actually starts the session: ALTER EVENT SESSION [CheckPoint] ON SERVER State = START; The session can now be seen in the session DMV: SELECT address, name FROM sys.dm_xe_sessions Result: address name 0x0000000080576081 system_health 0x0000000080360D61 CheckPoint Sure enough, there’s the CheckPoint session. SQL Server automatically started a default system_health session as well, which is worth exploring. Joining the session DMV with sys.dm_xe_session_targets will retrieve the event data from the ring_buffer target and expose the data for every captured event within a single XML value: SELECT CONVERT(XML, st.target_data) AS ring_buffer FROM sys.dm_xe_sessions s JOIN sys.dm_xe_session_targets st ON s.address = st.event_session_address WHERE NAME = ‘CheckPoint’ Result (abbreviated and opened in the browser window): <RingBufferTarget eventsPerSec="0" processingTime="0" totalEventsProcessed="13" eventCount="13" droppedCount="0" memoryUsed="546"> <event name="checkpoint_end" package="sqlserver" id="86" version="1" timestamp="2009-05-11T07:12:00.220Z"> <data name="database_id"> <type name="uint16" package="package0" /> <value>3</value> <text /> </data> </event> <event name="checkpoint_end" package="sqlserver" id="86" version="1" timestamp="2009-05-11T07:12:00.719Z"> <data name="database_id"> <type name="uint16" package="package0" /> <value>12</value> <text /> 1264 www.getcoolebook.com Nielsen c58.tex V4 - 07/21/2009 3:57pm Page 1265 Extended Events 58 </data> </event> Other types of events and actions will return other elements within the XML data. You can find addi- tional examples on the chapter’s downloadable SQL script. Extracting the data from the XML into relational data requires a little XPath: SELECT node.event_data.value(’(data/value)[1]’, ‘BIGINT’) AS source_database_id, node.event_data.value(’(timestamp)[1]’, ‘Datetime’) AS test FROM (SELECT CONVERT(XML, st.target_data) AS ring_buffer FROM sys.dm_xe_sessions s JOIN sys.dm_xe_session_targets st ON s.address = st.event_session_address WHERE NAME = ‘CheckPoint’ ) AS sq CROSS APPLY sq.ring_buffer.nodes(’//RingBufferTarget/event’) node (event_data) Result: source_database_id test 3 2009-05-11T07:12:00.220Z 12 2009-05-11T07:12:00.719Z To stop the session, use another ALTER command. Removing the session configuration requires a DROP command: ALTER EVENT SESSION [CheckPoint] ON SERVER State = STOP; DROP EVENT SESSION [CheckPoint] ON SERVER Summary Extended Events may seem cryptic at first, but with a few attempts, I think you’ll find they’re not that complicated. They offer better performance and granularity than SQL Trace, but at the cost of raw T-SQL and XPath. The key to working with Extended Events is exploring the objects so you know which events and actions to monitor. The next chapter also deals with a new monitoring technology in SQL Server 2008, Change Tracking. 1265 www.getcoolebook.com Nielsen c58.tex V4 - 07/21/2009 3:57pm Page 1266 www.getcoolebook.com Nielsen c59.tex V4 - 07/21/2009 3:58pm Page 1267 Change Tracking IN THIS CHAPTER Lightweight synchronization for data warehouse ETL and mobile applications Net changes Auto cleanup C hange Tracking is one of the best-named software features I’ve come across. All Change Tracking does is track changes; it says to the world: ‘‘This row was changed, here’s the PK.’’ Clean and simple, no fuss, no muss. It’s a piece of cake to configure and easy to query. While Change Data Capture (another of the new auditing technologies in SQL Server 2008, covered in the next chapter) is limited to only the Enterprise Edi- tion, Change Tracking is available in all the SQL Server editions, even SQL Server Express. Change Tracking occurs synchronously within the transaction. It simply records in an internal table the primary key values of the rows that are modified. Although there’s a cost to recording the changes within the transaction, it means that SQL Agent is not required. Optionally, Change Tracking can store which columns were changed, using a bit-mapped method similar to how triggers know which column was included in the DML code. The real purpose of Change Tracking is to support synchronization. By easily and reliably recording the primary keys of which rows were inserted, updated, or deleted since the last synchronization, it becomes much simpler to perform the synchronization. Change Tracking returns the net changes. If a row is inserted and updated since the last synchronization, then Change Tracking will list it as an insert. If the row is inserted and deleted, then it won’t even show in the Change Tracking results — which is perfect for applications that need synchronization. 1267 www.getcoolebook.com Nielsen c59.tex V4 - 07/21/2009 3:58pm Page 1268 Part VIII Monitoring and Auditing Several applications can benefit from using Change Tracking: ■ The Microsoft Synch Framework ■ ETL processes that keep a data warehouse in synch with the OLTP database ■ Caching data in middle tiers for performance ■ Synchronizing occasionally connected mobile applications There is a performance hit for using Change Tracking, but it’s a lightweight feature with about the same performance hit as adding an index. The performance cost depends on the size of the primary key and the number of rows affected by the transaction. A single column integer primary will have less perfor- mance cost than a wide composite primary key. Adding column tracking also adds to the performance overhead. As cool as Change Tracking is for applications that need synchronization, it isn’t adequate for OLTP auditing (who changed what rows to what new values when and from where?) for two reasons. First, Change Tracking does have a context option, but it’s not very pretty. Second, Change Tracking returns the net changes, so intermediate changes wouldn’t be audited. Configuring Change Tracking Compared to other optional SQL Server technologies, Change Tracking is relatively easy to turn on and configure. It’s simply turned on for the database, and then for each table. Enabling the database Change Tracking may be enabled for the database in Object Explorer’s Database Properties dialog, available from each database’s context menu (as shown in Figure 59-1). Changing the values in the drop-down boxes immediately changes the database settings when you close the properties dialog. Change Tracking may also be configured with T-SQL. I like that it uses normal SQL ALTER SET commands and not system stored procedures: ALTER DATABASE AdventureWorks2008 SET Change_tracking = on (change_retention = 24 hours, auto_cleanup = on); The current Change Tracking database configuration can be viewed in the Object Explorer Database Properties dialog, or by querying the sys.change_tracking_databases DMV: SELECT d.name, ct.is_auto_cleanup_on, ct.retention_period, ct.retention_period_units, ct.retention_period_units_desc FROM sys.change_tracking_databases ct JOIN sys.databases d ON ct.database_id = d.database_id; The database must be in 9.0 compatibility mode, and at least db_owner role permission is required, to enable the database for Change Tracking. 1268 www.getcoolebook.com Nielsen c59.tex V4 - 07/21/2009 3:58pm Page 1269 Change Tracking 59 FIGURE 59-1 The Database Properties’ Change Tracking page displays the current settings and may be used to enable or disable Change Tracking. Auto cleanup Change Tracking can create a lengthy audit trail, but it can also optionally automatically clean up old Change Tracking data. The retention period can be set to any number of Days, Hours,or Minutes. The default is to retain the data for two days (which is probably too short for most applications). Auto_cleanup and the retention period can be set when Change Tracking is initially enabled, or it can be modified later by reissuing the set Change_Tracking option with the new retention settings. In this situation, because Change Tracking is already enabled, re-enabling Change Tracking would generate an error. It’s only necessary to change the option setting: ALTER DATABASE [AdventureWorks2008] SET change_tracking (change_retention = 7 days) 1269 www.getcoolebook.com Nielsen c59.tex V4 - 07/21/2009 3:58pm Page 1270 Part VIII Monitoring and Auditing Be careful with the auto cleanup retention period. Change Tracking doesn’t know when synchroniza- tions occur. If the synchronization application doesn’t run before the retention period expires, then the changes will not be seen by the synchronization. If there’s a problem and the synchronization won’t occur, then use the ALTER DATABASE command to toggle auto cleanup: ALTER DATABASE [AdventureWorks2008] SET change_tracking (auto_cleanup = off) Best Practice E stimate the longest possible period between synchronizations and then triple that time. Other than the disk space usage, there’s no risk in a longer retention period, but there’s c onsiderable risk in a retention period that’s too short. Enabling tables The ease of configuring Change Tracking continues with enabling Change Tracking of each table. Using Management Studio, table Change Tracking is viewed or enabled/disabled in the Table Properties dialog, on the Change Tracking page, as shown in Figure 59-2. Like the database, Change Tracking is enabled using T-SQL for tables with an ALTER command: ALTER TABLE HumanResources.Department Enable Change_tracking With (track_columns_updated = on); The only option is to enable or disable whether Change Tracking tracks which columns were changed. By default, column tracking is disabled. There’s a section later in this chapter that explains how to use column tracking. The ALTER TABLE permission is required to enable the table for Change Tracking. Enabling Change Tracking for a table can affect other tasks: ■ The primary key constraint/index cannot be dropped while Change Tracking is enabled. ■ If the table is dropped, then Change Tracking is first removed from the table. ■ The table can’t use a partitioned table’s ALTER TABLE SWITCH PARTITION command. ■ Change Tracking does not track changes made by the TRUNCATE TABLE command. In this case, the synch target table should also be truncated. 1270 www.getcoolebook.com Nielsen c59.tex V4 - 07/21/2009 3:58pm Page 1271 Change Tracking 59 FIGURE 59-2 The Table Properties dialog may be used to view the table’s Change Tracking settings. To view the current tables with Change Tracking enabled, query the sys.change_tracking_tables DMV: SELECT s.name + ‘.’ + t.name as [table], ct.is_track_columns_updated_on,ct.min_valid_version, ct.begin_version, ct.cleanup_version FROM sys.change_tracking_tables ct JOIN sys.tables t ON ct.object_id = t.object_id JOIN sys.schemas s ON t.schema_id = s.schema_id ORDER BY [table]; 1271 www.getcoolebook.com . technologies in SQL Server 2008, covered in the next chapter) is limited to only the Enterprise Edi- tion, Change Tracking is available in all the SQL Server editions, even SQL Server Express. Change. Provides base functionality used by the other packages ■ sqlos: Works with the SQL/ OS events ■ sqlserver: Works with the rest of SQL Server Each of these packages contains several objects. To. equivalent of starting a SQL Trace is a session. Sessions can be launched, queried, and stopped using T -SQL code. The following example launches a simple event that tracks when SQL Server performs checkpoints. Using