Nielsen c55.tex V4 - 07/21/2009 3:50pm Page 1242 Part VIII Monitoring and Auditing FIGURE 55-2 The SQLServerTrace Counter Log is configured to write server performance counter data to the C:\PerfLogs directory. Summary PerfMon is a small but powerful tool in any DBA’s toolbox. Used alone, it provides a comprehensive overview of the server’s status — both Windows and SQL Server. Used in conjunction with Profiler (covered in the next chapter), it opens up a visual microscope into performance. The key to PerfMon is understanding the overwhelming number of counters. If you know which coun- ters to focus on, you’ll find that PerfMon can be efficient. If not, you’ll waste time and fail to come to any useful conclusion. The next chapter looks at Profiler, by far the most popular SQL Server monitoring tool, and for good reason. Profiler is indeed an excellent tool. With PerfMon, it’s even better. 1242 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1243 Tracing and Profiling IN THIS CHAPTER Viewing SQL Server events Establishing traces Merging Profiler data with SysMon I love SQL Server Profiler. When tuning my own application, or as a consultant when I arrive at a job site, one of the first things I do is to fire up Profiler to capture a full trace of every ad hoc SQL query and stored procedure call. (You can read more about my performance trace in Chapter 64, ‘‘Indexing Strategies.’’) I use a Dell Latitude E6400 notebook, which is great for traveling, but in the SQL Dungeon (my basement), I attach the notebook to a normal keyboard, a mouse, and a large monitor. The notebook screen becomes a second monitor off to the left. When I code, the second monitor runs Profiler and displays performance stats as I code. Of all the SQL Server programs and icons, I have only three pinned to my main Start menu: Management Studio, Books Online, and Profiler. Profiler is to the DBA what a camera is to a photographer. Any photographer who doesn’t have an affection for his favorite camera must not be much of a photogra- pher. (I just got a Nikon D60 for my fiftieth birthday!) L ikewise, any SQL Server professional needs to be intimately familiar with Profiler. When I help clients by doing tech interviews, I always ask the candidate if he or she loves Profiler. It’s one of the mandatory minimum skills. If you’re new to SQL Server, let me introduce you to SQL Server Profiler: ■ SQL Trace is a lightweight, but powerful, technology that can run on SQL Server; it collects performance data selected from hundreds of possible performance data points ranging from locks, to connections, to SQL DML statements, to recompiles, to you name it. 1243 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1244 Part VIII Monitoring and Auditing ■ SQL Server Profiler is a separate application that can configure, start, and stop SQL Trace as well as capture and display data from SQL Trace. You can think of SQL Server Profiler as the optional UI for SQL Trace. ■ Profiler data can scroll on the screen or be saved to a file or table for further analysis. ■ Data can be filtered in numerous ways, including viewing only events related to a specific database, excluding events from a specific application, or capturing only the queries that exceed a specified duration. ■ Profile configurations can be saved and reused later. ■ Profile event data can be merged with server counter data collected by Perfmon/SysMon for a great visual representation of what’s happening on the server. To be accurate, Profiler is actually just the front-end UI for SQL Trace. The two are different compo- nents and technologies. SQL Trace runs on the server, is lightweight, and collects data points to be passed to Profiler or written to a file. What’s New with SQL Server Profiler and Trace? T o be honest, there’s not a lot new with Profiler for SQL Server 2008. Management Studio adds the capability to open Profiler preset to filter for the current connection. To fill in the history of SQL Server Profiler, SQL Trace was first introduced with SQL Server version 6.5. The interface was enhanced with SQL Server 2005, merging the events and data points. SQL Server 2005 also introduced the Default Trace, a high-performance specific trace configuration. Running Profiler SQL Server Profiler can be opened from several locations: ■ From the Start menu, Profiler is under SQL Server 2008 ➪ Performance menu. ■ In Management Studio, Profiler is in the Tools menu. ■ Within Management Studio’s Query Editor, the context menu includes the option to Trace Query in SQL Server Profiler. This option opens Profiler filtered to the current connection. Defining a new trace When a new trace is created (with the New Trace toolbar button or File ➪ New Trace), a connection is created to a SQL Server, and the Trace Properties dialog (see Figure 56-1) is presented. The Trace Prop- erties General tab sets up the trace (name, file destination, etc.), and the Events Selection tab defines the events and data columns to be recorded, as well as the filter. If the trace is running, the properties may be viewed but not changed. A trace configuration can be saved a s a template to make creating new traces easier. 1244 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1245 Tracing and Profiling 56 FIGURE 56-1 This SQL Server Profiler is using the T-SQL Duration template and will write information to a file. A Profiler trace is primarily viewed interactively, but the data can also be written to a file or a SQL Server table. This is useful for further manual analysis, viewing alongside System Monitor counter data, or importing into the Database Engine Tuning Advisor. When Profiler data is written to a file, SQL Server writes 128-KB chunks of data at a time for perfor- mance. Conversely, writing data to a table involves a series of row inserts that doubles the transaction log workload and seriously hinders SQL Server’s performance. Never trace directly to a table on the server being traced, although writing to a different server is OK. Best Practice T o save Profiler data for further analysis, use the high-performance file method and a server-side trace (discussed later). If you want to analyze the data using T-SQL (and analyzing trace data with SQL aggregate queries and WHERE LIKE clauses is way cool), save the trace to a file; and after the trace is complete, open the trace file using Profiler and select File ➪ Save As ➪ Table. 1245 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1246 Part VIII Monitoring and Auditing Selecting events and data columns The Events Selection tab (see Figure 56-2) determines the data points that SQL Trace will capture. SQL Trace can monitor 179 key SQL Server events. The default templates configure a trace with a few prese- lected events, but there’s so much more. Two important details of the interface are easily overlooked: The Show all e vents and Show all columns check boxes in the lower-right side of the Events Selection tab enable viewing and selecting from the complete set of events and columns. Without those options checked, the form only displays the c ur- rently selected events and columns. While this can be useful to filter out noise, you must enable these options to select additional events or add columns to existing events. The following list shows the event categories, along with the number of events in each category and a comment: ■ Broker (13): The first category covers events related to Service Broker activity. ■ CLR (1): Only CLR assembly loads can be traced. ■ Cursors (7): These events are not related to T-SQL server-side cursors. They track ADO client-side cursor activity. ■ Database (6): Tracks database file activity such as autogrowth and mirror connections. ■ Deprecation (2): A very useful set of events. I always run these events when exercising code or running unit tests to highlight any deprecated features being used in the code. ■ Errors and Warnings (16): Any abnormal event or error will trigger these events; useful when watching for errors in an active system. ■ Full text (3): The Full text events only track Full-Text Search crawl activity. There are no events for Full-Text Search configuration changes. More information about Full-Text Search queries is hidden in the Performance: FullTextQuery events. ■ Locks (9): While enabling lock events can be great for learning about locks in a controlled setting, using these events in a production environment can instantly fill Profiler with thou- sands of events, so be careful with this category. A noteworthy exception is the Deadlock Graph event, which captures a full set of data about the deadlock and even displays a graph. ■ OLEDB (5): These events track OLE-DB distributed query calls made by SQL Server to other providers. ■ Objects (3): DDL events ( CREATE, ALTER, DROP) can be tracked with these events. ■ Performance (14): These events track data about query plans and plan guides. A notable event is the ShowPlan XML event, which can display the query execution plan. ■ Progress Report (1): This event category tracks the progress of online reindexing. ■ Query Notifications (4): These show information about query notification activity, including subscriptions activity. ■ Scans (2): When watching for performance issues, this category can trace index scans and table scans. ■ Security Audit (42):WithanodtotheHitchhiker’s Guide to the Galaxy, the Security Audit category includes numerous events to support C2 and Common Criteria compliance. ■ Server (3): This odd category includes mount tape, memory change, and trace stop events. 1246 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1247 Tracing and Profiling 56 ■ Sessions (1): This event fires when a trace is started and returns an event for every existing connection, including its properties. ■ Stored Procedures (15): This rich category includes a number of events related to stored procedure execution, compilation, and cache hits. ■ TSQL (9): These events fire for individual T-SQL statements. ■ Transactions (13): SQL transactions events at the level of begin transaction, commit trans- action, and rollback transaction are traced with this category. Playing with this event will reveal how much activity actually happens with SQL Server. Unfortunately, there’s no event to capture changing the transaction isolation level. ■ User Configurable (10): To gather custom data about the environment or application’s activ- ity, the application can call the sp_trace_generateevent system stored procedure to fire an event and pass custom data to SQL Trace. FIGURE 56-2 The Trace Properties Events Selection page enables you to select the events tracked by Profiler. Depending on the events, different data becomes relevant to the trace. The data columns automatically offer the appropriate data. Although the SPID data column appears optional, it’s only fooling you — it’s mandatory. A useful data column to capture is StartDate, which is required if the trace will be correlated with Perfmon data later. Filtering events Profiler can capture so much information that it can fill a drive with data. Fortunately, the Profiler Trace Filter (see Figure 56-3) can narrow the scope of your search to the data of interest. 1247 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1248 Part VIII Monitoring and Auditing FIGURE 56-3 The Edit Filter dialog serves as a where clause for the trace, restricting it to certain events only. Here, only events for the OBXKites database will be captured. The filter uses a combination of equal and like operators, depending on the data types captured. A frustrating aspect of the filter is that it only works against collected data, and the data collected for some columns may not be what was expected. For example, if you want to filter the trace to only those batches that reference a specific table or column, filtering by object name won’t work. Defining a like filter using wildcards on the text data column, however, will cause Profiler to select only those batches that include that column name. The Exclude system IDs check box sets the filter to select only user objects. Best Practice B y default, SQL Server Profiler filters out the calls made by SQL Server Profiler, but it’s a good idea to regularly also filter out the Reporting Services and SQL Agent applications to avoid any unnecessary event clutter. Organizing columns To add a column to the GROUP BY (not shown in a figure), select GROUP BY in the right-hand column before clicking the Add button. Columns can also be escalated to group status by means of the Up button. Any GROUP BY columns become the first columns in the trace window; and as new events are added to the trace window, those events are automatically added within their group. 1248 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1249 Tracing and Profiling 56 Running the trace Although Profiler is just a UI that consumes data generated by SQL Trace, it’s pretty good at controlling live traces. Profiler can start, pause, and stop the trace using the typical icons in the toolbar. A few details worth pointing out are as follows: ■ The eraser toolbar button will clear the results in Profiler. ■ If the trace is flying by too fast to view but you don’t want to pause or stop the trace, the Auto Scroll Window toolbar button enables Profiler to continue to add new events at the bottom of the list without scrolling the window. ■ While a trace is stopped, its events and data columns may be changed using the Properties button. Using the trace file Once the trace is captured it can be browsed through using the Profiler trace window, and a Find toolbar button can help navigate the data. However, the trace is likely to be so large that it will be difficult to manually use the data. The solution is to save the trace file to a SQL table using File ➪ Save As; the data can then be analyzed and manipulated as in any other SQL table. SQL Server Profiler has the ability to replay traces. However, the restrictions on the replay option are such that it’s unlikely to be useful for most databases. If the trace is to be replayed, certain events must be captured. For example, the SQL Batch Start event can be replayed, but SQL Batch Complete cannot. Also, if data inserts are replayed, you’ll want to reset the database to the same state as the beginning of the trace; otherwise, you’ll likely have unique data conflicts, and issues with identity column values. Additionally, the entire trace file can be submitted as a workload to the Database Engine Tuning Advi- sor so that it can tune for multiple queries. However, I’m not a big fan of the Database Engine Tuning Advisor, as I explain in Chapter 64, ‘‘Indexing Strategies.’’ Integrating Performance Monitor data Both System Monitor and Profiler present their own unique perspective on the state of the server. The two sets of information can be merged to produce a synchronized walk-through scenario viewing both perspectives using SQL Server Profiler. To set up the dual-perspective experience, simultaneously capture server performance logs using both Performance Monitor’s Counter Logs and SQL Server Profiler. These steps are specific: 1. Configure System Monitor with the exact counters you want to view later. Be sure to get the scale and everything just right. Set up the Counter Log to the exact same configuration. 2. Configure Profiler with the right set of trace events. They must include the start and end time data columns so Profiler can integrate the two logs later. Script the trace to T-SQL code. Close Profiler. 3. Manually start the Counter Log. Execute the T-SQL trace code to start the server-side trace. 1249 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1250 Part VIII Monitoring and Auditing 4. Exercise the server with the code and load you want to analyze. 5. When the test is complete, stop both the Counter Log and the server-side trace. 6. Open Profiler and then open the saved trace file. 7. Use the File ➪ Import Performance Data menu command to import the Counter Log. Profiler responds to the import by adding a pane that displays the System Monitor graph, as shown in Figure 56-4. Select a Profiler event or a time in the System Monitor graph; the two stay in sync. Cool, eh? FIGURE 56-4 SQL Server Profiler can integrate Performance Monitor data and move through the events in sync. Using SQL Trace SQL Server Profiler is usually used interactively, and for ad hoc data gathering this is probably sufficient. However, running Profiler o n a heavy transaction server can lead to problems: 1250 www.getcoolebook.com Nielsen c56.tex V4 - 07/21/2009 3:52pm Page 1251 Tracing and Profiling 56 ■ If Profiler can’t keep up, some events will be dropped. I’ve seen this happen frequently on heavy transaction servers with Profiler. ■ There’s a measurable performance impact on the server when running Profiler. The heavier the transaction traffic on the server, the greater the percentage of performance impact from Profiler. ■ The workstation gathering the events may run out of memory. The solution is to run the SQL Trace directly on the server without collecting the data using the Profiler UI. SQL Traces are started by the sp_trace_create system stored procedure. Once the trace exists, events are added to it using sp_tracesetevent. While it’s possible to code stored procedures to configure SQL Traces, the most common method is to define the trace in Profiler and then script the trace to run on the server. Once the trace is set up and tested in SQL Server Profiler, select File ➪ Export ➪ Script Trace Definition ➪ For SQL Server 2005 - 2008 to generate a T-SQL script that will launch the server-side trace. Best Practice F or production systems running server-side traces, writing to a file is the best way to collect performance data with the least overhead to the server. To view all the traces currently running in the server, query sys.traces: SELECT id, path, start_time, last_event_time, event_count, dropped_event_count FROM sys.traces t Result (abbreviated to fit): id start_time last_event_time event_count dropped_event_count 1 2009-04-27 03:07:49 2009-04-27 22:49:22 2770 NULL 2 2009-04-27 22:27:20 NULL 0 0 To programmatically view the events and data columns being collected by a trace, use the following query (you’ll need to modify the parameter for fn_trace_geteventinfo): SELECT tcat.name +‘:’ + te.name AS ‘Event’, tcol.NAME AS ‘Column’ FROM fn_trace_geteventinfo (2) tinfo JOIN sys.trace_events te ON tinfo.eventid = te.trace_event_id JOIN sys.trace_categories tcat ON te.category_id = tcat.category_id 1251 www.getcoolebook.com . minimum skills. If you’re new to SQL Server, let me introduce you to SQL Server Profiler: ■ SQL Trace is a lightweight, but powerful, technology that can run on SQL Server; it collects performance. with Profiler for SQL Server 2008. Management Studio adds the capability to open Profiler preset to filter for the current connection. To fill in the history of SQL Server Profiler, SQL Trace was first. Profiler, SQL Trace was first introduced with SQL Server version 6.5. The interface was enhanced with SQL Server 2005, merging the events and data points. SQL Server 2005 also introduced the Default