SQL Server DMVs in Action pptx

4 A glimpse into SQL Server’s internal data 7 Aggregated results 8 ■ Impact of running DMVs 8 Part of SQL Server 2005 onward 8 1.2 The problems DMVs can solve 9 Diagnosing problems 9 ■

Ian W Stirk

Better queries with

Dynamic Management Views

SQL Server DMVs in Action

SQL Server DMVs in Action

IAN W STIRK

M A N N I N GShelter Island

www.manning.com The publisher offers discounts on this book when ordered in quantity For more information, please contact

Special Sales Department

Manning Publications Co

20 Baldwin Road

PO Box 261

Shelter Island, NY 11964

Email: orders@manning.com

©2011 by Manning Publications Co All rights reserved

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in the book, and Manning

Publications was aware of a trademark claim, the designations have been printed in initial caps

or all caps

Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine

Manning Publications Co Development editor: Katharine Osborne

20 Baldwin Road Copyeditor: Linda Recktenwald

PO Box 261 Proofreader: Katie Tennant

Shelter Island, NY 11964 Typesetter: Dennis Dalinnik

Cover designer: Marija Tudor

ISBN: 9781935182733

Printed in the United States of America

1 2 3 4 5 6 7 8 9 10 – MAL – 17 16 15 14 13 12 11

To Joan, Karen, and Catherine, for yesterday, today, and tomorrow

brief contents

P ART 1 S TARTING THE JOURNEY 1

1 ■ The Dynamic Management Views gold mine 3

2 ■ Common patterns 31

P ART 2 DMV DISCOVERY 53

3 ■ Index DMVs 55

4 ■ Improving poor query performance 92

5 ■ Further query improvements 118

6 ■ Operating system DMVs 147

7 ■ Common Language Runtime DMVs 174

8 ■ Resolving transaction issues 196

9 ■ Database-level DMVs 226

10 ■ The self-healing database 257

11 ■ Useful scripts 285

contents

preface xix acknowledgements xx about this book xxii

P ART 1 S TARTING THE JOURNEY .1

1 The Dynamic Management Views gold mine 3

1.1 What are Dynamic Management Views? 4

A glimpse into SQL Server’s internal data 7 Aggregated results 8 ■ Impact of running DMVs 8 Part of SQL Server 2005 onward 8

1.2 The problems DMVs can solve 9

Diagnosing problems 9 ■ Performance tuning 10 Monitoring 10

1.3 DMV examples 13

Find your slowest queries 14 ■ Find those missing indexes 15 ■ Identify what SQL statements are running now 17 ■ Quickly find a cached plan 18

1.4 Preparing to use DMVs 22

Permissions 22 ■ Clearing DMVs 22

Parent Query 38 2.9 Identifying the database used by ad hoc queries 39 2.10 Calculating DMV changes 40

2.11 Reading cached plans 43

Targeting the area of concern 43 ■ Things to look out for 44

2.12 Building dynamic SQL 47 2.13 Printing the content of large variables 48 2.14 Common terms and acronyms 49

2.15 Known problems that may affect the scripts 50

SQL Server compatibility level set to below 2005 50

An OFFLINE database 50

2.16 Summary 51

P ART 2 DMV DISCOVERY .53

3 Index DMVs 55

3.1 The importance of indexes 56

Types of index 56 ■ Types of index access 57 Factors affecting index performance 58

3.2 Costly missing indexes 62

Finding the most important missing indexes 62 ■ The impact

3.5 Most-frequently used indexes 72

Finding the most-used indexes 72 ■ The importance

of the most-used indexes 74

3.6 Fragmented indexes 75

Finding the most-fragmented indexes 75 ■ The impact

of fragmented indexes 77

3.7 Indexes used by a given routine 78

Finding the indexes used by a given routine 78 The importance of knowing which indexes are used 81

3.8 Databases with most missing indexes 83

Finding which databases have the most missing indexes 84 The importance of other databases 84

3.9 Completely unused indexes 85

Finding which indexes aren’t used at all 85 ■ The importance

4 Improving poor query performance 92

4.1 Understanding executed queries 93

Aggregated results 93 ■ Clearing the cached plans 93

4.2 Finding a cached plan 94

How to find a cached plan 94

4.3 Finding where a query is used 97

Identifying where a query is used 97

4.4 Long-running queries 98

Finding the queries that take the longest time to run 99 The impact of long-running queries 101

4.5 Queries that spend a long time being blocked 103

Finding the queries that spend the longest time being blocked 104

4.8 Frequently executed queries 111

Finding the queries that have been executed the most often 111 Reducing the impact of queries that are executed most often 113

4.9 The last run of a query 114

Determining when a query was last run 114 ■ Variations on searching for queries 115

4.10 Summary 116

5 Further query improvements 118

5.1 Queries with missing statistics 119

Finding queries that have missing statistics 119 ■ The importance

of statistics 121 ■ Default statistics properties 122

5.2 Finding queries that have missing indexes 123 5.3 Finding queries that have implicit data type

5.6 Unused stored procedures (2008 only) 132

Finding unused stored procedures 133

5.7 Looking for SQL queries run during a

specific interval 134

What runs over a given time period? 134

5.8 Relationships between DMV snapshot deltas 137

Amalgamated DMV snapshots 137

5.9 Currently running queries 142

What’s running now? 142

5.10 Recompiled queries 144

Finding the most-recompiled queries 144

5.11 Summary 146

6 Operating system DMVs 147

6.1 Understanding server waits 148

6.2 Identifying your most common waits 148

Why are you waiting? 149 ■ Common wait types 151

6.3 Identifying your most common waits—snapshot

version 152

Why are you waiting? (snapshot version) 153

6.4 Identifying why queries wait 154

Discovering why your queries are waiting 154

6.5 Queries that are waiting 157

6.6 Finding what’s blocking running SQL 158

What’s blocking my SQL query? 158

6.7 SQL Server performance counters 160

Important non-SQL performance counters 162

6.8 Effect of running SQL queries on the performance

counters 164 6.9 How performance counters and wait states relate 165

6.10 SQL queries and how they change the performance

counters and wait states 168 6.11 Correlating wait states and performance counters 171

6.12 Capturing DMV data periodically 172

6.13 Summary 173

7 Common Language Runtime DMVs 174

7.1 Introducing the CLR 175 7.2 A simple CLR example 176

Creating a simple CLR class 176 ■ Using the SQL CLR regular expression functions 179

7.3 NET Framework performance concerns 181 7.4 Time-consuming CLR queries 183

Finding the queries that spend the most time in the CLR 183 Impact of time-consuming CLR queries 185

7.5 Queries spending the most time in the CLR

assemblies 193 7.9 Summary 195

8 Resolving transaction issues 196

8.1 Transaction overview 197 8.2 A simple transaction-based case study 198 8.3 Locks, blocks, and deadlocks 200

Locks 200 ■ Blocks 202 ■ Deadlocks 203

8.4 The ACID properties of transactions 205 8.5 Transaction isolation levels 205

8.6 Sessions, connections, and requests 206 8.7 Finding locks 207

8.8 Identifying the contended resources 209

Contended resources—basic version 209 ■ Contended resources— enhanced version 210

8.9 Identifying inactive sessions with open transactions 212

How idle sessions with open transactions arise 213 ■ How to find

an idle session with an open transaction 213

8.10 Waiting due to transaction locks 214

Waiting because of an idle session with an open transaction 215 ■ Waiting because of active session transactions only 217 ■ Waiting because of both active and idle session transactions 218

8.11 Queries waiting for more than 30 seconds 219

8.12 Lock escalation 220

8.13 How to reduce blocking 221

8.14 How to reduce deadlocks 224

8.15 Summary 225

9 Database-level DMVs 226

9.1 Space usage in tempdb 227

What is tempdb? 227 ■ Total, free, and used space

in tempdb 228 ■ Tempdb total space usage by object type 230

9.2 Session usage in tempdb 231

Session usage of tempdb space 232 ■ Space used and not reclaimed

in tempdb by session 233

9.3 Task usage in tempdb 235

Space used by running SQL queries 235 ■ Space used and not reclaimed by active SQL queries 237

9.4 Tempdb recommendations 239

9.5 Index contention 240

Indexes under row-locking pressure 241 Escalated indexes 243 ■ Unsuccessful index-lock promotions 244 ■ Indexes with the most page splits 245 ■ Indexes with most latch contention 247 Indexes with most page I/O-latch contention 248 ■ Indexes under row-locking pressure—snapshot version 249 ■ How many rows are being inserted/deleted/updated/selected? 252

9.6 Summary 256

10 The self-healing database 257

10.1 Self-healing database 258

10.2 Recompiling slow routines 259

Recompiling routines that are running slower than usual 259

10.3 Automatically rebuild and reorganize indexes 264

Rebuilding and reorganizing fragmented indexes 264

10.4 Intelligently update statistics 268

Simple intelligent statistics update 269 ■ Time-based intelligent statistics update 272

10.5 Automatically updating a routine’s statistics 276 10.6 Automatically implement missing indexes 279

Implementing missing indexes 279

10.7 Automatically disable or drop unused indexes 281

Disabling or dropping unused indexes 282

10.8 Summary 284

11 Useful scripts 285

11.1 Viewing everyone’s last-run SQL query 286

Find the last-run queries 286

11.2 A generic performance test harness 287

Using the generic performance test harness 288

11.3 Determining the impact of a system upgrade 290

Quantifying system upgrade impact 290

11.4 Estimating the finishing time of system jobs 293

Estimating when a job will end 294

11.5 Get system information from within SQL Server 295 11.6 Viewing enabled Enterprise features (2008 only) 297 11.7 Who’s doing what and when? 298

11.8 Finding where your query really spends its time 300

Locating where your queries are spending their time 300

11.9 Memory usage per database 303

Determining the memory used per database 303

11.10 Memory usage by table or index 304

Determining the memory used by tables and indexes 304

11.11 Finding I/O waits 307

I/O waits at the database level 307 ■ I/O waits at the file level 308 ■ Average read/write times per file, per database 310

11.12 A simple lightweight trace utility 311

11.13 Some best practices 313

11.14 Where to start with performance problems 314

Starting with a slow server or database 314 ■ Starting with

slow queries 315

11.15 Summary 316

index 317

preface

When I first discovered DMVs, I was enthralled because they made many difficult things so easy It was simple to identify, typically within seconds, the core perfor-mance problems that affect SQL Server databases For example, I could quickly discover which queries were taking the longest time to run, which indexes were miss-ing, and why queries were being blocked This was only the tip of the iceberg; the deeper I dug into DMVs, the more information they provided to help me fix perfor-mance problems

Although I was captivated by the power of DMVs, I was frustrated because there was very little awareness, even among experienced DBAs, of their existence In response to this I wrote an article for Microsoft’s MSDN magazine that was published in January

2008, which showed how useful DMVs could be I then waited, expecting someone to write a book about the subject

Time passed, and although several articles about DMVs were subsequently lished, the book I wanted to read was not forthcoming So late in 2009 I contacted Manning Publications to discuss the possibility of writing such a book You’re now holding the fruit of that conversation

I’m confident this book will help you successfully identify and target your mance problems as well as suggest solutions to these problems, giving you better-performing SQL Server databases

It’s heartening to hear comments from people when they first discover the power

of DMVs; they too are amazed at how easily DMVs can help identify problems and pose possible solutions to these problems Like me, they’re probably mystified why

pro-DMVs aren’t more widely used I hope this book will help correct this situation

acknowledgements

I’d like to start off by thanking the whole Manning team, in particular Katharine Osborne, Michael Stephens, Marjan Bace, Mary Piergies, Janet Vail, Linda Reckten-wald, Katie Tennant, and Dennis Dalinnik Thank you, Katharine, for your profession-alism and steering me in the right direction, and thank you Michael and Marjan for believing the subject of DMVs could make an important contribution to improving

SQL performance I’m indebted to the Manning production team, Mary, Janet, Linda, Katie, and Dennis, for guiding me through the production process and helping make this a better book

I’d like to express my thanks to Elvira Chierkoet, for checking and reading every sentence and helping ensure my ideas were sensible

To the technical reviewers, I want to thank you for your feedback and for making this a more accurate book: Tariq Ahmed, Christian Siegers, Nikander Bruggeman, Margriet Bruggeman, Amos Bannister, Richard Siddaway, Sumit Pal, Dave Corun, and Sanchet Dighe, and special thanks to the main technical reviewer, Deepak Vohra

I want to give a special thank-you to Karen Stirk, Catherine Stirk, and Charlie for their support and encouragement A special thank-you is owed to my grandparents, Joan and Bill Bridgewater, and the rest of the Bridgewater family (Karen, Timmy, Brenda, Caroline, Kenny, Patty, Jenny, Mary, Jacky, David, and Diane) And thanks also

to my old chemistry teacher, Jim Galbraith Without these people, I would have turned out a lesser person

I’ve been lucky enough to know some interesting and helpful people, both as friends and colleagues, and I’d like to thank you all: Tim Noakes, Dave Starkey,

Mark Hadley, Gerald Hemming, Albert Morris, Martin Gunning, Chris Conn, Roy Carter, Mark Woodward, Kevin Bowen, Lee Humphries, Steven Hines, Gus Oliver, Jason Hales, Marina Barbosa, Mark Barrett, Chris Ambrose, John Dillon, Jeremy Braithwaite, Ken Piddlesden, Steve Forrester, Maria Lynch, Ernie French, Chris Cuddy, Sean Farmer, Michael O’Boyle, Ione Farias, Suresh Konduru, Francis Spencer, Iain Roy, Paul Williams, Doug Victor, Paul Weeks, John Cousins, Dale Rainsford, Scott Eggert, Julie Mathews, Pierre Bressollette, Manuel Dambrine, Alexander Godschalk, Lars Grønkjær, Raimond Bakkes, Yan Huang, Chris Homer, Lasse Lundby Franck, Andy van Dongen, Shobha Mittal, Jeroen Ameling, Alek Kudic, Ruud Lemmers, Henk Leppers, Patricia Pena Torres, David Fanning, Mike Diment, Livia Raele, Raj Kissan, Alex Rougge, David Barker, Ron Finch, Tina Simon, John Predgen, Dave Fisher, Phil Fielding, Brian Wright, Maria Iturburu, Jerome Farnon, Harbans Heer, David Randall, Bruce Pitman, Lawrence Moore, Manal Koujan, Mike Bowler, Angela Dedeng, Russell Case, Cornelius van Berkel, Sarah Hamed, and Michael Hipkin

about this book

This book captures a wealth of experience that can be used along with code pets to immediately improve the performance of your databases SQL Server is find-ing its way into an increasing number of businesses Although most servers are conspicuous, some appear almost hidden, for example, SharePoint servers and Cus-tomer Relationship Management (CRM) servers In addition, increasing amounts of data are getting stored within SQL Server Both of these trends have a bearing on the performance of your SQL Server databases and queries You can use the advice and code snippets given in this book to fight back and reclaim your high-performing

snip-SQL Server

Who should read this book?

If you want to improve the performance of your SQL Server databases and the queries that run on them, you should buy this book

Anyone who wants to ensure their SQL Server databases are running as ciently as possible will find this book very useful The following groups of people in particular will find this book valuable: database administrators (DBAs), developers working with SQL Server, and administrators of SharePoint servers, CRM systems, and similar servers

When a new version of a software product appears, for example, Microsoft Word or

SQL Server, new features are typically added to the existing core Microsoft Word is still primarily used to enter and store text; this core functionality hasn’t changed, despite the numerous version releases Similarly, although this book is written primarily

for SQL Server 2005 and 2008, the core functionality of the DMVs is unlikely to change in future versions (for example, SQL Server 2011), and so it should be applica-ble to future versions too

DBAs need to ensure the databases under their command are running as efficiently

as possible Running the code snippets provided in this book will identify any problem areas and help provide solutions to these problems

Developers need to ensure their applications can retrieve and store data efficiently Using the supplied code snippets, developers will be able to ensure appropriate indexes are being used, the data is being retrieved efficiently, and any changes are tested for defined improvement

Increasingly, SharePoint servers, CRM servers, and similar servers that have SQL

Server as their underlying database are being installed in organizations with little thought for ongoing maintenance With time, the performance of these unattended servers can degrade Applying the code snippets included in this book will identify areas where performance can be improved

One final point: Often organizations install third-party applications on their SQL

Servers Although it’s usually not possible to change the code in these applications, it

is possible to run the code snippets in this book against these databases, with a view to either applying any missing indexes (if this is allowed) or providing feedback to the third party to make the required changes

Roadmap

This book contains 100-plus code snippets to help you investigate your SQL Server bases In addition to identifying the problem areas, potential solutions are discussed The book is divided into two sections The first section provides an overview of what DMVs are and how they can identify and solve problems easily and quickly In addition, this section contains details of common patterns that are used throughout the rest of the book The second section contains scripts and discussions for improv-ing performance relating to indexes, queries, the operating system, the Common Lan-guage Runtime (CLR), transactions, space usage, and much more Using the code snippets and advice given in this section will provide you with a more optimally per-forming SQL Server

Chapter 1 provides an overview of the power of DMVs It shows you what DMVs are and why they’re important Various examples are given to get you investigating your performance problems in seconds Structures such as indexes and statistics are dis-cussed in the context of DMVs Finally, DMVs are discussed in the context of other per-formance tools

Chapter 2 discusses common patterns that are used throughout the book Rather than describing these patterns everywhere, they are discussed once in this chapter and referenced in the rest of the book

Chapter 3 looks at index-based DMVs Indexes are a great tool for improving the performance of your SQL queries However, unused or little-used indexes can have a detrimental effect on performance The code snippets included in this chapter will help you improve your index usage, resulting in improved SQL queries.

Chapter 4 takes a look at DMVs that relate to your queries Code snippets are vided to identify your slowest-running queries, queries that are blocked the most, que-ries that use the most CPU, and queries that use the most I/O All these snippets allow you to investigate performance problems from differing viewpoints

Chapter 5 is an extension of chapter 4, discussing further aspects of how to improve the performance of your queries

Chapter 6 relates to operating system DMVs It discusses why your queries, as a whole, are not able to run, what resources they’re waiting for, and how these resources can be improved to give faster queries Windows performance counters are also exam-ined in relation to these collective queries

Chapter 7 focuses on the Common Language Runtime DMVs The use of the CLR

within SQL Server is illustrated with a CLR class that provides regular expression tionality for use within your own SQL queries

Chapter 8 opens with a look at transactions, locking, blocking, and deadlocks A small case study is provided to illustrate the transaction-based DMV code snippets Ways of reducing both blocking and deadlocking are explored

Chapter 9 discusses database-related DMVs The first section discusses the tance of tempdb and shows how to examine its usage when space problems arise The second section examines various aspects of index usage that can help you diagnose and improve your queries

Chapter 10 contains code snippets that can be used to automatically improve the performance of your SQL Server databases Snippets include intelligently updating statistics, recompiling slow routines, and implementing missing indexes

Chapter 11 has useful snippets that don’t fit into any of the other chapters The snippets include a generic test harness, estimating the finishing time of jobs, how memory is used by your database, and a simple lightweight DMV trace utility

Code conventions and downloads

All source code in listings or set off from the text is in a fixed-width font like this

to separate it from ordinary text Code annotations accompany many of the listings, highlighting important concepts In some cases, numbered bullets link to explana-tions that follow the listing

The source code for all of the examples in the book is available from the lisher’s website at www.manning.com/SQLServerDMVsinAction

Author Online

The purchase of SQL Server DMV s in Action includes free access to a private forum run

by Manning Publications where you can make comments about the book, ask cal questions, and receive help from the author and other users You can access and subscribe to the forum at www.manning.com/SQLServerDMVsinAction This page pro-vides information on how to get on the forum once you’re registered, what kind of help is available, and the rules of conduct in the forum

Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place It isn’t a commitment to any specific amount of participation on the part of the author, whose contributions to the book’s forum remain voluntary (and unpaid) We suggest you try asking the author some challenging questions, lest his interest stray! The Author Online forum and the archives of previous discussions will be accessi-ble from the publisher’s website as long as the book is in print

About the author

I love to investigate and discover new things, play around with ideas, and just spend time in thought The mind can be a wonderful playground I’m lucky enough that my inquisitive nature has found a natural home among the problems in the software industry As Churchill commented, “If you find a job you really love, you’ll never work again.” With this in mind, the boundary between work and play often dissolves

As an example of my curiosity, I remember as a child examining a droplet of water

on my fingertip and noticing that the droplet magnified the detail of my fingerprints

It made me wonder if an earlier civilization (such as the Romans, who had used glass) had also noticed this, and if they did, why they didn’t develop experiments that would have led to the earlier introduction of the study of optics and the advancement of sci-ence and civilization

I’ve worked in the software industry since 1987, using a variety of platforms and programming languages I’ve worked in a variety of business areas, including banking, insurance, health, telecoms, travel, finance, software, and consultancies Since 1995 I’ve worked freelance

My core competencies are primarily Microsoft-based technologies, with an sis on software performance, which naturally extends into database performance I’m also interested in the developing mobile technologies

In the course of my work I often create software utilities; when possible I author articles on these utilities to share with other developers I feel it’s important to give something back to the industry that provides me with a living

On a final note, I’m a freelance consultant, and I’m available to help improve the performance of your SQL Servers You can contact me for availability and cost at ian_stirk@yahoo.com

About the cover illustration

The figure on the cover of SQL Server DMV s in Action is captioned “Habit of

Aureng-zeeb” and is taken from the four-volume Collection of the Dresses of Different Nations by

Thomas Jefferys, published in London between 1757 and 1772 The collection, which includes beautifully hand-colored copperplate engravings of costumes from around the world, has influenced theatrical costume design ever since it was published Aurengzeb was the name given to the sixth Mughal Emperor of India, whose reign lasted from 1658 until his death in 1707 The name means “ornament of the throne.”

He was a warrior and conqueror, greatly expanding the reach of his empire during his lifetime His exploits were the topic of many poems, legends, and dramas

The diversity of the drawings in the Collection of the Dresses of Different Nations speaks

vividly of the richness of the costumes presented on the London stage over 200 years ago The costumes, both historical and contemporaneous, offered a glimpse into the dress customs of people living in different times and in different countries, bringing them to life for London theater audiences

Dress codes have changed in the last century and the diversity by region, so rich in the past, has faded away It’s now often hard to tell the inhabitant of one continent from another Perhaps, trying to view it optimistically, we’ve traded a cultural and visual diversity for a more varied personal life Or a more varied and interesting intel-lectual and technical life

We at Manning celebrate the inventiveness, the initiative, and the fun of the puter business with book covers based on the rich diversity of regional and historical costumes brought back to life by pictures from collections such as this one

com-Part 1 Starting the journey

You’re lucky You’re about to embark on a rewarding journey with the goal of improving your SQL Server performance problems using DMVs This part pro-vides an overview of what DMVs are and the range of problems they can solve You’ll be able to use the basic examples provided to immediately begin identify-ing and fixing your performance problems Various common patterns that are used repeatedly throughout the book are detailed here This section provides a solid foundation for the rest of the book

The Dynamic Management Views gold mine

Welcome to the world of Dynamic Management Views (DMVs) How would you like

to fix problems on your SQL Servers with little effort? Or fix problems before they become noticeable by irate users? Would you like to quickly discover the slowest

SQL queries on your servers? Or get details of missing indexes that could cantly improve the performance of your queries? All these things and more are eas-ily possible, typically in a matter of seconds, using DMVs

In a nutshell, DMVs are views on internal SQL Server metadata, and they can

be used to significantly improve the performance of your SQL queries, often by

an order of magnitude A more thorough definition of DMVs follows in the next section

The first part of fixing any problem is knowing what the underlying problem is

DMVs can give you precisely this information DMVs will pinpoint where many of your problems are, often before they become painfully apparent

This chapter covers

■ What Dynamic Management Views are

■ Why they’re impor tant

■ Ready-to-run practical examples

DMVs are an integral part of Microsoft’s flagship database SQL Server Although they have existed since SQL Server 2005, their benefits are still relatively unknown, even by experienced software developers and database administrators (DBAs) Hope-fully this book will help correct this deficit.

The aim of this book is to present and explain, in short snippets of prepackaged

SQL that can be used immediately, DMV queries that will give you a valuable insight into how your SQL Server and the queries running on it can be improved, often dra-matically, quickly and easily

In this chapter you’ll learn what DMVs are, the kinds of data they contain, and the types of problems DMVs can solve I’ll outline the major groups the DMVs are divided into and the ones we’ll be concentrating on I’ll provide several example code snippets that you’ll find immediately useful DMVs will be discussed briefly in the context of other problem-solving tools and related structures (for example, indexes and statistics)

I’m sure that after reading this chapter you’ll be pleasantly surprised when you discover the wealth of information that’s available for free within SQL Server that can be accessed via DMVs and the impressive impact using this information can have The DMV data is already out there waiting to be harvested; in so many ways it’s a gold mine!

1.1 What are Dynamic Management Views?

As queries run on a SQL Server database, SQL Server automatically records tion about the activity that’s taking place, internally into structures in memory; you can access this information via DMVs DMVs are basically SQL views on some pretty important internal memory structures

Lots of different types of information are recorded that can be used for quent analysis, with the aim of improving performance, troubleshooting problems, or gaining a better insight into how SQL Server works

subse-DMV information is stored on a per–SQL Server instance level You can, however, provide filtering to extract DMV data at varying levels of granularity, including for a given database, table, or query

DMV information includes metrics that relate to indexes, query execution, the ating system, Common Language Runtime (CLR), transactions, security, extended events, Resource Governor, Service Broker, replication, query notification, objects, input/output (I/O), full-text search, databases, database mirroring, change data cap-ture (CDC), and much more In addition, many corollary areas enhance and extend the DMV output I’ll discuss these a little later, in the section titled “DMV companions.” Don’t worry if you’re not familiar with all these terms; the purpose of this book is to help explain them and present examples of how you can use them to improve the performance and your understanding of your SQL queries and SQL

oper-Server itself

What are Dynamic Management Views?

Most sources categorize DMVs in the same manner that Microsoft has adopted, based on their area of functionality This book takes a similar approach A brief out-line of each of the DMV categories follows in table 1.1

Table 1.1 The major DMV groups

DMV group Description

Change data capture Change data capture relates to how SQL Ser ver captures change activity

(inser ts, updates, and deletes) across one or more tables, providing tralized processing It can be thought of as a combination of trigger and auditing processing in a central area These DMVs contain information relating to various aspects of change data capture, including transac- tions, logging, and errors This group of DMVs occurs in SQL Ser ver

cen-2008 and higher.

Common Language

Runtime

The Common Language Runtime allows code that runs on the database to

be written in one of the NET languages, offering a richer environment and language and often providing a magnitude increase in per formance These DMVs contain information relating to various aspects of the NET Common Language Runtime, including application domains (these are wider in scope than a thread and smaller than a session), loaded assemblies, prop-

er ties, and running tasks.

Database These DMVs contain information relating to various aspects of databases,

including space usage, par tition statistics, and session and task space information.

Database mirroring The aim of database mirroring is to increase database availability

Transac-tion logs are moved quickly between ser vers, allowing fast failover to the standby ser ver These DMVs contain information relating to various aspects of database mirroring, including connection information and page- repair details.

Execution These DMVs contain information relating to various aspects of quer y

execu-tion, including cached plans, connections, cursors, plan attributes, stored procedure statistics, memor y grants, quer y optimizer information, quer y statistics, active requests and sessions, SQL text, and trigger statistics Extended events Extended events allow SQL Ser ver to integrate into Microsoft’s wider

event-handling processes, allowing integration of SQL Ser ver events with logging and monitoring tools This group of DMVs occurs in SQL Ser ver

2008 and higher.

Full-text search Full-text search relates to the ability to search character-based data using

linguistic searches This can be thought of as a higher-level wildcard search These DMVs contain information relating to various aspects of full- text search, including existing full-text catalogs, index populations cur- rently occurring, and memor y buffers/pools

Index These DMVs contain information relating to various aspects of indexes,

including missing indexes, index usage (number of seeks, scans, and ups, by system or application, and when they last occurred), operational statistics (I/O, locking, latches, and access method), and physical statis- tics (size and fragmentation information).

look-Because this book takes a look at DMVs from a practical, everyday troubleshooting and maintenance perspective, it concentrates on those DMVs that the DBA and database developer will use to help solve their everyday problems With this in mind, it concen-trates on the following categories of DMV:

■ Index

■ Execution

■ SQL Server Operating System

Input/Output (I/O) These DMVs contain information relating to various aspects of I/O,

includ-ing vir tual file statistics (by database and file, number of reads/writes, amount of data read/written, and I/O stall time), backup tape devices, and any pending I/O requests.

Object These DMVs contain information relating to various aspects of dynamic

management objects; these relate to object dependencies.

Quer y notification These DMVs contain information relating to various aspects of quer y

notifi-cation subscriptions in the ser ver

Replication These DMVs contain information relating to various aspects of

replica-tion, including ar ticles (type and status), transactions, and schemas (table columns).

Resource Governor In the past, running inappropriate ad hoc queries on the database

sometimes caused timeout and blocking problems SQL Ser ver 2008 implements a resource governor that controls the amount of resources different groups can have, allowing more controlled access to resources These DMVs contain information relating to various aspects of Resource Governor, including resource pools, governor configuration, and workload groups This group of DMVs occurs in SQL Ser ver 2008 and higher.

Ser vice Broker Ser vice Broker is concerned with providing both transactional and

discon-nected processing, allowing a wider range of architectural solutions to be created These DMVs contain information relating to various aspects of Ser vice Broker, including activated tasks, for warded messages, connec- tions, and queue monitors.

SQL Ser ver Operating

Transaction These DMVs contain information relating to various aspects of

transac-tions, including snapshot, database, session, and locks.

Security These DMVs contain information relating to various aspects of security,

including audit actions, cr yptographic algorithms suppor ted, open

cr yptographic sessions, and database encr yption state (and keys).

Table 1.1 The major DMV groups (continued)

DMV group Description

What are Dynamic Management Views?

■ Common Language Runtime

1.1.1 A glimpse into SQL Server’s internal data

As an example of what DMV information is captured, consider what happens when you run a query An immense range of information is recorded, including the following:

■ The query’s cached plan (this describes at a low level how the query is executed)

■ What indexes were used

■ What indexes the query would like to use but can’t, because they’re missing

■ How much I/O occurred (both physical and logical)

■ How much time was spent executing the query

■ How much time was spent waiting on other resources

■ What resources the query was waiting on

Being able to retrieve and analyze this information will not only give you a better understanding of how your query works but will also allow you to produce better que-ries that take advantage of the available resources

In addition to DMVs, several related functions work in conjunction with DMVs,

named Dynamic Management Functions (DMFs) In many ways DMFs are similar to dard SQL functions, being called repeatedly with a DMV-supplied parameter For example, the DMV sys.dm_exec_query_stats records details of the SQL being pro-cessed via a variable named sql_handle If this sql_handle is passed as a parameter to the DMF sys.dm_exec_sql_text, the DMF will return the SQL text of the stored proce-dure or batch associated with this sql_handle

All DMVs and DMFs belong to the sys schema, and when you reference them you must supply this schema name The DMVs start with the signature of sys.dm_*, where the asterisk represents a particular subsystem For example, to determine what requests are currently executing, run the following:

SELECT * FROM sys.dm_exec_requests

Note that this query will give you raw details of the various requests that are currently running on your SQL Server; again, don’t worry if the output doesn’t make much sense at the moment I’ll provide much more useful and understandable queries that use sys.dm_exec_requests later in the book, in the chapter related to execution DMVs (chapter 5)

1.1.2 Aggregated results

The data shown via DMVs is cumulative since the last SQL Server reboot or restart Often this is useful, because you want to know the sum total effect for each of the que-ries that have run on the server instance or a given database

But if you’re interested only in the actions of a given run of a query or batch, you can determine the effect of the query by taking a snapshot of the relevant DMV data, run your query, and then take another snapshot of the DMV data Getting the delta between the two snapshots will provide you with details of the effect of the query that was run An example of this approach is shown later, in the chapter concerning com-mon patterns, section 2.10, “Calculating DMV changes.”

There are cases where the information isn’t initially or readily available in the

DMVs In these cases, the impact of running the query may be significant Luckily these DMVs are few in number, and I’ll highlight them in the relevant section One such DMV is used when calculating the degree of index fragmentation (sys.dm_db_ index_physical_stats)

In summary, compared with other methods of obtaining similar information, for example by using the Database Tuning Advisor or SQL Server Profiler, using DMVs is relatively unobtrusive and has little impact on the system performance

1.1.4 Part of SQL Server 2005 onward

DMVs and DMFs have been an integral part of SQL Server since version 2005 In SQL

Server 2005 there are 89 DMVs (and DMFs), and in SQL Server 2008 there are 136

DMVs With this in mind, this book will concentrate on versions of SQL Server 2005 and higher It’s possible to discover the range of these DMVs by examining their names, by using the following query:

SELECT name, type_desc FROM sys.system_objects WHERE name LIKE

➥'dm_%' ORDER BY name

In versions of SQL Server prior to 2005, getting the level of detailed information given

by DMVs is difficult or impossible For example, to obtain details of the slowest ries, you’d typically have to run SQL Trace (this is the precursor of SQL Server Pro-filer) for a given duration and then spend a considerable amount of time analyzing and aggregating the results This was made more difficult because the parameters for the same queries would often differ The corresponding work using DMVs can usually

que-be done in seconds

The problems DMVs can solve

1.2 The problems DMVs can solve

In the section titled “What are Dynamic Management Views?” I briefly mentioned the different types of data that DMVs record I can assure you that this range is matched by depth too DMVs allow you to view a great deal of internal SQL Server information that’s a great starting point for determining the cause of a problem and provide potential solutions to fix many problems or give you a much better understanding of

SQL Server and your queries

NOTE DMVs aren’t the sole method of targeting the source of a problem or improving subsequent performance, but they can be used with other tools to identify and correct concerns

The problems DMVs can solve can be grouped into diagnosing, performance tuning, and monitoring In the following sections I’ll discuss each of these in turn

1.2.1 Diagnosing problems

Diagnosing problems is concerned with identifying the underlying cause of a lem This is perhaps the most common use of DMVs It’s possible to query the DMVs to diagnose many common problems, including your slowest queries, the most common causes of waiting/blocking, unused indexes, files having the most I/O, and lowest reuse of cached plans Each of these areas of concern and more could be a starting point to improving the performance of your SQL Server, whether you’re a DBA main-taining a mature server environment or a developer working on a new project

It’s possible to view problem diagnosis at various levels, including from a server perspective, a database perspective, or investigating a particular known trouble-some query Applying the correct filtering will allow you to use the DMVs at each

of these levels

Sometimes, identified problems aren’t real problems For example, there may be queries that run slowly but they run at a time when it doesn’t cause anyone any con-cern So although you could fix them, it would be more appropriate to focus your problem-solving skills on issues that are deemed more important

No one ever says their queries are running too fast; instead, users typically report how slow their queries seem to be running Taking the slow-running query as an example of a performance problem, you can use the DMVs to inspect the query’s cached plan to determine how the query is accessing its data, how resources are being used (for example, if indexes are being used or table scans), or if the statistics are out

of date, as well as to identify any missing indexes and to target the particular statement

or access path that’s causing the slowness Later we’ll look at interpreting the cached plan with a view to identifying performance bottlenecks

Knowing the areas of the query that are slow allows you to try other techniques (for example, adding a new index) to see its effect on subsequent performance Applying these new features leads us into the area of performance tuning We’ll inves-tigate a great many ways of identifying problems in the rest of the book

One final point: sometimes if a query is too complicated and contains lots of tionality, you should try breaking it down into smaller steps Not only might this high-light the problem area with finer granularity, but it might also solve it! Maybe the optimizer has more choices available to it with simpler queries and generates a better plan You can see if this is the case by examining the relevant execution DMVs, as will become clear in chapter 5.

func-1.2.2 Performance tuning

Performance tuning is concerned with applying suggested remedies to problems identified by problem diagnosis with a view to improving performance Examination

of the information shown by the DMVs should highlight areas where improvement can

be made, for example, applying a missing index, removing contention/blocking, determining the degree of fragmentation, and so on Again, the query’s cached plan

is a primary source of ideas for improvement

Measurement of any improvement is typically reflected in time or I/O counts and can be made with traditional tools such as turning on STATISTICSIO or STATISTICS TIME SQL commands or using a simple stopwatch But for more comprehensive results, you can look at the time recording provided by the DMVs This includes, for each individual SQL statement, time spent on the CPU (worker_time) and total time (elapsed_time) A large difference between these two times indicates a high degree of waiting/blocking may be occurring Similarly, DMVs also record the amount of I/O

(reads/writes at both the physical and logical level) that can be used to measure the effectiveness of a query, because less I/O typically reflects a faster query

Again, you can examine the cached plan after the improvements have been made

to determine if a more optimal access method has been chosen Performance tuning

is an iterative process This new cached plan and DMV metrics could be used for ther improvements, but again you need to ask if any remaining problem is worth solv-ing, because you should always aim to fix what is deemed to be the most important problems first

You need to be careful of the impact performance-based changes can have on the maintainability of systems; often these two needs are diametrically opposed because complexity is often increased Rather than guess where optimization is needed, you should undertake appropriate testing first to determine where it’s needed As the renowned computer scientist Donald Knuth said, “We should forget about small effi-ciencies, say about 97% of the time: premature optimization is the root of all evil.”1 I’ll discuss this in more detail in chapter 5

The problems DMVs can solve

the status of the server instance and also its history Often this transient information is lost, but it’s possible to store it for later analysis (for example, into temporary or semi-permanent tables) An example of this is given in chapter 11, section 11.7, titled

“Who’s doing what and when?”

Sometimes you have problems with the overnight batch process, reported as a eout or slow-running queries, and it would be nice to know what SQL is running dur-ing the time of this problem, giving you a starting point for further analysis

Although you might know what stored procedure is currently running on your server (from your overnight batch scheduler or sp_who2), do you know what specific lines of SQL are executing? How are the SQL queries interacting? Is blocking occur-ring? You can get this information by using DMVs combined with a simple monitoring script I’ve used such a script often to examine problems that occur during an over-night batch run

NOTE This example uses routines I’ve created and fully documented in the web links given in the following code sample (so you see, not only is code reuse good but article reuse too) Rather than talk in detail about the con-tents of these two utilities, I’ll talk about them as black boxes (if you do want

to find out more about them, look here for the routine named Tracer: mng.bz/V5E3; and look here for the routine named dba_WhatSQLIs-Executing: mng.bz/uVs3) The code for both of these stored procedures is also available on the webpage for this book on the Manning website This way you’ll be able to adapt this simple monitor pattern and possibly replace the two utilities with your own favorite utilities Later in this chapter I’ll go through the code that forms the basis of one of the stored procedures (dba_WhatSQLIsExecuting)

dba_Block-The following listing shows the code for a simple monitor

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED

D

Show SQL running

E

Wait 15 seconds

F

Repeat (500 times)

G

is blocked, dbo.dba_BlockTracer displays information about both the blockers and the blocked items Additionally, if anything is blocked (and output produced) the variable @@ROWCOUNT will have a nonzero value D This causes it to output the date and time and list all the SQL that’s running E(including the batch/stored procedure and the individual SQL statement within it that’s running) The utility then waits a specified time (15 seconds in this example F) and repeats All this is repeated (except waiting until 7 p.m.) a number of times, as specified by the last GO

statement (500 in this example G)

The routines show not only what’s being blocked but also details of what SQL is running when any blocking occurs When a SQL query runs, SQL Server assigns it a

unique identifier, named the session id or the SQL Server process id (spid) You’ll notice

that the output in the various grids contain spids that can be used to link the output from the two utilities together An example of the type of output for this query is given

in figure 1.1

The first two grids show the root-blocking spid (this is the cause of the blocking) and the blocked spid This is followed by a grid showing the date and time the block-ing occurred Finally, details of everything that’s currently running are shown; these include the individual line of SQL that’s running together with the parent query (stored procedure or batch)

A special mention should be made about the humble GO command The GO mand will execute the batch of SQL statements that occurs after the last GO state-ments If GO is followed by a number, then it will execute that number of times This is useful in many circumstances; for example, after an INSERT statement if you put GO

com-50, the insert will occur 50 times

This GO number pattern can be extended to provide a simple

concurrency/block-ing/deadlock test harness If you enter a similar batch of SQL statements into two or more distinct windows within SQL Server Management Studio (SSMS), and the state-ments are followed with a GO 5000 and the SQL in all windows run at the same time, you can discover the effect of repeatedly running the SQL at the same time

Figure 1.1 Output showing if anything is blocked and what individual SQL queries are running

Queries often compete for resources, for example, exclusive access to a given set of rows in a table This competition causes related queries to wait until the resource is free This waiting affects performance You can query the DMVs to determine what queries are waiting (being blocked) the most and aim to improve them We’ll identify the most-blocked queries later, in chapter 4, section 4.5, “Queries that spend a long time being blocked.”

You can use the simple monitor utility discussed previously to determine why these identified queries are being blocked; the DMVs will tell you what is blocked, but they don’t identify what’s blocking them The monitoring utility can do this The monitor utility can be a powerful tool in identifying why and how the most-blocked queries are being blocked

Having looked at what kind of problems DMVs can help solve, let’s now dive into some simple but useful DMV example code that can be helpful in solving real-life pro-duction problems

immedi-NOTE All the examples are prefixed with a statement concerning isolation level This determines how the subsequent SQL statements in the batch inter-act with other running SQL statements The statement sets the isolation level

to read uncommitted This ensures you can read data without waiting for locks to be released or acquiring locks yourself, resulting in the query run-ning more quickly with minimal impact on other running SQL queries The statement used is

SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED

Wait until

7 p.m.

B

Show running SQL

C

Wait 15 seconds

D

Repeat (500 times)

E