4 – Managing data growth 115 Figure 4.23: Addition of covering indexes leads to an efficient index seek operation. So, while indexes do indeed take space, this space utilization is usually more than warranted when they are used correctly, and we see the desired pay-off in query performance. The issue with indexes arises when development teams adopt a scattergun approach to indexes, sometimes to the point of redundancy and harm to the database. Adding indexes arbitrarily can often do as much harm as good, not only because of the space that they take up, but because each index will need to be maintained, which takes time and resources. TempDB No DBA who has been working with SQL Server for long will have been immune to runaway TempDB growth. If this growth is left unchecked, it can eventually fill up a drive and prohibit any further activity in SQL Server that also requires the use of the TempDB database. SQL Server uses the TempDB database for a number of processes, such as sorting operations, creating indexes, cursors, table variables, database mail and user defined functions, to name several. In addition to internal processes, users have the ability to create temporary tables and have free reign to fill these tables with as much data as they wish, assuming that growth of the TempDB data file is not restricted to a specific value, which by default it is not. I do not recommend restricting growth for TempDB files, but I do recommend that you be aware of what will happen if TempDB does fill up. Many SQL Server processes, including user processes, will cease and an error message will be thrown, as I will show. The TempDB database is created each time SQL Server is restarted. It is never backed up nor can it be. It is always in Simple mode and the recovery model cannot be changed. 4 – Managing data growth 116 There are a couple of TempDB "properties", though, that you can and should change when configuring your server: • Its location • Its autogrowth rate By default, TempDB is created in the default data folder, which is set during SQL installation. It is highly recommended that, if possible, this location be changed so that TempDB resides on its own disk. Many DBAs also create multiple TempDB files, typically one per processor, with the aim of boosting performance still further. However, be warned that you will need to spread the load of these multiple files across multiple disks, in order to achieve this. Like all other databases, TempDB adopts the default configuration of the model database, which means that it will grow in 10% increments with unrestricted growth, unless you specify otherwise. In my opinion, having an autogrowth of 10% on TempDB is a bad idea because when rogue queries hit your server, calling for temporary tables, as they will do eventually, you do not want the TempDB database filling up the drive. Let's assume that you have a 30G TempDB database sitting on a 50G drive and autogrowing in 10% (i.e. 3G) increments. It would take only 6 growth events to fill the drive. Ideally, you will want to set a fixed growth rate of 3G for TempDB and use multiple TempDB data files across multiple disks. When loading multiple tens of millions of records into TempDB, bearing in mind that 1 million records is roughly equivalent to 1G, you can see how this can happen fairly easily. So, what happens when TempDB fills up? Let's find out! I'd have to generate a lot of TempDB activity to fill up 50GB of disk, so I am going to artificially restrict the data file for TempDB to a size of 200 MB, via the "maximum file size" property. Figure 4.24 shows the configuration. 4 – Managing data growth 117 Figure 4.24: Changing the TempDB maximum file size to 2 Gigabytes for simulation. Now that I've set the maximum file size for TempDB, it is time to fill it up and for that I will turn to our old friend, the endless loop. I have seen only a few of these in the wild but they do exist, I promise, and when you combine an endless loop with data or log space limitation, something has to give. Listing 4.4 shows the loopy code. CREATE TABLE #HoldAll ( Read_ID INT, Read_Date DATETIME, Person VARCHAR(100) ) GO DECLARE @cnt int = 1 WHILE @cnt = 1 BEGIN INSERT INTO #HoldAll SELECT Read_ID, Read_Date, Person FROM All_Books_Ever_Read.dbo.book_List WHERE Read_Date > '05/21/08' END GO Listing 4.4: The dreaded endless loop. 4 – Managing data growth 118 Notice that @cnt is given the value of 1, but nowhere subsequently is the value changed, so this query will run and run until it fills up a drive or surpasses a file size threshold, whichever comes sooner. In this example, the query runs for 3 minutes before we hit the 200MB file size limit, as shown in Figure 4.25, and get an error that the filegroup is full. Figure 4.25: Filling up TempDB. At this point the query fails, obviously, as will any other queries that need to use TempDB. SQL Server is still functioning properly, but as long as the temp table #HoldAll exists, TempDB will stay filled. Hopefully, you've got notifications and alerts set up to warn you of the imminent danger, before the file actually fills up (I will cover notifications, alerts and monitoring in depth in Chapter 6). In any event, you are likely to experience that DBA:M feeling, having spent half the night trying to track down the problem query and resolve the issue. Your three options, as a DBA, are to: • Restart SQL Server. • Try to shrink the TempDB database. • Find the errant query and eradicate it. Generally speaking, restarting is not always an option in a production system. Shrinking TempDB is a valid option, assuming that it can be shrunk. Sometimes, when there are open transactions, it is not possible. Therefore, finding and killing the offending query is the more likely course of action. The techniques you can use to do this are the focus of the very next chapter, on Troubleshooting. For now, I am going to simply close the query window which should force the temp table to be deleted and so allow the shrink operation to go ahead. Sure 4 – Managing data growth 119 enough, once I'd closed the connection I was able to select Tasks | Shrink |Database from within SSMS, and so shrink TempDB from 200 MB back down to its original size of 8K. Problem solved. Now, back to bed with a sleepy note to self to find the developer who wrote this code, and chastise him or her. Wait, I am the DBA who let this get into production in the first place, so new list … chastise self, get back to sleep, find the developer tomorrow and chastise him or her anyway; if they ask how it got into production … change subject. A query to determine current space utilization I have written a few articles about various queries that help me with my day to day job as a DBA. The following query is one that I use every single day to monitor potential space issues on my servers. If I notice a "danger signal" I can then dig deeper and determine the root cause, which is usually one of the issues discussed in this chapter i.e. log file growth due to incorrect recovery models, too many indexes, TempDB filling up, or just poor capacity planning. The SizeQuery query, shown in Listing 4.5, combines output from several sources, such as sp_MSForEachDB and xp_fixeddrives, and merges them to show how much data and log space is used, what drive that space is used on, and how much free space is available. Set NoCount On Check to see the temp table exists IF EXISTS ( SELECT Name FROM tempdb sysobjects Where name like '#HoldforEachDB%' ) If So Drop it DROP TABLE #HoldforEachDB_size Recreate it CREATE TABLE #HoldforEachDB_size ( [DatabaseName] [nvarchar](75) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL, [Size] [decimal] NOT NULL, [Name] [nvarchar](75) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL, [Filename] [nvarchar](255) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL, ) ON [PRIMARY] . working with SQL Server for long will have been immune to runaway TempDB growth. If this growth is left unchecked, it can eventually fill up a drive and prohibit any further activity in SQL Server. does fill up. Many SQL Server processes, including user processes, will cease and an error message will be thrown, as I will show. The TempDB database is created each time SQL Server is restarted COLLATE SQL_ Latin1_General_CP1_CI_AS NOT NULL, [Size] [decimal] NOT NULL, [Name] [nvarchar](75) COLLATE SQL_ Latin1_General_CP1_CI_AS NOT NULL, [Filename] [nvarchar](255) COLLATE SQL_ Latin1_General_CP1_CI_AS