ptg 1404 CHAPTER 38 Database Design and Performance nature and are not affected by the version of the database management system. This chapter focuses on those relatively unchanged principles. There are, however, some new features in SQL Server 2008 that will augment these basic principles. Filtered indexes, new query and table hints, plus other table-oriented features are just a few things you should consider when designing your database for performance. These features are discussed in detail in Chapter 24, “Creating and Managing Tables,” Chapter 34, “Data Structures, Indexes, and Performance,” and other chapters in Part V, “SQL Server Performance and Optimization.” Basic Tenets of Designing for Performance Designing for performance requires making trade-offs. For example, to get the best write performance out of a database, you must sacrifice read performance. Before you tackle database design issues for an application, it is critical to understand your goals. Do you want faster read performance? Faster write performance? A more understandable design? Following are some basic truths about physical database design for SQL Server 2008 and the performance implications of each: . It’s important to keep table row sizes as small as possible. Doing so is not about saving disk space. Having smaller rows means more rows fit on a single 8KB page, which means fewer physical disk reads are required to read a given number of rows. . You should use indexes to speed up read access. However, the more indexes a table has, the longer it takes to insert, update, and delete rows from the table. . Using triggers to perform any kind of work during an insert, an update, or a delete exacts a performance toll and decreases concurrency by lengthening transaction duration. . Implementing declarative referential integrity (via primary and foreign keys) helps maintain data integrity, but enforcing foreign key constraints requires extra lookups on the primary key table to ensure existence. . Using ON DELETE CASCADE referential integrity constraints helps maintain data integrity but requires extra work on the server’s part. Keeping tables as narrow as possible—that is, ensuring that the row size is as small as possible—is one of the most important things you can do to ensure that a database performs well. To keep your tables narrow, you should choose column data types with size in mind. You shouldn’t use the bigint data type if the int will do. If you have zero-to-one relationships in tables, you should consider vertically partitioning the tables. (See the “Vertical Data Partitioning” section, later in this chapter, for details on this scenario.) Cascading deletes (and updates) cause extra lookups to be done whenever a delete runs against the parent table. In many cases, the optimizer uses worktables to resolve delete and update queries. Enforcing these constraints manually, from within stored procedures, for example, can give better performance. This is not a wholehearted endorsement against ptg 1405 Logical Database Design Issues 38 referential integrity constraints. In most cases, the extra performance hit is worth the saved aggravation of coding everything by hand. However, you should be aware of the cost of this convenience. Logical Database Design Issues A good database design is fundamental to the success of any application. Logical database design for relational databases follows rules of normalization. As a result of normalization, you create a data model that is usually, but not necessarily, translated into a physical data model. A logical database design does not depend on the relational database you intend to use. The same data model can be applied to Oracle, Sybase, SQL Server, or any other rela- tional database. On the other hand, a physical data model makes extensive use of the features of the underlying database engine to yield optimal performance for the applica- tion. Physical models are much less portable than logical models. TIP If portability is a big concern to you, consider using a third-party data modeling tool, such as ERwin or ERStudio. These tools have features that make it easier to migrate your logical data models to physical data models on different database platforms. Of course, using these tools just gets you started; to get the best performance out of your design, you need to tweak the physical design for the platform you have chosen. Normalization Conditions Any database designer must address two fundamental issues: . Designing the database in a simple, understandable way that is maintainable and makes sense to its developers and users . Designing the database such that data is fetched and saved with the fastest response time, resulting in high performance Normalization is a technique used on relational databases to organize data across many tables so that related data is kept together based on certain guidelines. Normalization results in controlled redundancy of data; therefore, it provides a good balance between disk space usage and performance. Normalization helps people understand the relation- ships between data and enforces rules to ensure that the data is meaningful. TIP Normalization rules exist, among other reasons, to make it easier for people to under- stand the relationships between data. But a perfectly normalized database sometimes doesn’t perform well under certain circumstances, and it may be difficult to under- stand. There are good reasons to deviate from a perfectly normalized database. ptg 1406 CHAPTER 38 Database Design and Performance Normalization Forms Five normalization forms exist, represented by the symbol 1NF for first normal form, 2NF for second normal form, and so on. If you follow the rules for the first rule of normaliza- tion, your database can be described as “in first normal form.” Each rule of normalization depends on the previous rule for successful implementation, so to be in second normal form (2NF), your database must also follow the rules for first normal form. A typical relational database used in a business environment falls somewhere between second and third normal forms. It is rare to progress past the third normal form because fourth and fifth normal forms are more academic than practical in real-world environments. Following is a brief description of the first three rules of normalization. First Normal Form The first rule of normalization requires removing repeating data values and specifies that no two rows in a table can be identical. This means that each table must have a logical primary key that uniquely identifies a row in the table. Consider a table that has four columns— PublisherName, Title1, Title2, and Title3—for storing up to three titles for each publisher. This table is not in first normal form due to the repeating Title columns. The main problem with this design is that it limits the number of titles associated with a publisher to three. Removing the repeating columns so there is just a PublisherName column and a single Title column puts the table in first normal form. A separate data row is stored in the table for each title published by each publisher. The combination of PublisherName and Title becomes the primary key that uniquely identifies each row and prevents duplicates. Second Normal Form A table is considered to be in second normal form if it conforms to the first normal form and all nonkey attributes of the table are fully dependent on the entire primary key. If the primary key consists of multiple columns, nonkey columns should depend on the entire key and not just on a part of the key. A table with a single column as the primary key is automatically in second normal form if it satisfies first normal form as well. Assume that you need to add the publisher address to the database. Adding it to the table with the PublisherName and Title column would violate second normal form. The primary key consists of both PublisherName and Title, but the PublisherAddress attribute is an attribute of the publisher only. It does not depend on the entire primary key. To put the database in second normal form requires adding an additional table for storing publisher information. One table consists of the PublisherName column and PublisherAddress. The second table contains the PublisherName and Title columns. To ptg 1407 Logical Database Design Issues 38 retrieve the PublisherName, Title, and PublisherAddress information in a single result would require a join between the two tables on the PublisherName column. Third Normal Form A table is considered to be in third normal form if it already conforms to the first two normal forms and if none of the nonkey columns are dependent on any other nonkey columns. All such attributes should be removed from the table. Let’s look at an example that comes up often during database architecture. Suppose that an employee table has four columns: EmployeeID (the primary key), salary, bonus, and total_salary, where total_salary = salary + bonus. Existence of the total_salary column in the table violates the third normal form because a nonkey column ( total_salary) is dependent on two other nonkey columns (salary and bonus). Therefore, for the table to conform to the third rule of normalization, you must remove the total_salary column from the employee table. Benefits of Normalization Following are the major advantages of normalization: . Because information is logically kept together, normalization provides improved overall understanding of the system. . Because of controlled redundancy of data, normalization can result in fast table scans and searches (because less physical data has to be processed). . Because tables are smaller with normalization, index creation and data sorts are much faster. . With less redundant data, it is easier to maintain referential integrity for the system. . Normalization results in narrower tables. Because you can store more rows per page, more rows can be read and cached for each I/O performed on the table. This results in better I/O performance. Drawbacks of Normalization One result of normalization is that data is stored in multiple tables. To retrieve or modify information, you usually have to establish joins across multiple tables. Joins are expensive from an I/O standpoint. Multitable joins can have an adverse impact on the performance of the system. The following sections discuss some of the denormalization techniques you can use to improve the performance of a system. ptg 1408 CHAPTER 38 Database Design and Performance TIP An adage for normalization is “Normalize ’til it hurts; denormalize ’til it works.” To put this maxim into use, try to put your database in third normal form initially. Then, when you’re ready to implement the physical structure, drop back from third normal form, where excessive table joins are hurting performance. A common mistake is that devel- opers make too many assumptions and over-denormalize the database design before even a single line of code has been written to even begin to assess the database performance. Denormalizing a Database After a database has been normalized to the third form, database designers intentionally backtrack from normalization to improve the performance of the system. This technique of rolling back from normalization is called denormalization. Denormalization allows you to keep redundant data in the system, reducing the number of tables in the schema and reducing the number of joins to retrieve data. TIP Duplicate data is more helpful when the data does not change very much, such as in data warehouses. If the data changes often, keeping all “copies” of the data in sync can create significant performance overhead, including long transactions and excessive write operations. Denormalization Guidelines When should you denormalize a database? Consider the following points: . Be sure you have a good overall understanding of the logical design of the system. This knowledge helps in determining how other parts of the application are going to be affected when you change one part of the system. . Don’t attempt to denormalize the entire database at once. Instead, focus on the specific areas and queries that are accessed most frequently and are suffering from performance problems. . Understand the types of transactions and the volume of data associated with specific areas of the application that are having performance problems. You can resolve many such issues by tuning the queries without denormalizing the tables. . Determine whether you need virtual (computed) columns. Virtual columns can be computed from other columns of the table. Although this violates third normal form, computed columns can provide a decent compromise because they do not actually store another exact copy of the data in the same table. . Understand data integrity issues. With more redundant data in the system, main- taining data integrity is more difficult, and data modifications are slower. ptg 1409 Denormalizing a Database 38 . Understand storage techniques for the data. You may be able to improve perfor- mance without denormalization by using RAID, SQL Server filegroups, and table partitioning. . Determine the frequency with which data changes. If data is changing too often, the cost of maintaining data and referential integrity might outweigh the benefits provided by redundant data. . Use the performance tools that come with SQL Server (such as SQL Server Profiler) to assess performance. These tools can help isolate performance issues and give you possible targets for denormalization. TIP If you are experiencing severe performance problems, denormalization should not be the first step you take to rectify the problem. You need to identify specific issues that are causing performance problems. Usually, you discover factors such as poorly written queries, poor index design, inefficient application code, or poorly configured hardware. You should try to fix these types of issues before taking steps to denormalize data- base tables. Essential Denormalization Techniques You can use various methods to denormalize a database table and achieve desired perfor- mance goals. Some of the useful techniques used for denormalization include the following: . Keeping redundant data and summary data . Using virtual columns . Performing horizontal data partitioning . Performing vertical data partitioning Redundant Data From an I/O standpoint, joins in a relational database are inherently expensive. To avoid common joins, you can add redundancy to a table by keeping exact copies of the data in multiple tables. The following example demonstrates this point. This example shows a three-table join to get the title of a book and the primary author’s name: select c.title, a.au_lname, a.au_fname from authors a join titleauthor b on a.au_id = b.au_id join titles c on b.title_id = c.title_id where b.au_ord = 1 order by c.title You could improve the performance of this query by adding the columns for the first and last names of the primary author to the titles table and storing the information in the ptg 1410 CHAPTER 38 Database Design and Performance titles table directly. This would eliminate the joins altogether. Here is what the revised query would look like if this denormalization technique were implemented: select title, au_lname, au_fname from titles order by title As you can see, the au_lname and au_fname columns are now redundantly stored in two places: the titles table and authors table. It is obvious that with more redundant data in the system, maintaining referential integrity and data integrity is more difficult. For example, if the author’s last name changed in the authors table, to preserve data integrity, you would also have to change the corresponding au_lname column value in the titles table to reflect the correct value. You could use SQL Server triggers to maintain data integrity, but you should recognize that update performance could suffer dramatically. For this reason, it is best if redundant data is limited to data columns whose values are rela- tively static and are not modified often. Computed Columns A number of queries calculate aggregate values derived from one or more columns of a table. Such computations can be CPU intensive and can have an adverse impact on perfor- mance if they are performed frequently. One of the techniques to handle such situations is to create an additional column that stores the computed value. Such columns are called virtual columns, or computed columns. Since SQL Server 7.0, computed columns have been natively supported. You can specify such columns in create table or alter table commands. The following example demonstrates the use of computed columns: create table emp ( empid int not null primary key, salary money not null, bonus money not null default 0, total_salary as ( salary+bonus ) ) go insert emp (empid, salary, bonus) values (100, $150000.00, $15000) go select * from emp go empid salary bonus total_salary 100 150000.0000 15000.0000 165000.0000 By default, virtual columns are not physically stored in SQL Server tables. SQL Server inter- nally maintains a column property named iscomputed that can be viewed from the sys.columns system view. It uses this column to determine whether a column is computed. The value of the virtual column is calculated at the time the query is run. All columns ptg 1411 Denormalizing a Database 38 referenced in the computed column expression must come from the table on which the computed column is created. You can, however, reference a column from another table by using a function as part of the computed column’s expression. The function can contain a reference to another table, and the computed column calls this function. Since SQL Server 2000, computed columns have been able to participate in joins to other tables, and they can be indexed. Creating an index that contains a computed column creates a physical copy of the computed column in the index tree. Whenever a base column participating in the computed column changes, the index must also be updated, which adds overhead and may possibly slow down update performance. In SQL Server 2008, you also have the option of defining a computed column so that its value is physically stored. You accomplish this with the ADD PERSISTED option, as shown in the following example: Alter the computed SetRate column to be PERSISTED ALTER TABLE Sales.CurrencyRate alter column SetRate ADD PERSISTED SQL Server automatically updates the persisted column values whenever one of the columns that the computed column references is changed. Indexes can be created on these columns, and they can be used just like nonpersisted columns. One advantage of using a computed column that is persisted is that it has fewer restrictions than a nonper- sisted column. In particular, a persisted column can contain an imprecise expression, which is not possible with a nonpersisted column. Any float or real expressions are considered imprecise. To ensure that you have a precise column you can use the COLUMNPROPERTY function and review the IsPrecise property to determine whether the computed column expression is precise. Summary Data Summary data is most helpful in a decision support environment, to satisfy reporting requirements and calculate sums, row counts, or other summary information and store it in a separate table. You can create summary data in a number of ways: . Real-time—Every time your base data is modified, you can recalculate the summary data, using the base data as a source. This is typically done using stored procedures or triggers. . Real-time incremental—Every time your base data is modified, you can recalculate the summary data, using the old summary value and the new data. This approach is more complex than the real-time option, but it could save time if the increments are relatively small compared to the entire dataset. This, too, is typically done using stored procedures or triggers. . Delayed—You can use a scheduled job or custom service application to recalculate summary data on a regular basis. This is the recommended method to use in an OLTP system to keep update performance optimal. ptg 1412 CHAPTER 38 Database Design and Performance JanBill 1000000 records DecBill 1000000 records FebBill Monthly Billing Charges 90000000 records Attributes Acct# BillDate Balance FIGURE 38.1 Horizontal partitioning of data. Horizontal Data Partitioning As tables grow larger, data access time also tends to increase. For queries that need to perform table scans, the query time is proportional to the number of rows in the table. Even when you have proper indexes on such tables, access time slows as the depth of the index trees increases. The solution is splitting the table into multiple tables such that each table has the same table structure as the original one but stores a different set of data. Figure 38.1 shows a billing table with 90 million records. You can split this table into 12 monthly tables (all with the identical table structure) to store billing records for each month. You should carefully weigh the options when performing horizontal splitting. Although a query that needs data from only a single month gets much faster, other queries that need a full year’s worth of data become more complex. Also, queries that are self-referencing do not benefit much from horizontal partitioning. For example, the business logic might dictate that each time you add a new billing record to the billing table, you need to check any outstanding account balance for previous billing dates. In such cases, before you do an insert in the current monthly billing table, you must check the data for all the other months to find any outstanding balance. ptg 1413 Denormalizing a Database 38 TIP Horizontal splitting of data is useful where a subset of data might see more activity than the rest of the data. For example, say that in a healthcare provider setting, 98% of the patients are inpatients, and only 2% are outpatients. In spite of the small percent- age involved, the system for outpatient records sees a lot of activity. In this scenario, it makes sense to split the patient table into two tables—one for the inpatients and one for the outpatients. When splitting tables horizontally, you must perform some analysis to determine the optimal way to split the table. You need to find a logical dimension along which to split the data. The best choice takes into account the way your users use your data. In the example that involves splitting the data among 12 tables, date was mentioned as the optimal split candidate. However, if the users often did ad hoc queries against the billing table for a full year’s worth of data, they would be unhappy with the choice to split that data among 12 different tables. Perhaps splitting based on a customer type or another attribute would be more useful. NOTE You can use partitioned views to hide the horizontal splitting of tables. The benefit of using partitioned views is that multiple horizontally split tables appear to the end users and applications as a single large table. When this is properly defined, the optimizer automatically determines which tables in the partitioned view need to be accessed, and it avoids searching all tables in the view. The query runs as quickly as if it were run only against the necessary tables directly. For more information on defining and using partitioned views, see Chapter 27, “Creating and Managing Views.” In SQL Server 2008, you also have the option of physically splitting the rows in a single table over more than one partition. This feature, called partitioned tables, utilizes a par- titioning function that splits the data horizontally and a partitioning scheme that assigns the horizontally partitioned data to different filegroups. When a table is creat- ed, it references the partitioned schema, which causes the rows of data to be physical- ly stored on different filegroups. No additional tables are needed, and the table is still referenced with the original table name. The horizontal partitioning happens at the physical storage level and is transparent to the user. Vertical Data Partitioning As you know, a database in SQL Server consists of 8KB pages, and a row cannot span multiple pages. Therefore, the total number of rows on a page depends on the width of the table. This means the wider the table, the smaller the number of rows per page. You can achieve significant performance gains by increasing the number of rows per page, . the benefits provided by redundant data. . Use the performance tools that come with SQL Server (such as SQL Server Profiler) to assess performance. These tools can help isolate performance issues. 150000.0000 15000.0000 165000.0000 By default, virtual columns are not physically stored in SQL Server tables. SQL Server inter- nally maintains a column property named iscomputed that can be viewed. focuses on those relatively unchanged principles. There are, however, some new features in SQL Server 2008 that will augment these basic principles. Filtered indexes, new query and table hints,