x CONTENTS 1.4.4 CREATE TRIGGER Statement 52 1.4.5 CREATE PROCEDURE Statement 53 1.4.6 DECLARE CURSOR Statement 53 2 Normalization 61 2.1 Functional and Multivalued Dependencies 64 2.2 First Normal Form (1NF) 64 2.2.1 Note on Repeated Groups 66 2.3 Second Normal Form (2NF) 70 2.4 Third Normal Form (3NF) 71 2.5 Elementary Key Normal Form (EKNF) 72 2.6 Boyce-Codd Normal Form (BCNF) 73 2.7 Fourth Normal Form (4NF) 75 2.8 Fifth Normal Form (5NF) 76 2.9 Domain-Key Normal Form (DKNF) 78 2.10 Practical Hints for Normalization 87 2.11 Key Types 88 2.11.1 Natural Keys 89 2.11.2 Artificial Keys 89 2.11.3 Exposed Physical Locators 89 2.11.4 Practical Hints for Denormalization 91 2.11.5 Row Sorting 93 3 Numeric Data in SQL 101 3.1 Numeric Types 101 3.1.1 BIT, BYTE, and BOOLEAN Data Types 104 3.2 Numeric Type Conversion 105 3.2.1 Rounding and Truncating 105 3.2.2 CAST() Function 107 3.3 Four-Function Arithmetic 108 3.4 Arithmetic and NULLs 109 3.5 Converting Values to and from NULL 110 3.5.1 NULLIF() Function 110 3.5.2 COALESCE() Function 111 3.6 Vendor Math Functions 113 CONTENTS xi 3.6.1 Number Theory Operators 113 3.6.2 Exponential Functions 116 3.6.3 Scaling Functions 116 3.6.4 Converting Numbers to Words 117 4 Temporal Data Types in SQL 119 4.1 Notes on Calendar Standards 119 4.2 SQL Temporal Data Types 123 4.2.1 Tips for Handling Dates, Timestamps, and Times 124 4.2.2 Date Format Standards 124 4.2.3 Handling Timestamps 125 4.2.4 Handling Times 127 4.3 Queries with Date Arithmetic 128 4.4 The Nature of Temporal Data Models 129 4.4.1 Temporal Duplicates 129 4.4.2 Temporal Databases 135 4.4.3 Temporal Projection and Selection 137 4.4.4 Temporal Joins 139 4.4.5 Modifying Valid-Time State Tables 145 4.4.6 Current Modifications 146 4.4.7 Sequenced Modifications 150 4.4.8 Nonsequenced Modifications 155 4.4.9 Transaction-Time State Tables 156 4.4.10 Maintaining the Audit Log 158 4.4.11 Querying the Audit Log 160 4.4.12 Modifying the Audit Log 164 4.4.13 Bitemporal Tables 164 4.4.14 Temporal Support in Standard SQL 167 5 Character Data Types in SQL 169 5.1 Problems with SQL Strings 170 5.1.1 Problems of String Equality 170 5.1.2 Problems of String Ordering 171 5.1.3 Problems of String Grouping 172 5.2 Standard String Functions 172 xii CONTENTS 5.3 Common Vendor Extensions 174 5.3.1 Phonetic Matching 175 5.4 Cutter Tables 182 6 NULLs: Missing Data in SQL 185 6.1 Empty and Missing Tables 187 6.2 Missing Values in Columns 187 6.3 Context and Missing Values 189 6.4 Comparing NULLs 190 6.5 NULLs and Logic 190 6.5.1 NULLS in Subquery Predicates 191 6.5.2 Standard SQL Solutions 193 6.6 Math and NULLs 193 6.7 Functions and NULLs 193 6.8 NULLs and Host Languages 194 6.9 Design Advice for NULLs 195 6.9.1 Avoiding NULLs from the Host Programs 197 6.10 A Note on Multiple NULL Values 198 7 Multiple Column Data Elements 201 7.1 Distance Functions 201 7.2 Storing an IP Address in SQL 202 7.2.1 A Single VARCHAR(15) Column 203 7.2.2 One INTEGER Column 203 7.2.3 Four SMALLINT Columns 205 7.3 Currency and Other Unit Conversions 205 7.4 Social Security Numbers 206 7.5 Rational Numbers 209 8 Table Operations 211 8.1 DELETE FROM Statement 211 8.1.1 The DELETE FROM Clause 212 8.1.2 The WHERE Clause 212 8.1.3 Deleting Based on Data in a Second Table 216 8.1.4 Deleting within the Same Table 216 CONTENTS xiii 8.1.5 Deleting in Multiple Tables without Referential Integrity 220 8.2 INSERT INTO Statement 221 8.2.1 INSERT INTO Clause 221 8.2.2 The Nature of Inserts 222 8.2.3 Bulk Load and Unload Utilities 223 8.3 The UPDATE Statement 223 8.3.1 The UPDATE Clause 223 8.3.2 The WHERE Clause 224 8.3.3 The SET Clause 225 8.3.4 Updating with a Second Table 226 8.3.5 Using the CASE Expression in UPDATEs 228 8.4 A Note on Flaws in a Common Vendor Extension 231 8.5 MERGE Statement 232 9 Comparison or Theta Operators 235 9.1 Converting Data Types 236 9.2 Row Comparisons in SQL 238 10 Valued Predicates 241 10.1 IS NULL Predicate 241 10.1.1 Sources of NULLs 242 10.2 IS [NOT]{TRUE | FALSE | UNKNOWN} Predicate 242 10.3 IS [NOT] NORMALIZED Predicate 244 11 CASE Expressions 247 11.1 The CASE Expression 247 11.1.1 The COALESCE() and NULLIF() Functions 251 11.1.2 CASE Expressions with GROUP BY 252 11.1.3 CASE, CHECK() Clauses and Logical Implication 253 11.1.4 Subquery Expressions and Constants 257 11.2 Rozenshtein Characteristic Functions 258 12 LIKE Predicate 261 12.1 Tricks with Patterns 262 xiv CONTENTS 12.2 Results with NULL Values and Empty Strings 264 12.3 LIKE Is Not Equality 264 12.4 Avoiding the LIKE Predicate with a Join 264 12.5 CASE Expressions and LIKE Predicates 266 12.6 SIMILAR TO Predicates 267 12.7 Tricks with Strings 269 12.7.1 String Character Content 269 12.7.2 Searching versus Declaring a String 270 12.7.3 Creating an Index on a String 270 13 BETWEEN and OVERLAPS Predicates 273 13.1 The BETWEEN Predicate 273 13.1.1 Results with NULL Values 274 13.1.2 Results with Empty Sets 274 13.1.3 Programming Tips 274 13.2 OVERLAPS Predicate 275 13.2.1 Time Periods and OVERLAPS Predicate 275 14 The [NOT] IN() Predicate 287 14.1 Optimizing the IN() Predicate 288 14.2 Replacing ORs with the IN() Predicate 292 14.3 NULLs and the IN() Predicate 293 14.4 IN() Predicate and Referential Constraints 295 14.5 IN() Predicate and Scalar Queries 297 15 EXISTS() Predicate 299 15.1 EXISTS and NULLs 300 15.2 EXISTS and INNER JOINs 302 15.3 NOT EXISTS and OUTER JOINs 303 15.4 EXISTS() and Quantifiers 304 15.5 EXISTS() and Referential Constraints 305 15.6 EXISTS and Three-Valued Logic 306 CONTENTS xv 16 Quantified Subquery Predicates 309 16.1 Scalar Subquery Comparisons 310 16.2 Quantifiers and Missing Data 311 16.3 The ALL Predicate and Extrema Functions 313 16.4 The UNIQUE Predicate 314 17 The SELECT Statement 317 17.1 SELECT and JOINs 317 17.1.1 One-Level SELECT Statement 317 17.1.2 Correlated Subqueries in a SELECT Statement 324 17.1.3 SELECT Statement Syntax 326 17.1.4 The ORDER BY Clause 328 17.2 OUTER JOINs 336 17.2.1 Syntax for OUTER JOINs 337 17.2.2 NULLs and OUTER JOINs 342 17.2.3 NATURAL versus Searched OUTER JOINs 344 17.2.4 Self OUTER JOINs 345 17.2.5 Two or More OUTER JOINs 346 17.2.6 OUTER JOINs and Aggregate Functions 348 17.2.7 FULL OUTER JOIN 349 17.2.8 WHERE Clause OUTER JOIN Operators 350 17.3 Old versus New JOIN Syntax 351 17.4 Scope of Derived Table Names 353 17.5 JOINs by Function Calls 354 17.6 The UNION JOIN 356 17.7 Packing Joins 358 17.8 Dr. Codd’s T-Join 359 17.8.1 The Croatian Solution 363 17.8.2 The Swedish Solution 364 17.8.3 The Colombian Solution 364 18 VIEWs, Derived Tables, Materialized Tables, and Temporary Tables 369 18.1 VIEWs in Queries 370 18.2 Updatable and Read-Only VIEWs 371 xvi CONTENTS 18.3 Types of VIEWs 373 18.3.1 Single-Table Projection and Restriction 373 18.3.2 Calculated Columns 373 18.3.3 Translated Columns 373 18.3.4 Grouped VIEWs 374 18.3.5 UNIONed VIEWs 375 18.3.6 JOINs in VIEWs 377 18.3.7 Nested VIEWs 377 18.4 How VIEWs Are Handled in the Database System 379 18.4.1 View Column List 379 18.4.2 VIEW Materialization 379 18.4.3 In-Line Text Expansion 380 18.4.4 Pointer Structures 382 18.4.5 Indexing and Views 383 18.5 WITH CHECK OPTION Clause 383 18.5.1 WITH CHECK OPTION as CHECK() Clause 388 18.6 Dropping VIEWs 389 18.7 TEMPORARY TABLE Declarations 390 18.8 Hints on Using VIEWs and TEMPORARY TABLEs 391 18.8.1 Using VIEWs 392 18.8.2 Using TEMPORARY TABLEs 392 18.8.3 Flattening a Table with a VIEW 393 18.9 Using Derived Tables 395 18.9.1 Derived Tables in the FROM clause 395 18.9.2 Derived Tables with a VALUES Constructor 397 18.10 Derived Tables in the WITH Clause 397 19 Partitioning Data in Queries 401 19.1 Coverings and Partitions 401 19.1.1 Partitioning by Ranges 402 19.1.2 Partition by Functions 403 19.1.3 Partition by Sequences 404 19.2 Relational Division 406 19.2.1 Division with a Remainder 408 19.2.2 Exact Division 409 CONTENTS xvii 19.2.3 Note on Performance 410 19.2.4 Todd’s Division 410 19.2.5 Division with JOINs 413 19.2.6 Division with Set Operators 413 19.3 Romley’s Division 414 19.4 Boolean Expressions in an RDBMS 418 19.5 FIFO and LIFO Subsets 420 20 Grouping Operations 425 20.1 GROUP BY Clause 425 20.1.1 NULLs and Groups 427 20.2 GROUP BY and HAVING 427 20.2.1 Group Characteristics and the HAVING Clause 429 20.3 Multiple Aggregation Levels 431 20.3.1 Grouped VIEWs for Multiple Aggregation Levels 432 20.3.2 Subquery Expressions for Multiple Aggregation Levels 433 20.3.3 CASE Expressions for Multiple Aggregation Levels 434 20.4 Grouping on Computed Columns 435 20.5 Grouping into Pairs 436 20.6 Sorting and GROUP BY 437 21 Aggregate Functions 439 21.1 COUNT() Functions 440 21.2 SUM() Functions 443 21.3 AVG() Functions 444 21.3.1 Averages with Empty Groups 446 21.3.2 Averages across Columns 448 21.4 Extrema Functions 449 21.4.1 Simple Extrema Functions 449 21.4.2 Generalized Extrema Functions 451 21.4.3 Multiple Criteria Extrema Functions 460 21.4.4 GREATEST() and LEAST() Functions 462 21.5 The LIST() Aggregate Function 465 21.5.1 The LIST() Function with a Procedure 466 xviii CONTENTS 21.5.2 The LIST() Function by Crosstabs 467 21.6 The PRD() Aggregate Function 468 21.6.1 PRD() Function by Expressions 469 21.6.2 The PRD() Aggregate Function by Logarithms 470 21.7 Bitwise Aggregate Functions 473 21.7.1 Bitwise OR Aggregate Function 474 21.7.2 Bitwise AND Aggregate Function 475 22 Auxiliary Tables 477 22.1 The Sequence Table 477 22.1.1 Enumerating a List 479 22.1.2 Mapping a Sequence into a Cycle 481 22.1.3 Replacing an Iterative Loop 483 22.2 Lookup Auxiliary Tables 485 22.2.1 Simple Translation Auxiliary Tables 487 22.2.2 Multiple Translation Auxiliary Tables 487 22.2.3 Multiple Parameter Auxiliary Tables 488 22.2.4 Range Auxiliary Tables 489 22.2.5 Hierarchical Auxiliary Tables 490 22.2.6 One True Lookup Table 491 22.3 Auxiliary Function Tables 493 22.3.1 Inverse Functions with Auxiliary Tables 495 22.3.2 Interpolation with Auxiliary Function Tables 504 22.4 Global Constants Tables 506 23 Statistics in SQL 509 23.1 The Mode 510 23.2 The AVG() Function 512 23.3 The Median 512 23.3.1 Date’s First Median 513 23.3.2 Celko’s First Median 514 23.3.3 Date’s Second Median 516 23.3.4 Murchison’s Median 516 23.3.5 Celko’s Second Median 517 23.3.6 Vaughan’s Median with VIEWs 519 CONTENTS xix 23.3.7 Median with Characteristic Function 520 23.3.8 Celko’s Third Median 522 23.3.9 Ken Henderson’s Median 526 23.4 Variance and Standard Deviation 527 23.5 Average Deviation 528 23.6 Cumulative Statistics 528 23.6.1 Running Totals 529 23.6.2 Running Differences 530 23.6.3 Cumulative Percentages 531 23.6.4 Rankings and Related Statistics 533 23.6.5 Quintiles and Related Statistics 537 23.7 Cross Tabulations 538 23.7.1 Crosstabs by Cross Join 542 23.7.2 Crosstabs by Outer Joins 543 23.7.3 Crosstabs by Subquery 544 23.7.4 Crosstabs by CASE Expression 545 23.8 Harmonic Mean and Geometric Mean 545 23.9 Multivariable Descriptive Statistics in SQL 546 23.9.1 Covariance 546 23.9.2 Pearson’s r 547 23.9.3 NULLs in Multivariable Descriptive Statistics 548 24 Regions, Runs, Gaps, Sequences, and Series 549 24.1 Finding Subregions of Size (n) 550 24.2 Numbering Regions 551 24.3 Finding Regions of Maximum Size 552 24.4 Bound Queries 557 24.5 Run and Sequence Queries 557 24.5.1 Filling in Sequence Numbers 560 24.6 Summation of a Series 562 24.7 Swapping and Sliding Values in a List 565 24.8 Condensing a List of Numbers 567 24.9 Folding a List of Numbers 567 24.10 Coverings 568 . Crosstabs by Outer Joins 543 23.7.3 Crosstabs by Subquery 544 23.7.4 Crosstabs by CASE Expression 545 23.8 Harmonic Mean and Geometric Mean 545 23.9 Multivariable Descriptive Statistics in SQL. Differences 530 23.6.3 Cumulative Percentages 531 23.6.4 Rankings and Related Statistics 533 23.6.5 Quintiles and Related Statistics 537 23.7 Cross Tabulations 538 23.7.1 Crosstabs by Cross Join. Words 117 4 Temporal Data Types in SQL 119 4.1 Notes on Calendar Standards 119 4.2 SQL Temporal Data Types 123 4.2.1 Tips for Handling Dates, Timestamps, and Times 124 4.2.2 Date Format Standards