110 Chapter 4 Dissecting WHERE Clauses Dissecting WHERE Clauses If your WHERE clause isn’t working, you can debug it by displaying the result of each condition individually. To see the result of each comparison in Listing 4.29, for example, put each com- parison expression in the SELECT clause’s output column list, along with the values you’re comparing: SELECT type, type = ‘history’ AS “Hist?”, type = ‘biography’ AS “Bio?”, price, price < 20 AS “<20?” FROM titles; This query runs on Microsoft Access, MySQL, and PostgreSQL. If your DBMS interprets the = symbol as an assignment operator rather than as a comparison operator, you must substitute equivalent expressions for the logical comparisons. In Oracle, for example, you can replace type = ’history’ with INSTR(type,’history’) . The query’s result is: type Hist? Bio? price <20? ————————— ————— ——— ————— ——— history 1 0 21.99 0 history 1 0 19.95 1 computer 0 0 39.95 0 psychology 0 0 12.99 1 psychology 0 0 6.95 1 biography 0 1 19.95 1 biography 0 1 23.95 0 children 0 0 10.00 1 children 0 0 13.95 1 biography 0 1 NULL NULL psychology 0 0 7.99 1 biography 0 1 12.99 1 history 1 0 29.99 0 The comparison columns display zero if the comparison is false, nonzero if it’s true, or null if it’s unknown. ✔ Tips ■ The examples in this section show the AND , OR , and NOT operators used with com- parison conditions, but these operators can be used with any type of condition. ■ If your search condition contains only AND operators, your query will run faster if you put the conditions least likely to be true first. If col1=’A’ is less likely than col2=’B’ then WHERE col1=’A’ AND col2=’B’ is faster than WHERE col2=’B’ AND col1=’A’ because the DBMS won’t bother to evalu- ate the second expression if the first is false. For search conditions that contain only OR operators, do the reverse: Put the most likely conditions first. If the condi- tions are equally likely, put the least com- plex expression first. This logic depends on your DBMS’s opti- mizer reading WHERE clauses from left to right, which most do. Oracle’s cost-based optimizer (as opposed to its rule-based optimizer), however, reads right to left. ■ It’s a common error to type WHERE state = ‘NY’ OR ‘CA’ Illegal instead of WHERE state = ‘NY’ OR state = ‘CA’ continues on next page 111 Retrieving Data from a Table Combining and Negating Conditions ■ It’s easy to translate a correctly phrased spoken-language statement into an incorrect SQL statement. If you say, “List the books priced less than $10 and more than $30,” the and suggests the use of the AND operator: SELECT title_name, price FROM titles WHERE price<10 AND price>30; Wrong This query returns no rows, however, because it’s impossible for a book to be priced less than $10 and more than $30 simultaneously, as AND logic commands. The logical meaning of OR finds books that meet any of the criteria, not all the criteria at the same time: WHERE price<10 OR price>30 Correct ■ Table 4.6 demonstrates alternative ways of expressing the same condition. The first equivalency is double negation, the second two are De Morgan’s Laws, and the final two are the distributive laws. ■ Some DBMSs support the exclusive-or (or xor) logical operator, which yields true only if exactly one of its operands is true. p XOR q is equivalent to (p AND (NOT q)) OR ((NOT p) AND q) . ■ In MySQL 4.0.4 and earlier, false AND unknown evaluates to unknown, not false. 112 Chapter 4 Combining and Negating Conditions Table 4.6 Equivalent Conditions This Condition Is Equivalent To NOT (NOT p) p NOT (p AND q) (NOT p) OR (NOT q) NOT (p OR q) (NOT p) AND (NOT q) p AND (q OR r)(p AND q) OR (p AND r) p OR (q AND r)(p OR q) AND (p OR r) 113 Retrieving Data from a Table Re-expressing Conditions Re-expressing Conditions You must master the laws in Table 4.6 to become a competent programmer in SQL (or any language). They’re especially useful when you want to re-express conditions to make queries run faster. For example, the statement SELECT * FROM mytable WHERE col1 = 1 AND NOT (col1 = col2 OR col3 = 3); is equivalent to SELECT * FROM mytable WHERE col1 = 1 AND col2 <> 1 AND col3 <> 3; but the latter one will run faster if your DBMS’s optimizer isn’t smart enough to re-express the former internally. (The condition col1 = col2 is more expensive computationally than comparing col1 and col2 to literal values.) You also can use the laws to change a condition into its opposite. For example, the reverse of the condition WHERE (col1=’A’) AND (col2=’B’) is WHERE (col1<>’A’) OR (col2<>’B’) In this case, it would have easier just to negate the entire original expression with NOT : WHERE NOT ((col1=’A’) AND (col2=’B’)) But this simple approach won’t work with complex conditions involving multiple AND s, OR s, and NOT s. Here’s a problem to solve: Look at only the first code line below and see whether you can repeatedly apply equivalency rules to push the NOT operators inward until they apply to only the individual expressions p , q , and r : NOT ((p AND q) OR (NOT p AND r)) = NOT (p AND q) AND NOT (NOT p AND r) = (NOT p OR NOT q) AND (p OR NOT r) Matching Patterns with LIKE The preceding examples retrieved rows based on the exact value of a column or columns. You can use LIKE to retrieve rows based on partial information. LIKE is useful if you don’t know an exact value (“The author’s last name is Kel-something”) or you want to retrieve rows with similar values (“Which authors live in the San Francisco Bay Area?”). The LIKE condition’s important characteristics are: ◆ LIKE works with only character strings, not numbers or datetimes. ◆ LIKE uses a pattern that values are matched against. A pattern is a quoted string that contains the literal characters to match and any combination of wildcards. Wildcards are special characters used to match parts of a value. Table 4.7 lists the wildcard operators, and Table 4.8 lists some example patterns. ◆ String comparisons are case insensitive or case sensitive, depending on your DBMS; see the DBMS Tip in “Filtering Rows with WHERE ” earlier in this chapter. ◆ You can negate a LIKE condition with NOT LIKE . ◆ You can combine LIKE conditions and other conditions with AND and OR . 114 Chapter 4 Matching Patterns with LIKE Table 4.7 Wildcard Operators Operator Matches % A percent sign matches any string of zero or more characters. _ An underscore matches any one character. Table 4.8 Examples of % and _ Patterns Pattern matches ‘A%’ Matches a string of length ≥ 1 that begins with A, including the single letter A. Matches ‘A’ , ‘Anonymous’ , and ‘AC/DC’ . ‘%s’ Matches a string of length ≥ 1 that ends with s, including the single letter s. A string with trailing spaces (after the s) won’t match. Matches ‘s’ , ‘Victoria Falls’ , and ‘DBMSs’ . ‘%in%’ Matches a string of length ≥ 2 that contains in anywhere. Matches ‘in’ , ‘inch’ , ‘Pine’ , ‘linchpin’ , and ‘lynchpin’ . ‘____’ Matches any four-character string. Matches ‘ABCD’ , ‘I am’ , and ‘Jack’ . ‘Qua__’ Matches any five-character string that begins with Qua. Matches ‘Quack’ , ‘Quaff’ , and ‘Quake’ . ‘_re_’ Matches any four-character string that has re as its second and third characters. Matches ‘Tree’ , ‘area’ , and ‘fret’ . ‘_re%’ Matches a string of length ≥ 3 that begins with any character and has re as its second and third characters. Matches ‘Tree’ , ‘area’ , ‘fret’ , ‘are’ , and ‘fretful’ . ‘%re_’ Matches a string of length ≥ 3 that has re as the second and third characters from its end and ends with any character. Matches ‘Tree’ , ‘area’ , ‘fret’ , ‘red’ , and ‘Blood red’ . To filter rows by matching a pattern: ◆ Type: SELECT columns FROM table WHERE test_column [NOT] LIKE ‘pattern’; columns is one or more comma-separated column names, and table is the name of the table that contains columns. In the search condition, test_column is the name of a column in table (test_col- umn doesn’t have to be listed in columns), and pattern is the pattern that’s compared with the value in test_column. pattern is a string like one of the examples listed in Table 4.8. Specify NOT LIKE to retrieve rows with values that don’t match pattern (Listings 4.30 through 4.33, Figures 4.30 through 4.33). continues on next page 115 Retrieving Data from a Table Matching Patterns with LIKE Listing 4.30 List the authors whose last names begin with Kel. See Figure 4.30 for the result. SELECT au_fname, au_lname FROM authors WHERE au_lname LIKE 'Kel%'; Listing au_fname au_lname Christian Kells Kellsey Figure 4.30 Result of Listing 4.30. Listing 4.31 List the authors whose last names have ll (el-el) as the third and fourth characters. See Figure 4.31 for the result. SELECT au_fname, au_lname FROM authors WHERE au_lname LIKE '__ll%'; Listing au_fname au_lname Hallie Hull Klee Hull Christian Kells Kellsey Figure 4.31 Result of Listing 4.31. Listing 4.32 List the authors who live in the San Francisco Bay Area. (Zip codes in that area begin with 94.) See Figure 4.32 for the result. SELECT au_fname, au_lname, city, state, zip FROM authors WHERE zip LIKE '94___'; Listing au_fname au_lname city state zip Hallie Hull San Francisco CA 94123 Klee Hull San Francisco CA 94123 Kellsey Palo Alto CA 94305 Figure 4.32 Result of Listing 4.32. Listing 4.33 List the authors who live outside the 212, 415, and 303 area codes. This example shows three alternative patterns for excluding telephone numbers. You should favor the first alternative because single- character matches ( _ ) are faster than multiple- character ones ( % ). See Figure 4.33 for the result. SELECT au_fname, au_lname, phone FROM authors WHERE phone NOT LIKE '212-___-____' AND phone NOT LIKE '415-___-%' AND phone NOT LIKE '303-%'; Listing au_fname au_lname phone Sarah Buchman 718-496-7223 Kellsey 650-836-7128 Paddy O'Furniture 941-925-0752 Figure 4.33 Result of Listing 4.33. You can search for values that contain the special wildcard characters. Use the ESCAPE keyword to specify an escape character that you can use to search for a percent sign or underscore as a literal character. Immediately precede a wildcard character with an escape character to strip the wildcard of its special meaning. If the escape character is ! , for exam- ple, !% in a pattern searches values for a literal % . (Unescaped wildcards still have their special meaning.) The escape character can’t be part of the value that you’re trying to retrieve; if you’re searching for ‘50% OFF!’ , choose an escape character other than ! . Table 4.9 shows some examples of escaped and unescaped patterns; the designated escape character is ! . To match a wildcard character: ◆ Type: SELECT columns FROM table WHERE test_column [NOT] LIKE ‘ pattern ’ ESCAPE ‘ escape_char ’ ; The syntax is the same as the SELECT statement in “To filter rows by matching a pattern,” earlier in this chapter, except for the ESCAPE clause. escape_char is a single character. Any character in pattern that follows escape_char is interpreted literally; escape_char itself is not consid- ered to be part of the search pattern (Listing 4.34 and Figure 4.34). ✔ Tips ■ test_column can be an expression. ■ The NOT that can precede LIKE is inde- pendent of the NOT that can precede test_column (see “The NOT operator” earlier in this chapter). This clause WHERE phone NOT LIKE ‘212-%’ is equivalent to this one: WHERE NOT phone LIKE ‘212-%’ 116 Chapter 4 Matching Patterns with LIKE Table 4.9 Escaped and Unescaped Patterns Pattern Matches ‘ 100% ’ Unescaped. Matches 100 followed by a string of zero or more characters. ‘ 100!% ’ Escaped. Matches ‘ 100% ’ . ‘ _op ’ Unescaped. Matches ‘ top ’ , ‘ hop ’ , ‘ pop ’ , and so on. ‘ !_op ’ Escaped. Matches ‘_op’ . Listing 4.34 List the titles that contain percent signs. Only the % that follows the escape character ! has its literal meaning; the other two percent signs still act as wildcards. See Figure 4.34 for the result. SELECT title_name FROM titles WHERE title_name LIKE '%!%%' ESCAPE '!'; Listing title_name Figure 4.34 Result of Listing 4.34. An empty result. No title names contain a % character. You even can write this silly double nega- tion, which retrieves everyone with a 212 area code: WHERE NOT phone NOT LIKE ‘212-%’ ■ Wildcard searches are time-consuming— particularly if you use % at the start of a pattern. Don’t use wildcards if another type of search will do. ■ In the simplest case in which a pattern contains no wildcards, LIKE works like an = comparison (and NOT LIKE works like <> ). In many cases WHERE city LIKE ‘New York’ is equivalent to WHERE city = ‘New York’ But these comparisons will differ if your DBMS takes trailing spaces into account for LIKE but not for = . If that’s not impor- tant, the = form usually is faster than LIKE . ■ Microsoft Access doesn’t sup- port the ESCAPE clause. Instead, surround a wildcard character with brackets to render it a literal character. To run Listing 4.34, replace the WHERE clause with: WHERE title_name LIKE ‘ %[%]% ’ Some DBMSs let you use regular expres- sions to match patterns. Microsoft SQL Server, for example, supports a limited variant of POSIX-style regular expressions. The [] wildcard matches any single char- acter within a range or set, and the [^] wildcard matches any single character not within a range or set. Table 4.10 lists some examples. The SQL standard uses the SIMILAR operator for regex matching. Regex support varies by DBMS; search your DBMS documentation for LIKE, regular expressions, or pattern matching. Some DBMSs let you use LIKE to search numeric and datetime columns. 117 Retrieving Data from a Table Matching Patterns with LIKE Table 4.10 Examples of [] and [^] Patterns Pattern Matches ‘ [a-c]at ’ Matches ‘ bat ’ and ‘ cat ’ but not ‘ fat ’ . ‘ [bcf]at ’ Matches ‘ bat ’ , ‘ cat ’ , and ‘ fat ’ but not ‘ eat ’ . ‘ [^c]at ’ Matches ‘ bat ’ and ‘ fat ’ but not ‘ cat ’ . ‘ se[^n]% ’ Matches strings of length ≥2 that begin with se and whose third character isn’t n. Range Filtering with BETWEEN Use BETWEEN to determine whether a given value falls within a specified range. The BETWEEN condition’s important charac- teristics are: ◆ BETWEEN works with character strings, numbers, and datetimes. ◆ The BETWEEN range contains a low value and a high value, separated by AND . The low value must be less than or equal to the high value. ◆ BETWEEN is a convenient, shorthand clause that you can replicate by using AND . WHERE test_column BETWEEN low_value AND high_value is equivalent to: WHERE (test_column >= low_value) AND (test_column <= high_value) ◆ BETWEEN specifies an inclusive range, in which the high value and low value are included in the search. To specify an exclusive range, which excludes end- points, use > and < comparisons instead of BETWEEN : WHERE (test_column > low_value) AND (test_column < high_value) ◆ String comparisons are case insensitive or case sensitive, depending on your DBMS; see the DBMS Tip in “Filtering Rows with WHERE ” earlier in this chapter. ◆ You can negate a BETWEEN condition with NOT BETWEEN . ◆ You can combine BETWEEN conditions and other conditions with AND and OR . 118 Chapter 4 Range Filtering with BETWEEN Listing 4.35 List the authors who live outside the zip range 20000–89999. See Figure 4.35 for the result. SELECT au_fname, au_lname, zip FROM authors WHERE zip NOT BETWEEN '20000' AND '89999'; Listing au_fname au_lname zip Sarah Buchman 10468 Hallie Hull 94123 Klee Hull 94123 Christian Kells 10014 Kellsey 94305 Figure 4.35 Result of Listing 4.35. Listing 4.36 List the titles priced between $10 and $19.95, inclusive. See Figure 4.36 for the result. SELECT title_id, price FROM titles WHERE price BETWEEN 10 AND 19.95; Listing title_id price T02 19.95 T04 12.99 T06 19.95 T08 10.00 T09 13.95 T12 12.99 Figure 4.36 Result of Listing 4.36. To filter rows by using a range: ◆ Type: SELECT columns FROM table WHERE test_column [NOT] BETWEEN low_value AND high_value; columns is one or more comma-separated column names, and table is the name of the table that contains columns. In the search condition, test_column is the name of a column in table (test_column doesn’t have to be listed in columns), and low_value and high_value specify the endpoints of the range that is compared with the value in test_column. low_value must be less than or equal to high_value, and both values must be the same as or comparable to the data type of test_column. Specify NOT BETWEEN to match values that lie outside the range (Listings 4.35 through 4.37, Figures 4.35 through 4.37). 119 Retrieving Data from a Table Range Filtering with BETWEEN Listing 4.37 List the titles published in 2000. See Figure 4.37 for the result. SELECT title_id, pubdate FROM titles WHERE pubdate BETWEEN DATE '2000-01-01' AND DATE '2000-12-31'; Listing title_id pubdate T01 2000-08-01 T03 2000-09-01 T06 2000-07-31 T11 2000-11-30 T12 2000-08-31 Figure 4.37 Result of Listing 4.37. . “Bio?”, price, price < 20 AS “<20?” FROM titles; This query runs on Microsoft Access, MySQL, and PostgreSQL. If your DBMS interprets the = symbol as an assignment operator rather than as a comparison. Conditions ■ It’s easy to translate a correctly phrased spoken-language statement into an incorrect SQL statement. If you say, “List the books priced less than $10 and more than $30,” the and suggests. of its operands is true. p XOR q is equivalent to (p AND (NOT q)) OR ((NOT p) AND q) . ■ In MySQL 4.0.4 and earlier, false AND unknown evaluates to unknown, not false. 112 Chapter 4 Combining