Data Mining and Knowledge Discovery Handbook, 2 Edition part 57 potx

540 Yoav Benjamini and Moshe Leshno Storey J.D., Taylor J.E. and Siegmund D., (2004). Strong control, conservative point esti- mation, and simultaneous conservative consistency of false discovery rates: A unified approach. Journal of the Royal Statistical Society Series B, 66:187–205. Therneau T.M. and Grambsch P.M., (2000). Modeling Survival Data, Extending the Cox Model. Springer. Tibshirani R. and Knight K., (1999). The covariance inflation criterion for adaptive model selection. Journal of the Royal Statistical Society Series B, 61:Part 3 529–546. Zembowicz R. and Zytkov J.M., (1996). From contingency tables to various froms of knowl- edge in databases. In U.M. Fayyad, R. Uthurusamy, G. Piatetsky-Shapiro and P. Smyth (editors) Advances in Knowledge Discovery and Data Mining (pp. 329-349). MIT Press. Zytkov J.M. and Zembowicz R., (1997). Contingency tables as the foundation for concepts, concept hierarchies and rules: The 49er system approach. Fundamenta Informaticae, 30:383–399. 26 Logics for Data Mining Petr H ´ ajek Institute of Computer Science Academy of Sciences of the Czech Republic 182 07 Prague, Czech Republic hajek@cs.cas.cz Summary. Systems of formal (symbolic) logic suitable for Data Mining are presented, main stress being put to various kinds of generalized quantifiers. Key words: logic, Data Mining, generalized quantifiers, GUHA method Introduction Data Mining, as presently understood, is a broad term, including search for “association rules”, classification, regression, clustering and similar. Here we shall restrict ourselves to search for “rules” in a rather general sense, namely general dependencies valid in given data and expressed by formulas of a formal logical language. The present theoretical approach is the result of a long development of the GUHA method of automated generation of hypotheses (General Unary Hypotheses Automaton, see a paragraph in Section 26.2) but is believed to be fully relevant for contemporary mining of association rules and its possible generalization. See (Agrawal et al., 1996,Hoppner, 2005, Adamo, 2001) for association rules. Data are assumed to have the form of one or more tables, matrices or relations. A rectan- gular matrix may be understood as giving data on objects (corresponding to rows of the matrix) and their attributes (columns). Or rows may correspond to objects from one set, columns to objects of the same or a different set and the whole matrix is understood as one binary at- tribute. In the former case we have variables x,y for objects and a unary predicate for each column (P i for i-th column, say); P i (x) denotes the value of P i for the object x. In the latter case we have variables for objects from the first set (x, say), other variables for objects from the other set (y, say) and one binary predicate P; then P(x,y) denotes the value of the attribute for the pair (x,y) of objects. For example, rows correspond to patients, the first column corresponds to having feaver (yes – value 1, no – value 0). Then patient Nov ´ ak satisfies P 1 (x) if he has feaver. Secondly, the matrix describes the relation “being a married couple” and we take MC for the predicate. Then the couple (Nov ´ ak, Nov ´ akov ´ a) satisfies P(x,y) if they are a married couple, thus the corresponding field in the matrix has value 1. These were examples of Boolean (0-1-valued) data; more gener- ally, they may take values from some set of values (reals, colours, ). O. Maimon, L. Rokach (eds.), Data Mining and Knowledge Discovery Handbook, 2nd ed., DOI 10.1007/978-0-387-09823-4_26, © Springer Science+Business Media, LLC 2010 542 Petr H ´ ajek It should be clear what is the value of P(x) for an object in the former case and value of P(x,y) for a pair of objects in the latter. Logic enables us to construct composed formulas from atomic formulas as above using some connectives (as conjunction, disjunction, implication, negation in the Boolean logic, no- tation: ∧, ∨, →,¬) and quantifiers (universal ∀ and existential ∃ in classical Boolean logic). Our main message is that in Data Mining we have to deal with generalized quantifiers of some particular kind and that their logical properties are very important. For simplicity we re- strict ourselves to two-valued (0-1-valued) data. Our approach generalizes easily to categorical (finitely valued) data when we work with atomic formulas of the form (X)P(x) (or (X)P(x,y) etc.) where X is a subset of the domain of values of the attribute P and an object o satisfies (X)P(x) iff the P-value of o is in X, similarly for (X)P(x,y). (For example let P denote age in years 0, ,100, let X be the set of numbers 0 ≤n ≤ 30.) The reader having some knowledge of classical propositional and predicate calculus will have no problems with the mentioned notions; the reader having difficulties is recommended to have a look to a textbook of mathematical logic, e. g. (Ebbinghaus et al., 1984). 26.1 Generalized quantifiers We shall present the notion of a generalized quantifier and supply several examples. (In the next section we shall study various classes of quantifiers.) For simplicity we shall work with data having the form of a rectangular boolean matrix, rows corresponding to object and columns to (yes-no) attributes. Predicates P 1 , ,P n are names of the attributes; we have an object variable x, and P 1 (x), ,P n (x) are atomic formulas. For each formula ϕ , we have its negation ¬ ϕ ; an object satisfies ¬ ϕ if it does not satisfy ϕ . For each pair ϕ , ψ of formulas we have their conjunction ϕ ∧ ψ and disjunction ϕ ∨ ψ . An object satisfies ϕ ∧ ψ if it satisfies both ϕ and ψ ; it satisfies at least one of ϕ , ψ . Similarly for conjunction/disjunction of three, four. formulas. An object satisfies implication ϕ → ψ if it satisfies ψ or does not satisfy ϕ . Formulas built from atomic formulas using the connectives ¬,∧∨,→ are open: each open formula ϕ defines an attribute: each object of our data either satisfies ϕ or does not satisfy it. This is uniquely determined by the data. Thus ϕ defines two numbers: r – the number of objects satisfying ϕ and s – the number of objects satisfying ¬ ϕ . The pair (r,s) may be called the two-fold table of ϕ (given by the data); r+s = m is the number of objects in the data (rows of the matrix). A (one-dimensional) quantifier Q applied to an open formula ϕ describes the behaviour of the attribute defined by ϕ in the data as a whole, i.e. gives a global characterization of the attribute (in the data). The reader surely knows the classical quantifiers ∀ (universal) and ∃ (existential). The formula (∀x) ϕ is true in the data if each object satisfies ϕ (thus s = 0); the formula (∃x) ϕ is true in the data of at least one object satisfies ϕ (thus r ≥ 1). You see that truth of such a quantified formula does not depend on any ordering of the rows of the data matrix, just it is given by the two-fold table of ϕ . A (one-dimensional) quantifier Q is determined by its truth function Tr Q assigning to each two-fold table (r,s) either 1 (true) or 0 (false). For each open formula ϕ , the closed formula (Qx) ϕ is true in the data iff the two-fold table (r,s) of ϕ (given by the data) satisfies Tr Q (r, s)=1. Clearly, Tr ∀ (r, s)=1iffs = 0 and Tr ∃ (r, s)=1iffr > 0. Some other examples: Majority: Tr Maj (r, s)=1iffr > s; (Maj x) ϕ says that the majority of objects satisfy ϕ . Many: Let 0 < p < 1; Tr Many p (r, s)=1iffr/(r + s) ≥ p; (Many p x) ϕ says that the relative 26 Logics for Data Mining 543 frequence of objects satisfying ϕ is at least p. At least: Tr ∃≥n (r, s)=1iffr ≥ n (at least n objects satisfy ϕ ). Odd: Tr Odd (r, s)=1iffr is an odd number. The reader may produce many more examples. We shall be more general: A two-dimensional quantifier Q, applied to a pair ϕ , ψ of open formulas describes the behaviour of the pair of attributes defined by ϕ and ψ in the data as a whole, thus gives a global characterization of the mutual relation of ϕ , ψ (in the data). The closed formula given by Q, ϕ , ψ is written as (Qx)( ϕ , ψ ). Its truth/falsity in the data is determined by the four-fold table (a,b, c, d) where a, b, c,d denotes the number of objects in the data satisfying ϕ ∧ ψ , ϕ ∧¬ ψ , ¬ ϕ ∧ ψ , ¬ ϕ ∧¬ ψ respectively. This is often displayed as ψ ¬ ψ ϕ abr ¬ ϕ cds klm where r = a + b (number of objects satisfying ϕ ), s = c + d, k = a + c, l = b + d (marginal sums), m = a+b+c+d = r+s = k+l. Thus the truth function of a two-dimensional quantifier Q assigns to each four-fold table (a,b,c,d) the value Tr Q (a,b,c,d) ∈{0,1}. All (two-dimensional) ϕ (x) ⇒ ψ says “all ϕ ’s are ψ ’s)”; Tr ⇒ (a,b,c,d)=1iffb = 0. This is definable by one-dimensional ∀ and the connective →, namely ϕ (x) ⇒ ψ says the same as (∀x)( ϕ → ψ ). Many: for 0 < p ≤ 1, ϕ (x) ⇒ p ψ says “p-many ϕ ’s are ψ ’s”, i.e. the relative frequence of objects satisfying ψ among those satisfying ϕ is ≥ p, thus Tr ⇒ p (a,b,c,d)=1iffa/(a + b) ≥ p. Caution: this is not the same as (Many p x)( ϕ → ψ ) : for example if (a, b,c,d) is (2,2,5,5) then a/(a + b)=1/2 but the number of objects satisfying ϕ → ψ is a + c + d = 12, thus (a + c + d)/m = 12/14 = 6/7. Thus if p = 0.8 then ϕ (x) ⇒ ψ is false but (∀x)( ϕ → ψ ) is true. But ϕ (x) ⇒ ψ can be also written as (Many p x) ϕ / ψ , understood as saying that the formula (Many p x) ϕ is true in the subtable consisting of rows satisfying ψ . p-equivalence: ϕ (x) ⇔ ψ ( ϕ is p-equivalent to ψ ) is true in the data if both ϕ (x) ⇒ p ψ and ¬ ϕ (x) ⇒ p ¬ ψ is true, thus a/(a +b) ≥ p and d/(c+ d) ≥ p. Foundedness: Let t be a natural number. (Fdd x)( ϕ , ψ ) says that at least t objects satisfy ϕ ∧ ψ , i.e. a ≥t. Similarly: Support: Let 0 < σ < 1. (Supp σ x)( ϕ , ψ ) is Many σ ( ϕ ∧ ψ ), thus says that the relative frequence of ϕ ∧ ψ in the data is at least σ . Founded implication: (FIMPL p,t x)( ϕ , ψ ) (or just ϕ ⇒ p,t ψ ) is (Many p x)( ϕ , ψ ) and Fdd t s)( ϕ ∧ ψ ), hence Tr ⇒ p,t (a,b,c,d)=1iff a/(a + b) ≥ p and a ≥t. Agrawal: ϕ ⇒ Agr p, σ ψ is Many p ( ϕ , ψ ) and Supp σ ( ϕ , ψ ), hence Tr ⇒ Agr p, σ (a,b,c,d)=1iffa/(a + b) ≥ p and a/(a,b, c, d) ≥ σ . Clearly the last two quantifiers differ only very little: ϕ ⇒ ∗ p,t ψ is equivalent to ϕ ⇒ Agr p, σ for σ = t/(a + b + c + d). Now ⇒ Agr is the quantifier of the association rules of Agrawal; it is little known and has to be stressed that the “almost the same” quantifier of founded implication 544 Petr H ´ ajek was used in GUHA to generate “association rules” in the presently common sense as soon as in mid-sixties of the past century (H ´ ajek et al., 1966). The reader may play by defining more and more two-dimensional quantifiers; clearly not all of them are relevant for Data Mining. We close this section by two important remarks. Closed formulas. Each formula (of the present formalism, with unary predicates and just one object variable) beginning with a quantifier is closed, i.e. does not refer to any particular object but expresses some global pattern found in the data. Further closed formulas result from those beginning by a quantifier using connectives (e.g. we have seen that ϕ (x) ⇔ p ψ is equivalent to ( ϕ (x) ⇒ p ψ ) ∧(¬ ϕ (x) ⇒ p ¬ ψ ), etc.). A closed formula is a tautology (logical truth) if it is true in each data. To give a trivial example, observe that if p 1 ≤ p 2 then the formula ( ϕ (x) ⇒ p 2 ψ ) → ( ϕ (x) ⇒ p 1 ψ ) is a tautology. Predicates of higher arity. If our data contain information on relations of higher arity (bi- nary, ternary,. . . ) we have to use predicates of higher arity and more than one object variable. A quantifier always binds (quantifies) a variable. This leads to the logical notion of free and bound variables of a formula, free variables varying over arbitrary objects of the data. For ex- ample, take a binary predicate P; P(x, y) is a formula in which x,y are free, (Many p y)P(x,y) is a formula in which only x is free. An object o satisfies the last formula iff it is P-related with p- many objects. (Let P(x,y) say “x knows y”, let p be 0.8. An object o satisfies (Many 0.8 y)P(x,y) if he knows at least 80% objects (from the data). We may form composed formulas using sev- eral quantifiers binding different variables, e.g. (∀x)((Many 0.8 y)P(x,y) →R(x)) (saying “each object knowing at least 80% objects has the property R”) etc. This sort of formulas is used in relational Data Mining (D ˇ zeroski and Lavra ˇ c, 2001). We shall not go into details. 26.2 Some important classes of quantifiers 26.2.1 One-dimensional Let us call a one-dimensional quantifier Q multitudinal if its truth function Tr Q is not decreas- ing in its first argument and non-increasing in the second, i.e. for any two-fold tables (r 1 ,s 1 ), (r 2 ,s 2 ) whenever r 1 ≤ r 2 , s 1 ≥ s 2 and Tr Q (r, s 1 )=1 then Tr Q (r 1 ,s 2 )=1. This means that the formula (Qx) ϕ says, in some sense given by Tr Q , that sufficiently many objects satisfy ϕ . “Sufficiently many” may mean “all” (∀), at least one (∃), at least 7 (∃ 7 ), at least 100p% (Many p ) etc. Very important: The quantifier may correspond to a statistical test of high probabil- ity. Telegraphically: Our hypothesis is that the probability of the attribute ϕ is bigger than p (under frame assumptions saying that all objects have the same probability of having ϕ and are mutually independent). Take a small α (e.g. 0.05 – significance level). The number ∑ r+s i=r ( r+s i )p i (1 − p) r+s−i is the probability that at least r objects (form our r +s objects) will have ϕ , assuming that the probability of ϕ is p. If this sum is ≤ α then we can reject the (null) hypothesis saying that the probability of ϕ is p or less (since if it were then what we have observed would be improbable). This is the (simplified) idea of statistical hypothesis testing. We get a quantifier of testing high probability, HProb p, α . Tr HProb p, α (r, s)=1iff r+s ∑ i=r  r +s i  p i (1 − p) r+s−i ≤ α . 26 Logics for Data Mining 545 This is an example of a statistically motivated one-dimensional quantifier; it can be proved to be multitudinal). See Chapter 31.4.6 for statistical hypothesis testing and (H ´ ajek and Havr ´ anek, 1978) for its logical foundations. 26.2.2 Two-dimensional Recall that a two-dimensional quantifier is given by its truth function assigning to each four- fold table (a,b,c,d) a truth value (1 or 0). In Data Mining we are especially interested in two-dimensional quantifiers expressing in some sense a kind of association of two attributes (described by two open formulas). In some sense, the formula (Qx)( ϕ , ψ ) should say that there are sufficiently many coincidences in (the truth values of) ϕ , ψ and not too many differences. This leads to the following definition (H ´ ajek and Havr ´ anek, 1978): A two-dimensional quantifier is associational if it satisfies the fol- lowing for each pair (a 1 ,b 1 ,c 1 ,d 1 ), (a 2 ,b 2 ,c 2 ,d 2 ) of four-fold tables: a 2 ≥ a 1 ,b 2 ≤ b 1 ,c 2 ≤ c 1 ,d 2 ≥ d 1 and Tr Q (a 1 ,b 1 ,c 1 ,d 1 )=1 implies Tr Q (a 2 ,b 2 ,c 2 ,d 2 )=1. In other words: if ψ 1 ¬ ψ 1 ϕ 1 a 1 b 1 ¬ ϕ 1 c 1 d 1 ψ 2 ¬ ψ 2 ϕ 2 a 2 b 2 ¬ ϕ 2 c 2 d 2 are four-fold tables of the pairs ( ϕ 1 , ψ 1 ), ( ϕ 2 , ψ 2 ) of open formulas in given data, if (Qx)( ϕ 1 , ψ 1 ) is true in the data and the above inequalities hold (a 2 ≥a 1 ,b 2 ≤b 1 ,c 2 ≤c 1 ,d 2 ≥ d 1 ), then (Qx)( ϕ 2 , ψ 2 ) is also true. The second table has more coincidences (a 2 ≥a 1 ,d 2 ≥d 1 ) and less differences (b 2 ≤ b 1 ,c 2 ≤ c 1 ). A quantifier Q is locally associational if the above condition holds for all (a 1 ,b 1 ,c 1 ,d 1 ), (a 2 ,b 2 ,c 2 ,d 2 ) satisfying the additional assumption a 1 + b 1 + c 1 + d 1 = a 2 + b 2 + c 2 + d 2 (i.e. the tables correspond to two data matrices of the same cardinality; in particular, think of ϕ 1 , ψ 1 , ϕ 2 , ψ 2 evaluated in the same data matrix). We shall deal with (locally) associational quantifiers of two important kinds: implicational and comparative. We give examples and state general (deductive) properties of quantifiers in these classes. Implicational quantifiers formalize the association formulated as “many ϕ ’s are ψ ’s”. (They could be also called two-dimensional multitudinal quantifiers.) The definition reads as follows: A two-dimensional quantifier Q is implicational if each pair (a 1 ,b 1 , c 1 ,d 1 ), (a 2 ,b 2 ,c 2 ,d 2 ) of four fold tables satisfies the following condition: If a 2 ≥ a 1 ,b 2 ≤ b 1 and Tr Q (a 1 ,b 1 ,c 1 ,d 1 )=1 then Tr Q (a 2 ,b 2 ,c 2 ,d 2 )=1 Q is locally implicational if this condi- tion is satisfied for each pair of four-fold tables with the same sum (a 1 + b 1 + c 1 + d 1 = c 2 + b 2 + c 2 + d 2 ). Clearly, the quantifier ⇒ p (p-many) is implicational: a 2 ≥a 1 and b 2 ≤b 1 imply a 2 /(a 2 + b 2 ) ≥a 1 /(a 1 +b 1 ). The quantifier ⇒ ∗ p,t of founded implication is also implicational: if a 2 ≥a 1 and a 1 ≥ t then trivially a 2 ≥ t. The “almost the same” Agrawal’s quantifier ⇒ Agr p, σ is locally implicational: if the tables have equal sum and a 2 ≥a 1 then trivially a 2 /(a 2 +b 2 +c 2 +d 2 ) ≥ a 1 /(a 1 + b 1 + c 1 + d 1 ). 546 Petr H ´ ajek Note the statistical parallel of ⇒ p : The hypothesis of P( ψ | ϕ ) ≥ p (conditional probability of ψ , given ϕ ) is tested using the statistic ∑ a+b i=a ( a+b i ) p i (1 − p) a+b−i . The corresponding quantifier ⇒ ! p, α of likely p-implication (with significance level α ) is Tr ⇒ ! p, α (a,b,c,d)=1iff a+b ∑ i=a  a + b i  p i (1 − p) a+b−i ≤ α . This is also an implicational quantifier (see (H ´ ajek and Havr ´ anek, 1978), where also another statistically motivated implicational quantifier is discussed). For each (locally) implicational quantifier (denote it ⇒ # ) the following two deduction rules are sound (in the sense that when- ever the assumption is true in your data, the consequence is also true): ( ϕ 1 ∧ ϕ 2 ) ⇒ # ψ ϕ 1 ⇒ # ψ ∨¬ ϕ 2 , ϕ ⇒ # ψ 1 ϕ ⇒ # ( ψ 1 ∨ ψ 2 ) For example, if the formula “p-many probands being smokers and older then 50 have cancer” is true in your data then the following is true too: “p-many probands being smokers have cancer or are not older than 50”. Second: If the “association rule” “x buys Lidov ´ e noviny ⇒x is Czech” is 90%-true with support 1000 then also x buys Lidov ´ e noviny ⇒(x is Czech or x is Slovak) is 90% true with the same support. These deduction rules are extremely useful for optimizing search for formulas (“rules”) of the form ϕ ⇒ # ψ where ⇒ # is an implicational quantifier, ϕ is an elementary conjunction (conjunction of atomic open formulas and negated atomic formulas containing each predicate at most once, e.g. P 1 (x) ∧¬P 3 (x) ∧P 7 (x)) and ψ is an elementary disjunction (similar definition, e.g. ¬P 2 (x) ∨¬P 10 (x)). Caution: For the “classical” quantifier ⇒ 1 ( ϕ ⇒ 1 ψ saying “all ϕ ’s are ψ ’s”) the first rule can be converted, thus truth of ϕ 1 ⇒ 1 ψ ∨¬ ϕ 2 implies truth of ( ϕ 1 ∧ ϕ 2 ) ⇒ 1 ψ . But this is not true for ⇒ p and other mentioned implicational (locally implicational) quantifiers. Let us stress once more that implicational quantifiers formalize, in various possible ways, what we mean saying “many ϕ ’s are ψ ’s”. Agrawal’s association rules are a particular case, with one particular implicational quantifier and also with specific open formulas (no negation allowed, just conjunction of atoms). Even if this may be the most used case, the reader is invited to consider broader, more general and more powerful possibilities. We now turn our attention to a very important class of quantifiers that we shall call com- parative. The intuitive meaning of association expressed by a comparative quantifier is that the formula (Qx)( ϕ , ψ ) should say that presence of ϕ positively contributes to the presence of ψ . This does not mean that many ϕ ’s are ψ ’s, thus that the relative frequence of ψ among ϕ (denoted Freq( ψ | ϕ )) is big but that Freq( ψ | ϕ ) is (sufficiently) bigger than Freq( ψ |¬ ϕ ). For example, imagine that 30% of smokers have an illness and only 5% of non-smokers have the same illness. The simplest quantifier of this kind is called the simple associational quantifier, denoted SIMPLE or ∼ 0 (see (H ´ ajek and Havr ´ anek, 1978, H ´ ajek et al., 1995) or other GUHA papers); the truth function is Tr ∼ 0 (a,b,c,d)=1ifad > bc. A trivial compu- tation shows that ad > bc is equivalent both to a a+b > a+c a+b+c+d (if (a,b, c, d) is the four- fold table of ϕ , ψ then this says that, in the data, Freq( ψ | ϕ ) > Freq( ψ )) and to a a+b > c c+d (Freq( ψ | ϕ ) > Freq( ψ |¬ ϕ )). You may make this quantifier parametric, demanding ad > h.bc, for some h ≥1. Thus let us accept the following definition: A two-dimensional quantifier Q is comparative if Tr Q (a,b,c,d)=1 implies ad > bc. The statistical counterpart is Fisher quantifier ∼ F α based 26 Logics for Data Mining 547 on the test of the hypothesis P( ψ | ϕ ) > P( ψ ) (against the null hypothesis P( ψ | ϕ ) ≤P( ψ )), with significance α . The formula is: Tr ∼ α (a,b,c,d)=1ifad > bc and min(a+b,a+c) ∑ i=a  a + b i  b + d a + b −i  a + b + c +d a + b  ≤ α . If we adopt the usual notation a + b = r, a+ c = k, b + d = l, a + b + c + d = m then the last formula becomes min(r,k) ∑ i=a  k i  l r −i  /  m r  ≤ α . This is a rather complicated formula; there are non-trivial algorithms for computing the sum in question. Fisher quantifier can be proved to be associational (H ´ ajek and Havr ´ anek, 1978). Let us mention that another comparative associational statistically motivated quantifier is based on the statistical chi-square test. Indeed, Tr ∼ CHISQ α (a,b,c,d)=1ifad > bc and m(ad −bc) 2 rskl ≥ χ 2 α , where χ 2 α is a constant (the (1- α )-quantile of the χ 2 distribution function) Now let us present three deduction rules and ask if our quantifiers obey them. Once more, it means that whenever the assumption (above the line) is true in the data then the conclusion (below the line) is true. Here ∼stands for a quantifier; we write ϕ ∼ ψ instead of (∼x)( ϕ , ψ ). Rule of symmetry: (SYM) ϕ ∼ ψ ψ ∼ ϕ Rule of negation: (NEG) ϕ ∼ ψ ¬ ϕ ∼¬ ψ Rule of conversion: (CNVS) ϕ ∼ ψ ¬ ψ ∼¬ ϕ Fact The simple quantifier ∼ 0 , the Fisher quantifier ∼ F α as well as the chi-square quantifier ∼ CHI α obey all the rules (SYM), (NEG), (CNVS). For a proof see again (H ´ ajek and Havr ´ anek, 1978), observing that if any quantifier obeys (SYM) and (NEG) then it automatically obeys (CNVS). Now we present three more quanti- fiers occuring in the literature, each obeying just one of our present rules. The quantifier of pure p-equivalence ≡ p (Rauch, see e.g. (Rauch, 1998A)). The formula ϕ ≡ p ψ is true if both ϕ ⇒ p ψ and ¬ ϕ ⇒ p ¬ ψ are true, thus Tr ≡ p (a,b,c,d)=1ifa/(a +b) ≥ p and d/(c + d) ≥ p. For p > 1 2 this quantifier is comparative. (Indeed, if a/(a + b) > 1 2 and d/(c +d) > 1 2 then c/(c + d) < 1 2 < a/(a + b), which gives bc < ad.) 548 Petr H ´ ajek The quantifier of conviction (Adamo, 2001). ϕ ∼ conv h ψ is true if (a + b)(b + d) > h.b(a + b + c + d), or equivalently, (rl)/(bm) > h, where h is a parameter, h ≥1. An elementary computation gives that the last inequality for h = 1 (and hence for each h ≥1) implies ad < bc; the quantifier is comparative. The quantifier “above average” is a variant of SIMPLE (used in the program 4FT-miner (lispminer)). ϕ ∼ AA h ψ is true if a/(a + b) > h.(a + c)/(a + b + c + d) (thus a/r > h.k/m), which means that is Fr( ψ / ϕ ) > h.Fr( ψ ). For h = 1 this is equivalent to the simple quantifier with h = 1; evidently, for each h ≥ 1, the AA quantifier is comparative. But these last three quantifiers differ as far as our deduction rules are concerned: Fact (1) The quantifier AA obeys symmetry but for h > 1 neither negation nor conversion. (2) The quantifier of pure p-equivalence obeys negation but for p < 1 neither symmetry nor conver- sion. (3) The quantifier of conviction obeys conversion but for h > 1 neither symmetry nor negation. The positive claims are verified by easy computations; the negative claims can be all witnessed e.g. by the table (9,1,10, 80). Let us also mention the quantifier of double p-implication ⇔ p (Rauch): ϕ ⇔ p ψ is true if both ϕ ⇒ p ψ and ψ ⇒ p ϕ is true. Show that this quantifier is not comparative (consider e.g. (9,1,1,0)); it obeys symmetry but (for p < 1) neither negation nor conversion. The study of deductive rules is important for interpretation of results of Data Mining as well as for optimization of mining algorithms. To close this section let us mention that each two-dimensional quantifier ∼ can be used to define a three-dimensional quantifier by partializing: the formula ( ϕ ∼ ψ )/ χ is true in the data matrix in question iff ϕ ∼ ψ is true in the submatrix of objects satisfying χ . Cf. (H ´ ajek, 2003). 26.3 Some comments and conclusion Using four-fold tables Even is we have dealt with logical aspects of Data Mining we feel obliged to stress once more the importance of the statistical side of the game. We already referred to (Giudici, 2003) ; let us make some further references. Glymour’s (Glymour et al., 1996) is a good reading on the prehistory of Data Mining, namely exploratory data analysis and of some dangers of using statistically motivated notions in Data Mining. (Zytkow and Zembowicz, 1997) deal with gen- erating knowledge from four-fold tables (and describe their database discovery system “49- er”). Recently, the chi-square statistic was used to define “generalized association rules” (we would say: using a comparative quantifier) by Hegland (Hegland, 2001) and Brin (Brin et al., 1998). Papers by Rauch (et al.) discuss several further classes of quantifiers, see references. Two generalizations First, the logical approach to mining generalized association rules can be and has been gener- alized to fuzzy logic. We refer to Hole ˇ na’s papers ( (Hole ˇ na, 1996) – (Hole ˇ na, 1996)) and also to Chen et. al. (Chen et al., 2003). Second, we have only mentioned relational Data Mining 26 Logics for Data Mining 549 and its techniques of inductive logic programming. Besides D ˇ zeroski and Lavra ˇ c(D ˇ zeroski and Lavra ˇ c, 2001) the reader may consult e.g. Dehaspe and Toivonen (Dehaspe and Toivonen, 1999). The GUHA method The reader should be informed on the more then 30 years old story of the GUHA method of automated generation of hypotheses (General Unary Hypotheses Automa- ton) which is undoubtely one of the oldest methods of computerized exploratory data analysis (or, if you want, mining of association rules) starting with (H ´ ajek et al., 1966) from 1966. We already mentioned above that formulas almost identical with Agrawal’s association rules were considered and algorithms for their generation were dis- cussed in that paper. This was followed by a long period of research culminating in 1978 by the monograph (H ´ ajek and Havr ´ anek, 1978) by H ´ ajek and Havr ´ anek, presenting a logical and statistical fundations that are still relevant for contemporary Data Mining. (Note that the book is presently available on web, see references.) The research has continued; see (H ´ ajek et al., 2003) and (H ´ ajek and Hole ˇ na, 2003, H ´ ajek, 2001) for a survey of the present state and relation to other Data Mining methods. The GUHA approach offers observational logical calculi (based on generated quantifiers as presented here), logical foundations of statistical inference (theoretical logical calculi), theory of some auxiliary (helpful) quantifiers good for compression of results, three semantics of missing information and several other facts, no- tions and techniques. There have been several implementations; for two presently available see (GUHA+-, lispminer). It is regrettable that the mainstream of Data Mining has neglected the GUHA approach ( (Liu et al., 2000) being one of few exceptions); this subsection is a small attempt to change this. Conclusion The study of logical aspects of Data Mining is interesting and useful: it gives an exact abstract approach to “association rules” based on the notion of (generalized) quantifiers, important classes of quantifiers, deductive properties of associations expressed using such quantifiers as well as other results not mentioned here (as e.g. results on computational complexity). Hopefully the present chapter will help the reader to enjoy this. Acknowledgments Partial support of the COST Action 274 (TARSKI) is recognized. References Adamo, J. M. Data Mining for association rules and sequential patterns. Springer 2001. Agrawal, R., Mannila, H., Srikant, R., Toivonen, H., and A. I. Verkamo. “Fast discovery of association rules.” In: Advances in knowledge discovery and Data Mining. Fayyad U. M. et al., ed., AAAI Press/MIT Press,1996 . (a 1 ,b 1 ,c 1 ,d 1 ), (a 2 ,b 2 ,c 2 ,d 2 ) of four-fold tables: a 2 ≥ a 1 ,b 2 ≤ b 1 ,c 2 ≤ c 1 ,d 2 ≥ d 1 and Tr Q (a 1 ,b 1 ,c 1 ,d 1 )=1 implies Tr Q (a 2 ,b 2 ,c 2 ,d 2 )=1. In other words:. if ψ 1 ¬ ψ 1 ϕ 1 a 1 b 1 ¬ ϕ 1 c 1 d 1 ψ 2 ¬ ψ 2 ϕ 2 a 2 b 2 ¬ ϕ 2 c 2 d 2 are four-fold tables of the pairs ( ϕ 1 , ψ 1 ), ( ϕ 2 , ψ 2 ) of open formulas in given data, if (Qx)( ϕ 1 , ψ 1 ) is true in the data and the above. hold (a 2 ≥a 1 ,b 2 ≤b 1 ,c 2 ≤c 1 ,d 2 ≥ d 1 ), then (Qx)( ϕ 2 , ψ 2 ) is also true. The second table has more coincidences (a 2 ≥a 1 ,d 2 ≥d 1 ) and less differences (b 2 ≤ b 1 ,c 2 ≤ c 1 ). A