Chapter Soft Computing Based Theory and Techniques 4.1 Introduction In many multimedia data mining applications, it is often required to make a decision in an imprecise and uncertain environment For example, in the application of mining an image database with a query image of green trees, given an image in the database that is about a pond with a bank of earth and a few green bushes, is this image considered as a match to the query? Certainly this image is not a perfect match to the query, but, on the other hand, it is also not an absolute mismatch to the query Problems like this example, as well as many others, have intrinsic imprecision and uncertainty that cannot be neglected in decision making Traditional intelligent systems fail to solve such problems, as they attempt to use Hard Computing techniques In contrast, a Soft Computing methodology implies cooperative activities rather than autonomous ones, resulting in new computing paradigms such as fuzzy logic, neural networks, and evolutionary computation Consequently, soft computing opens up a new research direction for problem solving that is difficult to achieve using traditional hard computing approaches Technically, soft computing includes specific research areas such as fuzzy logic, neural networks, genetic algorithms, and chaos theory Intrinsically, soft computing is developed to deal with pervasive imprecision and uncertainty of real-world problems Unlike traditional hard computing, soft computing is capable of tolerating imprecision, uncertainty, and partial truth without loss of performance and effectiveness for the end user The guiding principle of soft computing is to exploit the tolerance for imprecision, uncertainty, and partial truth to achieve a required tractability, robustness, and low solution cost We can easily come to the conclusion that precision has a cost Therefore, in order to solve a problem with an acceptable cost, we need to aim at a decision with only the necessary degree of precision, not exceeding the requirements In soft computing, fuzzy logic is the kernel The principal advantage of fuzzy logic is the robustness to its interpolative reasoning mechanism Within soft computing, fuzzy logic is mainly concerned with imprecision and approximate reasoning, neural networks with learning, genetic algorithms with 143 © 2009 by Taylor & Francis Group, LLC 144 Multimedia Data Mining global optimization and search, and chaos theory with nonlinear dynamics Each of these computational paradigms provides us with complementary reasoning and searching methods to solve complex, real-world problems The interrelations between these paradigms of soft computing contribute to the theoretical foundation of Hybrid Intelligent Systems The use of hybrid intelligent systems leads to the development of numerous manufacturing systems, multimedia systems, intelligent robots, and trading systems, well beyond the scope of multimedia data mining 4.2 Characteristics of the Paradigms of Soft Computing Different paradigms of soft computing can be used independently and more often in combination In soft computing, fuzzy logic plays a unique role Fuzzy sets are used as a universal approximator, which is often paramount for modeling unknown objects However, fuzzy logic in its pure form may not necessarily always be useful for easily constructing an intelligent system For example, when a designer does not have sufficient prior information (knowledge) about the system, the development of acceptable fuzzy rules becomes impossible; further, as the complexity of the system increases, it becomes difficult to specify a correct set of rules and membership functions for adequately and correctly describing the behavior of the system Fuzzy systems also have the disadvantage of the inability to automatically extract additional knowledge from the experience and to automatically correct and improve the fuzzy rules of the system Another important paradigm of soft computing is neural networks Artificial neural networks, as a parallel, fine-grained implementation of non-linear static or dynamic systems, were originally developed as a parallel computational model A very important advantage of these networks is their adaptive capability, where “learning by example” replaces the traditional “programming” in problem solving Another important advantage is the intrinsic parallelism that allows fast computations Artificial neural networks are a viable computational model for a wide variety of problems, including pattern classification, speech synthesis and recognition, curve fitting, approximation, image compression, associative memory, and modeling and control of non-linear unknown systems, in addition to the application of multimedia data mining The third advantage of artificial neural networks is the generalization capability, which allows correct classification of new patterns A significant disadvantage of artificial neural networks is their poor interpretability One of the main criticisms addressed to neural networks concerns their black box nature Evolutionary computing is a revolutionary paradigm for optimization One component of evolutionary computing — genetic algorithms — studies the al- © 2009 by Taylor & Francis Group, LLC Soft Computing Based Theory and Techniques 145 Table 4.1: Comparative characteristics of the components of soft computing c Reprint from [8] 2001 World Scientific Fuzzy sets Artificial neu- Evolutionary ral networks computing, Genetic algorithms Weakness Knowledge Black box inter- Coding; Compuacquisition; pretability tational speed Learning Strengths Interpretability; Learning; Adap- Computational Transparency; tation; Fault efficiency; Plausibility; tolerance; Curve Global optiModeling; fitting; General- mization Reasoning; ization ability; Tolerance to Approximation imprecision ability gorithms for global optimization Genetic algorithms are based on the mechanisms of natural selection and genetics One advantage of genetic algorithms is that they effectively implement a parallel, multi-criteria search The mechanism of genetic algorithms is simple Simplicity of operations and powerful computational effect are the two main principles for designing effective genetic algorithms The disadvantages include the convergence issue and the lack of strong theoretic foundation The requirement of coding the domain variables into bit strings also seems to be a drawback of genetic algorithms In addition, the computational speed of genetic algorithms is typically low Table 4.1 summarizes the comparative characteristics of the different paradigms of soft computing For each paradigm of soft computing, there are appropriate problems where this paradigm is typically applied 4.3 Fuzzy Set Theory In this section, we give an introduction to fuzzy set theory, fuzzy logic, and their applications in multimedia data mining 4.3.1 Basic Concepts and Properties of Fuzzy Sets DEFINITION 4.1 Let X be a classic set of objects, called the universe, with the generic elements denoted as x The membership of a classic subset © 2009 by Taylor & Francis Group, LLC 146 Multimedia Data Mining FIGURE 4.1: Fuzzy set to characterize the temperature of a room A of X is often considered as a characteristic function µA mapped from X to {0,1} such that iff x ∈ A µA (x) = iff x ∈ /A where {0,1} is called a valuation set; indicates membership while indicates non-membership If the valuation set is allowed to be in the real interval [0,1], A is called a fuzzy set µA (x) is the grade of membership of x in A: µA : X −→ [0, 1] The closer the value of µA (x) is to 1, the more x belongs to A A is completely characterized by the set of the pair: A = {(x, µA (x)), x ∈ X} Solutions to many real-world problems can be developed more accurately using fuzzy set theory Figure 4.1 shows an example regarding how fuzzy set representation is used to describe the natural drift of temperature DEFINITION 4.2 Two fuzzy sets A and B are said to be equal, A = B, if and only if x X àA (x) = àB (x) â 2009 by Taylor & Francis Group, LLC Soft Computing Based Theory and Techniques 147 In the case where universe X is infinite, it is desirable to represent fuzzy sets in an analytical form, which describes the mathematical membership functions There are several mathematical functions that are frequently used as the membership functions in fuzzy set theory and practice For example, a Gaussian-like function is typically used for the representation of the membership function as follows: (x − a)2 ) b which is defined by three parameters, a, b, and c Figure 4.2 summarizes the graphical and analytical representations of frequently used membership functions An appropriate construction of the membership function for a specific fuzzy set is the problem of knowledge engineering [125] There are many methods for an appropriate estimation of a membership function They can be categorized as follows: µA (x) = c exp(− Membership functions based on heuristics Membership functions based on reliability concepts with respect to the specific problem Membership functions based on a certain theoretic foundation Neural networks based construction of membership functions The following rules which are common and valid in the classic set theory also apply to fuzzy set theory • De Morgan’s law: and A∩B =A∪B A∪B =A∩B • Associativity: and (A ∪ B) ∪ C = A ∪ (B ∪ C) (A ∩ B) ∩ C = A ∩ (B ∩ C) • Commutativity: and A∪B =B ∪A A∩B =B ∩A • Distributivity: A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C) and A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) © 2009 by Taylor & Francis Group, LLC 148 Multimedia Data Mining c FIGURE 4.2: Typical membership functions Reprint from [8] 2001 World Scientific © 2009 by Taylor & Francis Group, LLC Soft Computing Based Theory and Techniques 4.3.2 149 Fuzzy Logic and Fuzzy Inference Rules In this section fuzzy logic is reviewed in a narrow sense as a direct extension and generalization of multi-valued logic According to one of the most widely accepted definitions, logic is an analysis of methods of reasoning; in studying these methods, logic is mainly taken in the form, not in the content, of the arguments used in a reasoning process Here the main issue is to establish whether the truth of the consequence can be inferred from the truth of premises Systematic formulation of the correct approaches to reasoning is one of the main issues in logic Let us define the semantic truth function of fuzzy logic Let P be a statement and T(P) be its truth value, where T (P ) ∈ [0, 1] Negation values of the statement P are defined as T (¬P ) = − T (P ) The implication connective is always defined as T (P → Q) = T (¬P ∨ Q) and the equivalence is always defined as T (P ↔ Q) = T [(P → Q) ∧ (Q → P )] Based on the above definitions, we further define the basic connectives of fuzzy logic as follows • T (P ∨ Q) = max(T (P ), T (Q)) • T (P ∧ Q) = min(T (P ), T (Q)) • T (P ∨ (P ∧ Q)) = T (P ) • T (P ∧ (P Q)) = T (P ) ã T (ơ(P Q)) = T (ơP ơQ) ã T (ơ(P ∨ Q)) = T (¬P ∧ ¬Q) It is shown that multi-valued logic is the fuzzification of the traditional propositional calculus (in the sense of the extension principle) Here each proposition P is assigned a normalized fuzzy set in [0,1]; i.e., the pair {µP (0), µP (1)} is interpreted as the degree of false or true, respectively Since the logical connectives of the standard propositional calculus are functionals of truth, i.e., they are represented as functions, they can be fuzzified Let A and B be fuzzy sets of the subsets of the non-fuzzy universe U ; in fuzzy set theory it is known that A is a subset of B iff µA ≤ µB , i.e., ∀x ∈ U , µA (x) ≤ µB (x) In fuzzy set theory, great attention is paid to the development of fuzzy conditional inference rules This is connected to the natural language understanding where it is necessary to have a certain number of fuzzy concepts; therefore, we must ensure that the inference of the logic is made such that the preconditions and the conclusions both may contain such fuzzy concepts It © 2009 by Taylor & Francis Group, LLC 150 Multimedia Data Mining is shown that there is a huge variety of ways to formulate the rules for such inferences However, such inferences cannot be satisfactorily formulated using the classic Boolean logic In other words, here we need to use multi-valued logical systems The conceptual principle in the formulation of the fuzzy rules is the Modus Ponens inference rule that states: IF(α→ β) is true and α is true, THEN β must also be true The methodological foundation for this formulation is the compositional rule suggested by Zadeh [231, 232] Using this rule, he has formulated the inference rules in which both the logical preconditions and consequences are conditional propositions, including the fuzzy concepts 4.3.3 Fuzzy Set Application in Multimedia Data Mining In multimedia data mining, fuzzy set theory can be used to address the typical uncertainty and imperfection in the representation and processing of multimedia data, such as image segmentation, feature representation, and feature matching Here we give one such application in image feature representation as an example in multimedia data mining In image data mining, the image feature representation is the very first step for any knowledge discovery in an image database In this example, we show how different image features may be represented appropriately using the fuzzy set theory In Section 2.4.5.2, we have shown how to use fuzzy logic to represent the color features Here we show the fuzzy representation of texture and shape features for a region in an image Similar to the color feature, the fuzzification of the texture and shape features also brings a crucial improvement into the region representation of an image, as the fuzzy features naturally characterize the gradual transition between regions within an image In the following proposed representation scheme, a fuzzy feature set assigns weights, called the degree of membership, to feature vectors of each image block in the feature space As a result, the feature vector of a block belongs to multiple regions with different degrees of membership as opposed to the classic region representation, in which a feature vector belongs to exactly one region We first discuss the fuzzy representation of the texture feature, and then discuss that of the shape feature We take each region as a fuzzy set of blocks In order to propose a unified approach consistent with the fuzzy color histogram representation described in Section 2.4.5.2, we again use the Cauchy function to be the fuzzy membership function, i.e., (4.1) µi (f ) = ˆ + ( d(f,σfi ) )α where f ∈ Rk is the texture feature vector of each block with k as the dimensionality of the feature vector; fˆi is the average texture feature vector of region i; d is the Euclidean distance between fˆi and any feature f ; and σ © 2009 by Taylor & Francis Group, LLC Soft Computing Based Theory and Techniques 151 represents the average distance for texture features among the cluster centers obtained from the k-means algorithm σ is defined as: σ= C−1 C X X kfˆi − fˆk k C(C − 1) i=1 (4.2) k=i+1 where C is the number of regions in a segmented image, and fˆi is the average texture feature vector of region i With this block membership function, the fuzzified texture property of region i is represented as X T f µi (f ) (4.3) f~i = f ∈U T where U T is the feature space composed of texture features of all blocks Based on the fuzzy membership function µi (f ) obtained in a similar fashion, we also fuzzify the p-th order inertia as the shape property representation of region i as: P ˆ)2 + (fy − yˆ)2 ]p/2 µi (f ) f ∈U S [(fx − x l(i, p) = (4.4) [N ]1+p/2 where fx and fy are the x and y coordinates of the block with the shape feature f , respectively; x ˆ and yˆ are the x and y central coordinates of region i, respectively; and N is the number of blocks in an image and U S is the block feature space of the images Based on Equation 4.4, we have obtained S the fuzzy representation for the shape feature of each region, denoted as f~i 4.4 Artificial Neural Networks Historically, in order to “simulate” the biological systems to make nonsymbolic computations, different mathematical models were suggested The artificial neural network is one such model that has shown great promise and thus attracted much attention in the literature 4.4.1 Basic Architectures of Neural Networks Neurons represent a special type of nervous cells in the organism, having electric activities These cells are mainly intended for the operative control of the organism A neuron consists of cell bodies, which are enveloped in the membrane A neuron also has dendrites and axons, which are its inputs and outputs Axons of neurons join dendrites of other neurons through synaptic contacts Input signals of the dendrite tree are weighted and added in the © 2009 by Taylor & Francis Group, LLC 152 Multimedia Data Mining c FIGURE 4.3: Mathematical model of a neuron Reprint from [8] 2001 World Scientific cell body and formed in the axon, where the output signal is generated The signal’s intensity, consequently, is a function of a weighted sum of the input signal The output signal is passed through the branches of the axon and reaches the synapses Through the synapses the signal is transformed into a new input signal of the neighboring neurons This input signal can be either positive or negative, depending upon the type of the synapses The mathematical model of the neuron that is usually utilized under the simulation of the neural network is represented in Figure 4.3 The neuron receives a set of input signals x1 , x2 , , xn (i.e., vector X) which usually are output signals of other neurons Each input signal is multiplied by a corresponding connection weight w — analogue of the synapse’s efficiency Weighted input signals come to the summation module corresponding to the cell body, where their algebraic summation is executed and the excitement level of the neuron is determined: n X I= xi Wi 1=1 The output signal of a neuron is determined by conducting the excitement level through the function f , called the activation function: y = f (I − θ) where θ is the threshold of the neuron Usually the following activation functions are used as function f : • Linear function (see Figure 4.4), y = kI, k = const • binary (threshold) function (see Figure 4.5), if I ≥ θ y= if I < θ © 2009 by Taylor & Francis Group, LLC