DSpace at VNU: A new effective learning rule of Fuzzy ART

2012 Conference on Technologies and Applications of Artificial Intelligence A new effective learning rule of Fuzzy ART Nong Thi Hoa, The Duy Bui Human Machine Interaction Laboratory University of Engineering and Technology Vietnam National University, Hanoi such as document clustering [7] [8], classification of multivariate chemical data [9], Analysing gene expression [10] Therefore, developing a new effective Fuzzy ART is essential for clustering applications In this paper, we propose a new effective learning rule of Fuzzy ART that learns many types of datasets as well as clusters data better than previous models Our learning rule updates weights of categories based on the ratio of the input to the weight of chosen category and a learning rate The learning rate presents the speed of increasing/decreasing the weight of chosen category It is changed by the following rule: the number of inputs is larger, value is smaller We have conducted experiments with ten typical datasets to prove the effectiveness of our novel model Results of experiments show our novel model clusters better than exiting model, including Original Fuzzy ART, Complement Fuzzy ART, Kmean algorithm, Euclidean ART The rest of the paper is organized as follows The next section shows some background Related works are presented in Section II In section VI, we present our learning rule and discussions Section VII shows experiments with ten datasets Abstract—Unsupervised neural networks are known for their ability to cluster inputs into categories based on the similarity among inputs Fuzzy Adaptive Resonance Theory (Fuzzy ART) is a kind of unsupervised neural networks that learns training data until satisfying a given need In the learning process, weights of categories are changed to adapt to noisy inputs In other words, learning process decides the quality of clustering Thus, updating weights of categories is an important step of learning process We propose a new effective learning rule for Fuzzy ART to improve clustering Our learning rule modifies weights of categories based on the ratio of the input to the weight of chosen category and a learning rate The learning rate presents the speed of increasing/decreasing the weight of chosen category It is changed by the following rule: the number of inputs is larger, value is smaller We have conducted experiments on ten typical datasets to prove the effectiveness of our novel model Result from experiments shows that our novel model clusters better than existing models, including Original Fuzzy ART, Complement Fuzzy ART, K-mean algorithm, Euclidean ART Index Terms—Fuzzy Adaptive Resonance Theory; Clustering; Learning rule; I I NTRODUCTION Clustering is an important tool in data mining and knowledge discovery because clustering discovers hidden similarity and key concepts base on the ability of grouping similar items together Moreover, clustering summarizes a large amount of data into a small number of groups Therefore, it is useful for comprehending a large amount of data Fuzzy ART is a artificial neural networks that clusters data into categories by using AND operators of fuzzy logic The most important advantage of Fuzzy ART is learning training data until reaching to given conditions Meaning, weights of categories are updated until they completely adapt to training data As a result, the learning process decides the quality of clustering Thus, designing a learning rule that allows Fuzzy ART to learn various types of datasets as well as to cluster data better is always on demand Studies about learning process of Fuzzy ART models usually focus on designing new effective learning rules Capenter’s model maximized code generalization by training system several times with different orderings of input set [1] Simpson incorporated new data and new clusters without retraining [2] Tan showed Adaptive Resonance Associative Map with the ability of hetero-associative learning [3] Lin addressed the on-line learning algorithms for realizing a controller [4] Isawa proposed an additional step, Group Learning, to present connections between similar categories [5] Yousuf proposed an algorithm that allows updating multiple matching clusters [6] Moreover, Fuzzy ART has applied for many applications 978-0-7695-4919-4/12 $26.00 © 2012 IEEE DOI 10.1109/TAAI.2012.60 II R ELATED WORKS Studies about theory of Fuzzy ART can be divided into three categories, including developing new models of Fuzzy ART; studying properties of Fuzzy ART; optimizing the performance of Fuzzy ART In the first category, new models of Fuzzy ART were proposed to improve clustering/classifying data into categories Capenter proposed Fuzzy ARTMAP for incremental supervised learning of recognition categories and multidimensional maps from arbitrary sequences of inputs [1] This model minimized predictive error and maximized code generalization by training system several times with different orderings of input set Simpson presented a fuzzy min-max clustering neural network with unsupervised learning [2] Pattern clusters were fuzzy sets associating a membership function Simpson’s model have three advantages including stabilizing into pattern clusters in only a few passes; reducing to hard cluster boundaries; incorporating new data and add new clusters without retraining Tan showed a neural architecture termed Adaptive Resonance Associative Map that extends unsupervised ART systems for rapid, yet stable, hetero-associative learning [3] 216 224 Pattern pairs were coded explicitly and recalled perfectly Moreover, this model produces the stronger noise immunity Lin addressed the structure and the associated on-line learning algorithms of a feed forward network for realizing the basic elements and functions of a traditional fuzzy logic controller [4] The input and output spaces were parted on-line based on the training data by tuning membership functions and finding proper fuzzy logic rules Isawa proposed an additional step, called Group Learning, for the Fuzzy ART in order to obtain more effective categorization [5] The important feature of the group learning was creating connections between similar categories Kenaya employed the Euclidean neighbourhood to decide the said pertinence and patterns mean value for category training [11] This model calculated the Euclidean distance and decides a new pattern in an existing category or a new category Isawa proposed Fuzzy ART combining overlapped categories in connections to void the category proliferation problem [12] The important feature of this study was arranging the vigilance parameters for every category and varying them in learning process Yousuf proposed an algorithm that compares all weights to the input and allows updating multiple matching clusters [6] This model mitigated the effects and supervision of updating clusters for the wrong class In the second category, important properties of Fuzzy ART were studied to choosing suitable parameters for a new Fuzzy ART Huang presented some important properties of the Fuzzy ART that distinguished into a number of categories[13] Properties includes template, access, reset, and other properties for weight stabilization Moreover, the effects of choice parameter and vigilance parameter on the functionality of Fuzzy ART were presented clearly Geogiopoulos provided a geometrical and clearer understanding of why, and in what order, categories are chosen for various ranges of choice parameter of Fuzzy ART [14] This study was useful to develop properties of learning that pertain to the architecture of neural networks Moreover, he commented the orders according to which categories were chosen Anagnostopoulos introduced novel geometric concepts, namely category regions, in the original framework of Fuzzy ART and Fuzzy ARTMAP These regions had the same geometrical shape and shared a lot of common and interesting properties [15] He proved properties of learning and showed the training and performance phases did not depend on the particular choices of the vigilance parameter in one special state of the vigilance-choice parameter space In the third category, studies focused on ways to increase the performance of FART Cano generated function identifiers for noisy data [16] Thus, FARTs trained on noisy data without changing the structure or data preprocessing Burwick discussed implementations of ART on a nonrecursive algorithm to decrease algorithmic complexity of Fuzzy ART [17] Therefore, the complexity dropped from Figure Architecture of an ART network O(N*N)+O(M*N) down to O(NM) where N be the number of categories and M be the input dimension Dagher introduced an ordering algorithm that identified a fixed order of training pattern presentation based on the maxmin clustering method to improve generalization performance of FART [18] Kobayashi proposed a new reinforcement learning system that used fuzzy ART to classify observed information and construct effective state space [19] Then, this system was used to solving partially observable Markov decision process problems Fuzzy ART has applied for many applications such as document clustering [7] [8], classification of multivariate chemical data [9], Analysing gene expression [10], quality control of manufacturing process [20], classification with missing data in a wireless sensor network [21] III BACK GROUND [22] A ART Network Adaptive Resonance Theory (ART) neural networks are developed by Grossberg to address the problem of stabilityplasticity dilemma The general structure of an ART network is shown in the Figure A typical ART network consists of two layers: an input layer (F1) and an output layer (F2) The input layer contains N nodes, where N is the number of input patterns The number of nodes in the output layer is decided dynamically Every node in the output layer has a corresponding prototype vector The networks dynamics are governed by two sub-systems: an attention subsystem and an orienting subsystem The attention subsystem proposes a winning neuron (or category) and the orienting subsystem decides whether to accept it or not This network is in a resonant state when the orienting system accepts a winning category, meaning, the winning prototype vector matches the current input pattern close enough B Fuzzy ART Algorithm [22] Input vector: Each input I is an M-dimensional vector (I1 , .IM ), where each component li is in the interval [0, 1] Weight vector: Each category (j) corresponds to a vector wj = (Wj1 , , wjM ) of adaptive weights, or LTM traces The number of potential categories N(j = i, , N) is arbitrary Initially 225 217 Wj1 = = wjM = Fast-commit slow-recode option: For efficient coding of noisy input sets, it is useful to set β = when J is an uncommitted node, and then to take β < after the category is (new) committed Then wj = I the first time category J becomes active (1) and each category is said to be uncommitted Alternatively, initial weights wji may be taken greater than Larger weights bias the system against selection of uncommitted nodes, leading to deeper searches of previously coded categories After a category is selected for coding it becomes committed As shown below, each LTM trace wji is monotone non-increasing through time and hence converges to a limit Parameters: Fuzzy ART dynamics are determined by a choice parameter α 0; a learning rate parameter β ∈ [0, 1]; and a vigilance parameter θ ∈ [0, 1] Category choice: For each input I and category j, the choice function Tj is defined by Tj (I) = |I ∧ wj | α + |wj | C Fuzzy ART with complement coding [22] Moore [23] described a category proliferation problem that can occur in some analog ART systems when a large number of inputs erode the norm of weight vectors Proliferation of categories is avoided in Fuzzy ART if inputs are normalized; that is, for some γ > |I| = γ for all inputs I Normalization can be achieved by preprocessing each incoming vector a A normalization rule, called complement coding, achieves normalization while preserving amplitude information Complement coding represents both the on-response and the off-response to a To define this operation in its simplest form, let a itself represent the onresponse The complement of a, denoted by ac , represents the off-response, where (2) where the fuzzy AND (Zadeh, 1965) operator ∧ is defined by (x ∧ y)i = min(xi , yi ) (3) and where the norm |.| is defined by aci = − M |x| = |xi | (4) For notational simplicity, Tj (I) in Equation is often written as Tj when the input I is fixed The category choice is indexed by J, where I = (ai , aci ) = (a1 , , aM , ac1 , , aM i ) Wj1 = = wj2M = IV K- MEANS C LUSTERING [24] (6) k (7) N J= (j) xi − C j (13) j=1 i=1 Then the value of the choice function Tj is reset to −1 for the duration of the input presentation to prevent its persistent selection during search A new index J is chosen, by Equation The search process continues until the chosen J satisfies Equation Learning: The weight vector wj is updated according to the equation wjnew = β(I ∧ wjold ) + (1 − β)wjold (12) K-means is one of the simplest unsupervised learning algorithms that solve the clustering problem The procedure follows a simple and easy way to classify a given data set through a certain number of clusters (assume k clusters) This algorithm aims at minimizing a squared error function by the following equation: Learning then ensues, as defined below Mismatch reset occurs if |I ∧ wj | ≤ρ |I| (11) After normalization, |I| = M so inputs preprocessed into complement coding form are automatically normalized Where complement coding is used, the initial condition is replaced by (5) If more than one Tj is maximal, the category j with the smallest index is chosen In particular, nodes become committed in order j = 1, 2, 3, Resonance or reset: Resonance occurs if the match function of the chosen category meets the vigilance criterion; that is, if |I ∧ wj | ≥ρ |I| (10) The complement coded input I to the recognition system is the 2M-dimensional vector i=1 Tj = max{Tj : j = N } (9) where N be the number of points that is in cluster j (j) In the other words, xi − Cj is a chosen distance (j) measure between a data point xi and the cluster centre Cj , is an indicator of the distance of the n data points from their respective cluster centres The algorithm is composed of the following steps: • Step 1: Place k points into the space represented by the objects that are being clustered These points represent initial group centroids (8) 226 218 Step 2: Assign each object to the group that has the closest centroid • Step 3: When all objects have been assigned, recalculate the positions of the k centroids • Step 4: Repeat Steps and until the centroids no longer move This produces a separation of the objects into groups from which the metric to be minimized can be calculated Although the procedure will always terminate, the K-means algorithm does not necessarily find the most optimal configuration, corresponding to the global objective function minimum • categories Our novel model with the new effective learning rule greatly clusters better than exiting studies Our novel model consists of following steps: • Step 1: Set up connection weights Wj , the choice parameter α and the vigilance parameter ρ • Step 2: Choose a suitable category for the input according to Equation 2-5 • Step 3: Test the current state that can be resonance or reset by Equation and • Step 4: Learning is performed by our learning rule: wjnew = wjold + β(I − wjold ) V E UCLIDEAN ART A LGORITHM [11] The Euclidean ART is a clustering technique that evaluates the Euclidean distance between patterns and cluster centrers to decide clustering membership of patterns The pattern membership is dependent on the parameter Rth ,the Euclidean threshold The Euclidean ART algorithm consist of the following steps: • Step 1: Present a normalized and complement coded pattern to Euclidean ART module • Step 2: Calculate the Euclidean distance between this pattern and the entire existing cluster centers by Equation 14 Those Euclidean distances are considered as an activation value of each cluster center with respect to the presented pattern If there is no cluster center yet, consider this pattern to be the first one • (xi − wj )2 (14) j=1 • • where j is the category index found in Euclidean ART network and i is the index of the current presented pattern Step 3: Find d(J), where d(J) = min(d) Step 4: If d(J) ≤ Rth then – Include the presented pattern xk in the winning cluster whose center is wJ – Start the learning process; calculate the new cluster center according to learning equation 15 • L k=1 VII E XPERIMENTS We select 10 datasets from UCI database [25] and Shape database [26], including Iris, Wine, Jain, Flame, R15, Glass, Pathbased, Compound, Aggregation, and Spiral These datasets are different from each other by the number of attributes, the number of categories, the number of patterns, and distribution of categories Table I shows parameters of selected databases Our novel models are compare to Fuzzy ARTs [22], Kmean [24], and Euclidean ART [11] Six models are coded to assess the effective of our novel models, including Original Fuzzy ART (OriFART), Complement Fuzzy ART (ComFART), Original New Fuzzy ART (OriNewFART), Complement New Fuzzy ART (ComNewFART), K-mean (Kmean), and Euclidean ART (EucART) xJk (15) L where xJk is the pattern member k of cluster J and L is the number of cluster members Else xi becomes a new category wN +1 Step 5: Jump back to Step to accept a new pattern if there are more patterns to test Else training is over and resulting Euclidean ART matrix is the trained Euclidean ART network wJ new = where β be learning rate The learning rate is change base on the number of patterns of datasets Meaning, the number of patterns is larger, β is smaller In other words, adding an input to a category, the weight vector of this category is increased/decreased to adapt to the new input In Fuzzy ART, wj is in [0,1] Therefore, if wj < then assign wj = and wj > then assign wj = Fast-commit slow-recode option is similar to Fuzzy Original ART of Carpenter B Discussion In previous studies, learning rules are similar with two terms, including the percent of old weight of the chosen category and the percent of the ratio of the input to old weight of the chosen category Learning parameter β is used to present the percent of terms In our learning rule, learning parameter shows the rate of learning process is quick or slow Therefore, our learning rule is different from previous studies Our novel model can coded into two models, including Original New Fuzzy ART without normalizing inputs and Complement New Fuzzy ART with normalizing inputs Therefore, we have two models in experiments Similarly with the model of Carpenter [22] With Complement Fuzzy ARTs, category proliferation problem is not happened by normalizing inputs Moreover, choosing a suitable value of β (enough small), Original Fuzzy ARTs void the category proliferation problem Thus, not solve this problem in experiments N d(j) = (16) VI O UR APPROACH A Our novel model Our goal is creating a new Fuzzy ART that clusters better We propose a new effective rule for updating weights of 227 219 Table V T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN G LASS DATASET Table I F EATURES OF DATASETS Index 10 Dataset Name Iris Glass Wine Jain Pathbased Spiral R15 Flame Compound Aggregation #Categories 3 15 #Attribute 13 2 2 2 #Reocords 150 214 178 373 300 312 600 240 399 788 #Records 50 % 100 % 150 % 200 % 214 % 1–50 51–100 Category Index Distribution 101–150 207–312 Table VII T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN S PIRAL DATASET T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN I RIS DATASET OriFART 30 100.0 55 91.7 65 72.2 88 73.3 118 78.7 ComFART 30 100.0 60 100.0 83 92.2 111 92.5 139 92.7 EucART 30 100.0 58 96.7 81 90.0 108 90.0 135 90.0 #Records 50 % 100 % 150 % 200 % 250 % 312 % K-mean 30 100.0 60 100.0 85 94.4 112 93.3 140 93.3 Table IV T HE DISTRIBUTION OF CATEGORIES IN G LASS DATASET 147–163 102–206 Table X shows the distribution of categories is not uniform with categories Table XI shows Complement New Fuzzy ART are greatly better than other models in all sub-tests, excepting Euclidean ART model in three first sub-tests However, Euclidean ART model is greatly lower Complement New Fuzzy ART in the sub-test with 788 patterns Table III 71–146 1–101 E Experiment 5: Testing with Aggregation dataset Table IV shows the distribution of categories is not uniform with categories, especially, the distribution of the fourth category is Table V shows our novel model is better some sub-tests with the large number of testing patterns and not better some sub-tests with the small one 1–70 K-mean 41 82.0 53 53.0 53 35.3 73 36.5 87 40.7 Table VIII shows the distribution of categories is not uniform with categories Table IX shows Original New Fuzzy ART are greatly better than other models in all sub-tests, excepting Original Fuzzy ART model in the sub-test with 240 patterns B Experiment 2: Testing with Glass dataset Category Index Distribution EucART 8.0 25 25.0 46 30.7 73 36.5 79 36.9 D Experiment 4: Testing with Flame dataset Table II shows the distribution of categories is uniform with categories Table III shows the number of successful clustering patterns in Iris dataset Data are sorted by the number of testing patterns Table III shows Complement New Fuzzy ART is better in all sub-tests ComNewFART 30 100.0 60 100.0 87 96.7 115 95.8 143 95.3 ComFART 12.0 25 25.0 61 40.7 67 33.5 80 37.4 Table VI shows the distribution of categories is uniform with categories Table VII shows Original New Fuzzy ART are greatly better than other models in all sub-tests, excepting Euclidean ART model in the sub-test with 312 patterns A Experiment 1: Testing with Iris dataset OriNewFART 30 100.0 59 98.3 84 93.3 114 95.0 144 96.0 OriFART 12.0 42 42.0 54 36.0 79 39.5 92 43.0 C Experiment 3: Testing with Spiral dataset Data of each datasets are normalized to values in [0,1] In all experiments, we choose a random vector of each category to be the initial weight vector Values of parameters are chosen to reach to highest performance of models In most datasets and most models, α = 0.8, β = 0.1, ρ = 0.5 In Euclidean ART with all datasets, ρ = 0.4 #Records 30 % 60 % 90 % 120 % 150 % ComNewFART 2.0 26 26.0 53 35.3 87 43.5 99 46.3 Table VI T HE DISTRIBUTION OF CATEGORIES IN S PIRAL DATASET Table II T HE DISTRIBUTION OF CATEGORIES IN I RIS DATASET Category Index Distribution OriNewFART 16.0 49 49.0 84 56.0 106 53.0 119 55.6 164–176 177-185 OriNewFART 44 88.0 71 71.0 72 48.0 85 42.5 101 40.4 120 38.5 ComNewFART 4.0 49 49.0 50 33.3 84 42.0 84 33.6 118 37.8 OriFART 4.0 37 37.0 38 25.3 59 29.5 69 27.6 103 33.0 ComFART 11 22.0 25 25.0 32 21.3 78 39.0 88 35.2 88 28.2 EucART 2.0 48 48.0 49 32.7 82 41.0 99 39.6 133 42.6 Table VIII T HE DISTRIBUTION OF CATEGORIES IN F LAME DATASET 186-214 Category Index Distribution 228 220 1–87 88–240 K-mean 22 44.0 22 22.0 25 16.7 64 32.0 80 32.0 102 32.7 Table IX T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN F LAME Table XIII T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN W INE DATASET DATASET #Records 50 % 100 % 150 % 200 % 240 % OriNewFART 50 100.0 98 98.0 148 98.7 190 95.0 203 84.6 ComNewFART 50 100.0 87 87.0 131 87.3 153 76.5 159 66.3 OriFART 50 100.0 87 87.0 121 80.7 171 85.5 211 87.9 ComFART 38 76.0 83 83.0 127 84.7 127 63.5 133 55.4 EucART 44 88.0 94 94.0 142 94.7 148 74.0 152 63.3 #Records 30 % 60 % 90 % 120 % 150 % 178 % K-mean 39 78.0 54 54.0 104 69.3 154 77.0 188 78.3 Table X T HE DISTRIBUTION OF CATEGORIES IN AGGREGATION DATASET Category Index Distribution 1–45 46–215 216–317 318–590 591–624 625–754 OriNewFART 30 100.0 59 98.3 75 83.3 92 76.7 116 77.3 138 77.5 ComNewFART 30 100.0 59 98.3 80 88.9 101 84.2 128 85.3 156 87.6 OriFART 30 100.0 59 98.3 77 85.6 99 82.5 124 82.7 149 83.7 ComFART 22 73.3 41 68.3 58 64.4 73 60.8 97 64.7 124 69.7 Table XIV T HE DISTRIBUTION OF CATEGORIES IN R15 755–788 Category Index Distribution 281–320 1–40 321–360 41–80 10 361–400 81–120 11 401–440 121–160 12 441–480 EucART 30 100.0 59 98.3 60 66.7 60 50.0 61 40.7 61 34.3 K-mean 30 100.0 60 100.0 81 90.0 104 86.7 128 85.3 156 87.6 DATASET 161–200 13 481–520 201–240 14 521–560 241–280 15 561–600 Experiment 6: Testing with Wine dataset Table XII shows the distribution of categories is uniform with categories Table XIII shows Complement New Fuzzy ART is approximate K-mean and better than other models Table XIX shows Complement New Fuzzy ART is lower K-mean with all sub-tests and Euclidean ART with two first sub-tests, and better than other models F Experiment 7: Testing with R15 dataset Table XX shows the distribution of categories is not uniform with categories Data from Table XXI shows Complement New Fuzzy ART are better than Original Fuzzy ART and Euclidean ART However, Complement New Fuzzy ART is a bit lower than K-mean and Complement Fuzzy ART in two final sub-tests In summary, although several sub-tests of other models are better than our novel model, our novel model is better than exiting models in many sub-tests and in most datasets I Experiment 10: Testing with Jain dataset Table XIV shows the distribution of categories is uniform with 15 categories Table XV shows Complement New Fuzzy ART is approximate Euclidean ART, equal to Complement Fuzzy ART, and better than K-mean and Original Fuzzy ART G Experiment 8: Testing with Compound dataset Table XVI shows the distribution of categories is not uniform with categories Table XVII shows Original New Fuzzy ART is better than other models, excepting Original Fuzzy ART In two first subtests, Original New Fuzzy ART is better than Original Fuzzy ART but lower in two final sub-tests VIII C ONCLUSION In this paper, we proposed a new effective learning rule for Furry ART Our novel model updates weights of categories base on the ratio of the input to the weight of chosen category, and a learning rate The learning parameter shows the rate of learning process is quick or slow Changing learning rate is made by the following rule: The number of inputs is larger, H Experiment 9: Testing with Pathbased dataset Table XVIII shows the distribution of categories is uniform with categories Table XV T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN R15 Table XI T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN AGGREGATION #Records 100 % 200 % 300 % 400 % 500 % 600 % DATASET #Records 200 % 400 % 600 % 788 % OriNewFART 196 98.0 355 88.8 502 83.7 545 69.2 ComNewFART 193 96.5 367 91.8 558 93.0 615 78.0 OriFART 167 83.5 273 68.3 357 59.5 404 51.3 ComFART 167 83.5 328 82.0 509 84.8 538 68.3 EucART 200 100.0 392 98.0 573 95.5 577 73.2 K-mean 163 81.5 265 66.3 391 65.2 417 52.9 1–59 60–130 ComNewFART 98 98.0 191 95.5 287 95.7 387 96.8 487 97.4 587 97.8 OriFART 95 95.0 187 93.5 265 88.3 347 86.8 447 89.4 547 91.2 ComFART 98 98.0 191 95.5 287 95.7 387 96.8 487 97.4 587 97.8 EucART 100 100.0 192 96.0 287 95.7 387 96.8 487 97.4 587 97.8 K-mean 100 100.0 146 73.0 161 53.7 256 64.0 356 71.2 456 76.0 Table XVI T HE DISTRIBUTION OF CATEGORIES IN C OMPOUND DATASET Table XII T HE DISTRIBUTION OF CATEGORIES IN W INE DATASET Category Index Distribution OriNewFART 96 96.0 191 95.5 286 95.3 384 96.0 484 96.8 584 97.3 DATASET Category Index Distribution 131–178 229 221 1–50 51–142 143–180 181–225 226–383 384–399 Table XVII T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN C OMPOUND Table XXI T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN JAIN DATASET DATASET #Records 100 % 200 % 300 % 399 % OriNewFART 66 66.0 109 54.5 162 54.0 205 51.4 ComNewFART 69 69.0 103 51.5 155 51.7 197 49.4 OriFART 63 63.0 107 53.5 177 59.0 208 52.1 ComFART 56 56.0 69 34.5 84 28.0 141 35.3 EucART 69 69.0 105 52.5 158 52.7 201 50.4 #Records 100 % 200 % 300 % 373 % K-mean 52 52.0 74 37.0 74 24.7 90 22.6 Table XVIII T HE DISTRIBUTION OF CATEGORIES IN PATHBASED DATASET Category Index Distribution 1–110 111–207 208–300 ACKNOWLEDGEMENTS This work is supported by Nafosted research project No 102.02-2011.13 R EFERENCES [1] G A Capenter, S Grossberg, and N Markuron, “Fuzzy artmap-an addaptive resonance architecture for incremental learning of analog maps,” 1992 Table XIX T HE NUMBER OF SUCCESSFUL CLUSTERING PATTERNS IN PATHBASED DATASET OriNewFART 30 60.0 30 30.0 55 36.7 96 48.0 142 56.8 192 64.0 ComNewFART 33 66.0 33 33.0 57 38.0 98 49.0 148 59.2 198 66.0 OriFART 27 54.0 27 27.0 42 28.0 42 21.0 85 34.0 135 45.0 ComFART 40 80.0 40 40.0 51 34.0 51 25.5 77 30.8 127 42.3 EucART 29 58.0 29 29.0 51 34.0 89 44.5 139 55.6 189 63.0 K-mean 39 78.0 50 50.0 91 60.7 128 64.0 178 71.2 228 76.0 Table XX T HE DISTRIBUTION OF CATEGORIES IN JAIN DATASET Category Index Distribution 1–276 ComNewFART 100 100.0 200 100.0 293 97.7 352 94.4 OriFART 99 99.0 199 99.5 209 69.7 258 69.2 ComFART 100 100.0 200 100.0 300 100.0 372 99.7 EucART 100 100.0 114 57.0 129 43.0 177 47.5 K-mean 100 100.0 200 100.0 300 100.0 365 97.9 [2] P Simpson, “Fuzzy min-max neural networks - part 2: Clustering,” IEEE Transactions on Fuzzy Systems, vol 1, no 1, p 32, 1993 [3] A.-H Tan, “Adaptive resonance associative map,” Elsevier Science, Neural Network, vol 8, no 3, pp 437–446, 1995 [4] C.-t Lin, C.-j Lin, and C Lee, “Fuzzy adaptive learning control network with on-line neural learing,” Elsevier Science-Fuzzy sets and Systems, vol 71, pp 25–45, 1995 [5] H Isawa, M Tomita, H Matsushita, and Y Nishio, “Fuzzy adaptive resonance theory with group learning and its applications,” 2007 International Symposium on Nonlinear Theory and its Applications, no 1, pp 292–295, 2007 [6] A Yousuf and Y L Murphey, “A supervised fuzzy adaptive resonance theory with distributed weight update,” Springer-Verlag Berlin Heidelberg, vol Part I, LN, no 6063, pp 430–435, 2010 [7] P Pardhasaradhi, R P Kartheek, and C Srinivas, “A evolutionary fuzzy art computation for the document clustering,” International Conference on Computing and Control Engineering, 2012 [8] R Kondadadi and R Kozma, “A modified fuzzy art for soft document clustering,” Proceedings of the 2002 International Joint Conference on Neural Networks IJCNN’02, pp 2545–2549, 2002 [9] X.-H Song, P K Hopke, M Bruns, D a Bossio, and K M Scow, “A fuzzy adaptive resonance theorysupervised predictive mapping neural network applied to the classification of multivariate chemical data,” Chemometrics and Intelligent Laboratory Systems, vol 41, no 2, pp 161–170, 1998 [10] S Tomida, T Hanai, H Honda, and T Kobayashi, “Gene expression analysis using fuzzy art,” Genome Informatic, vol 12, pp 245–246, 2001 [11] R Kenaya and K C Cheok, “Euclidean art neural networks,” Proceedings of the World Congress on Engineering and Computer Science 2008, no 1, 2008 [12] H Isawa, H Matsushita, and Y Nishio, “Improved fuzzy adaptive resonance theory combining overlapped category in consideration of connections,” IEEE Workshop on Nonlinear Circuit Networks, pp 8– 11, 2008 [13] J Huang, M Georgiopoulos, and G L Heileman, “Fuzzy art properties,” Elsevier Science, Neural Network, vol 8, no 2, pp 203–213, 1995 [14] M Geogiopoulos, H Fernlund, G Bebis, and G Heileman, “Fart and fartmap-effects of the choice parameter,” pp 1541–1559, 1996 [15] G C Anagnostopoulos and M Georgiopoulos, “Category regions as new geometrical concepts in fuzzy-art and fuzzy-artmap,” Elsevier Science, Neural Network, vol 15, pp 1205–1221, 2002 [16] M Cano, Y Dimitriadis, E Gomez, and J Coronado, “Learning from noisy information in fasart and fasback neuro-fuzzy systems,” Elsevier Science, Neural Network, vol 14, pp 407–425, 2001 [17] T Burwick and F Joublin, “Optimal algorithmic complexity of fuzzy art,” Kluwer Academic Publisher-Neural Processing Letters, vol 7, pp 37–41, 1998 [18] I Dagher, M Georgiopoulos, G L Heileman, and G Bebis, “An ordering algorithm for pattern presentation in fuzzy artmap that tends to improve generalization performance.” IEEE transactions on neural networks, vol 10, no 4, pp 768–78, 1999 [19] K Kobayashi, S Mizuno, T Kuremoto, and M Obayashi, “A reinforcement learning system based on state space construction using fuzzy art,” Proceedings of SICE Annual Conference, vol 2005, no 1, pp 3653– 3658, 2005 [20] M Pacella, Q Semeraro, and A Anglani, “Adaptive resonance theorybased neural algorithms for manufacturing process quality contro,” International Journal of Production Research, no 21, pp 4581–4607 [21] L E Parker, “Classification with missing data in a wireless sensor network,” IEEE SoutheastCon 2008, pp 533–538, Apr 2008 this parameter is smaller We proposed our novel model with the new learning rule to compare to exiting models Moreover, we have conducted experiments with ten datasets to prove the effectiveness of our novel model Experiments show our novel model is the best with four datasets (Iris, Glass, Spiral, Flame), better than or equal to other models with four datasets (Aggregation, Wine, R15, Compound) However, Kmean is better than our models with two datasets (Pathbase, Jain) and complement Fuzzy ART with Jain dataset From data of experiments, we obtain two important conclusions, including (i) our novel model clusters correctly from 80% to 100% with formal small datasets that categories distribute uniformly and (ii) from 50% to 80% with formal small datasets that its categories distribute non-uniformly and consist of many categories #Records 50 % 100 % 150 % 200 % 250 % 300 % OriNewFART 99 99.0 199 99.5 289 96.3 353 94.6 277–373 230 222 [22] G Carpenter, S Grossberg, and D B Rosen, “Fuzzy art : Fast stable learning and categorization of analog patterns by an adaptive resonance system,” Pergamon Press-Neural network, vol 4, pp 759–771, 1991 [23] B.Moore, “Art and pattern clustering,” Proceedings of the 1988 Connectionist Models Summer School, Morgan Kaufmann Publishers, pp 174–1985, 1989 [24] J.B.MacQueen, “Some methods for classification and analysis of multivariate observations,” Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, no 1, pp 281–297, 1967 [25] “Uci database,” Avaliable at: http://archive.ics.uci.edu/ml/datasets [26] “Shape database,” Avaliable at: http://cs.joensuu.fi/sipu/datasets/ 231 223 ... effective of our novel models, including Original Fuzzy ART (OriFART), Complement Fuzzy ART (ComFART), Original New Fuzzy ART (OriNewFART), Complement New Fuzzy ART (ComNewFART), K-mean (Kmean),... distribution of categories is not uniform with categories Data from Table XXI shows Complement New Fuzzy ART are better than Original Fuzzy ART and Euclidean ART However, Complement New Fuzzy ART is a bit... model calculated the Euclidean distance and decides a new pattern in an existing category or a new category Isawa proposed Fuzzy ART combining overlapped categories in connections to void the category