Bài giảng Khai phá dữ liệu: Chương 5 - TS. Võ Thị Ngọc Châu

Bài giảng Khai phá dữ liệu - Chương 5: Gom cụm dữ liệu cung cấp cho người học các kiến thức: Tổng quan về gom cụm dữ liệu, gom cụm dữ liệu bằng phân hoạch, gom cụm dữ liệu bằng phân cấp, gom cụm dữ liệu dựa trên mật độ,... Mời các bạn cùng tham khảo.

Khoa Khoa Học & Kỹ Thuật Máy Tính Trường Đại Học Bách Khoa Tp Hồ Chí Minh Chương 5: Gom cụm liệu Cao Học Ngành Khoa Học Máy Tính Giáo trình điện tử Biên soạn bởi: TS Võ Thị Ngọc Châu (chauvtn@cse.hcmut.edu.vn) Học kỳ – 2011-2012 1 Tài liệu tham khảo ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ [1] Jiawei Han, Micheline Kamber, “Data Mining: Concepts and Techniques”, Second Edition, Morgan Kaufmann Publishers, 2006 [2] David Hand, Heikki Mannila, Padhraic Smyth, “Principles of Data Mining”, MIT Press, 2001 [3] David L Olson, Dursun Delen, “Advanced Data Mining Techniques”, Springer-Verlag, 2008 [4] Graham J Williams, Simeon J Simoff, “Data Mining: Theory, Methodology, Techniques, and Applications”, Springer-Verlag, 2006 [5] Hillol Kargupta, Jiawei Han, Philip S Yu, Rajeev Motwani, and Vipin Kumar, “Next Generation of Data Mining”, Taylor & Francis Group, LLC, 2009 [6] Daniel T Larose, “Data mining methods and models”, John Wiley & Sons, Inc, 2006 [7] Ian H.Witten, Eibe Frank, “Data mining : practical machine learning tools and techniques”, Second Edition, Elsevier Inc, 2005 [8] Florent Messeglia, Pascal Poncelet & Maguelonne Teisseire, “Successes and new directions in data mining”, IGI Global, 2008 [9] Oded Maimon, Lior Rokach, “Data Mining and Knowledge Discovery Handbook”, Second Edition, Springer Science + Business Media, LLC 2005, 2010 2 Nội dung ‡ Chương 1: Tổng quan khai phá liệu ‡ Chương 2: Các vấn đề tiền xử lý liệu ‡ ‡ Chương 3: Hồi qui liệu Chương 4: Phân loại liệu ‡ Chương 5: Gom cụm liệu ‡ Chương 6: Luật kết hợp ‡ Chương 7: Khai phá liệu công nghệ sở liệu Chương 8: Ứng dụng khai phá liệu ‡ ‡ ‡ Chương 9: Các đề tài nghiên cứu khai phá liệu Chương 10: Ôn tập 3 Chương 5: Gom cụm liệu ‡ 5.1 Tổng quan gom cụm liệu ‡ 5.2 Gom cụm liệu phân hoạch ‡ 5.3 Gom cụm liệu phân cấp ‡ 5.4 Gom cụm liệu dựa mật độ ‡ 5.5 Gom cụm liệu dựa mơ hình ‡ 5.6 Các phương pháp gom cụm liệu khác ‡ 5.7 Tóm tắt 4 Chương 5: Gom cụm liệu Phần 5 5.0 Tình – Outlier detection Người sử dụng thẻ ID = 1234 thật chủ nhân thẻ tên trộm? 6 5.0 Tình - Làm liệu ‡ Nhận diện phần tử biên (outliers) giảm thiểu nhiễu (noisy data) „ Giải pháp giảm thiểu nhiễu ‡ Phân tích cụm (cluster analysis) 7 5.0 Tình 8 5.0 Tình 9 5.0 Tình 10 10 Self-Organizing Map (SOM) ‡ SOM architecture „ A typical SOM network has two layers of nodes, an input layer and output layer (sometimes called the Kohonen layer) „ Each node in the input layer is fully connected to nodes in the low (one, two, three-) dimensional output layer „ The number n of nodes in the input layer is corresponding to the number of input variables (i.e the length/dimensionality of an input vector x) „ Weights w Input x The number m of output nodes depends on the specific problem and is determined by the user ‡ This number of neurons in the rectangular array should be large enough to allow a sufficient number of clusters to form [9], part 21.3.3, pp 434 102 102 Self-Organizing Map (SOM) ‡ SOM architecture „ A winning neuron k: the neuron corresponding to weight wk (an nx1 vector) that has the minimum distance to the input x randomly selected in a training step „ The neighborhood Nk around a winning neuron k: the collection of all nodes with the same radial distance Fig 21.5 A 5x5 Kohonen Layer with two neighborhood sizes at radius of and [9], part 21.3.3, pp 435 103 103 Self-Organizing Map (SOM) ‡ Self-organizing (competitive, unsupervised) learning „ Neighboring cells in a neural network compete in their activities by means of mutual lateral interactions, and develop adaptively into specific detectors of different signal patterns „ The cells become specifically tuned to various input signal patterns or classes of patterns through the learning process „ Only one cell or local group of cells at a time gives the active response to the current input ‡ winner-take-all strategy 104 104 Self-Organizing Map (SOM) ‡ SOM learning procedure initialize the weights w to small random values and the neighborhood size large enough to cover half the nodes select an input pattern x randomly from the training set and present it to the network find the best matching or “winning” node k whose weight vector wk is closest to the current input vector x using the vector distance update the weights of nodes in the neighborhood of k using the Kohonen learning rule decrease the learning rate slightly repeat steps 1-5 with a number of cycles and then decrease the size of the neighborhood Repeat until weights are stabilized 105 105 Self-Organizing Map (SOM) ‡ The vector distance where ||.|| represents the Euclidean distance ‡ The Kohonen learning rule winew = wiold + αhik ( x − wi ) winew = wiold if i is in N k if i is not in N k (10) where α is the learning rate between and and hik is a neighborhood kernel centered on the winning node and can take Gaussian form as where ri and rk are positions of neurons i and k on the SOM grid and σ is the neighborhood radius 106 106 Self-Organizing Map (SOM) As the number of cycles of training (epochs) increases, better formation of the clusters can be found ‡ Eventually, the topological map is fine-tuned with finer distinctions of clusters within areas of the map ‡ After the network has been trained, it can be used as a visualization tool to examine the data structure ‡ Once clusters are identified, neurons in the map can be labeled to indicate their meaning ‡ „ Assignment of meaning usually requires knowledge on the data and specific application area 107 107 Self-Organizing Map (SOM) ‡ Notes on SOM „ Dimensionality (1D, 2D, 3D, ?) of the output layer „ The size of the map (the number of neurons of the output layer) ‡ The number of clusters „ Not restricted to any particular form of preprocessing „ Capable of noise handling „ Originally proposed for numeric data ‡ Extended versions for categorical data 108 108 Self-Organizing Map (SOM) ‡ SOM’s Applications „ market segmentation, „ customer targeting, „ business failure categorization, „ credit evaluation, „ document retrieval, „ group technology „ … 109 109 Self-Organizing Map (SOM) Source: Teuvo Kohonen, “The Self-Organizing Map,” Proceedings of the IEEE, vol 78, no 9, pp 1464-1480, September 1990 110 110 Self-Organizing Map (SOM) 111 111 Self-Organizing Map (SOM) Source: Teuvo Kohonen, “The Self-Organizing Map,” Proceedings of the IEEE, vol 78, no 9, pp 1464-1480, September 1990 112 112 Self-Organizing Map (SOM) ‡ Example on SOM „ The number of training vectors: „ The length of each training vector/weight: „ The number of clusters (output neurons): „ Neighborhood size: „ Learning rate: ‡ η(t) = 0.6;