WM WM trí tuệ nhân tạo cao hoàng trứ chương ter11 machine learning sinhvienzone com

m e co en Z on Machine Learning Si nh Vi Chapter 11 SinhVienZone.com https://fb.com/sinhvienzonevn e co m Machine Learning Si nh Vi en Z on •  What is learning? Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn e co m Machine Learning •  What is learning? en Z on •  “That is what learning is You suddenly understand something you've understood all your life, but in a new way.” Si nh Vi (Doris Lessing – 2007 Nobel Prize in Literature) Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn e co m Machine Learning •  Arthur Samuel (1959): en Z on "Field of study that gives computers the ability to learn without being explicitly programmed” Vi •  Tom Mitchell (1997): Si nh "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E” Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn e co m Machine Learning Si nh Vi en Z on •  How to construct programs that automatically improve with experience Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn e co m Machine Learning en Z •  Learning problem: on •  How to construct programs that automatically improve with experience Si nh Vi –  Task T –  Performance measure P –  Training experience E Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn e co m Machine Learning •  Chess game: en Z on –  Task T: playing chess games –  Performance measure P: percent of games won against opponents Si nh Vi –  Training experience E: playing practice games againts itself Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn e co m Machine Learning •  Handwriting recognition: en Z on –  Task T: recognizing and classifying handwritten words –  Performance measure P: percent of words correctly classified classifications Si nh Vi –  Training experience E: handwritten words with given Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn Example Experience GRAY? + + + + + + + + - - + + + - + + e co + - + + + + - (Mouse) + + - (Giraffe) + - + - (Dinosaur) + + - + + + - + ? en Z on + Vi Elephant + Si nh Prediction MAMMAL? LARGE? VEGETARIAN? WILD? m Example + + - + - + ? + + + - - ? Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn m Example Example Sky Sunny Warm Normal Sunny Warm High Rainy Cold Sunny Warm Humidity Wind Water Forecast EnjoySport Warm Same Yes Strong Warm Same Yes High Strong Warm Change No High Strong Cool Change Yes Low Weak Cold High Strong Warm Change ? Vi en Z on Strong Si nh Prediction AirTemp e co Experience Rainy Sunny Warm Normal Strong Warm Same ? Sunny Warm Low Strong Cool Same ? Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 10 15 November 2011 https://fb.com/sinhvienzonevn Decision Trees High e co Normal Sky on Yes en Z Sunny AirTemp Sunny Warm Sunny Warm Rainy Cold Sunny Humidity Wind Si nh Sky Vi Yes No m Humidity Rainy AirTemp No Water Forecast Enjoy Normal Strong Warm Same Yes High Strong Warm Same Yes High Strong Warm Change No Warm High Strong Cool Change Yes Cloudy Warm High Weak Cool Same Yes Cloudy Cold High Weak Cool Same No 57 May 3, 2014 SinhVienZone.com Cloudy Warm Cold Yes Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn No + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - Vi en Z + + + - - - Si nh A1 = v1 on + + + - - - e co m Decision Trees A2 = v2 58 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Homogenity of Examples Si nh Vi en Z on •  Entropy(S) = - p+log2p+ - p-log2p- 0.5 59 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn impurity measure Si nh Vi en Z on •  Entropy(S) = ∑i=1,c- pilog2pi e co m Homogenity of Examples 60 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Information Gain en Z on •  Gain(S, A) = Entropy(S) - ∑v∈Values(A)(|Sv|/|S|).Entropy(Sv) Si nh Vi A Sv1 Sv2 61 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Example •  Entropy(S) = - p+log2p+ - p-log2p- = - (4/6)log2(4/6) - (2/6)log2(2/6) on = 0.389 + 0.528 = 0.917 en Z •  Gain(S, Sky) = Entropy(S) - ∑v∈{Sunny, Rainy, Cloudy}(|Sv|/|S|)Entropy(Sv) Si nh Vi = Entropy(S) - [(3/6).Entropy(SSunny) + (1/6).Entropy(SRainy) + (2/6).Entropy(SCloudy)] = Entropy(S) - (2/6).Entropy(SCloudy) = Entropy(S) - (2/6)[- (1/2)log2(1/2) - (1/2)log2(1/2)] = 0.917 - 0.333 = 0.584 62 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Example •  Entropy(S) = - p+log2p+ - p-log2p- = - (4/6)log2(4/6) - (2/6)log2(2/6) on = 0.389 + 0.528 = 0.917 en Z •  Gain(S, Water) = Entropy(S) - ∑v∈{Warm, Cool}(|Sv|/|S|)Entropy(Sv) Si nh Vi = Entropy(S) - [(3/6).Entropy(SWarm) + (3/6).Entropy(SCool)] = Entropy(S) - (3/6).2.[- (2/3)log2(2/3) - (1/3)log2(1/3)] = Entropy(S) - 0.389 - 0.528 =0 63 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn Rainy on Sunny e co Sky m Example Yes en Z No Cloudy ? Si nh Vi •  Gain(SCloudy, AirTemp) = Entropy(SCloudy) - ∑v∈{Warm, Cold}(|Sv|/|S|)Entropy(Sv) =1 •  Gain(SCloudy, Humidity) = Entropy(SCloudy) - ∑v∈{Normal, High}(|Sv|/|S|)Entropy(Sv) =0 64 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Inductive Bias Si nh Vi en Z on •  Hypothesis space: complete! 65 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Inductive Bias •  Hypothesis space: complete! en Z on •  Shorter trees are preferred over larger trees Si nh Vi •  Prefer the simplest hypothesis that fits the data 66 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Inductive Bias en Z ⇒ Preference bias on •  Decision Tree algorithm: searches incompletely thru a complete hypothesis space Si nh Vi •  Cadidate-Elimination searches completely thru an incomplete hypothesis space ⇒ Restriction bias 67 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Overfitting Si nh Vi en Z on •  h∈H is said to overfit the training data if there exists h’∈H, such that h has smaller error than h’ over the training examples, but h’ has a smaller error than h over the entire distribution of instances 68 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Overfitting en Z on •  h∈H is said to overfit the training data if there exists h’∈H, such that h has smaller error than h’ over the training examples, but h’ has a smaller error than h over the entire distribution of instances: Vi –  There is noise in the data Si nh –  The number of training examples is too small to produce a representative sample of the target concept 69 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Overfitting tập kiểm tra Si nh Vi en Z on Lỗi tập huấn luyện học h học h’ 70 May 3, 2014 SinhVienZone.com Thời gian học Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn e co m Homework Exercises Si nh Vi en Z on 3-1→3.4 (Chapter 3, ML textbook) 71 May 3, 2014 SinhVienZone.com Cao Hoang Tru CSE Faculty - HCMUT https://fb.com/sinhvienzonevn ... m Machine Learning Si nh Vi en Z on •  What is learning? Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 15 November 2011 https://fb.com/sinhvienzonevn e co m Machine Learning •  What is learning? ... Training Data Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 11 15 November 2011 https://fb.com/sinhvienzonevn e co m Machine Learning Si nh Vi en Z on •  What is learning? Cao Hoang Tru... m Machine Learning en Z on •  What is learning? Learner Hypothesis Si nh Vi Experience Cao Hoang Tru CSE Faculty - HCMUT SinhVienZone.com 13 15 November 2011 https://fb.com/sinhvienzonevn m Machine