Phân tích hồi quy bằng support vector machines (SVM)

BO• GIAO DUC VA DAO TAO • • TRUONG DAI HOC KINH TE TP.HO CHi MINH • • DE TAl NGHIEN ciru KHOA HOC CAP CO SO • ' PHAN TiCH HOI QUY BANG SUPPORT VECTOR MACHINES (SVM) MA SO: CS-2007-01 BQ GIAO DUC.DAOTAO- ,TRIJONG D/,11 HQC KINH r{rP.HCM THU'VItN r-l19c:- CN: ThS GVC HuYNH VAN DUC TP HO CHi MINH NAM 2009 I' BO• GIAO DUC VA DAO TAO • • TRUONG DAI HOC KINH TE TP.HO CHi MINH • • DE TAl NGHIEN Ciru KHOA HOC CAP CO SO • ' " , ;t, ~ PHAN TICH HOI QUY BANG SUPPORT VECTOR MACHINES (SVM) MA SO: CS-2007-01 CHU NHI¢M: THANH VrEN: ThS GVC HUYNH VAN DUC ThS GV NGUYEN CONG TRi TP HO CHi MINH NAM 2009 Ngay chung ta dang dung tru&c mot kh6i luQ'ng du li~u kh6ng 16 fin chua ben nhi~u quy Iuat cha dUQ'C kham pha Cung v&i S\1' phat triSn cua khoa hoc, S\1' hiSu bi~t cua chung ta vS nhiSu d6i tUQ'ng, S\1' vat dUQ'C dfiy du han va chi ti~t han M6i quan gifra cac y8u tfi theo d6 cang them phuc t~;tp Mot thuoc tinh c6 thS c6 m6i quan h~ v&i rAt nhiSu nhung thuoc tinh khac, ddn d8n du li~u quan sat duQ'c thuang c6 s6 chiSu rAt l&n lam cho cac phuang phap truySn th6ng gap nhiSu kh6 khan Sau thai ky hoang kim cua thJng ke rieng phdn (cac thap nien 1930- 1960, v&i phuang phap clfe dc,zi likelihood Fisher dS xuAt vao dfiu thap nien 1930, v6n lam vi~c rAt t6t v&i dfr lieu c6 s6 chiSu nh6), ngucri ta b~t dfiu quay l~;ti v&i thJng ke t6ng quat [1] Ngay Iap tuc mot nguyen ly chung dUQ'C chAp nhan rong rai, nguyen ly qrc tidu t6n thdt thlfc nghi~m (Empirical Risk Minimization- ERM) V&i dii li~u c6 s6 chiSu Ian, khong gian gia thi8t tra nen phuc t~;tp Lam th8 nao vira kiSm soat dUQ'C khong gian gia thi8t vira bao dam tinh vfrng cua cac u&c luQ'ng? Nguyen ly qrc tidu tbn thdt cdu true (Structural Risk Minimization - SRM) da duQ'c d~ xufit vao gifra thap nien 1970 nh~m thvc hi~n nguyen ly ERM c6 kiSm soat S\1' phuc t~;tp cua khong gian gia thi8t Sau d6 (1990), cac mo hinh Support Vector Machines (SVM) duQ'c gi&i thi~u nhu Ia mot phuang phap cai dat nguyen ly SRM Tu d6 d~n nay, cac thuat toan SVM da chung to duQ'c kha nang lam vi~c hi~u qua v&i dii li~u c6 s6 chiSu l&n Trang dS tai nay, chung toi gi&i thi~u mo hinh SVM nhu Ia mot(phuang phap h5i quy hi~u qua cho dfr li~u nhi~u chi~u c6 tinh phi tuy~n cao Trong khuon kh6 cita m9t dS tai cAp 00 sa, chung toi khong c6 tham vong l&n, khong dua bfit ky mot nghien cuu m&i hoac mot ung d1,mg thvc t8 hi~u qua nao ca Chung toi tap trung trinh bay mot each c6 M th6ng cac khai ni~m, cac bai toan va cac thuat toan huAn luy~n cho thAy SVM dang dS chung ta dfiu tu nghien Cll'U sau han vS n6 Ngmli chung toi cling da cai ti~n mot thuat toan huAn luy~n SVM, da trinh bay t~;ti Hoi thao Qu6c gia lfin thu Ill Nghien c(ru ca ban va ung d1,1ng Cong ngh~ thong tin nam 2007 (Hoi thao F AIR07), va xay dvng mot chuang trinh minh hoa Chung toi da dung chuang tiinh ch~;ty du li~u thvc t8 lAy mot dS tai nghien cuu cfip bo [20] tu CAu true cua dS tai g5m ba chuang va mot ph1,1 l1,1c - Chuang phac thao mot hue tranh toan canh, cling gi&i thi~u dong CO' nghien Cll'U Chuang chi ti8t vi~c xay dvng mo hinh - Chuang trinh bay mot thuat toan huAn luy~n chi ti8t d8n muc c6 thS cai dat duQ'c d~ dang - Phfin ph1,1l1,1c trinh bay cac k8t qua ch~;ty thvc nghi~m, bao g5m du li~u Ifiy tu [20] PHAN TiCH HOI QUY BANG SVM Toi xin g&i loi cam an chan thimh d~n Phong Qufm ly khoa hQc - HQ'p tcic quBc t~ da t~o di~u kien cho chung toi hoan tAt d~ tai nay; Cam an cac d6ng nghiep khoa Tin hQc quan ly, cac d6ng nghiep tu Khoa Cong ngM thong tin, d~i hQc Khoa hQc t1,r nhien TpHCM, da tham gia va dong g6p cac y ki~n quy bau cac bu6i seminar duQ'c t6 chuc cho d~ tai nhung d~ tai duQ'c thl,l'c hien ch~c ch~n nhi~u khi~m khuy~t Chung toi nghiem tuc d6n nhan cac g6p y gAn xa Du rAt n6 h,rc bam sat m1,1c tieu, Tp.H6 Chi Minh, 24/04/2009 Nh6m tac gia 11 Mucluc MO'diu i MIJC II}C iii ChU'O'Dg 1: D't vftn d~ Chwung 2: MO hinh SVM Mo hinh SVM tach tuydn tlnh Bai toan tach Mo hinh toan hoc Mo hinh chiu 16i 12 Mo hinh tach phi tuyin 15 Mo hinh hdi quy SVM 19 ' khAong g1an • glc:t 'At}'J Cau tnic uet 20 Mo hinh toan hoc 21 J ChU'O'Dg 3: Thu~t toan huftn luy~n SM0 25 Mota thuqt toan : 26 K 1em A J • ' dJ :t tra ti'nh to1 uu cua p huong an 01 ngau 26 Di~u chinh phuong an 27 Xay d\fng bang tinh toan 29 Minh hQa 29 Minh hQa trubng hQl> phi tuy~n 31 Thu~t toan SMO cua Platt [25] 32 Heuristic tim i 33 Heuristic ti1n j , 33 Thugt toan SMO cho biti toim hdi quy 34 Xay d\fng bang tinh toan '"""' 37 Minh hQa 38 K@t luij.n 41 Tai li~u tham khao ._ 43 Phi} I~.IC 1: Thl}'C nghi~m 47 Bai toan tach 47 Bai to an hdi quy 48 Bai toan thl,fc ti 49 Du lieu 49 K~t qua ch~y thir nghiem 50 Chi ml}c • .• • • • 52 Ill ChU'ong 1: D(it vftn d@ Bai toan suy luqn quy nqp da c6 tu han 2000 nam qua Tuy nhien mai d~n thS ky XVIII, mf>i lien he gifra nganh khoa h(JC thl!C nghi~m va CRC nganh khoa h(JC chinh Xac khac nhu toan, logic mai duqc d~t (D Hume va I Kant, bai toan phan bi~t - demarcation problem)[ 1] C6 th~ n6i S\1' phat tri&n cita khoa hrc va cu(jc each mqng v~ c6ng ngh~ thong tin th~ k)r XX da la ti8n d8 cho viec xu~t hi en cite y tuc:'Yng m6i suy luan th6ng ke M~c du cite ySu t6 cua suy Iuan th6ng ke da tan tl;li each day han thS ky, cite cong viec cua Gauss va Laplace, nhung n8n tang that S\1' cua ly thuySt chi dUQ'C b~t ddu vito cu6i thap nien 1920 thai di~m d6, cite th6ng ke mota hfiu nhu daddy du v6i nhi8u quy luqt th6ng ke cho phep mota t6t cite biSn c6 xay thS gi6i thuc Cling vao nhung nam 1920 nay, cite mo hinh ca cho ca hai tiSp can: thf>ng ke c6 di~n (con dUQ'C goi la th6ng ke tham s6) Ifin th6ng ke t6ng quat cling da hinh [1] Su phat tri~n cua khoa hoc hien d~;ti b~t ddu vito cu6i thS ky XIX da lam thay d6i su hi~u biSt cua chung ta v8 mo hinh t6ng quat cua thS giai thuc tu mo hinh mang tinh xac dinh sang mo hinh co tinh ngdu nhien Cite y tuc:'Yng mai c6 y nghla cho suy Iuan th6ng ke xu~t hien thai ky la cua Karl Popper, Glivenko, Cantelli, Andrei N Kolmogorov va Ronald A Fisher a sa [1] Karl Popper, vito nhung nam dfiu cua thap ky 1930, da xem xet bai toan quy n~;tp tU khia c~;tnh triSt hoc Nguyen ly phiin bi~t cua ong r~t t6ng quat, dua tren khai niem v8 kha nang sai (falsifiability) cua ly thuySt Lfin dfiu tien ong da lien kSt kha nang t6ng quat h6a v6i khai niem dung lut;mg (capacity) Cling vito nhung nam ddu cua thap ky 1930 nay, Andrei N Kolmogorov l~;ti xet bai toan quy n~;tp tu khia c~;tnh th6ng ke ly thuySt Cong viec cua ong dua vito hai k~t qua chinh: S\1' h()i 1\1 cua phan ph6i thuc nghiem dSn phan ph6i thuc (Glivenko va Cantelli, 1933) va t6c d() h()i 41 nhanh co ham mil va d()c tap v6i ph~n phf>i (Kolmogorov, 1933) Hai k~t qua la ca SO chfnh cua S\1' phat tri~n cua nguyen ly thJng ke tJng quat Cling thai ky nay, Ronald A Fisher da xet bai toan quy n~;tp tu khia c~;tnh thf>ng ke ung d1.mg Do ap luc cong viec luc b~y gia cAn c6 cite k~t qua tinh toan nhanh, dan gian va hieu qua, R Fisher da d8 nghi m()t ti~p can mang tinh rieng phdn, U'cYC llf(J11g cac tham sJ cua ham mat d() Ti~p can da chia khoa hoc thf>ng ke hai nhitnh thf>ng ke t6ng quat va th6ng ke ~ieng phdn, dUQ'C goi la th6ng ke tham s61 Trong luc mo hinh th6ng ke t6ng quat phat tri~n cham, thi mo hinh th6ng ke tham sf> l~;ti phat tri~n r~t nhanh B~t ddu tu thap nien 1930, chi vong 10 nam sau cite y~u tf> chinh cua mo hinh thf>ng Th~t ngfr dung cila n6 Ia th6ng ke parametric PHAN TiCH HOI QUY BANG SVM ke tham s6 da dtrQ'c dua Khoang thai gian tir 1930 dSn 1960 Ia thai Icy vang son cua tiSp can Cac gia thiSt chinh cua mo hinh th&ng ke tham sf> Ia [1]: D~ tim mot quan h~ phlJ thu(Jc ham tir dfr Ii~u, cac nha th6ng ke dinh nghla mot tap cac ham phl,l thuoc tham sf>, v&i sf> it cac tham s6 va tuy~n tinh theo tham s6; Lu~it th&ng ke cua phdn ng~u nhien, Ia sai s6 giua mo hinh va du li¢u thl!C, tuan thea Iuat phan ph&i chuAn; voi gia thiSt 2, phuang phap Cf!C dc;Ii likelihood Ia phuang phap t5t Ngay n6i dSn luQ'c dB cua Fisher nguai ta hay goi Ia th5ng ke c6 di~n Th5ng ke cfl di8n di giai ba bai toan: U'cYC lu()11g ham m(it dQ, U'cYC lw;mg hJi quy va U'cYC lu()11g ham phan bi¢t dung cac mo hinh tham sf> khac (Phuang phap Cf!C dqi likelihood, R.A.Fisher, 1930) v&i CO' sa toan vfrng ch~c (Mathematical Methods of Statistics, Harold Cramer, 1946) Mot each tflng quat, suy Iuan thf>ng ke di giai mot bai toan qt'c tidu phidm ham dva vao du Ii~u thvc nghi~m V&i each Ic\m rieng phdn cua Fisher, ly thuySt th5ng ke c6 di8n da khong xem xet mot each chi tiSt bai toan Cl,lC ti8u phiSm ham Ngoai ra, u&c luQ'ng ham gia tri thvc tir dfr li~u duQ'c xem nhu bai toan trung tam cua thf>ng ke trng d1,1ng Ky thuat chinh dtrQ'c sir dt,mg aday Ia phuang phap t6ng binh phUV11g be nhdt va phuang phap t6ng modul be nhdt dtrQ'c Gauss va Laplace dS xufit thai gian dai qua khu Tuy nhien nhfrng phan tich vS cac phuang phap chi m&i thvc hi~n thS ky XX Thea d6 thf>ng ke c6 di~n chu dSn cac u&c ltrQ'ng khong ch~ch Gia thiSt vS u&c luQ'ng khong ch~ch b~t ddu duQ'c xem xet4 sau James va Stein (1961) xay dvng mOt u&c ltrQ'ng ky vong cua mot vecta ng~u nhien (n ;::: 3) c6 phan ph&i chudn v&i rna tran tuang quan dan vi U'&c ltrQ'ng cMch va v&i kich thu&c quan sat c5 dinh u&c IUQ'ng dSu t6t han trung binh m~u (mot u&c luQ'ng khong ch~ch cua· ky vong) v~ sau Baranchik da dua mot tap cac u&c ltrQ'ng nhu vay, baa gBm u&c ltrQ'ng cua JamesStein Them vao d6, cac bai toan thvc tS, khong phai tfit ca sac gia thiSt cua mo hinh th6ng ke tham s6 d~u duQ'c thoa man Cac bai toan c6 sf> chi~u rfit Ion d~n dSn S\1 bung n6 tA hQ'p cua cac tham sf> Ngoai quy Iuat cua phdn ng~u nhien c6 th8 khong thea phan ph6i chudn (Tukey) va phuang phap eve d~i likelihood cling khong Ia phuang phap t6t nhfit (James va Stein) [1] Da c6 nhfrng c5 g~ng VUQ'tqua h~n chS nay: P Huber (1960) phat tri~n tiSp can robust cho phep Io~i gia thiSt phan ph5i chudn cua phdn ng~u nhien; Bill toan qrc ti~u phi~m ham da tn'l' tlllinh bai toan chinh lien quan d~n xAp xi ham va giai tich ham Trong s6 cac phuang pMp u&c luinh Iy lam phat sinh hai cau hoi: Tim lai giai t6i uu? Bai toan tach Ia each t6t nhAt d~ di~u khi~n SIJ t6ng quat? vs sau, vao nhfrng nam cua thap nien 1980, mot nhfrng bai toan n~n tang (bai toan Glivenko-Cantelli) da ddn d~n ly thuySt thdng ke tdng quat, dva vao du true cua ho cac khong gian gia thiJt IBng [1] Theo d6, ben c~nh chAt lUQ'ng cua xAp xi, tiSp can quan tam dSn SIJ phuc t~p cua cac khong gian gia thiSt Nhu vay viec ki~m soat cac khong gian gia thiSt Ia mot nhfrng cong Cl,l chinh cua tiSp can Lam thS nao ki~m soat duQ'c phuc t~p cua khong gian gia thiSt? Theo Iuat s6 Ion c6 di~n, t~n suAt cua mot biSn c6 se hoi tl,l dSn xac suAt xay biSn c6 m\y Tuy nhien voi mot ho cac biSn c6, SIJ h(Ji {1f aJu c6 dam bao hay khong thi khong ch~c Vi~c ki~m soat KhoaTHQL PHAN TiCH HOI QUY BANG SVM d9 phuc t~;tp cua khong gian gia thi~t c6 lien quan d~n ly thuy~t v~ sv h9i tl,l d~u C6 ba khai niem v~ d9 phuc t~;tp cua khong gian gia thi~t duqc d~ cap (xem [1 ], chuang 2) la d(j h6n d(m (Annealed Entropy), ham tang truong (Growth Function) va s6 chi~u VC (VC dimension) Ly thuy~t v~ Sl,l' h()i tl,l d~u da duqc xay dvng vao cu6i nam 1960 (Vapnik va Chervonenkis, 1968, 1971) v6i honda tang la ho cac khai niem dung luqng (capacity) cua tap cac ham chi thj (indicator functions, cac ham nhan gia tri ho~c 1) dUC)'C goi la sJ chiJu VC Nguyen ly eve ti~u ham l6i v6i s6 chi~u VC nho duqc goi la nguyen ly qrc tiJu thi~t hqi cdu true (Structural Rist Minimization- SRM) Su phat tri~n ti~p tl,lc cua nguyen ly da d~n d~n m9t lo~;ti thuat toan m6i duqc goi la may vectO' t~u5 (Support Vector Machines- SVM) [1, 2] Gi6ng v6i mo hinh perceptron, cac thuat toan SVM cung t6ng quat h6a tu viec giai bai toan tach tuy~n tinh Tu mo hinh perceptron va kha nang t6ng quat h6a cua n6 (F Rosenblatt, 1958), mo hinh m~;tng neuron nhan t~;to (Artificial Neural Network - ANN) da phat tri~n va c6 cac ung dl,lng hieu qua nhi~u linh vvc khac [3, 4, 5, 6, 7] Nhfrng gi m~;tng neuron lam duqc thi SVM cung hlm duqc, tham chi hieu qua han [2, 8, 9] Nhfrng cong cua cac mo hinh SVM khac da chung to kha nang cua lo~;ti thuat toan [8, 9, 10, 11] I>~c biet m9t di~u tra gfin day [9] (Xindong Wu, 2007) da x~p SVM n~m top 10 cac thuat toan khai khoang du li~u Ngay luqng du li~u tang gftp doi sau m6i 20 thang (Sever Hayri, 1998) Rftt nhi~u quy luat An chua ben kh6i luqng du li~u vo cling l6n d6 cfin duQ'c phat hi~n Llnh Vl,l'C Kinh t~ cung khong la ngo~;ti l~ I>i~u gi xay n~u m9t cong ty n~m b~t dUC)'C hanh vi cua khach hang? Ch~c ch~n m()t chi~n luqc kinh doanh hieu qua se dUO'C d~t Trong khoa hoc kinh t~, viec xu ly du li~u la cong vi~c h~t sue quan Nhi~u giai do~;tn qua trinh dua ramo hinh, ki~m dinh mo hinh d~u cfin phai xu ly du li~u v~ phuang di~n nao d6 nghien cuu kinh t~ c6 th~ d6ng nhAt v6i dfr li~u · Hai phuang phap chinh dung d~ phan tich du li~u duQ'c su dl,lng kinh t~ la phuang phap ky thuat va phuang phap ca ban [20] Phuang phap nao cung dva tren ca cua ly thuy~t xac xuAt Chung ta d~u bi~t, bai toan chinh cua ly thuy~t xitc suftt la nghien ClrU t6ng cua cac d~;ti luQ'ng ngftu nhien d()c lap c6 phuang sai d~u sa Dfr li~u ph1;1m vi h~p va ng~n h1;1n c6 th~ thoa man di~u ki~n nay, nhien v6i kh6i luQ'ng du li~u d6 s9 hi~n di~u d6 khong ch~c dung nfra V6i cac phuang phap n6i tren vftn d~ lAy m~u chinh xac anh huang l6n d~n k~t qua Lam th~ nao lfty mftu phu hO'P v&i vAn d~ nghien cuu ngfr canh nay? Them vao d6, cling v6i vi~c toan du hoa n~n kinh t~, nhi~u y~u t6 rAt m6i dang tac d()ng vao cac n~n kinh t~ Vai tro tac d()ng cua cMng dang An chua du li~u rna vi~c lAy mfiu khong chinh xac co th~ lam sai l~ch k~t qua phan tich Thi truong chung khoan tu lau da duqc xem la llnh vvc dfiu tu c6 lO'i nhuan cao Bai toan dl,l' bao gia chung khoan chiu anh huang bai tuang tac gifra cac lo~;ti hinh kinh t~, chinh sach, tham chi tam ly quan h~ rAt phuc t~;tp nen rftt kh6 khan dv bao C6 chung Mot s6 tai li~u ti~ng Vi~t dung thu~it ngfr may vecta h6 tr(Y DE TAl CAP Cd sd Chuang 1: DA TVAN DE cu cho riing (Yunos, Zaid, Jamaluddin, Shamsuddin, Sallehuddin, & Alwi, 2001) phan tich ky thuat khong c6 kha nang du bao chinh xac gia chung khoan GAn day ky thuat tinh toan m~m nhu Granular computing, Rough sets, Neural networks, Fuzzy sets, Genertic algorithms dUQ'C Slr dt,mg rong fiii d~ cai thi~n chinh xac CUa du baa cling nhu hi~u qua tinh toan t6t han so v6i phan tich ky thuat M~ng ncr ron da chung to tinh hi~u qua bai toan du baa gia chung khoan (Yoon & Swales, 1991 ), c6 kha nang giai rna tinh phi tuy~n cua du lieu, mo ta cac dl,ic trung cua thi truang chung khoan (Lapedes & Farber, 1987), du baa chi s6 thi truang (Chong & Kyoung, 1992.) (Freisleben, 1992), nhan d~ng cac miu cac bi~u d6 thuang m~i (Dutta & Shekhar, 1990), lai sufit cua trai phiSu cong ty, uac luqng gia Iua chn (Li, 1994)va chi baa mua ban (Chapman, 1994) (Margarita, 1992) Nhu cAu c6 them cac phuang phap va ky thuat mai viec xu ly dfr li~u cang Ian Nhi~u phuang phap va ky thuat khai pha dfr li~u d~ phat hi~n tri thuc da dang vase COn dUQ'C dua da chtrng to tinh hi~u qua CUa chung nhi~U l'inh VUC khac nhau, d6 c6 kinh t~ Cac phuang phap va ky thuat c6 th~ k~ dSn nhu: SVM, tim Iuat kSt hqp, ly thuy~t tap tho, Chung toi tim thfiy cac thuat toan SVM duqc xay dung dua tren nguyen ly SRM vai n~n tang toan hc vfrng ch~c Ngoai cac mo hinh SVM da duqc chung to tinh nang hi~u qua cua n6 so vai mo hinh m~ng ncrron nhan t~o va nhi~u mo hinh th6ng ke khac [21] Chung toi hy vng cac mo hinh SVM cung cfip them nhi~u cong Cl,l hi~u qua cho nhu du rfit Ian viec tim cac quan he ham tir dfr lieu linh vue kinh t~ hi~n GAndayc6rfitnhi~umohinhSVMduqcd~nghi [11, 12, 13, 14, 15, 16, 17, 18, 19] Tinh hi~u qua cua nhi~u mo hinh phAn nhi~u duqc thuy~t ph1,1c thong qua vi~c hc cac tap dfr li~u miu D~ c6 th~ tra lai cac cau hoi tren mot each thea dang chung ta cAn quay trcr l~i ly thuy~t va lam cac nghien cuu mang tinh CO' ban cao C6 nhu vay chung ta mai c6 CO' d~ dua mo hinh mai va ap dl,lng duqc n6 cac bai toan thuc t~ sa Mot each t\1' nhien c6 mot s6 cau hoi dl,it cho mot mo hinh SVM Cl,l th~ Ia: Mo hinh li~u c6 vfrng khong? _ Lam thB nao ki~m soat duqc cac khong gian gia thi~t If>ng nhau? Do phuc t~p cua thuat toan hufin luy~n cua mo hinh? Day Ia mot cong vi~c phuc t~p Trang ph~m vi cua mot d~ tai cfip CO' thuc hien mot sf> nghien cuu h~n ch~ M1,1c tieu dM cho d~ tai Ia: sa, chung toi chi Cac mo hinh SVM CO' ban Giai thieu thuat toan hufin luy~n nhanh Xay dung mot cai dl,it thl'r nghi~m Thong qua d~ tai chung toi mu6n giai thi~u mot lo~i m6 hinh cho bai toan hf>i quy ap d1,1ng cho bcU toan uac luqng quan h~ ham tir dfr li~u cua kinh t~ Dfr lieu ch~y thuc nghi~m duqc Ifiy tir mot d~ tai nghien cuu cfip bo (2007), d6 cac tac gia da dung mo hinh hf>i quy tuy~n tinh thea ti~p can th6ng ke tham sf> [20] Cac mo hinh SVM ca ban duqc trinh bay cac tai li~u [1, 2, 21, 22, 23, 24] Thuat toan hufin luy~n nhanh la thuat toan SMO [25, 26, 27, 28] duqc chn d~ trinh bay vi cac ly do: tu KhoalHQL PHAN TiCH HOI QUY BANG SVM Truong hqp TJ > 0, vi g kha vi t~i nhfrng gia tri khac v6'i m va M, tinh 1JA- (t g'().) = + 2c), ).6 Nhu An, M(H Cai Ti~n /:)'MO Tang Tdc Hudn Luy?n SVM, Bao cao t~i Hoi thao Quae gia IAn v~ Nghien Ctru Ca Ban va Ung Dt,mg Cong NgM Thong Tin, F AIR07, Tp Nha Trang, 2007 (dinh kern toan van) Khoa THQL 45 Ph1}11}c: Thvc nghi~m D1,Ul vao k& qua cua d€ tai chUng toi cai d~t mQt phdn m€m nho danh cho vi~c thu nghi~m Phl) ll)c khong giai thi~u ban thi~t k~ chuong trinh, chi dua mQt s6 k~t qua minh hQa Bai tmin tach Sau khoang 20 IAn l~p ta thu dugc k~t qua (bnng hinh anh) sau Problem c e Kernel -· ~ -·-·-] ,v ~Polynorminal -~ -~- :.:: Para [ Train J [ Step ,-~~-ii1so64Sci384615~ao· ~:3~soo406ao47337iioso6ss37asss2249o.455983n7a1 GJ[§~ [ill SVM Problem I ··-····-···-··· -·· -·· .·-.- ·1 Example2 v ·· -~ ~ ~ - c [~~~-~~ ~-= ~~·~-] e [O.~T~ ~~~~~~-~-~~] Kernel lii~~~~~-~~ ~~l Para Train J [ Step ~~~~y~~~~IO.~~~~i?~~j_s;~~!3.~Yi1~i~~?l~~~~~Ji~~f-~~j3f~5~!~~~~Il 47 PHAN TICH HOI QUY BANG SVM c e [~~~1-~ ~-~-~ ~-~:_] Kernel [i~-~~~~~-~lii-: ,3J Para [j~~~-~~~ -~ -~ ::.J Train J [ Q Q Q Q Step Bai toan hAi quy Q Q Q Q Q Q Q Q Q Q u'J Q SVM ,\ ; Sample [ReD;;-] Kemels Curve a:( -1.84616416324362 0.20530287732 ·: Normal Sf,1CJ Polynomial ·~· Gauss · Inverse ' • Tanh 48 0.2 :i 50 ( Train 0.1 0.1 59.2855 11.f = 14.2172615942263 DE TAl cAP Cd so Ph1,1l\lc 1: THl)'C NGHIBM Bang du li~u sau duQ'c Ifiy tu [2] v&i m1,1c tieu tim quan h~ ham gifra d~i luQ'ng PO ph1,1 thuoc vito cac d~i luQ"ng l~i Trong d8 tai chung tOi khong quan Him d~n y nghia cua cac d~i luQ'ng a 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 KhoaTHQL PO 10.64065 10.47729 10.28534 9.994242 9.729134 9.350102 9.680344 9.729134 9.648595 10.32548 10.35774 9.846917 9.011889 9.595603 9.680344 9.740969 9.609116 9.560997 9.433484 9.10498 9.220291 10.37661 10.37349 10.43705 11.14908 10.859 10.5321 10.03889 10.07744 10.20359 10.40123 10.17351 9.975808 9.918376 10.35774 9.717158 9.918376 10.35774 9.786954 10.43412 10.07744 9.717158 10.75577 10.52943 9.985068 10.33527 10.32548 9.740969 9.798127 9.55393 9.55393 P1 10.47729 10.28534 10.23996 9.729134 9.350102 9.661416 9.729134 9.648595 9.85744 10.35774 9.846917 9.830917 9.595603 9.680344 9.852194 9.609116 9.560997 9.798127 9.10498 9.220291 9.642123 10.37349 10.43705 10.6736 10.859 10.5321 10.57132 10.07744 10.20359 10.49681 10.17351 9.975808 9.971146 10.35774 9.717158 10.04759 10.35774 9.786954 10.08581 10.07744 9.717158 9.893437 10.52943 9.985068 10.33202 10.32548 9.740969 9.717158 9.55393 9.55393 9.581904 P2 10.28534 10.23996 10.30895 9.350102 9.661416 10.16585 9.648595 9.85744 10.12663 9.846917 9.830917 10.06476 9.680344 9.852194 9.994242 9.560997 9.798127 10.47729 9.220291 ·9.642123 10.25414 10.43705 10.6736 10.54534 10.5321 10.57132 10.65726 10.20359 10.49681 10.91509 9.975808 9.971146 9.975808 9.717158 10.04759 9.975808 9.786954 10.08581 10.41631 9.717158 9.893437 10.38282 9.985068 10.33202 10.43998 9.740969 9.717158 10.12663 9.55393 9.581904 10.08163 01 6.684612 7.495542 7.090077 7.090077 7.090077 6.214608 7.31322 7.31322 6.55108 7.31322 7.31322 6.55108 7.090077 7.090077 7.090077 7.130899 6.109248 6.29693 7.090077 5.703782 5.703782 7.37759 7.37759 6.907755 7.78324 7.78324 7.090077 7.600902 7.600902 7.090077 7.31322 7.783224 7.31322 7.595514 7.549609 6.214608 7.495542 7.495542 6.55108 7.244228 7.17012 5.703782 7.377759 7.377759 6.684612 7.377759 7.377759 6.39993 6.907755 6.745236 5.703782 02 7.495542 7.090077 7.783224 7.090077 6.214608 6.214608 7.31322 6.55108 7.34601 7.31322 6.55108 7.31322 7.090077 7.090077 7.090077 6.109248 6.29693 7.377759 5.703782 5.703782 6.907755 7.37759 6.907755 7.824046 7.78324 7.090077 8.070906 7.600902 7.090077 8.006368 Z83224 7.31322 7.244228 7.549609 6.214608 7.377759 7.495542 6.55108 7.495542 7.17012 5.703782 7.090077 7.377759 6.684612 7.377759 7.377759 6.39993 7.377759 6.745236 5.703782 6.802395 03 7.090077 7.783224 7.090077 6.214608 6.214608 7.090077 6.55108 7.34601 7.090077 6.55108 7.31322 7.16858 7.090077 7.090077 7.090077 6.29693 7.377759 7.377759 5.703782 6.907755 6.907755 6.907755 7.824046 8.073403 7.090077 8.070906 8.006368 7.090077 8.006368 7.783224 7.31322 7.244228 7.17012 6.214608 7.377759 7.377759 6.55108 7.495542 7.31322 5.703782 7.090077 7.31322 6.684612 7.377759 7.31322 6.39993 7.377759 6.802395 5.703782 6.802395 6.39693 49 PHAN TiCH HOI QUY BANG SVM 52 53 54 55 56 57 58 59 60 10.66896 10.37036 10.40729 10.22557 9.92818 9.933046 10.19242 9.775654 9.752665 10.37036 10.40729 10.49957 9.92818 9.933046 10.06476 9.775654 9.752665 9.825526 10.40729 10.49957 10.69422 9.933046 10.06476 10.29215 9.752665 9.825526 9.998798 7.31322 7.31322 7.003065 7.31322 7.31322 6.684612 6.39693 7.31322 6.39693 7.31322 7.003065 7.696213 7.31322 6.684612 7.495542 7.31322 6.39693 7.377759 7.003065 7.696213 7.495542 6.684612 7.495542 7.495542 6.39693 7.377759 7.090077 K~t qua ch~y thii· nghi~m Dung ham kernel Ia Gauss (hfii quy phi tuy€n vai s6 chi8u rAt cao), thu~t tmin h()i tl,l sau 198ldn ~p vai t6ng binh phuong sai s6 bfulg 0.18 ali SVM ·.:~,-.- - ·:~ -~- Sample c:: ,, · · 2C J Curve 'sr·.·10 Train ReDraw Kernels C' Normal Poi)'Tlomial '~' Gauss 0.2 ·~ · Inverse ;:, Tanh 50 50 0.1 0.1 198 129750 60 f = 6.09191374948641 I:JE TAI cAP co so Ph\ll\lc 3: TIEP c~ D6I NGAU Dung ham kernel hi tich vo huang (hbi quy tuyen tinh), thu~t toan h{)i tv sau 7936 ldn l?p v&i tAng binh phmmg sai s6 bing 4.17 u~ lq.IJ§ll~ SVM _,_:;r,,_,.t;·~k-.