Phân tích hồi quy bằng Support Vector Machines (SVM) Đề tài nghiên cứu khoa học cấp cơ sở

BO GIAO DUC VA DAO TAO • • • TRUONG DAI HOC KINH TE TP.HO CHi MINH • • DE TAl NGHIEN ciru KHOA HOC CAP CO SO • . ' . PHAN TiCH HOI QUY BANG SUPPORT VECTOR MACHINES (SVM) MA SO: CS-2007-01 BQ GIAO DUC.DAOTAO- . . ,- TRIJONG D/,11 HQC KINH r{rP.HCM THU'VItN I r-l19c:- ' CN: ThS. GVC HuYNH VAN DUC TP. HO CHi MINH NAM 2009 BO GIAO DUC VA DAO TAO • • • TRUONG DAI HOC KINH TE TP.HO CHi MINH • • DE TAl NGHIEN Ciru KHOA HOC CAP CO SO • ' " , ;t,. PHAN TICH HOI QUY BANG SUPPORT VECTOR MACHINES (SVM) MA SO: CS-2007-01 CHU NHI¢M: ThS. GVC HUYNH VAN DUC THANH VrEN: ThS. GV NGUYEN CONG TRi TP. HO CHi MINH NAM 2009 Ngay nay chung ta dang dung tru&c mot kh6i luQ'ng du kh6ng 16 fin chua ben trong quy Iuat cha dUQ'C kham pha. Cung v&i S\1' phat triSn cua khoa hoc, S\1' hiSu cua chung ta vS nhiSu d6i tUQ'ng, S\1' vat dUQ'C dfiy du han va chi han. M6i quan gifra cac y8u tfi theo d6 cang them phuc Mot thuoc tinh c6 thS c6 m6i quan v&i rAt nhiSu nhung thuoc tinh khac, ddn d8n du quan sat duQ'c thuang c6 s6 chiSu rAt l&n lam cho cac phuang phap truySn th6ng gap nhiSu kh6 khan. Sau thai ky hoang kim cua thJng ke rieng phdn (cac thap nien 1930- 1960, v&i phuang phap clfe dc,zi likelihood do Fisher dS xuAt vao dfiu thap nien 1930, v6n lam rAt t6t v&i dfr lieu c6 s6 chiSu nh6), ngucri ta dfiu quay v&i thJng ke t6ng quat [1]. Ngay Iap tuc mot nguyen ly chung dUQ'C chAp nhan rong rai, nguyen ly qrc tidu t6n thdt thlfc (Empirical Risk Minimization- ERM). V&i dii c6 s6 chiSu Ian, khong gian gia thi8t tra nen phuc Lam th8 nao vira kiSm soat dUQ'C khong gian gia thi8t vira bao dam tinh vfrng cua cac u&c luQ'ng? Nguyen ly qrc tidu tbn thdt cdu true (Structural Risk Minimization - SRM) da duQ'c xufit vao gifra thap nien 1970 thvc nguyen ly ERM c6 kiSm soat S\1' phuc cua khong gian gia thi8t. Sau d6 (1990), cac mo hinh Support Vector Machines (SVM) duQ'c gi&i nhu Ia mot phuang phap cai dat nguyen ly SRM. Tu d6 nay, cac thuat toan SVM da chung to duQ'c kha nang lam qua v&i dii c6 s6 chiSu l&n. Trang dS tai nay, chung toi gi&i mo hinh SVM nhu Ia mot(phuang phap h5i quy qua cho dfr c6 tinh phi cao. Trong khuon kh6 cita m9t dS tai cAp 00 sa, chung toi khong c6 tham vong l&n, khong dua ra bfit ky mot nghien cuu m&i hoac mot ung d1,mg thvc t8 qua nao ca. Chung toi tap trung trinh bay mot each c6 M th6ng cac khai cac bai toan va cac thuat toan huAn cho thAy SVM dang dS chung ta dfiu tu nghien Cll'U sau han vS n6. Ngmli ra chung toi cling da cai mot thuat toan huAn SVM, da trinh bay Hoi thao Qu6c gia lfin thu Ill Nghien c(ru ca ban va ung d1,1ng Cong thong tin nam 2007 (Hoi thao F AIR07), va xay dvng mot chuang trinh minh hoa. Chung toi da dung chuang tiinh nay du thvc t8 lAy tu mot dS tai nghien cuu cfip bo [20]. CAu true cua dS tai g5m ba chuang va mot ph1,1 l1,1c. - Chuang 1 phac thao mot hue tranh toan canh, cling gi&i dong CO' nghien Cll'U. - Chuang 2 chi ti8t xay dvng mo hinh. - Chuang 3 trinh bay mot thuat toan huAn chi ti8t d8n muc c6 thS cai dat duQ'c dang. - Phfin ph1,1l1,1c trinh bay cac k8t qua thvc bao g5m du Ifiy tu [20]. PHAN TiCH HOI QUY BANG SVM Toi xin g&i loi cam an chan thimh Phong Qufm ly khoa hQc - HQ'p tcic quBc da kien cho chung toi hoan tAt tai nay; Cam an cac d6ng nghiep trong khoa Tin hQc quan ly, cac d6ng nghiep tu Khoa Cong ngM thong tin, hQc Khoa hQc t1,r nhien TpHCM, da tham gia va dong g6p cac y quy bau trong cac bu6i seminar duQ'c t6 chuc cho tai nay. Du rAt n6 h,rc bam sat m1,1c tieu, nhung tai duQ'c thl,l'c hien con Chung toi nghiem tuc d6n nhan cac g6p y gAn xa. 11 Tp.H6 Chi Minh, ngay 24/04/2009 Nh6m tac gia Mucluc . . MO'diu i MIJC II}C iii ChU'O'Dg 1: D't vftn 1 Chwung 2: MO hinh SVM 7 1. Mo hinh SVM tach tuydn tlnh 9 Bai toan tach 9 Mo hinh toan hoc 9 Mo hinh chiu 16i 12 2. Mo hinh tach phi tuyin 15 3. Mo hinh hdi quy SVM. 19 C J. ' khA • 'At}'J. 20 au tnic ong g1an glc:t uet . Mo hinh toan hoc 21 ChU'O'Dg 3: toan huftn SM0 25 1. Mota thuqt toan : 26 K A ti'nh J • h ' dJ :t 26 1em tra to1 uu cua p uong an 01 ngau . chinh phuong an 27 Xay d\fng bang tinh toan 29 Minh hQa 29 Minh hQa trubng hQl> phi 31 toan SMO cua Platt [25]. 32 Heuristic tim i 33 Heuristic ti1n j , 3 3 2. Thugt toan SMO cho biti toim hdi quy 34 Xay d\fng bang tinh toan '"""' 37 Minh hQa 38 K@t luij.n 41 Tai tham khao _ 43 Phi} 1: Thl}'C 47 1. Bai toan tach 47 2. Bai to an hdi quy 48 3. Bai toan thl,fc ti. 49 Du lieu 49 qua thir nghiem 50 Chi ml}c • • • • • 52 Ill ChU'ong 1: D(it vftn d@ Bai toan suy luqn quy nqp da c6 tu han 2000 nam qua. Tuy nhien mai thS ky XVIII, mf>i lien he gifra nganh khoa h(JC thl!C va CRC nganh khoa h(JC chinh Xac khac nhu toan, logic mai duqc ra (D. Hume va I. Kant, bai toan phan - demarcation problem)[ 1]. C6 n6i S\1' phat tri&n cita khoa hrc va cu(jc each mqng c6ng thong tin trong k)r XX da la ti8n d8 cho viec hi en cite y tuc:'Yng m6i trong suy luan th6ng ke. du cite ySu t6 cua suy Iuan th6ng ke da tan tl;li each day han 2 thS ky, trong cite cong viec cua Gauss va Laplace, nhung n8n tang that S\1' cua ly thuySt chi dUQ'C ddu vito cu6i thap nien 1920. a thai d6, cite th6ng ke mota hfiu nhu daddy du v6i nhi8u quy luqt th6ng ke cho phep mota t6t cite biSn c6 xay ra trong thS gi6i thuc. Cling vao nhung nam 1920 nay, cite mo hinh ca sa cho ca hai tiSp can: thf>ng ke c6 (con dUQ'C goi la th6ng ke tham s6) Ifin th6ng ke t6ng quat cling da hinh thanh [1]. Su phat cua khoa hoc hien ddu vito cu6i thS ky XIX da lam thay d6i su biSt cua chung ta v8 mo hinh t6ng quat cua thS giai thuc tu mo hinh mang tinh xac dinh sang mo hinh co tinh ngdu nhien. Cite y tuc:'Yng mai c6 y nghla cho suy Iuan th6ng ke hien trong thai ky nay la cua Karl Popper, Glivenko, Cantelli, Andrei N. Kolmogorov va Ronald A. Fisher [1]. Karl Popper, vito nhung nam dfiu cua thap ky 1930, da xem xet bai toan quy tU khia triSt hoc. Nguyen ly phiin cua ong t6ng quat, dua tren khai niem v8 kha nang sai (falsifiability) cua ly thuySt. Lfin dfiu tien ong da lien kSt kha nang t6ng quat h6a v6i khai niem dung lut;mg (capacity). Cling vito nhung nam ddu cua thap ky 1930 nay, Andrei N. Kolmogorov xet bai toan quy tu khia th6ng ke ly thuySt. Cong viec cua ong dua vito hai qua chinh: S\1' h()i 1\1 cua phan ph6i thuc nghiem dSn phan ph6i thuc (Glivenko va Cantelli, 1933) va t6c d() h()i 41 nay nhanh co ham mil va d()c tap v6i phf>i (Kolmogorov, 1933). Hai qua nay la ca SO chfnh cua S\1' phat cua nguyen ly thJng ke tJng quat. Cling trong thai ky nay, Ronald A. Fisher da xet bai toan quy tu khia thf>ng ke ung d1.mg. Do ap luc cong viec luc gia cAn c6 cite qua tinh toan nhanh, dan gian va hieu qua, R. Fisher da d8 nghi m()t can mang tinh rieng phdn, U'cYC llf(J11g cac tham sJ cua ham mat d(). can nay da chia khoa hoc thf>ng ke thanh hai nhitnh thf>ng ke t6ng quat va th6ng ke phdn, con dUQ'C goi la th6ng ke tham s6 1 . Trong luc mo hinh th6ng ke t6ng quat phat cham, thi mo hinh th6ng ke tham sf> phat nhanh. ddu tu thap nien 1930, chi trong vong 10 nam sau cite tf> chinh cua mo hinh thf>ng 1 ngfr dung cila n6 Ia th6ng ke parametric. PHAN TiCH HOI QUY BANG SVM ke tham s6 da dtrQ'c dua ra. Khoang thai gian tir 1930 dSn 1960 Ia thai Icy vang son cua tiSp can nay. Cac gia thiSt chinh cua mo hinh th&ng ke tham sf> Ia [1]: 1. tim mot quan phlJ thu(Jc ham tir dfr cac nha th6ng ke dinh nghla mot tap cac ham phl,l thuoc tham sf>, v&i sf> it cac tham s6 va tinh theo tham s6; 2. th&ng ke cua thanh phdn nhien, Ia sai s6 giua mo hinh va du li¢u thl!C, tuan thea Iuat phan ph&i chuAn; 3. v oi gia thiSt 2, phuang phap Cf!C dc;Ii likelihood Ia phuang phap t5t. Ngay nay khi n6i dSn luQ'c dB cua Fisher nguai ta hay goi Ia th5ng ke c6 Th5ng ke cfl di8n di giai ba bai toan: U'cYC lu()11g ham m(it dQ, U'cYC lw;mg hJi quy va U'cYC lu()11g ham phan bi¢t dung cac mo hinh tham sf> khac nhau (Phuang phap Cf!C dqi likelihood, R.A.Fisher, 1930) v&i CO' sa toan vfrng (Mathematical Methods of Statistics, Harold Cramer, 1946). Mot each tflng quat, suy Iuan thf>ng ke di giai mot bai toan qt'c tidu phidm ham dva vao du thvc V&i each Ic\m rieng phdn cua Fisher, ly thuySt th5ng ke c6 di8n da khong xem xet mot each chi tiSt bai toan Cl,lC ti8u phiSm ham nay 2 . Ngoai ra, u&c luQ'ng ham gia tri thvc tir dfr duQ'c xem nhu bai toan trung tam cua thf>ng ke trng d1,1ng. Ky thuat chinh dtrQ'c sir dt,mg a day Ia phuang phap t6ng binh phUV11g be nhdt va phuang phap t6ng modul be nhdt dtrQ'c Gauss va Laplace dS xufit trong thai gian dai trong qua khu. Tuy nhien nhfrng phan tich vS cac phuang phap nay chi m&i thvc trong thS ky XX. Thea d6 thf>ng ke c6 chu trong dSn cac u&c ltrQ'ng khong Gia thiSt vS u&c luQ'ng khong ddu duQ'c xem xet 4 sau khi James va Stein (1961) xay dvng mOt u&c ltrQ'ng ky vong cua mot vecta nhien (n ;::: 3) c6 phan ph&i chudn v&i rna tran tuang quan dan vi. U'&c ltrQ'ng nay cMch va v&i kich thu&c quan sat c5 dinh u&c IUQ'ng nay dSu t6t han trung binh (mot u&c luQ'ng khong cua· ky vong). sau Baranchik da dua ra mot tap cac u&c ltrQ'ng nhu vay, baa gBm u&c ltrQ'ng cua James- Stein. Them vao d6, trong cac bai toan thvc tS, khong phai tfit ca sac gia thiSt cua mo hinh th6ng ke tham s6 duQ'c thoa man. Cac bai toan ngay nay c6 sf> rfit Ion dSn S\1 bung n6 tA hQ'p cua cac tham sf>. Ngoai ra quy Iuat cua thanh phdn nhien c6 th8 khong thea phan ph6i chudn (Tukey) va phuang phap eve likelihood cling khong Ia phuang phap t6t nhfit (James va Stein) [1]. Da c6 nhfrng c5 VUQ'tqua chS nay: 1. P. Huber (1960) phat tiSp can robust cho phep gia thiSt phan ph5i chudn cua thanh phdn nhien; 2 Bill toan qrc ham da tn'l' tlllinh bai toan chinh lien quan xAp xi ham va giai tich ham. 3 Trong s6 cac phuang pMp u&c lu<;Yng kh6ng ch?ch thi phuang pMp t6ng binh phuang be nhAt Ia phuang fhaP c6 phuang sai be nhAt. Vao nhfrng nc1m 1960 ly cac bill toan y6u (ill-posed problems) dua ra mot phuang phap xay d\l'ng cac u&c luQ'ng cMch. sau y tu&ng nay duQ'c dung cho bill toan u&c Im;mg h6i quy cua Iy hoc thflng ke. Thflng ke c6 hung vao bai toan I \fa chon mo hinh. 2 BE TAl CAP CO sd Chuang 1: DA TV AN DE 2. J. Nedler (1970) xuAt mo hinh tuySn tinh t6ng quat cho phep chon mo hinh t6t nhAt; 3. L. Breiman, P. Huber va J. Friedman xet ham phi tuySn theo tham s6 va dung phuang phap ClfC tiJu ham thi¢t hqi thlfc nghi¢m (Empirical Risk Minimization- ERM) thay cho eve likelihood. Cuoc each cong nghe thong tin 50 nam sau d6 da tac dong to Ion dai s6ng, rna ra cac CO' hoi moi cho phep c6 sang trong cac cong viec hang ngay. Trong th6ng ke c6 s6 tham s6 cua mo hinh Ia nho do d6 kSt qua cua n6 chi gioi trong cac hiun c6 s6 nho. Ngay sau khi cuoc each cong nghe thong tin cung dp cac ca hoi uoc luQ'ng cac ham voi s6 chiSu Ion, nguai ta xem xet biSu d6 cua Fisher va quay th6ng ke t6ng quat. I>a c6 c6 trong giai bai toan voi s6 chiSu Ion. Truoc nam 1970 can chinh cho bai toan uoc luQ'ng hBi quy chiSu Ia phuang phap t6ng binh phuang be nhAt va phuang phap t6ng modul be nhAt voi cac ham tuySn tinh theo tham s6. Trong cac nam cua thap nien 1970 cac ham tuySn tinh t6ng quat duQ'c dung voi hy vong tim dUQ'C s6 nho cac ham CO' sa. Thap nien 1980-1990 hien phuang phap tl,l diSn, voi s6 Ion cac ham cho truoc, dung du lieu xac dinh mot s6 nho cac ham va uoc luQ'ng cac he s6. Phuong phap nay gBm Projection Pursuit (Friedman va Stuetzle (1981), Huber (1985)); MARS (Multivariate Adaptive Regression Spline) (see Friedman (1991)) rAt thu hut va tra thanh cong Cl,l chinh trong phan tich nhiSu chiSu. Tra hti can t6ng quat da bi quen lang trong su6t 20 nam. Nam 1958 F. Rosenblatt, mot nha sinh ly hoc, dadS mo hinh perceptron cho bai toan tach tuySn tinh va co ths t6ng quat h6a dUQ'C. Mo hinh perceptron phan anh SIJ sinh ly hoc kinh c6 trong CO' chS hoc nhu Ia SIJ tuang tac gifra s6 Ion cac tac nhan dan gian (mo hinh naron cua McCulloch-Fitts). Ngay Iap tuc mot nguyen ly chung duQ'c chAp nhan, chinh Ia nguyen ly ERM. Sau d6 ly thuySt ERM cho bai toan nhan mdu da duQ'c xay dvng vao cu6i nam 1960. Nam 1963, Novikoff dua ra dinh ly SIJ hoi tl,l cua thuat toan perceptron (Hoi nghi hoc may Vien Khoa hoc diSu Moscow) c6 anh huang dSn nhfrng nguai tham dv. Dung may tinh va cac thuat toan dan gian chu6c each lam cua con nguai, dong vat va w nhien giai quySt bai toan. I>inh Iy nay lam phat sinh hai cau hoi: 1. Tim lai giai t6i uu? 2. Bai toan tach Ia each t6t nhAt SIJ t6ng quat? vs sau, vao nhfrng nam cua thap nien 1980, mot trong nhfrng bai toan tang (bai toan Glivenko-Cantelli) da ddn ly thuySt thdng ke tdng quat, dva vao du true cua ho cac khong gian gia thiJt IBng nhau [1]. Theo d6, ben chAt lUQ'ng cua xAp xi, tiSp can nay con quan tam dSn SIJ phuc cua cac khong gian gia thiSt. Nhu vay viec soat cac khong gian gia thiSt Ia mot trong nhfrng cong Cl,l chinh cua tiSp can nay. Lam thS nao soat duQ'c do phuc cua khong gian gia thiSt? Theo Iuat s6 Ion c6 suAt cua mot biSn c6 se hoi tl,l dSn xac suAt xay ra biSn c6 m\y. Tuy nhien voi mot ho cac biSn c6, SIJ h(Ji {1f aJu c6 dam bao hay khong thi khong soat KhoaTHQL 3 PHAN TiCH HOI QUY BANG SVM d9 phuc cua khong gian gia c6 lien quan ly sv h9i tl,l C6 ba khai niem d9 phuc cua khong gian gia duqc cap (xem [1 ], chuang 2) la d(j h6n d(m (Annealed Entropy), ham tang truong (Growth Function) va s6 VC (VC dimension). Ly Sl,l' h()i tl,l da duqc xay dvng vao cu6i nam 1960 (Vapnik va Chervonenkis, 1968, 1971) v6i honda tang la ho cac khai niem dung luqng (capacity) cua tap cac ham chi thj (indicator functions, cac ham nhan gia tri 0 1) con dUC)'C goi la sJ chiJu VC. Nguyen ly eve ham l6i v6i s6 VC nho duqc goi la nguyen ly qrc tiJu hqi cdu true (Structural Rist Minimization- SRM) Su phat tl,lc cua nguyen ly nay da m9t thuat toan m6i duqc goi la may vectO' (Support Vector Machines- SVM) [1, 2]. Gi6ng v6i mo hinh perceptron, cac thuat toan SVM cung t6ng quat h6a tu viec giai bai toan tach tinh. Tu mo hinh perceptron va kha nang t6ng quat h6a cua n6 (F. Rosenblatt, 1958), mo hinh neuron nhan (Artificial Neural Network - ANN) da phat va c6 cac ung dl,lng hieu qua trong linh vvc khac nhau [3, 4, 5, 6, 7]. Nhfrng gi neuron lam duqc thi SVM cung hlm duqc, tham chi con hieu qua han [2, 8, 9]. Nhfrng thanh cong cua cac mo hinh SVM khac nhau da chung to kha nang cua thuat toan nay [8, 9, 10, 11]. biet trong m9t tra gfin day [9] (Xindong Wu, 2007) da SVM trong top 10 cac thuat toan khai khoang du Ngay nay luqng du tang gftp doi sau m6i 20 thang (Sever Hayri, 1998). Rftt quy luat An chua ben trong kh6i luqng du vo cling l6n d6 cfin duQ'c phat Llnh Vl,l'C Kinh cung khong la gi xay ra m9t cong ty dUC)'C hanh vi cua khach hang? m()t luqc kinh doanh hieu qua se dUO'C ra. Trong khoa hoc kinh viec xu ly du la cong sue quan trong. giai trong qua trinh dua ramo hinh, dinh mo hinh cfin phai xu ly du phuang nao d6 nghien cuu trong kinh c6 d6ng nhAt v6i dfr · Hai phuang phap chinh dung phan tich du duQ'c su dl,lng trong kinh la phuang phap ky thuat va phuang phap ca ban [20]. Phuang phap nao cung dva tren ca sa cua ly xac xuAt. Chung ta bai toan chinh cua ly xitc suftt la nghien ClrU t6ng cua cac luQ'ng ngftu nhien d()c lap c6 phuang sai Dfr trong ph1;1m vi va h1;1n c6 thoa man nay, tuy nhien v6i kh6i luQ'ng du d6 s9 nay d6 khong con dung nfra. V6i cac phuang phap n6i tren vftn lAy chinh xac anh huang l6n qua. Lam nao lfty mftu phu hO'P v&i vAn nghien cuu trong ngfr canh nay? Them vao d6, cling v6i toan du hoa kinh t6 rAt m6i dang tac d()ng vao cac kinh Vai tro tac d()ng cua cMng dang An chua trong du rna lAy mfiu khong chinh xac co lam sai qua phan tich. Thi truong chung khoan tu lau da duqc xem la llnh vvc dfiu tu c6 lO'i nhuan cao. Bai toan dl,l' bao gia chung khoan chiu anh huang bai tuang tac gifra cac hinh kinh chinh sach, tham chi tam ly trong quan rAt phuc nen rftt kh6 khan trong dv bao. C6 chung 5 Mot s6 tai dung ngfr may vecta h6 tr(Y. 4 DE TAl CAP Cd sd Chuang 1: DA TV AN DE cu cho riing (Yunos, Zaid, Jamaluddin, Shamsuddin, Sallehuddin, & Alwi, 2001) phan tich ky thuat khong c6 kha nang du bao chinh xac gia chung khoan. GAn day ky thuat tinh toan nhu Granular computing, Rough sets, Neural networks, Fuzzy sets, Genertic algorithms dUQ'C Slr dt,mg rong fiii cai do chinh xac CUa du baa cling nhu qua tinh toan t6t han so v6i phan tich ky thuat. ncr ron da chung to tinh qua trong bai toan du baa gia chung khoan (Yoon & Swales, 1991 ), c6 kha nang giai rna tinh phi cua du lieu, mo ta cac dl,ic trung cua thi truang chung khoan (Lapedes & Farber, 1987), du baa chi s6 thi truang (Chong & Kyoung, 1992.) (Freisleben, 1992), nhan cac miu trong cac d6 thuang (Dutta & Shekhar, 1990), lai sufit cua trai phiSu cong ty, uac luqng gia Iua ch<;>n (Li, 1994)va chi baa mua ban (Chapman, 1994) (Margarita, 1992). Nhu cAu c6 them cac phuang phap va ky thuat mai trong viec xu ly dfr ngay cang Ian. phuang phap va ky thuat khai pha dfr phat tri thuc da dang vase COn dUQ'C dua ra da chtrng to tinh qua CUa chung trong l'inh VUC khac nhau, trong d6 c6 kinh Cac phuang phap va ky thuat c6 dSn nhu: SVM, tim Iuat kSt hqp, ly tap tho, Chung toi tim thfiy cac thuat toan SVM duqc xay dung dua tren nguyen ly SRM vai tang toan h<;>c vfrng Ngoai ra cac mo hinh SVM da duqc chung to tinh nang qua cua n6 so vai mo hinh ncrron nhan va mo hinh th6ng ke khac [21]. Chung toi hy v<;>ng cac mo hinh SVM cung cfip them cong Cl,l qua cho nhu du rfit Ian trong viec tim cac quan he ham tir dfr lieu trong linh vue kinh nay. [11, 12, 13, 14, 15, 16, 17, 18, 19]. Tinh qua cua mo hinh phAn duqc ph1,1c thong qua h<;>c cac tap dfr miu. c6 tra lai cac cau hoi tren mot each thea dang chung ta cAn quay trcr ly va lam cac nghien cuu mang tinh CO' ban cao. C6 nhu vay chung ta mai c6 CO' sa dua ra mo hinh mai va ap dl,lng duqc n6 trong cac bai toan thuc Mot each t\1' nhien c6 mot s6 cau hoi dl,it ra cho mot mo hinh SVM Cl,l Ia: 1. Mo hinh nay c6 vfrng khong? _ 2. Lam thB nao soat duqc cac khong gian gia If>ng nhau? 3. Do phuc cua thuat toan hufin cua mo hinh? Day Ia mot cong phuc Trang vi cua mot tai cfip CO' sa, chung toi chi thuc hien mot sf> nghien cuu M1,1c tieu dM ra cho tai nay Ia: 1. Cac mo hinh SVM CO' ban 2. Giai thieu thuat toan hufin nhanh 3. Xay dung mot cai dl,it thl'r Thong qua tai chung toi mu6n giai mot m6 hinh cho bai toan hf>i quy ap d1,1ng cho bcU toan uac luqng quan ham tir dfr cua kinh Dfr lieu thuc duqc Ifiy tir mot tai nghien cuu cfip bo (2007), trong d6 cac tac gia da dung mo hinh hf>i quy tinh thea can th6ng ke tham sf> [20]. Cac mo hinh SVM ca ban duqc trinh bay tu cac tai [1, 2, 21, 22, 23, 24]. Thuat toan hufin nhanh la thuat toan SMO [25, 26, 27, 28] duqc ch<;>n trinh bay vi cac ly do: KhoalHQL 5 [...]... cang kh&p v&i thvc Trong bai toan tach tinh, lai giai (sieu tach) c6 thuang khong duy nhat can may vecta t11cl (Support Vector Machines- SVM) dua ra mot tieu chuAn t5i uu cho phep chi ra lai giai t5t nhat trong s5 cac lai giai kha di biet can.nay xac dinh sieu tach qua cac vecta t11a (Support Vector) thay vi mot phucrng trinh tuang minh V&i cac vectcr tva chung ta c6 phep thay d6i khong gian bai toan... xr w - y + be ;:::: -t:e - z+ -xrw + y- be;:::: -t:e- z{ z+,z-;:::: 0 vay mo hinh toan hoc cho SVM h6i quy tinh Ia: 1 -WTW + CeT(z+ + z-) 2 Yi - wr xi - b ::;; t:e + zt -yi + wr xi+ b ::;; t:e + zi{ zt,zi-;:::: 0 Khoa THQL ( 2.16) 21 PMN TiCH HOI QUY BANG SVM T6ng quat v6i ham ap d1,mg cho ca h6i quy phi 1 -WTW 2 + CeT(z+ +z-) + zt -yi + wT ci>(xi) + b ::; ee + ziYi - wT (xi)- b ::; ee { cand6i... Ia bai toan quy buoc tinh va rna tran cua toan phuang Ia ntra xac djnh duang, vdn tham s6 c dung soat khong gian gia thiSt Mo hinh h6i quy SVM giai bai l&n va mang tinh phi h6i quy hieu qua ngay ca v&i du lieu c6 s6 tuyen cao Bi€u kien KKT chinh hi di€u kien Kulm Tucker, se duQ"c gioi tlrieu chi ti€t trong phvlvc 3 24 DE TAI cAP co so ChU"ong 3: tmin huftn SMO phuang phap giai bai toan quy toan phuang... n = 2 Bai toan quy 1 2 y 1 1 1 -1 -1 toan phuong luc nay: 2 2 (w1 + w2 ) +w2 +2w 2 +4w 2 -5w 2 -6w2 w1 3w1 4w1 -w1 -3w1 1 +b 1 +b 1 +b -b ;::: 1 -b ;::: 1 Hinh sau cho thfty cac vecto tl,l'a la M3, M4 va M5 ; ···:···· ··: :· ::· ·:· ·.· ·:··.·.··®··.=·· Giai ·=·· phuong trinh ung vai cac vecto tl,l'a +b dUQ'C WI= KhoaTHQL 0.4, Wz = -0.8 1 -b -b 1 1 Va b = 2.6 11 PHAN TiCH HOI QUY BANG SVM qua... - b = -£ < Yi - wT xi - b < C yi - wT xi - b = £ < yi - wT xi - b -£ < t: £ V&i ki h1 vector cot thu i cua rna tr{in K Ta c6 tieu chuAn kh6ng chua w: -c y.- aTki- b < -£ -c < ai < 0 yi- aTki- b = -£ 0 = ai -£ < Yi- aTki- b < £ I l 0 < ai < C ai = C Khoa THQL £ - yi - aT ki - b = c < yi- aTki- b 23 PHAN TiCH HOI QUY BANG SVM - - - 9 Bai to{m tach tinh c6 lcri ghii, lcri giai Ia khong duy V&i khai niem... bai cua y la 16p c6 chAt lam toan tach, y nhan gia tti 1 -1, bai toan h6i quy lam v6i gia tti thanh 2 gia tti lien we Mot each trvc giac chung ta se tap dfr xay dung mo hinh dua vao SVM Vi d1,1 sau cho chung ta thAy SVM, ban tren hai 16p, c6 giai bai to{m h6i quy nhu nao Xet bang du y X 1 1 1 3 2 3 3 3 4 5 4 6 7 6 Phuang trinh h6i quy la: ' : :J ,j ·0 \ , 1• • • • ••.••• • , • •• •.••• each nhan... lam nay chi khac khong a hai chi s6 i vaj Dij.t s = YiYj ta c6 flaj =-sA !l.E =-flu= -Ayi(ki -·kj) v&i k i la vector cot thu i cua K 0 m6i bu&c, 'Ala loi giai cua bai toan t6i uu sau: 1 2 z11A - Yi(Ei min -ai :::;; A :::;; C - ai { -aj :::;; -sA :::;; C - aj V&i Khoa THQL 27 PHAN TiCH HOI QUY BANG SVM I>ay hi bai toim t6i uu mot va dS dang giai duqc Tru&c l < r, Ia giao cua 2 tUy vao s: I = [l, r]... so v&i tach tuy8n tinh 1 -aTDa2 min yTa = 0 {o:s;;a:s;;c trir viec thay th8 cac thanh phAn cua rna tran d6i xung XTX, cac phAn tir (xi' xj)' b&i cac phAn tu mai (( xi), ct>(xj)> Ky hieu Khoa THQL 17 PHAN TiCH HOI QUY BANG SVM Ma tran K Thvc = (kij) duQ'c goi la rna tran kernel chung ta khong cAn phai xac dinh , rna dung mot ham goi la ham kernel kij = kernel(xi,xi) Trong nhi8u trucrng hQ'p nola... duqc xac dinh bai sieu sa quan trong cua SVM bao gam sieu ph&ng tdi uu, w!cta Hinh sau minh hoa cac khai tl:fa va M Theo d6 sieu t6i uu la sieu tach c6 lon nhAt va duqc xac djnh qua cac vecta fl:fa (Support Vectors) Mot each trvc giac sieu t6i uu la duy nhk Tuy nhien phuang trinh cua mot sieu la khong duy nhAt Chung ta se di tim mot se duqc goi la chinh sao cho chinh cua sieu t6i uu la duy nhdt Cho tapJifr... hai 16p ········•·········•························· ······························ ··············· fi :: ······························ 1;01 •1 KhoaTHQL i , oo m ·······•····•····••••·················· _.,., , § ? s 19 PHAN TiCH HOI QUY BANG SVM Tu true giac tren chung ta di xay dvng mo hinh toan hoc Hip nhu nao, chung ta xet chu6i Bai toan SVM cho vee to du mai thAy mo hinh toan . TE TP.HO CHi MINH • • DE TAl NGHIEN ciru KHOA HOC CAP CO SO • . ' . PHAN TiCH HOI QUY BANG SUPPORT VECTOR MACHINES (SVM) MA SO: CS-2007-01 BQ GIAO DUC.DAOTAO- TP.HO CHi MINH • • DE TAl NGHIEN Ciru KHOA HOC CAP CO SO • ' " , ;t,. PHAN TICH HOI QUY BANG SUPPORT VECTOR MACHINES (SVM) MA SO: CS-2007-01 CHU NHI¢M: ThS. GVC. c6 kiSm soat S1' phuc cua khong gian gia thi8t. Sau d6 (1990), cac mo hinh Support Vector Machines (SVM) duQ'c gi&i nhu Ia mot phuang phap cai dat nguyen ly SRM. Tu d6