Tuyen tap bao cao khoa hgc Hgi nghi Khoa hgc ky thuat Do ludng tuan qudc lan thfl IV Hd Ndi, 11 - 2005 ^MO RONG MANG NORON TRUYEN T H A N G Nguyen Mgnh Tiing Tnfdng Dgi hgc ky thugt cdng nghiep Thdi Nguyen Torn tdt: Bdi bdo phdn tich mang tniyen thang md rgng mang truyen thdng nhieu Idj) thdnh mgng tong qudt han, ifng dung thudt hgi lan truyen ngifgc cho mgng niff rdng nhiiu Iff]), ddu butt cdp nhdt trgng cho cdc Idii Abstract: This paper analyses the feed-forward neural networks then develops it into more general network to apply the back-propagahon learning algorithm for multilayer feedforwaid networks and infer (draw) weigh-iipdate rules for the layers I DAT VAN DE Mang truyfin thdng tfnh don gian, nen dugc flng dung nhieu cac bai loan nhan mdu, phan Idp, cung nhu cac bai toan ludng, difiu khidn Mang truydn Ihdng chudn cd kha nang phdn Idp, nhd mdu, nhimg kha nang phdn Idp khdng cao, ddng Ihdi mang cd mdi sd gia tri dua vao khdng cd kha nang nhan bifi't, ngoai kha ndng luu mdu ciia mang chudn tha'p [2], [3] D6 khac phuc didu dd, cd thd xay dung mang vdi ddu vao bac cao, bac ba hoac bac cao hon (phi tuye'n) Dua trfin mang truyen thdng chudn cd thfi phat trifin mang truyfin thdng bdc cao, neu su tuong quan gitta mang truydn thdng chudn va mang bdc cao, cdc kha nang ndi troi ciia mang bac cao cung dugc phan tich va khdng dinh [1] II MANG TRUYEN T H A N G CHUAN 2.1 Phuong trinh mang truyen th^ng chuan Mang truydn thang vdi phuong trinh [3] y;'=g?[l'w;yr'+iM (1) la mang bac hai flng vdi cac ddu vao tuyen tfnh trongdd: y|'"' ddu cua noron thfl i, ldp thfl q-1, i,j = I, i w^ trpng lien ke'l ddu yj ldp q-1 den dau vao noron thfl i Idp q I^ ngudng cua noron thfl i, ldp q n*"' so phdn tu "noron'' ciia ldp q-1 q Idp noron thflq, q = 1, ••-, Q Q sd Idp cLia mang noron Cdc noron ldp thfl nhat nhan cac ddu vao ngoai y = u (u la ddu vao ngodi) Dau cua cac noron ldp cud'i cung la cac ddu ciia mang noron y = y Mang duac md la bdi (1) la mang truyfin thang chu£n, mang cd ham sai sd: E(w) = it('d,-'yi)' (2) k-i Vdi luat hgc Windrow - Hoff dua trfin phucfng phdp gradient ta cd cdng thflc dd chinh trgng va ngUong la: w J ( k + l ) = wJ(k) + A w J = w J ( k ) - n i;i(k + i) = i ; i ( k ) + A i ; ' - i ; ' ( k ) - T i dE (3) aw!! dE (4) 2.2 Algorithm truyen ngugc 2.2.1 Casd Ggi tdng cdc dau vao cua noron thfl i la: x|' = V ^lly'!'' ^ ^? (5) i-^i Thi cdng thflc (1) vie't lai y?=g;' 'i:'w;yr'+i?]=g:'(x^) (6) Mang can hoc vdi tap m^u {yi, d,!, ly-,, d,}, jy^, d J , y^ la cac ddu cc d^ la cac dau mong muon Tim dao ham ridng cua (6) cho (3), (4): Ma: _5E _aE_sx2^ gE _ 5E ax;* Sw' ~ dx'! dw'!' dl'; ~ dx; di; (7) 5x' ,-1 sx; — - = y , — - = Sw' -' 61' Dinh nghia ; ! = ^ = ^ dx; thi: ^ = s;'yf' Sw; 'J (8) SE , SI; ' (9) vay chung ta co cong thirc de chinh va ngirong la; w;(k+i) = w;(k)-Tis;y;'; i;(k+i) = i;(k)-Tis; ViSci dang ma tran: W ' ( k + 1) = W ' ( k ) - r i S ' ( y ' " ' y I ' ( k + l) = I ' ( k ) - T i S ' (10) (11) (12) SE Sx? SE ddSy: S' SE Sx' Sx' SE Sx' (13) 2.2.2 Caih linh S' Sir dung ma tran Jacobian &;*' "SJ,'*' Sx;*' Sx:; Sx| Srf Sxf &; SxJ Sx'*,', &,',H &',H Sx; Sxf ^I' Sf'*' _ Sx' Sxf (14) Vifi't bidu Ihflc cho ma trdn vdi i,j la cac phdn tfl ciia ma tran: (x^ v'*'M Sx; " Sx' Sx' 5x' Sx' w;;'g'(x;) (15) sg'(x;) g'(x;) = Oday; (16) Sx' Vi vay, ma tran Jacobian co the viet; G'(x')= Trong dd; +1 • Sx'* (17) — =W'*G'(x') Sx' i'Cx?) 0 g''(x',) (18) Bay gid chiing ta co th^ vi6't each ti'nh S'' dudi dang ma tran: SE "Sx' SE , Sx' j Sx'*' (Sx T SE G ' ' ( x ' ) ( w ' * ' ) ' ' ' - ? = ^ = ( ' ( x ' ) ( w ' * ' ) ' ' S ' * ' (19) ' Sx'*' Nhu vay, din ti6n ta phai tim dudc S*^ sau se tim dudc S r6i tie'n ddn d6'n S /a S S tUdng ling vdi ldp mang cud'i cung co ddu mong mud'n d ta cd: o _ SE _ f S ( d - y ) ' ( d - y ) Sx? SxP V S5:(dj-y.)^ Sx? -2(d,-y,) (20) Ta cd thd via: s^ = - ( d , - y,)g'^(x^) (22) Viet ddang ma Iran: S^ = - G ^ ( x ' ^ ) ( d - y ) (23) III M R O N G M A N G TRUYEN THANG Tfl mang truydn thdng chudn, ta cd the phat tridn md rdng cac mang bac cao, mang ldng quat hon Cac mang bdc cao tdng quat dfi dang xfl ly cac bai toan phdn loai phi tuyfi'n [1], cung nhu kha ndng nhd mdu Idn 3.1 Phuong trinh mang • Mang truydn thang dugc md la bdi hfi phuong trinh >'?=gi[Zw;,y;-'y;-'+i;J Trong dd: w^^ (24) trgng lifin kfi't cdc ddu yj.y^ ldp q-1 de'n ddu vao noron Ihfl i ldp q; i,j,k = 1, , n la mang truyen thing bac ba • Mang truydn thdng dugc md ta bdi hfi phuong trinh = g^(Zw3y;-'+Xw;y;-'yr'+I?l Trong dd: w^ (25) trgng lifin kfi't ddu y • ldp q-1 dfi'n ddu vao noron thfl i ldp q 'w'^J^^ trgng lien kfi't cac ddu yj,y[; ldp q-1 de'n ddu vao noron thfl i Idp q; i,j,k = 1, , n la mang truyen thdng bdc ba ldng quat • Mang truydn thdng dugc md ta bdi he phuong trinh y?=g'[ i'_wjzyr'yr'-yr'+ir] (26) Trong dd: w^ ^ trgng lien kfi't cac ddu yj,y|(, ,y2 Idpq-I de'n ddu vao noron thfl i ldp q; i,j, ,z = I, , n la mang truydn thing bdc cao • Mang truydn thdng dugc md ta bdi he phuong trinh y?=g?["i:'w;yr' + Zw;^yr'y;-'+ i ' w;j,yj-'y;^' y;-'+i;l (27) l^ J=l 1.1^=1 j,k, ,z=l I Trong dd: w^ trgng lifin kfi't cac ddu y^ ldp q-1 dfi'n ddu vao noron thfl i ldp q wj^ trgng lifin kfi'l cac ddu yj,y,^ Idp q-1 dfi'n ddu vao noron thfl i ldp q wJ ^ Ugng lifin kfi't cdc ddu yj, y ^ , - , y^ Idp q-1 dfih ddu vao noron thfl i Idp q; i,j, ,z = I, , n la mang truyen Ihdng bdc cao long qudt 829 3.2 Algorithm truyen ngugc Vdi mang tdng quat, theo luat hgc Windrow - Hoff dfla trfin phuong phdp gradient ta SE ' 5E 5w,j .Aw,j , = Sw,j^ SE ^ I J (28) z AI, = - r i — SI, Cdng thirc de chinh trpng va ngudng la: (29) SF w;(k+i)=w;(k)+Aw; = w;(k)-Ti— SE w ; ( k + l) = w;,(k) + A w ; , = w J , ( k ) - r , - — SE w; z(k+i) = w; ,(k)+Aw; , = w ; ^ ( k ) - i i ^ - ^ oo SE i;(k+i)=i;(k)+Ai;=i;(k)-ri— PD Cac dao hamrifingtrong (30), (31) dE dE axl" 5E dE dx"! dE dwl ax'' dwl dwl, dx^ a w l ' "" dwl , dE dx^ dx'} dwl SE SE Sx; SI; ~ Sx; SI; Ma: Sx; -yV Dinh nghTa; thi; SE = s;y'1 Sw; 3x; (33) , -'yr'; Sx; Sw' = yr'yr'-yr' Sx; =1 SI; SE s; = Sx; (34) (35) SE ' (32) ^ 'yrvr ^^ 5w' SE , SI; ' vay chiing ta cd cdng thiic de chinh va ngudng la; w;(k + I) = w;(k)-ris;y;-'' — = s' s;y;-' yr'; (36) (37) w ; ( k + i ) = w;^(k)-,is;y;-'y;"' w; z(k + i) = w;^(k)-Tis;y;-' y;-' (38) i;(k + l) = i;(k)-ris; (39) 830 IV KET LUAN Tfl vific phdn tich mang truyfin thdng chudn dd ddn mang truyen thang bdc cao tdng quat Mang truydn thang bdc cao long quat tang kha nang nhdn bie't mdu va luu mdu ldn Viec dp dung thudt hgc lan truyen ngugc cho mang truydn thang bdc cao tdng qual tang kha nang sfl dung cua mang Thuat hgc truydn ngugc cho ta mdt trinh tu rd rang de tfnh cdc trgng cua mang noron nhieu Idp tfl dd ta cd the dd dang ldp trinh cho qud trinh hgc cua mang noron truyen thing nhifiu Idp cung nhu mang truydn thdng bdc cao tdng quat, gdp phdn lam don gian vific flng dung mang noron vao cdc ITnh vuc ky thuat Tdi lieu tham khdo: [1] Nguyin Quang Hoan Nguyen Mgnh Tdng, Phgm Thugng Hdn (2002), "idig dung mgng naron tuffng tdc bdc cao cho hdi todn phdn ldp cd gidi hgn " , Tuyen tap hdo cdo khoa hgc Hdi nghi todn qudc ldn thd ndm vi tif dgng hod, Hd ndi, tr.126-131 [2} Jacek M Zurada (1997), Introduction to Artificial Neural Systems Jaico Pubishing House [3] Chin Teng Lin + C.S.George Lee (1996) Neural Fuzzy Systems, Prentice - Hall International Editions [4] George A Rovithakis (1999), "Robustifying Nonlinear Systems Using High-Order Neural Network Controllers", IEEE Transacdons on Autoniadc Control, 44(1) pp.102-108 [5] Q Gan and CJ Harris (1999), " Linearization and State Estimation of Unknown Discrete - Time Nonlinear Dynamic Systems Using Recurrent Neurofuzzy Networks", IEEE Transacdons on Systems, Man and Cybernetics, 29(6), pp.80291 831