PPt3 - Derivation of Delta Rules pps

1 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule  Nguồn gốc của Delta rule cho mạng sau  Cần tìm NN 1 }2,1,0{ cho ∈ ∂ ∂ −=∆ i w E w i i η 2 22110 2 210 ))(( 2 1 )( 2 1 ),,( dxwxwwdywwwE −++=−= 1 x2 x1 w2 w1 w0 )()( 22110 xwxwwnety ++== ϕϕ , , 210 w E w E w E ∂ ∂ ∂ ∂ ∂ ∂ 2 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule  Tương tự ta nhận được: và 1 1 22110 1 22110 22110 1 2 22110 1 210 )('))(( )( )('))(( ))(( ))((2 2 1 ))(( 2 1),,( xnetdnet w dxwxww netdnet w dxwxww dxwxww w dxwxww w wwwE ϕϕ ϕϕ ϕ ϕ ϕ −= ∂ −++∂ −= ∂ −++∂ −++= ∂ −++∂ = ∂ ∂ 2 2 210 )('))(( ),,( xnetdnet w wwwE ϕϕ −= ∂ ∂ )('))(( ),,( 0 210 netdnet w wwwE ϕϕ −= ∂ ∂ 3 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule  Do đó luật update cho weight là: NN 3 22 11 0 )('))(( )('))(( )('))(( xnetnetdw xnetnetdw netnetdw ϕϕη ϕϕη ϕϕη −=∆ −=∆ −=∆ 4 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule NN 4 )(y , )(y , )(y , Let 2211 2222112 1122111 netuyuynet netxvxvnet netxwxwnet ϕ ϕ ϕ =+= =+= =+= x2 x1 w2 w1 v2 v1 u1 u2 )()(y 2211 uyuynet +== ϕϕ • Vi phân delta rule cho mạng sau: 5 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule 1 1 2211 1 2211 2211 1 2 2211 1 )(')( )( )(')( ))(( ))((2 2 1))(( 2 1 ynetdy u duyuy netdy u duyuy duyuy u duyuy u E ϕϕ ϕ ϕ ϕ −= ∂ −+∂ −= ∂ −+∂ −+= ∂ −+∂ = ∂ ∂ Tương tự ta có: 2 2 )(')( ynetdy u E ϕ −= ∂ ∂ 1 2211 1 2211 2211 1 2 2211 1 )( )(')( ))(( ))((2 2 1))(( 2 1 w duyuy netdy w duyuy duyuy w duyuy w E ∂ −+∂ −= ∂ −+∂ −+= ∂ −+∂ = ∂ ∂ ϕ ϕ ϕ ϕ 6 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule NN 6 111 1 11 11 1 2211 1 1 1 1 1 11 1 2211 )(' )( )(' ))(()()( xnetu w xw netu w xwxw u w y u w uy w duyuy ϕϕ ϕ = ∂ ∂ = ∂ +∂ = ∂ ∂ = ∂ ∂ = ∂ −+∂ Do vậy nhận được:: 111 1 2211 1 )(')(')( )( )(')( xnetunetdy w duyuy netdy w E ϕϕ ϕ −= ∂ −+∂ −= ∂ ∂ Tương tự: 222 2 122 1 211 2 )(')(')( ,)(')(')( )(')(')( xnetunetdy v E xnetunetdy v E xnetunetdy w E ϕϕϕϕ ϕϕ −= ∂ ∂ −= ∂ ∂ −= ∂ ∂ . 1 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule  Nguồn gốc của Delta rule cho mạng sau  Cần tìm NN 1 }2,1,0{. )()(y 2211 uyuynet +== ϕϕ • Vi phân delta rule cho mạng sau: 5 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule 1 1 2211 1 2211 2211 1 2 2211 1 )(')( )( )(')( ))(( ))((2 2 1))(( 2 1 ynetdy u duyuy netdy u duyuy duyuy u duyuy u E ϕϕ ϕ ϕ ϕ −= ∂ −+∂ −= ∂ −+∂ −+= ∂ −+∂ = ∂ ∂ Tương. 1 1 22110 1 22110 22110 1 2 22110 1 210 )('))(( )( )('))(( ))(( ))((2 2 1 ))(( 2 1),,( xnetdnet w dxwxww netdnet w dxwxww dxwxww w dxwxww w wwwE ϕϕ ϕϕ ϕ ϕ ϕ −= ∂ −++∂ −= ∂ −++∂ −++= ∂ −++∂ = ∂ ∂ 2 2 210 )('))(( ),,( xnetdnet w wwwE ϕϕ −= ∂ ∂ )('))(( ),,( 0 210 netdnet w wwwE ϕϕ −= ∂ ∂ 3 Faculty of Electronics and Telecommunications, HUT Bangkok, Jun. 14 – 23, 2006 Derivation of Delta Rule  Do đó luật update cho weight là: NN 3 22 11 0 )('))(( )('))(( )('))(( xnetnetdw xnetnetdw netnetdw ϕϕη ϕϕη ϕϕη −=∆ −=∆ −=∆