cài đặt thực nghiệm - kết quả

trình bày phần cài đặt thực nghiệm - kết quả

CHUaNG 5: CAI D~T THtjC NGHItM - KET QU~4.

Voi mo hlnh BTL dU<;fCxay dvng trong chuang 3 cling voi cac b~iiroan co lien quail cua no da dU<;fcgiai quy€t trong chuang 4, trong chuang 5 nay, chung toi ti€n hanh di d~t thvc nghi~m mo hlnh BTL do Vi~c di d~t dU<;fCtach lam 2 ph~n: ph~n gall nhan ngon ngu cho ngu li~u song ngu EVe va ph~n dling lu~t chuy~n d6i rut ra du<;fCtu EVe d~ gall nhan cho cau ti€ng Anh moi va chuy~n sang ti€ng Vi~t

Do khuon kh6 gioi h<;lnv~ s6 trang cua mQt lu~n an ti€n S1,Den chung toi khong th~ trlnh bay chi ti€t nhung di d~t ma~g tinh ky thu~taday, chung toi chi neu so d6 kh6i cling cac thong s6 ky thu~t, qui trlnh dich va k€t qua thvc nghi~m cua h~ dich EVT (English-Vietnamese Translator) cua chung rei

H~dich g6m cac co sa du li~u (database) va cac kh6i (module) sail:

1 Ca sa d/1li~u tit diin: tu di~n Anh-Vi~t, Vi~t-Anh, tu di~n lOp ngu nghla,

2 Ca sa tri thue: d~ gall nhan (ca sa) v~ ngu phap, ngu nghla, chuy~n d6i,

3 Ca sa d/1li~u t4p lu4t: t~p lu~t dl1<;fCrUt ra tu EVe d~ gall nhan ngon ngu

4 Kh6'iphon rich h1nhthai:ti~n xli'ly, tach cau,nh~n di~n thanh ngu/ten rieng.

5 Kho'ipho.n rich ng/1philp: gall nhan tu lo<;li,cu phap, quail h~ ngu phap.

6 Kho'iphan rich ng/1nghza: gall nhan ngu nghla

7 Kho'iehuyin d5i e6y eLlphcip:chuy~n cay cu phap ti~ng Anh sang cay cu

phap ti€ng Vi~t (chuy~nd6i tr~t tv tu)

8 Kho'i sinh eo.u tie'ng Vi~t: t~o call ti€ng Vi~t tu cay Cll phap ti€ng Vi~t, gall

nghla ti€ng Vi~t dva tren nhan ngu nghla, hi~u dinh hlnh thai hQc ti€ng

Vi~t

GP

/ tie-fig Anh/

+

T~p lu~t

chuy€n d6i

Hlnh thai

KFTBL

Ngu li~u song

ngu da gall nhan

H~u hi~u dinh

1::::::,:::::::::::,"""'"""""'""""""""""""""""" ,~" ,,:1,~:]

EGT

Coso Tri thuc, Tti' di€n

Hlnh 501: So d6 khoi chuang trlnh dich EVT Tieng Anh nh~p VaG se du<;jcdua VaG khoi phan rich hinh thai d€ xu 19 so bQ,nhu tach cali (neu la van ban), tach tti' (nh~n di~n thanh ngu), nh~n di~n ten riengoSau d6 chuy€n qua khoi phan rich ngu phap (gan nhan tti' lo<:ti,CD phap, quailh~ ngu phap)o Khoi nay du<;jcgall nhan co so b~ng bQ cong Cl.).EGT (EnglishGrammar Tagger) cua My (phien ban 2001) Ke d6 la khoi gall nhan ngunghla, r6i chuy€n d6i cay CDphap tieng Anh sang cay CDphap tieng Vi~t Cu6icling, Cali tieng Vi~t du<;jchinh thanh tren co so cay CDphap va nhan ngu nghia.Cac khoi tren ho<:ttdQng tren co so cac lu~t du<;jcrut ra tti' EVC b~ng giai thu~tKFTBL va cac ngu6n co sa tri thuc cling nhu tti' di€no

" " ,,?

5.2 m:: TV DIEN TRONG EVT

Tii' di6n la ml)t h~ co sa tri thUGquail trQng trong mQi h~ djch, n6 chua

dl,tngh~u he't cac thong tin da bie't v~ ngon ngu Trong h~ djch EVT, chung toi t6 chilecac tit di6n theo nhung nguyen t~c va ca'u truc du li~u nhu sail:

5.2.1

A ~ ,:> ' " A?

NGUYEN TAC TO CHUC T(1 BIEN

v~ mat chuffn ngon n~u tie'ng Viet:

-Phuong ngu: du li~u titng Vi~t trong tit di6n nay duQc dt;ratren phuong ngu Ha NQi,vi dl.,l:blanket: m~n, chan, pig: lqn, heo Tuy nhien, cac bie'n th6 cia phuongthong d\lllg khac vlin duQc lUlltrong tit di6n voi tu cach la d6ng nghla

~.

-Lu~t chinh ta: dt;ratheo Qui dinh 240-1984 cua BI) Giao dl.,lcv~ chuffn boa chinhta tie'ng Vi~t trong nganhGiao dl.,lc

-BQma va font: bl) ma ABC (1 byte) theo tieu chuffn TCVN-5712, VNI-Times -Tieu chi It;rachQn ml.,lctit tie'ng Vi~t: day la va'n d~ kh6 khan do chu'a co st;r nha't quan, tuy nhien, chung toi cling da t~m thai dua ra cac tieu chi trong [14]

v~ mat chuffn ngon ngu tie'ng Anh:

- Dung tie'ng Anh cua My (American English), tuy nhien, bell c~nh d6, v~n cha'pnh~n va lUlltru tie'ngAnh cua Anh (British English) nhu nhung don vi tU'ongduong Vi dl.,l:"mau" -7 color (Am.Eng) va colour (Bri.Eng); "di~n toaDboa " -7 computerize (Am.Eng) va compute rise (Bri.Eng).

- Tieu chi It;rachQn ml.,lctu tie'ng Anh: cac tit trong tit di6n Anh - Vi~t thong

thU'ong (35.000 tit), tit gbep (compound), ngu co dint (phrasal), thanh ngu

(idiom), tl.,lcngu (proverb), tit vie't ta:t (abbreviation), ten rieng (Proper Name),

, 144

Call truc vi mo(microstructure) cua tIT di€n song ngii' Anh-Vi~t (EVD:

~lish-Vietnamese Dictionary) cua EVT baa g5m nhii'ng thong tin du'Qc trinh m9t each co h~ th6ng trong moi m1,lctU [38] nh~m di€u khi€n qua trinh elich, :h€ nhu' sail:

Thong tin v€ hinh thai:

)~ng cua tIT(word form), vi d1,l:"program", "book",

ria v€ d~ng cua tIT(word form): tu nguyen g6c, d~ng bat quy t~c,

ria d~c tinh hint thai: ma gap doi ph1,lam, ma ket hc;1pvdi ph1,lt6

ria vi tri cua tIT(word order): tITnay dung 0 vi tri nao trong ngii', trang cau,

-fhong tin v€ ngu phaD:

'ITlo~i (Parts-of-speech) cua tIT,nhu': danh tIT,dQng tIT,tinh tiI,

:hung lo~i (Subcategory): lo~i danh tITdem du'Qc/khongdem du'Qc,

)~c tint cu phap (syntactic feature): thi (tense), th€ (voice), gi6ng (gender),

)~c tint Call truc (structure): tITnay dung trong Call truc DaD,mftu call naCo

Jgii'di kern (collocation/phra~e/idiom): tITnay hay di kern vdi nhii'ng tITnaco

fhong tin v€ ngu n.ghla:

\Jghla tieng Vi~t (meaning) cua tu.

:>~c di€m tieng Vi~t: hi~u chinh gi v€ hint thailnghla tieng Vi~t khi dich.

\Jhan ngii' nghla cua tu (sernatic class): nhu' HUM, ANI, (LLOCE/WordNet) Vai va rang buQc trang van ph~m each (case): Agent(Hum), Instrument (obj)

fhong tin v€ ngii' dun.g:

MilImh vlfc sa dl,lDg(field): tin hQc,tOaDhQC,y hQc,

f~n s6 xuat hi~n (frequency): thee tITdi€n t~n s6 [27]

.,.

Mil v€ tlnh thai (modality): trinh trQng, than m~t, thong tt,lc,

5.2.3 CAD TRUC DU L!tU CUA TVDIEN EVD

EVD la tU dign quail trQng nhat (chila nhi~u thong tin nhat va du'QCtruy c~p thu'dng xuyen nhat) Bg co th~ hja chQn cali truc dG'1i~uphil hQp cho vi~c lu'u trG'

va truy xuat tll di~n Anh-Vi~t nay, chung toi da kh?w sat cac thong so cua tu vlfngtie"ngAnh tren illQtso IOnvan b?mtie"ngAnh va co du'QCcac nh~n dinh sail:

- H~u he"tcac tu thong d\lng nhat d~u la nhG'ngtu chilcDang(function words),

nhu': the, of, a, va thu'dng co chi~u dai nho hOD4 (thong ke tren Brown

corpus)

- Chi~u dai trung blnh cua cac tUtie"ngAnh la 8,5: Tu I 11l~ % I

15000

10000

i I

!-+-Length i.

Quantity!!

25000

20000

5000

0

1 4 7 10 13 16 19 22 25 28

Iis 1,5 j

Hlnh 5.2: Thong ke chi~u dai (hlnh trai) va t~n so (hlnh phai) cac tu tie"ngAnh

Do yeti du t6 chile co sa dG'li~u tu di~n El ph,E dam baa t6c dQ truy xuat nhanh

va dung lu'Qng lu'u tru nho, Den sail khi xem xet cac kef qua thong ke a tIeD

(ffinh 5.2), chung toi da chQneach t6 chile tu dign tie"ngAnh theo 2 cap nhu' sail: Cap 1: dung bang barn (hash table) cho 3 ky tlf d~u lien vdi ham barn du'di day:

2

;=0

trong do key la chuoi ky tlf (tu) dn tra, i la chi so ky t9'trong chuoi (co so

27la tinh tIeD26 m§u tlf tie"ngAnh va 1 ky tlf g?ch du'di)

Cap2: sti' d9ng cay tlm kie"msO'DST (Digital Search Tree) cho cac ky tlf con l?i Chitiel v~ t6 chile dG'li~u cua tu dign na y da du'Qccong b6 trong [11] Cac tu di€nkhac cling du'Qct6 chilc thee ki~u phan tich va lu'u trG'nhu'tren

2,5 i

I 2,5

to I

So Ten

1 EVD

3 EVDie

4 VED

5 IVDic

I

6 !ECD

7 IVCD

8 CED

9 CVD

10 lEWD

Chuc Dang

Tu di€n dieh IDa y ehua h~ u

he't thong tin da bie't v€ ngon ngil' d€ di€u khi€n vi~c dich T?p lu?t chuy€n dbi tr?t tl1 tu d€ gall nhan ehuy€n dbi IT?t

W tucd sa eho EVT

Tra tu tie'ng Anh d€ co cac nghla tie'ng Vi~t (co tu lOC;li)

Tra tu tie'ng Vi~t~d~ co cae nghla tie'ng Anh (co tu lo~i)

Tu di€n ta'td cac tu co th~

co trong tie'ngVi~t

Tra tu tie'ng Anh xern thuQc lOp d5ng nghia naG

Tra tu tie'ng Vi~t xern thuQc lOp d5ng nghla naG

Tra lOp d5ng nghla xern co

tu tie'ng Anh na0 Tra lOp d5ng nghla xern co

tu tie'ng Vi~t naG

Tu di€n lien ke't lOp synset trong Wordnet voi cac tu tie'ng Anh

N<Tu5n <TckI:> I:> Dung

luQn

15.000 tU tu di€n 35.000 tu va

nhi€u tU di€n kh,k

215 lu~t

65.000tU

70.000tU

I

85.000tU

40.000 tu

tu EVT 1.0

Tu di€n Anh- Vi~t clia DB Naoai nail'o. 0

Ha NQi, NXB GD-1997

Tudi€n Vi~t-Anh cua Blii Ph\,mg, NXB The' £':ioi2000 '-Dl1a tren tu di~n tie'ng Vi~t clia Hoang Phe va cae

tu di€n khac

dl1a thee CEDic 50.000tU Idl1athee CVDic

2441 lOp

2441 lOp

100.000 lOp

LLOCE va Wordnet I

LLOCE va tu di~n d5ng nghla tie'ng Vi~t Du'dng Ky Due (1999)

Tu khe dil'li~u Wordnet phien ban 1.7 Dam 2001 Ghi ehu: tu di€n EVD la tu di~n chinh cua h~ diehEVT, con TFRULla tu

l' di6n ehU'a tri tMe d6 gan nUn cd ,Ii cho vi~c ehuy6n d6i tr~t tit tit trong EVT,

Cac tu di~n con lC;lichi phl;lCVl;lgiall tie'p trong cae bai toan xii'19 cua BTL

5.3 KHO NGU LltU SONGNGU ANH-VltT Eve

THUTHA-PNGULItU SONGNGUANH-Vn'tTEve THO

5.3.1

Kho ngu li~u song ngu EVe Quc;!cduc;!cthu th~p tU nhi~u ngu6n van ban song ngu (sach~ tu di€n, ngu li~u, ) khac nhau thuQc lint vl,l'cKhoa hQc, ky thu~t, thong thuong, Vi~c thu th?p phai tuan theo cac tieu chi nhat qUail v~ m~t ngon ngu, v~ van phong, v~ lInh vt,l'c, (da trlnh bay chi tie't trong [13])

~ a:nnoUJ1cement dt lai loan baa,

thong CaD,CaDthL lai tOyeD b6 The

announcement of the royal birth

was broadcast to the nation wi loan

baa stj ra d(jj cUa cilia con hoEing tQc

da dLrqc truy\~n thanh tren toan

quOc Announcements of births,

marriages and deaths appear in

some newspapers NhiXng thong baa

v~ sl,l'ra dai, c1I6iboi, tang ch~xu~"t

hi~n tren illQt vai tcr baa

Hlnh 5.3 Trich do~illcac vi d\l trong cac tu di€n Anh-Vi~I va Vi~t-Anh

gift minh not be caught napping gift nguyen keep intact; preserve;

nhilng sTjt1jdo,ccW cM dQva l6i song

freedoms, the systems and way ~l.life

""ill be p~erved; kit qua la kMng mi)t dong new con dlLqc gift nguyen the

result is that hardly a line bas been left untouched.

gift nUOc defend the country; defend-ing the country; safeguarding the

Sail khi thu th?p cac van biin song ngu tu cac ngu6n khac nhau, chung t6i phai chugn boa v~ dq,ng van ban (text only), ma lull tru (TCVN3), font (Vni-times),'r6i tie'n tanh lien ke't cali (sentence alignment) va chugn hoa chinh ta nhu'sau:

*DO2:01323:The announcement of the royal birth was broadcast to the nation +DO2:01323: Lai loan bao SI! ra aai cua Qua con hoang tQc da dU:(Jctruy~n thanh tren toim que/c.

*DO2:01324: Announcements of births, marriages and deaths appear in some newspapers.

+DO2:01324: Nhiing thong bao v~ SI! ra aai, cu:6i hoi tang cht xu{/t hi~n tren nIQtvai ta bao.

Trong do, cae ma scldgu thalli chie'u de'n Cali tu'ong ling trong ngu li~u EVe.

Bang 5.1 Ke't qua t6ng hQpeua kho ngu li~u EVC

diu(Anh

-Vi~t) 9.475 33.078 174.906 6.269 12.120 4.953 9.210 250.011

S6luQ't s6 Iu<.tt

(Anh) (Vi~t) 165.042 239.984 312.655 410.760 1110.003 1460.010 128.000 181.781 226.953 297.920 79.927 101.023 126.060 160.585

I 2148.640 I 2852.063 Bang 5.2 Khao sat ngu6n ngu li~u song ngu Anh-Vi~t EVC

1

2

Bt) saeh CADASA2

Tli di~n LLOCE3

Cae tITdi~n khae4

Ngu li~u SUSANNE)

Bt) saeh vi~n thong6

Bt) bach khoa Tf(~em7

Cae sach khac8

Chi~u d~li (Anh) 17,42 9,45 6,35

20,42

18,73 16,14 13,69 8,59

-T?

I" I

1 ~ I

(s6 tit) 7,68 14,55 51,66 5,96 10,56 3,72 5,87 100%

2 BQ stich song ngu Anh-Vi~t "Hay den voi the gidi may vi tinh" g6m 12 U~pcua

3

4

5

6

7

T6ng cong:

CADASA do T.Nguy~n The Hung ehu bien, NXB Thong ke, 2002.

3 Cae cau vi d1,ltrong tu ditin LLOCE (xin xem ph1,ll1,le8.1).

4 Cae cau vi d\I song ngu dU'c;1e trleh tu cae tu ditin Anh-Vi~t, Vi~t-Anh, ru ditin thanh ngu, eua Vi~n NGON NGU HQC, NXB GD.

5 Xin xem ph1,lll;1e8.6.

6 B9 giao trlnh song ngu 10 t~p nay dU'c;1eNTT va KDD (Nh~t) t?ng, hi~n dang dU'c;1ecae trU'ong DH dqy v~ Vi~n thong CiVi~t Nam sii' d1,lng B9 saeh nay do T6ng e\Ie

1997.

BU'udi~n ehu trl bien djeh va dU'c;1e xua't ban bCiiNXB Khoa hQe Ky thu~t vao Dam

7 TrQn b9 8 cuon, nhU'nghi~n nay moi xua'r ban dU'c;1c 3 cuon va eel 3 cuon nay dil

dU'c;1c nMp vao B9 stich nay dU'c;1c NXB Giao d\Ie cM trl bien djch tu b9 THE GOLDEN BOOK ENCYCLOPEDIA cua Golden Presss - New York.

8 Cae stich v~ KHKT, hQc tieng Anh, cac mau cali Anh-Vi~t, tieng Anh eho tin hQe, kien thuc bact khoa, dqng song ngii'Anh-Vi~t.

35000

30000

25000

20000

15000

10000

5000

a

3 5 7 9 11 13 15 17 19 21 23 25 27 29

Hlnh5.4 D8 thi khao sat so lu<;5ngdiu thee dQdai

Thong sf) Ghi trj I ghichu

i

So cali tieng Anh= So cali tieng Vit

250.011cp I

I

So lu'Qttu tieng Anh 2148.640 l11'<;1t

So tu tieng Anh (tu goc) 39.207 tU khong tinh eac tu

bien "each Chiu dai trung blnh Cali tieng Anh 8,59 tU/cau

1

Bang 5.3 Khao sat so lu'QngCali thee dQdai

I

So cali 0 1023 9034 20433 33398 36003 31189 249931 18002 14556

So ca u 12405 10335 8034 6234 4987 3984 21911 17811 1447 I 1137

IIChiu dai 21 22 23 24 I 25- 26 27 I 28 I 29 I 30

So Cali 995 I 847 17161 700 I 764 I 594 I 569 I 531 I 483 I 423

5.3.2 GAN NRAN NGON NGU CRO NGU LI~U SONGNGU EVC

Sail khi chugn boa ngii' lit$u va lien ke't Cali cho Eve, chung toi da tie'n hanh gall nhan ngon ngu cho EVe theo qui trinh nhl1sail:

5.3.2.1 OAN NHAN HINHTHAi - LIEN KET TU

Nh~m xac dinh raub gioi tu n5i lien ke't tu tie'ngAnh voi tu tie'ng Vit$td6i ung Do s1,1'khac bit$tv~ lo~i hinh ngon ngu, Den khong phai tat ca cae tu d~u

du'<;jcc6 S1,1' d6i ung 1-1, nhung chung toi cling chi dn chu trQng de'n cae th1,1'ctll' (content word).

Hinh 5.5 Vi dl,llien ke't tu trong EVe

Trang ngu lit$usong ngu, cae m6i lien ke't nay dl1<;jcma boa thanh cae chi so' th(( tlf (cua tu chinh fa) cua tu d6i ung nhl1d~ng sail:

They are carrying on their

researching work now

,-,

Hi~n nay, hQ dang xuc tieD

cong vit$c nghien CUllcua hQ.

{They 2} {are 3} {carrying on 4,5} {their 10,11} {researching 8,9} {work 6,7} {now" 0,1}{Hit$nnay7} {hQO} {dang I} {xuctie'n 2,3} {cong vit$c 6 Llng!1i~Il CUll5} {~uahQ.:t}

Bang 5.4 Ket qua lien kef tll' trang EVe

N eru6n saner n eru I::> I::> I::> so' cp S6tu so' tll'c6 so' lien DQchinh

2 Tll'din LLOCE (0) 3.575 34.590 32.189 30.296 94,12 %

T6n?: con?:: 871.087 81.224 71.991 67.211 88,63 %

~ ~ ::r V1 0"\

~ ~, ~ ::r, ~ ::r 0 ::r §: ~ (Jq

,, ~ ::r (Jq ~, ~ ~ ::r ~I ~ g, 0 ,~

J"'" 0

"D ::r

"D -<

(Jq s::;,1

I=< ' -"

:::! (Jq

0>

Sail khi gall TItan tli lo<:li,chung toi tie'n h~mh gall TItan cu phap (hay con gQi la phan rich cu phap - parse) Vi~c gall TItancu phap nh~mxay dvng cay cu phap va danh da'u rant gidi cac ngu (phrase) trong cali Du'di day la ke't qua gall TItan cu phap cua me>tcali trong ngiI' li~u SUSANNE (t~p AOl, cali 02) cua chu'dng trlnh VCLParser 1.0 do nh6m nghien CUllchung toi tv thvc hi~n (theo TItanPTB da trlnh bayaphl,ll1,lC8.5.2):

[S [NP [NPL The jury] [ADJP further]] [VP said [PP in [NP [NPL term-end presentments] [PP that [NPL the City Executive Committee] ]]], [SS [NPL which] [VP had [NP [NPL over-all charge] [PP of [NP [NPL the election] , " [SS [VPdeserves [NPLthe praise and thanks] [PP of [NP [NPL the City] [PP of [NPL Atlanta] ]))))n ))) [PP for [NPLthe manner) ] [PP in [NP [NPL which] [SBAR [SS [NPL the election] [VP was [VP conducted ]]]]]]]]] ]

Cuoi cung, chung toi tie'n hanh gall TItan quaD h~ ngiI' phap (Grammatical

Relations)nh~m xac dint cac moi quaDh~ nhu' S-V, v-a, A-N,Modifiers,

Vi~c gall nhan ngiI' phap cd sa d~u du'<;1ctht!c hi~n b~ng be> cong Cl,lEGT

(cua My) r6i sau d6 dung KFTBL ke't h<;1pvdi thong tin tie'ng Vi~t d~ sua loi

Trong vi~c gall TItan ngiI' nghTa cd sa, chung t6i ke' thlia ke't qua lien ke't tli theo lOpngiI' nghTa, sail 'd6 dung cae ngu6n tri thuc khac d~ sua loi Ke't qua t6ng hQp

nhu'sau:

Bang 5.5 Ke't qua gall nhan ngiI'phap va ngiI'nghla tie'ng Anh trong EVC

Vi~c thli nghi~m - danh gia cac ke't qua gall TItan nay du'cJctht!c hi~n b~ng each so sanh vdi 10% ngiI' li~u hua'n luy~n "vang" cua SUSANNE ph~n J (Technical)ma kh6ng dung d~ hua'n luy~n (g6m 3.200 tli)

Ngu6n song ngG' tli lo<:li I cu phap Quan h np Nt:>aiI'nt:>ahTa

1 Gan nhan cd sa 96,11 %I 89,62 % 90,12 % 75,23 %

2 Gan TItan tht!c 98,63 %I 90,45 % 90,12 % 96,36 %