xây dựng mô hình từ điển điện tử cho tiếng việt

Viện công nghệ thông tin Báo cáo tổng kết khoa học công nghệ đề tài nhánh xây dựng mô hình từ điển điện tử cho tiếng việt thuộc đề tài cấp nhà nớc nghiên cứu phát triển côngnghệ nhận dạng, tổng hợp xử lý ngôn ngữ tiếng việt Mã số: KC 01.03 Chủ nhiệm đề tài: gs.tskh bạch hng khang 6455-4 07/8/2007 Hà Nội- 2004 ti KC01 - 03: BO CO K THUT V Mễ HèNH T IN IN T VMTD Ngi thc hin: GS TSKH H Tỳ Bo, Japan Advanced Institute of Science and Technology KS Nghiờm Anh Tun, Vin Cụng Ngh Thụng Tin MC LC Gii thiu Cu trỳc chung ca t in VMTD 1.1 T in t 1.2 T in khỏi nim 1.3 T in ng hin din 1.4 T in song ng 1.5 Corpus 1.6 Mi quan h gia cỏc t in VMTD Cu trỳc cỏc t in VMTD 2.1 T in t 2.2 T in khỏi nim 10 2.2.1 T in gii thớch khỏi nim 10 2.2.2 T in phõn loi khỏi nim 11 2.2.3 T in mụ t khỏi nim 11 2.3 T in song ng 12 2.4 T in ng hin din 13 2.5 Corpus 16 Cỏc bc xõy dng t in VMTD 18 3.1 Xõy dng t in gii thớch khỏi nim v phõn loi khỏi nim 18 3.2 Xõy dng t in t 18 3.3 Xõy dng corpus 19 3.3.1 Phõn tỏch t 19 3.3.2 Phõn tớch cu trỳc ng phỏp 20 3.3.3 Tỡm ngha ca t 20 3.3.4 Phõn tớch cu trỳc ng ngha 20 Kt lun 21 Ti liu tham kho 22 Ph lc A: Bng mó t ca t in t ting Anh 23 Ph lc B: Bng mó t ca t in t ting Vit 34 Ph lc C: Cỏc bi bỏo liờn quan 39 Gii thiu Mt cỏc mc tiờu quan trng ca ngnh Cụng ngh thụng tin l lm cho mỏy tớnh cú kh nng giao tip vi ngi bng ngụn ng ca ngi ( ngụn ng t nhiờn ) Tng t vic ngi cn n t in hc v s dng mt ngụn ng, mỏy tớnh cn cú t in ca riờng mỡnh cú th hiu v s dng cỏc t mt ngụn ng t nhiờn T in in t cung cp ngun tri thc giỳp mỏy tớnh cú th hiu c ngụn ng ngi v úng vai trũ nn tng cho cỏc nghiờn cu v ngụn ng t nhiờn Khỏc vi cỏc t in trờn mỏy tớnh dnh cho ngi nh Lc Vit T in, Click and See hay Kim t in t in in t c thit k riờng cho cỏc ng dng x lý ngụn ng t nhiờn nh dch mỏy, tr li t ng Vỡ vy h thng ng ngha (cỏch biu din ngha ca t) t in in t khụng c lu tr di dng ngụn ng t nhiờn nh t in thụng thng m phi mt s dng c bit mỏy tớnh cú th x lý c nh mng ng ngha, frame cú th thy rừ hn vai trũ ca t in in t ta hóy xột mt s vớ d sau õy: Xõy dng engine tỡm kim da trờn ng ngha: vi cỏc engine tỡm kim thụng dng nh Google hay Yahoo, ta cú th tỡm c nhng bn cú cha mt t khúa no ú Tuy nhiờn, vi cỏc t khúa a ngha nh table (l bn hoc bng biu) v nu ngi dựng ch mun tỡm cỏc bn cú cha t table vi ngha bng biu thỡ cỏc engine tỡm kim hin s tr v rt nhiu ti liu khụng liờn quan Trong trng hp ny nu ta thc hin vic ch mc cỏc bn khụng phi theo s xut hin ca t khúa m theo ngha ca t thỡ ta cú th d dng gii quyt nờu trờn Xõy dng h qun tr c s d liu cho phộp truy da trờn ng ngha: Gi s ta cú cõu truy sau: Hóy tỡm tt c nhng ngi trớ thc ang sng khu ph X Vi mt h qun tr c s d liu thụng thng iu kin ta ch cú trng mụ t ngh nghip, ta khụng th thc hin c cõu truy ny bi c s d liu khụng lu tr bn ghi no cú giỏ tr trng ngh nghip l trớ thc c Tuy nhiờn, vi s h tr ca t in in t, ta cú th bit rng bỏc s, k s, nh vn, nh th l nhng ngh nghip ca gii trớ thc Vỡ vy ta cú th tỡm tt c cỏc bn ghi cú cha nhng t ny Trờn th gii, ó cú rt nhiu d ỏn ln kộo di nhiu nm nghiờn cu v t in in t nh d ỏn WORDNET ti i hc Princeton, d ỏn Cyc phỏt trin bi cụng ty CYCORP, d ỏn EDR ca Vin nghiờn cu v t in in t ca Nht bn Ti Vit Nam, t in in t cng ó bt u c s dng mt s ng dng x lý ngụn ng t nhiờn ting Vit Mc dự vy, cỏc t in ny c thit k chuyờn bit cho tng ng dng c th nờn chỳng khú cú th c ỏp dng mt cỏch rng rói Hn na, vic thiu nhng nghiờn cu chuyờn sõu v t in in t ó phn no nh hng n cht lng ca cỏc t in ny Chớnh vỡ vy, yờu cu t l cn tin hnh nghiờn cu cỏc mụ hỡnh t in in t trờn th gii, t ú xut mt mụ hỡnh phự hp cho t in in t ting Vit v cui cựng l a quy trỡnh thc hin vic xõy dng t in Ti liu ny gii thiu mt mụ hỡnh ca t in in t ting Vit phỏt trin khuụn kh ti KC01-03 Ti liu trung gii thiu cu trỳc ca t in in t cho ting Vit VMTD, gm bn phn chớnh nh sau: Gii thiu cu trỳc chung ca VMTD: cỏc t in cựng mi liờn h gia chỳng Gii thiu chi tit cu trỳc tng bn ghi ca cỏc t in Xỏc nh cỏc bc cn thc hin cng nh cỏc cn gii quyt xõy dng VMTD Kt lun Cu trỳc chung ca t in VMTD VMTD bao gm cỏc t in sau: - T in t - T in khỏi nim - T in song ng - T in ng hin din - Corpus Mi t in cú hai phiờn bn cho ting Anh v ting Vit 1.1 T in t Cha cỏc thụng tin v mt cu to t v c tớnh ng phỏp ca t Bờn cnh ú, t in t cũn cha cỏc tr khỏi nim liờn kt t vi ngha (khỏi nim) tng ng ca nú t in khỏi nim 1.2 T in khỏi nim Biu din cỏc khỏi nim ca ngi di dng mng ng ngha T in khỏi nim gm cú hai t in con: t in phõn loi khỏi nim v t in miờu t khỏi nim T in miờu t khỏi nim lu tr tt c cỏc mi liờn h gia cỏc khỏi nim Nú l mt mng ng ngha ú cỏc khỏi nim c liờn kt vi thụng qua 18 loi mi liờn h khỏc Cỏc mi liờn h ny c la chn cho vic s dng chỳng cú th biu din c hu ht mi liờn h gia cỏc khỏi nim mt cõu Vớ d cõu Tụi n cm, gia cỏc khỏi nim tụi, n v cm ta cú hai mi liờn h sau: (Tụi _He_believed_not Allows no passive Ex He lacks motivation * Motivation is lacked by him Occurs only in the passive form Ex John was said to be a good teacher * They said him to be a good teacher * Indicates an ill-formed construction Does not occur in the progressive tense Ex * I am knowing him for a long time Occurs only in the progressive tense Ex The baby is teething Code Combinations Note: The codes on the left must be accompanied by a code indicated on the right [EVIO0]->[EVDO0] [EVOC0]->[EVDO0] [EVDO0]->[EVDO1-9 ] One code from EVDO1-9 [EVSC0]->[EVC10-93] One code from EVC10-93 [EVOC0]->[EVC10-93] One code from EVC10-93 [EVSA0]->[EVSA2-6 ] One code from EVSA2-6 [EVSA2]-> Specific Preposition Code 29 Thuc tớnh ng phỏp ca danh t (Nouns: Grammatical Attributes) Attribute Code Countability 47 48 49 Collectivity 50 Gender 51 52 53 54 Verb Agree- 55 ment 56 57 58 Cooccurrence with Articles 59 60 61 62 63 64 65 Word Form Restrictions 66 67 Comment/Explanation Countable Uncountable Uncountable noun that can be instantiated Note: Only one code is assigned Word form and right adjacency attribute are given based on countable usage for ENUC records All other coding for ENUC nouns is based on noun when noun is uncountable Collective noun (Ex people) Note: Code is given only when applicable Referent of noun is male (Ex man) Referent of noun is female (Ex woman) Referent of noun is neutral (Ex book, baby) Referent of noun can be either male or female (Ex student, baby) Note: More than one code can be assigned when applicable Always treated as singular in subject - verb agreement Always treated as plural in subject - verb agreement Treated either singular or plural in subject -verb agreement Note: For non-count nouns, only one code is given Verb agreement coding is given to count nouns only when the noun in the singular form takes either a plural form verb or a singular form word Does not have restrictions on the article Always takes an article Must be preceded by a definite article Must be preceded by an indefinite article Never occurs with a definite article Never occurs with an indefinite article Never occurs with an article Note: One code only is given to each noun A count noun that is not coded is interpreted as ENWAR and a non-count noun that is not coded is interpreted as ENNOINF Nouns occurs in the singular form only Nouns occurs in the plural form only 30 Thuc tớnh ng phỏp ca tớnh t (Adjectives: Grammatical attributes) Code 68 69 70 Explanation Does not occur in the positive degree form Does not occur in the comparative degree form Does not occur in the superlative degree form Thuc tớnh ng phỏp ca trng t (Adverbs: Grammatical attributes) Code 71 72 73 Explanation Does not occur in the positive degree form Does not occur in the comparative degree form Does not occur in the superlative degree form Chc nng v v trớ ca hn nh t (Determiner: Function and Position Information) Code Explanation 74 May follow an indefinite article 75 May not follow a definite article Note: Code is given only when applicable 76 May be followed by a countable singular noun 77 May be followed by a countable plural noun 78 May be followed by an uncountable noun Note: Code is given only when applicable 79 May be followed by a noun phrase beginning with an indefinite article 80 May be followed by a noun phrase begining with a definite article 31 Bng mó t chc nng Function Word Codes: Preposition Equivalents Code Preposition Equivalent concerning considering excepting excluding following including involving pending regarding 10 respecting 11 according to 12 along with 13 as for 14 as regards 15 as to 16 based on 17 based upon 18 because of 19 consisting of 20 down to 21 prior to Code Preposition Equivalent 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 due to out of by means of by way of in front of in respect to in terms of in view of on account of on behalf of on top of with regard to for the benefit of for the purpose of for the sake of in the course of in the matter of in the middle of in the way of on the basis of up to Function Word Codes: Be-Verb, Auxiliary Verbs, Auxiliary Verb Equivalents Code Function Word 43 be 44 can 45 cannot 46 could 47 48 dare 49 have 50 may 51 might 52 must 53 need 54 Shall Code Function_Word 55 56 57 58 59 60 61 62 63 64 65 should will would be to had better have to ought to used to be able to be about to be going to Function Word Codes:Coordinate Conjunctions, Corrdinate Conjunction Equivalents, Subordiate Conjunctions, Subordinate Conjunction Equivalents, and Conjunctive Adverbs Code Function Word Code Function_Word 32 66 After 91 only 67 Against 92 once 68 Also 93 otherwise 69 And 94 or 70 As 95 provided 71 Because 96 providing 72 Before 97 since 73 Beside 98 so 74 Besides 99 suppose 75 Both 100 supposing 76 But 101 than 77 Directly 102 that 78 Either 103 then 79 Else 104 though, although 80 Except 105 till, until 81 If 106 unless 82 Immediately 107 while 83 Instantly 108 yet 84 Lest 109 as if 85 Like 110 as though 86 Moreover 111 even if 87 Namely 112 even though 88 Neither 113 in order that 89 Nor 114 so that 90 now Function Words: Relative Pronouns, Interrogative Pronouns, Relative Adverbs, Interrogative Adverbs Code Function Word 115 how 116 that 117 whenever 118 what 119 whatever 120 when 121 where 122 wherever Code Function Word 123 124 125 126 127 128 129 130 whether which whichever who whoever whom whose why Other Function Words Code Function Word 131 never 132 not 133 more 134 most Code Function Word 135 136 137 that to (verb infinitive) not to 33 Ph lc B: Bng mó t loi ca t in t ting Vit Bng t loi ting Vit Loi Danh t T loi - Danh t riờng - Danh t chung +) Danh t ch loi th +) Danh t ch o lng Chớnh xỏc Khụng chớnh xỏc Mó Vớ d H Ni, H Chớ Minh cỏi,con,cn,quyn,s,cuc, +) Danh t ch cht liu +) Danh t ch ngi Ch quan h thõn thuc Ch chc v, ngh nghip thc, trc, phn, lớt cc, ming, mu, on, mnh, toỏn, dóy, tp, m, st than, chỡ, m, tht, mui 10 11 12 13 14 15 16 17 -ng t ch trng thỏi tip thu -ng t cm ngh, núi nng 18 19 -ng t tỡnh thỏi 20 -ng t quan h S t +) Danh t ch vt Ch vt Ch ng vt Ch thc vt +) Danh t ch khỏi nim tru tng - Danh t ch hin tng thiờn nhiờn - Thut ng chuyờn mụn -ng t ngoi hng -ng t gõy khin -ng t xut hin, tn li, tiờu tan Tớnh t cha, m,anh,cu,cụ bỏc s, cụng nhõn, niờn giỏm c, giỏo s -ng t ni hng ng t 21 bn, gh, nh chú, mốo,g, cam, quýt, tre thiờn nhiờn, xó hi tri,mõy,giú,bóo t bo, mng, lm, m, mua, bỏn,n, ra, vo lờn, tng bin, np, vay, thc, ng, ci, ựa, nhỡn, nm, bũ lm, , bt cũn, cú, ht, mt, xut hin ny, mc, ni b, c, chu bit, thy, khen, chờ, bo, nhn nh, tin tng, mun, toan, nh, nờn, dỏm phi, chu, l, lm, hoỏ, ging, khỏc Tớnh t ch c im bờn ngoi ca s vt +) Mu sc +) Hỡnh th +) Dung lng +) Kớch thc -Tớnh t ch c tớnh bờn v trng thỏi -Tớnh t miờu t mc - S t ch s lng chớnh xỏc - S t ch s lng ỏng chng - S t ch s th t 22 23 24 25 26 27 28 29 30 xanh, , tớm, to nh, trũn, vuụng nh, nng, cng di, ngn, cao, thp, tt, xu, hin, to gan, nhanh y, vi, nhiu,ớt,dy, tha mt, hai phn ba, my, vi ba, dm, mt vi, nht, nhỡ, th mi, 34 i t Phú t (trng t) Quan h t (kt t) Tr t Thỏn t Cm ng t Cm danh t Cm tớnh t - i t nhõn xng - i t ch nh s vt - i t ch nh v trớ khụng gian, thi gian - i t ch trng thỏi - i t ch s lng - i t hi +) Hi v s vt +) Hi v v trớ khụng gian +) Hi v hot ng, trng thỏi +) Hi v s lng - Biu th s lng ton th hay riờng l - Biu th ý ngha thi gian - Biu th ý ngha ph nh - Biu th ý ngha yờu cu, sai khin, khớch l - Biu th ý ngha ng nht hay liờn tc - Biu th mc - Biu th s din bin - Biu th s kt thỳc hnh ng - Quan h t chớnh ph 31 32 33 34 35 tụi, chỳng tụi, nú, ny, n, kia, y, õy,y, ú kia, nay, bõy gi th, vy, by nhiờu, ht thy, c, tt c, 36 37 38 39 40 41 42 43 44 45 46 47 48 ai, chi, gỡ, no õu no, bao gi, th no, sao, my, bao nhiờu, nhng, cỏi, mi, mi, tng, ang,ng, s, ó, va, mi, khụng, cha, chng, hóy, i, ng, ch cng, u, vn, cũn, li, c rt, khỏ, hi, khớ, quỏ, lm, cng, li, luụn,mói,bốn,bng, xong, ri ca, bng, vi, v, n, hi, bi, , 49 50 51 52 53 54 55 v, cựng, vi, rng, l, thỡ, m thỡ, vỡ do, nhng hay, hoc, thỡ, l, c, chớnh,t,nhng,cỏi,thỡ,c, 56 57 58 59 60 61 62 63 64 65 kia, õu, y, chng, h, nh, , i no, thụi, vi, õu, y c, kia, nhộ hi, i, ờ, ny võng, d, , phi ụi, chao, khin, tri, t chy thc mang Cm t vi danh t lm trung tõm 66 xe mỏy Cm t vi tớnh t lm trung tõm 67 hon hn - Quan h t liờn hp +)Song song +)Ph thuc +)Sau ng t cm ngh, núi nng +)T ni, cp t ni +) La chn - Tr t cho t - Tr t cho cm t - Tr t cho cõu +) Nhn mnh +) Hoi nghi +) Ngc nhiờn +) Cu mong +) Dt khoỏt +) Nng nu - Thỏn t lm ting gi - Thỏn t lm ting ỏp - Thỏn t lm ting than Cm t vi ng t lm trung tõm Thuc tớnh ng phỏp ca ng t Mó Gii thớch 35 -Khụng cn kt hp vi ph t: ng t ni hng - Phi kt hp vi ph t: +) Ph t l danh t: ng t ngoi hng +) Ph t l cm CV: ng t gõy khin, cm ngh, núi nng +) Ph t l ng t: ng t tỡnh thỏi - Cú hoc khụng kt hp vi ph t: ng t xut hin, tn ti, tiờu tan -Lm v ng cõu Thuc tớnh ng phỏp ca danh t Mó Gii thớch 36 10 11 12 13 14 15 16 17 18 19 20 21 22 23 -Kt hp vi danh t +)Danh t ch quan h xó hi, gia ỡnh: danh t riờng +)Danh t khỏc: danh t ch loi th, o lng +)Danh t ch cht liu: danh t ch o lng +)Danh t ch n v o lng: danh t ch cht liu -Khụng kt hp vi danh t +)Ch loi th: danh t ch cht liu, khỏi nim tru tng (ớt) -Kt hp vi i t ch nh: danh t ch loi th, cht liu, thi gian, ngi, phng hng, v trớ (tr ụng, tõy, nam, bc), vt, khỏi nim tru tng - Khụng kt hp vi i t ch nh: danh t riờng - Kt hp vi s t: danh t ch n v o lng, thi gian, ch ngi, vt, khỏi nim tru tng - Khụng kt hp vi s t: danh t riờng, ch n v o lng khụng chớnh xỏc, phng hng v trớ (tr phớa, phng, bờn, hng) - Kt hp vi i t ch s lng: danh t ch ngi (chc v ngh nghip), ch vt, loi th, cht liu, khỏi nim tru tng - Khụng kt hp vt i t ch s lng: danh t riờng, o lng, thi gian, ngi (quan h thõn thuc) - Kt hp vi nh t: danh t ch loi th, ch ngi, vt - Kt hp vi tớnh t: danh t ch ngi, vt, cht liu, khỏi nim tru tng - Lm ch ng Thuc tớnh ng phỏp ca tớnh t Mó 24 25 26 27 28 Gii thớch -Kt hp vi trng t: tt c tr cụng", "t", "riờng", chung" - Kt hp vi danh t: tớnh t ch c im bờn ngoi, c im bờn v trng thỏi - Kt hp vi ng t thnh cm ng t: tớnh t ch c im bờn ngoi - Kt hp vi tớnh t ch tớnh cht: tớnh t ch mu sc - Lm v ng: tr cụng, "t", "riờng", "chung" Thuc tớnh ng phỏp ca s t Mó 29 30 31 32 Gii thớch - S t lm tin t ph cm danh t - S t lm v ng cõu: s th t, s lng chớnh xỏc+"l" - S t ch s lng phng chng - S t ch s lng tng trng: ba, trm nghỡn Thuc tớnh ng phỏp ca i t Mó Gii thớch 37 33 34 35 36 -i t lm ch ng: i t nhõn xng, ch nh s vt (y, ny), hi -i t lm nh t: ch nh s vt, hi -i t thay th cho mt n v ng phỏp: ch nh s vt trng thỏi -i t thay th cho s t ch s lng: i t ch s lng Thụng tin t chc nng ca phú t Mó Gii thớnh -i kốm vi danh t: phú t biu th s lng ton th hay riờng l -i kốm vi ng t, tớnh t: phú t biu th ý ngha trung gian, ph nh, din bin - Trc ng t: phú t biu th ý ngha yờu cu, sai khin, ng nht hay liờn tc - Trc tớnh t: phú t ch mc , ng nht hay liờn tc - Sau ng t: phú t biu th kt thỳc hnh ng, phú t biu th din bin Thụng tin t chc nng ca quan h t Mó 10 Gii thớch - Trong cm danh t: ca, bng (chớnh ph) - Trong cm ng t: ca (chớnh ph), vi, n, v - Trong cm tớnh t: v (chớnh ph) + ng t: ch mc ớch, i tng: õu, , cho, n - Trong cõu ghộp: quan h t liờn hp Thụng tin t chc nng ca tr t Mó 11 12 13 Gii thớch - Tr t cho t - Tr t cho cm t - Tr t cho cõu Thụng tin t chc nng ca thỏn t Mó 14 Gii thớch Thnh phn ph bit lp cõu Cỏch s dng Cỏch s dng Abbreviation Slang Mó Gii thớch Hỡnh thc rỳt gn ca mt t Khụng phự hp vi núi ni cụng cng hoc bn Vớ d HQG m 38 Ph lc C: Cỏc bi bỏo liờn quan - Nghiem Anh Tuan, Ho Chi Kien, Ho Tu Bao Issues in Construction of a Vietnamese Machine Tractable Dictionary in Proceeding of APF, Japan, 2002 39