Nghiên cứu xây dựng tài nguyên song ngữ anh việt ứng dụng cho dịch máy theo miền

ĐAI HOC QUOC GIA HÀ N®I TRƯèNG ĐAI HOC KHOA HOC TU NHIÊN ——————— NGUYEN TIEN HÀ NGHIÊN CÚU XÂY DUNG TÀI NGUYÊN SONG NGU VIfiT-ANH ÚNG DUNG CHO D±CH MÁY THEO MIEN LU¾N ÁN TIEN SĨ TỐN HOC Hà N®i - 2020 ĐAI HOC QUOC GIA HÀ N®I TRƯèNG ĐAI HOC KHOA HOC TU NHIÊN ——————— NGUYEN TIEN HÀ NGHIÊN CÚU XÂY DUNG TÀI NGUYÊN SONG NGU VIfiT-ANH ÚNG DUNG CHO D±CH MÁY THEO MIEN Chuyên ngành: Cơ so tốn Mã so: 9460117.02 HQc cho tin HQ c LU¾N ÁN TIEN SĨ TOÁN HOC NGƯèI HƯéNG DAN KHOA HOC: TS Nguyen Th% Minh Huyen PGS.TS Nguyen HEu NgE LèI CAM ĐOAN Tơi xin cam đoan n®i dung trình bày lu¾n án ket qua nghiên cúu cna tơi, đưoc thnc hi¾n dưói sn hưóng dan cna TS Nguyen Th% Minh Huyen PGS TS Nguyen Huu Ngn Các n®i dung trích dan tù nghiên cúu cna tác gia khác trình bày lu¾n án đưoc ghi rõ nguon phan tài li¾u tham khao Nguyen Tien Hà LèI CAM ƠN Tơi xin gui lòi cam ơn sâu sac đen TS Nguyen Th% Minh Huyen PGS.TS Nguyen Huu Ngn trnc tiep hưóng dan, chi bao t¾n tình, ln ho tro tao nhung đieu ki¾n tot nhat cho tơi q trình HQc t¾p nghiên cúu Tơi xin gui lịi cam ơn đen thay/cơ giáo o Khoa Tốn - Cơ - Tin HQ c, Trưòng Đai HQc Khoa HQc Tn nhiờn, HQc Quoc gia H Nđi, ắc biắt l cỏc thay/cụ giỏo o Bđ mụn Tin HQc, nhung ngưòi trnc tiep giang day giúp đõ tơi q trình HQc t¾p nghiên cúu o trưịng Tơi xin gui cam ơn đen TS Nguyen Văn Vinh, PGS TS Nguyen Phương Thái, PGS TS Phan Xuân Hieu Trưịng Đai HQc Cơng ngh¾, Đai HQc Quoc gia Hà N®i; TS Tran Th% Oanh khoa Quoc te, Đai HQc Quoc gia Hà N®i; PGS TS Lê Thanh Hương, TS Đo Th% NGQc Di¾p Trưịng Đai HQc Bách khoa Hà N®i; PGS TS Đo Trung Tuan, TS Đo Thanh Hà, TS Lê Hong Phương, PGS TS Lê TRQNG Vĩnh, TS Nguyen Th% Bích Thny, TS Vũ Tien Dũng Trưịng Đai HQc Khoa HQc Tn nhiên, Đai HQc Quoc gia Hà N®i, thay/cơ có nhung góp ý chinh sua đe tơi hồn thi¾n lu¾n án Tơi xin gui lòi cam ơn đen tat ca anh, ch%, em o B® mơn Tin HQc, Khoa TốnCơ-Tin HQc, Trưịng đai HQc khoa HQ c Tn nhiên, Đai HQc Quoc gia Hà N®i B® mơn Khoa HQc máy tính, Khoa Cơng ngh¾ thơng tin, Trưịng Đai HQ c Cơng ngh¾, Đai HQc Quoc gia Hà N®i giúp đõ tơi thịi gian làm nghiên cúu sinh Cuoi cùng, tơi xin gui lòi cam ơn đen tat ca thành viên gia đình, ban bè, đong nghi¾p nơi tơi cơng tác ln nng h®, chia se, đ®ng viên khích l¾ tơi HQc t¾p, nghiên cúu Mnc lnc Danh mnc chE viet tat Ma đau Tong quan ve d%ch máy tài nguyên ngôn ngE 1.1 1.2 15 Tőng quan ve d%ch máy .15 1.1.1 L%ch su ve d%ch máy 16 1.1.2 Kien trúc cna h¾ thong d%ch máy 19 1.1.3 Các phương pháp d%ch máy 22 1.1.4 Các h¾ thong d%ch máy có the su dung đe thnc nghi¾m 30 1.1.5 Đánh giá h¾ thong d%ch máy 32 Tài nguyên ngơn ngu cho h¾ thong d%ch máy 35 1.2.1 Tài nguyên đa ngu cho d%ch máy 35 1.2.2 Tài nguyên song ngu Vi¾t-Anh 38 1.3 Thích úng mien d%ch máy .41 1.4 Các công cu tien xu lý văn ban 43 1.5 Ket lu¾n chương 45 Xây dEng kho ngE li¾u song ngE Vi¾t - Anh dóng hàng mÉc câu theo mien 2.1 47 Xây dnng kho ngu li¾u song ngu Vi¾t-Anh theo mien 48 2.1.1 Phương pháp thu th¾p ngu li¾u song ngu dóng hàng câu48 2.1.2 2.2 2.3 Xây dnng kho ngu li¾u song ngu Vi¾t - Anh mien du l%ch 49 Dóng hàng văn ban song ngu Vi¾t-Anh 52 2.2.1 Phương pháp dóng hàng văn ban song ngu múc câu 52 2.2.2 Cai tien cơng cu dóng hàng câu XAlign 54 Úng dung kho ngu li¾u du l%ch song ngu Vi¾t-Anh cho h¾ thong d %ch máy 64 2.4 2.3.1 Ket qua thnc nghi¾m .65 2.3.2 M®t so loi cna h¾ thong d%ch .68 Ket lu¾n chương 70 Xây dEng kho ngE li¾u tÈ, cnm tÈ song ngE Vi¾t-Anh 3.1 72 Xây dnng tn đng kho tự vnng song ngu Viắt - Anh 73 3.1.1 Xây dnng kho tù vnng song ngu 73 3.1.2 Phương phỏp xõy dnng tn đng tự vnng song ngu Viắt-Anh77 3.1.3 Phương pháp xây dnng tn đ®ng tù vnng song ngu Vi¾tAnh mien du l%ch 79 3.1.4 3.2 Thnc nghi¾m ket qua .84 Trích rút thu¾t ngu song ngu Vi¾t-Anh tù văn ban đơn ngu tieng Vi¾t dna vào t¾p lu¾t .88 3.2.1 Các cơng trình nghiên cúu có liên quan .90 3.2.2 Phương pháp trích rút thu¾t ngu song ngu Vi¾t-Anh tù văn ban đơn ngu tieng Vi¾t 93 3.2.3 3.3 Thnc nghi¾m 104 Ket lu¾n chương 106 Khai thác kho ngE li¾u song ngE Vi¾t-Anh cho d%ch máy 4.1 108 Tien xu lý du li¾u huan luy¾n d%ch máy nơ-ron 108 4.1.1 Phương pháp tien xu lý câu dài d%ch máy nơ-ron 110 4.1.2 Phương pháp trích rút cum tù ExtPhrase 112 4.1.3 4.2 Thnc nghi¾m ket qua .115 Phương pháp sinh tn đng chỳ giai tieng Viắt cho hỡnh anh 119 4.2.1 Các cơng trình có liên quan đen sinh giai cho anh 119 4.2.2 Đe xuat quy trình xây dnng h¾ thong sinh giai tieng Vi¾t cho anh 121 4.3 Ket lu¾n chương 128 Ket lu¾n 131 Danh mnc cơng trình khoa hQ C CUA tác gia liên quan đen lu¾n án 133 Tài li¾u tham khao 135 Danh mnc chE viet tat ALPAC Automatic Language Processing Advisory Committee (H®i đong co van xu lý ngơn ngu tn đng) BiTES Bilingual Term Extraction System (Hắ thong trích rút thu¾t ngu song ngu) BLEU BiLingual Evaluation Understudy (Chi so đánh giá chat lưong d%ch song ngu) CNN Convolutional Neural Network (Mang nơ-ron tích ch¾p) DTW Dynamic Time Warping (Thuắt toỏn cn chinh thũi gian đng) GRU Gated Recurrent Unit (Đơn v% hoi quy cőng) LSTM Long Short Term Memory (B® nhó dài ngan han) MI Mutual Information (Thông tin tương ho) NLP Natural Language Processing (Xu lý ngôn ngu tn nhiên) NMT Neural Machine Translation (D%ch máy mang nơ-ron) OPUS The open parallel corpus (Kho ngu li¾u song song mo) PBSMT Phrase-Based Statistical Machine Translation (D%ch máy dna cum tù) PER Position-independent word Error Rate (Ty lắ loi tự đc lắp v% trớ) RNN Recurrent Neural Network (Mang nơ-ron hoi quy) SMT Statistical Machine Translation (D%ch máy thong kê) SALM Suffix Array tool kit for empirical Language Manipulations (Công cu LQc bang cum tù Moses) TER Translation Error Rate (Ty l¾ loi d%ch) TV Television (Truyen hình) VLSP Vietnamese Language Speech Processing (Xu lý ngơn ngu tieng nói tieng Vi¾t) WER Word Error Rate (Ty l¾ loi tù) Danh sách hình ve 1.1 Tam giác Vauquois 20 1.2 Mơ hình d%ch trnc tiep 21 1.3 Mơ hình d%ch qua ngôn ngu trung gian 22 1.4 Mơ hình d%ch máy thong kê 24 1.5 Cau trúc cna h¾ thong d%ch máy dna mang nơ-ron 27 1.6 Cau trúc cna h¾ thong d%ch máy MOSES 30 3.1 Phương pháp xõy dnng tn đng tự vnng Viắt-Anh .77 3.2 Phng phỏp xõy dnng tn đng tự ien Viắt-Anh mien du l%ch 82 3.3 Mơ hình trích rút thu¾t ngu song ngu Vi¾t-Anh tù văn ban tieng Vi¾t 94 3.4 Mơ hình áp dung lu¾t đe lna chQN úng viên thu¾t ngu song ngu Vi¾t-Anh .98 4.1 Mơ hình ý tồn cuc 111 4.2 Mơ hình ý cuc b® 111 4.3 Mơ hình tien xu lý câu dài 30 tù huan luy¾n h¾ thong d%ch máy 113 4.4 Điem BLEU cna h¾ thong theo đ di tự tieng Viắt oc coi l câu tieng Vi¾t dài 118 4.5 Mơ hình giai tieng Vi¾t cho anh 123 4.6 So sánh chat lưong d%ch máy vói Google 125 [76] Ngo Quoc-Hung and Werner Winiwarter (2012), “Building an English- Vietnamese Bilingual Corpus for Machine Translation”, Proceedings of the 2012 International Conference on Asian Language Processing, pp 157– 160 [77] Nguyen Th% Minh Huyen and Mathias Rossignol (2006), “A language- independent method for the alignement of parallel corpora”, Proceedings of 20th Pacific Asia Conference on Language, Information and Computation (PACLIC), pp 223–230 [78] Oriol Vinyals and Alexander Toshev and Samy Bengio and Dumitru Erhan (2015), “Show and Tell: A Neural Image Caption Generator”, Proceedings of The 2015 IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), DOI: 10 1109 / CVPR 2015 7298935, arXiv: 1411 4555v2 [cs.CV] [79] Peter F Brown, Jennifer C Lai, and Robert L Mercer (1991), “Aligning Sentences in Parallel Corpora”, Proceedings of the 29th Annual Meeting on Association for Computational Linguistics, 169– 176, DOI: 10 3115 / 981344.981366, URL: https://doi.org/10.3115/981344.981366 [80] Peter F Brown et al (1990), “A Statistical Approach to Machine Trans- lation”, Comput Linguist., 16 (2), pp 79–85, ISSN: 0891-2017, URL: http: //dl.acm.org/citation.cfm?id=92858.92860 [81] Pham Ngoc-Quan et al (2019), “Improving Zero-shot Translation with Language-Independent Constraints”, Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pp 13– 23, URL: http://www.aclweb.org/anthology/W19-5202 [82] Philipp Koehn (2002), “Europarl: A Multilingual Corpus for Evaluation of Machine Translation”, Draft [83] Philipp Koehn (2020), Moses Statistical Machine Translation System User Manual and Code Guide, Statistical Machine Translation, URL: http:// www.statmt.org/moses/manual/manual.pdf [84] Philipp Koehn and Franz Josef Och and Daniel Marcu (2003), “Statistical phrase-based translation”, Proceedings of HLT-NAACL 2003, pp 127–133 [85] Philipp Koehn and Rebecca Knowles (2017), “Six Challenges for Neural Machine Translation”, CoRR, abs/1706.03872, arXiv: 1706 03872, URL: http://arxiv.org/abs/1706.03872 [86] Philip Resnik (1998), “Parallel Strands: A Preliminary Investigation into Mining the Web for Bilingual Text”, Machine Translation and the Informa- tion Soup, ed by David Farwell, Laurie Gerber, and Eduard Hovy, pp 72– 82 [87] Philip Resnik (1999), “Mining the Web for Bilingual Text”, Proceedings of the 37th Annual Meeting of the Association for Computational Linguis- tics, pp 527–534, DOI: 10 3115 / 1034678 1034757, URL: https :// www aclweb.org/anthology/P99-1068 [88] Rob Gaizauskas et al (2015), “Extracting bilingual terms from the Web”, Terminology International Journal of Theoretical and Applied Issues in Specialized Communication, Vol.21, pp 205–236, DOI: 10.1075/term.21 2.04gai [89] Rui Wang et al (2017), “Sentence Embedding for Neural Machine Trans- lation Domain Adaptation”, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp 560–566, DOI: 10.18653/v1/P17-2089 [90] Saba Amsalu (2006), “Data-driven Amharic-English Bilingual Lexicon Ac- quisition”, Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06), URL: http://www.lrecconf.org/ proceedings/lrec2006/pdf/666_pdf.pdf [91] SANJIKA HEWAVITHARANA and Stephan Vogel (2016), “Extracting parallel phrases from comparable data for machine translation”, Natural Language Engineering, Vol.22, pp 549–573, DOI: 10.1017/S1351324916000139 [92] Sara Ebrahim et al (2015), “English-Arabic Statistical Machine Trans- lation: State of the Art”, Computational Linguistics and Intelligent Text Processing, ed by Alexander Gelbukh, pp 520–533 [93] Sergei Nirenburg et al (2002), “Machine Translation: A Knowledge- Based Approach”, Morgan Kaufmann Publishers Inc.340 Pine Street, Sixth Floor- San Francisco CA United States [94] Shaohui Kuang and Deyi Xiong (2016), “Automatic Long Sentence Seg- mentation for Neural Machine Translation”, Proceeding of the 24th Inter- national Conference on Computer Processing of Oriental Languages, IC- CPOL 2016, pp 162–174 [95] Shengxiang Gao et al (2018), “A Method to Chinese-Vietnamese Bilingual Metallurgy Term Extraction Based on a Pivot Language”, Proceedings of the 6th CCF Conference, Big Data, pp 3–20, DOI: 10.1007/978-981-132922-7_1 [96] Shuoyang Ding, Hainan Xu, and Philipp Koehn (2019), “Saliency- driven Word Alignment Interpretation for Neural Machine Translation”, Proceed- ings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pp 1–12, URL: http://www.aclweb.org/anthology/W19-5201 [97]S K Jang and J.S Chang (1997), “A Class-based Approach to Word Alignment”, Computational Linguistics, Vol.23(2), pp 313–343 [98] Solomon Teferra Abate et al (2018), “Parallel Corpora for bi- Directional Statistical Machine Translation for Seven Ethiopian Language Pairs”, Pro- ceedings of the First Workshop on Linguistic Resources for Natural Lan- guage Processing, pp 83–90, URL: https://www.aclweb.org/anthology/ W18- 3812 [99] Stanley F Chen (1993), “Aligning Sentences in Bilingual Corpora Using Lexical Information”, Proceedings of the 31st Annual Meeting on Associa- tion for Computational Linguistics, 9–16, DOI: 10.3115/981574.981576, URL: [100] https://doi.org/10.3115/981574.981576 Stephan Vogel et al (2000), “Statistical Methods for Machine Transla- tion”, Springer, Berlin, Heidelberg, DOI: 10 1007 / 978 - - 662 - 04230 - 4_27 [101] Stig-Arne Grăonroos, Sami Virpioja, and Mikko Kurimo ((2015)), “Tuning Phrase-Based Segmented Translation for a Morphologically Complex Tar- get Language”, Proceedings of the Tenth Workshop on Statistical Machine Translation, pp 105–111, URL: http :// aclweb org / anthology / W15 - 3010 [102] TAN Min, DUAN Xiangyu, ZHANG Min (2019), “Neural Machine Trans- lation Domain Adaptation Based on Domain Features”, Journal of Chinese Information Processing, Vol.33 (7), p 56, URL: http://jcip.cipsc.org cn/EN/abstract/article_2798.shtml Thomas M Cover and [103] Joy A Thomas (1991), Elements of Information Theory, New York : Wiley, Ⓧc 1991., ISBN: 978-0-471-24195-9 [104] Trieu Hai Long and Nguyen Le Minh (2017), “A Multilingual Parallel Corpus for Improving Machine Translation on Southeast Asian Languages”, Machine Translation Summit XVI [105] Trieu Hai-Long, Nguyen Phuong-Thai, and Nguyen Le-Minh (2015), “A New Feature to Improve Moore’s Sentence Alignment Method”, VNU Jour- nal of Science: Comp Science & Com, Eng Vol 31 No 1, 32–44 [106] Van Ngoc Sang (2007), “Building Vietnamese -Jrai; Jrai - Vietnamese dictionary”, Research project and technology, Ministry level, Vietnam [107] Van Ngoc Sang, Mohamad Bin Bilal Ali, Noor Dayana Abd Halim (2016), “Building Cham - Vietnamese Electronic Dictionary”, Journal Pendidikan Nusantara, ISSN 2289 -9375 (Print) Special Edition, No 1, pp 215–223 [108]V I Levenshtein (1966), “Binary Codes Capable of Correcting Deletions, Insertions and Reversals”, Soviet Physics Doklady, Vol.10, pp 707–710 [109]Vincent Vandeghinste et al (2006), “METIS-II: Machine Translation for Low Resource Languages”, Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06) [110] Warren Weaver (1953), “Recent Contributions to the Mathematical The- ory of Communication”, ETC: A Review of General Semantics, Vol.10, No.4, pp 261–281 [111] Wei Yang, Jinghui Yan, and Yves Lepage (2016), “Extraction of Bilingual Technical Terms for Chinese-Japanese Patent Translation”, Proceedings of the NAACL Student Research Workshop, pp 81–87, DOI: 10 18653 / v1 / N16- 2012, URL: https://www.aclweb.org/anthology/N16- 2012 [112] William A Gale and Kenneth Ward Church (1991), “A program for Align- ing sentences in bilingual corpora”, Proceedings of the 29th Annual Meeting of the Association of Computational Linguistics (ACL) [113] Wujie Zheng and Wenyu Wang and Dian Liu and Changrong Zhang and Qinsong Zeng and Yuetang Deng and Wei Yang and Tao Xie (2018), “Oracle-free Detection of Translation Issue for Neural Machine Translation”, CoRR, abs/1807.02340 [114] Xabier Saralegi and In˜aki San Vicente and Antton Gurrutxaga (2008), “Automatic Extraction of Bilingual Terms from Comparable Corpora in a Popular Science Domain”, Proceeding of the Workshop on Comparable Corpora, LREC 2008, pp 27–32 [115] Yasuhiro Ogawa and Makoto Nakamura and Tomohiro Ohno and Kat- suhiko Toyama (2018), “Extraction of legal bilingual phrases from the Japanese Official Gazette, English Edition”, Journal of Information and Telecommunication, Vol.2 (4), pp 359–373, DOI: 10 1080 / 24751839 2017 1380272, eprint: https :// doi org / 10 1080 / 24751839 2017 1380272, URL: https://doi.org/10.1080/24751839.2017.1380272 [116] Yonghui Wu and Mike Schuster and Zhifeng Chen and Quoc V Le and Mohammad Norouzi and Wolfgang Macherey and Maxim Krikun and Yuan Cao and Qin Gao and Klaus Macherey and Jeff Klingner and Apurva Shah and Melvin Johnson and Xiaobing Liu and L- ukasz Kaiser and Stephan Gouws and Yoshikiyo Kato and Taku Kudo and Hideto Kazawa and Keith Stevens and George Kurian and Nishant Patil and Wei Wang and Cliff Young and Jason Smith and Jason Riesa and Alex Rudnick and Oriol Vinyals and Greg Corrado and Macduff Hughes and Jeffrey Dean (2016), “Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation”, CoRR, abs/1609.08144, arXiv: 1609 08144v2 [cs.CL] [117] Yorick Wilks (1979), “MACHINE TRANSLATION AND ARTIFICIAL INTELLIGENCE"”, Translating and the Computer, B.M Snell (ed.) [118] Yun-Chuang Chiao et al (2006), “Evaluation of multilingual text align- ment systems: the ARCADE II project”, Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06), URL: http :// www lrec conf org / proceedings / lrec2006 / pdf / 506 _ pdf.pdf [119] Zi-Yi Dou et al (2019), “Domain Differential Adaptation for Neural Ma- chine Translation”, Proceedings of the 3rd Workshop on Neural Generation and Translation (WNGT 2019), pp 59–69, DOI: 10.18653/v1/D19-5606 Phn lnc Mđt so thuắt ngu su dung xây dnng kho ngu li¾u: Kho ngE li¾u: Là t¾p hop manh ngôn ngu đưoc cHQN lna sap xep theo m®t so tiêu chí ngơn ngu HQc rõ ràng đe đưoc su dung m®t mau cna ngơn ngu Kho ngE li¾u so: Là kho ngu li¾u đưoc mã hóa theo m®t chuan nhat đ%nh đong nhat đe có the khai thác cho úng dung khác Kho ngE liắu song ngE: L mđt ban đưoc viet bang hai ngơn ngu Kho ngE li¾u song song a ngụn ngE: L mđt ban đưoc viet bang nhieu ngơn ngu Dóng hàng văn ban song ngE: • Dóng hàng múc văn ban: văn ban kho ngu li¾u đưoc ánh xa vói nhau, tài li¾u ban d%ch cna tài li¾u • Dóng hàng múc đoan: đoan hai văn ban đưoc ánh xa vói nhau, m®t vài đoan se l ban d%ch cna mđt vi oan ã Dóng hàng múc câu: câu hai văn ban đưoc ánh xa vói nhau, câu ban d%ch cna câu • Dóng hàng múc cum tù: cum tù hai văn ban đưoc ánh xa vói nhau, cum tù ban d%ch cna cum tù • Dóng hàng múc tù: tù hai văn ban đưoc ánh xa vói nhau, tù ban d%ch cna tù dóng hàng múc cum tù múc dóng hàng chi tiet nhat kho ngu li¾u song ngu ... song ngu Vi¾t -Anh" đưoc su dung Hi¾n kho ngu li¾u song ngu Vi¾t -Anh đưoc xây dnng chn yeu dna vào khai thác nguon tài nguyên song ngu Vi¾t -Anh Nhưng c¾p ngơn ngu Vi¾t -Anh khơng có nhieu tài nguyên. .. câu cho c¾p ngơn ngu Vi¾t -Anh xây dnng kho ngu li¾u song ngu Vi¾t -Anh theo mien Nghiên cúu đe xuat phng phỏp trớch rỳt tn đng cỏc cắp tự, cum tù song ngu đe xây dnng kho ngu li¾u tù v cum tự song. .. ve d%ch máy tài nguyên ngôn ngu đưoc su dung d%ch máy: l%ch su ve d%ch máy; kien trúc cna h¾ thong d%ch máy; hưóng tiep c¾n cna d%ch máy; đánh giá h¾ thong d%ch máy; tài ngun ngơn ngu tài ngun