Tự động đặt tiêu đề cho đoạn văn

IH TR NG GIA TP HCM I H C BÁCH KHOA TÔ THÀNH NHÂN Chuyên ngành: 8.48.01.01 LU N V N TH tháng 08 -HCM - H PGS.TS - PGS.TS lý chuyên KH MÁY TÍNH - - NHI M V LU Tô Thành Nhân .MSHV: 1970022 I : 8.48.01.01 : / Automatically title paragraphs II : III : 22/02/2021 IV NGÀY HO V CÁN B : 13/06/2021 : NG D N CH NHI M B i MÔN L IC trình Tính Tơ Thành Nhân ii TĨM T T LU hình PGN- tài này, phân PGN- iii ABSTRACT In the current era, the amount of information from the Internet in general and the electronic press in particular has increased rapidly and has extremely useful information value in all aspects of life, leading to the exploitation of values from the Internet This news source to serve many purposes in society is increasingly interested by people This information can be personal blog posts, comments or autobiography of an individual However, most of these articles are often untitled, or will be assigned by editors during compilation With the goal of coming up with impressive titles, often the author will choose words that never appear in the original passage, and this is a huge challenge for previous techniques Stemming from this need to automatically title, we will introduce and proposed the PGN-LM model, a system capable of automatically summarizing and titleing text The architecture of the PGNLM model is built by combining modern natural language processing models, which stands out for its ability to create impressive titles, with words that have never been seen before appear in the original text We tested our approach with real data and got positive results, based on algorithmic automated evaluation and manual human evaluation In this thesis, Chapter will introduce an overview of the research topic, present the reason for the birth of the PGN-LM model and introduce the problem of automatic heading Next, Chapter will examine the research works related to this topic, analyze the advantages and disadvantages of the approaches to come up with a suitable solution for the problem Chapter will present the relevant theoretical foundations used to build the PGN-LM system Chapter aims to present and analyze the PGN-LM model in detail Continuing with Chapter 5, we will show the process of implementing the topic, including preparing the data set, and providing methods to evaluate the accuracy of the PGN-LM model And finally, a summary of the obtained results and directions for further research will be presented in Chapter iv L Tô Thành Nhân v M CL C i ii .iii ABSTRACT iv v vi viii viii ix 1.1 1.2 1.3 CƠNG TRÌNH LIÊN QUAN 2.1 2.2 3.1 Word Embedding 3.1.1 Mơ hình Skip-Gram 10 3.1.2 Mơ hình CBOW 11 3.2 Mơ hình Sequence-to- 12 3.3 Mơ hình Pointer Generator Network 14 3.4 18 3.4.1 18 3.4.2 19 20 4.1 20 vi 4.2 n trúc mơ hình PGN-LM 21 4.2.1 21 4.2.2 Mơ hình Pointer Generator Network 22 4.2.3 22 26 5.1 26 5.2 28 5.3 29 5.4 31 5.5 iá 32 5.5.1 32 5.5.2 34 37 6.1 6.2 37 H 37 39 42 vii 5.2 K t qu hu n luy n Base-Seq2Seq PGN-LM Hình 18 cho Base-Seq2Seq the bigelow Base-Seq2Seq Hình 18 -Seq2Seq 28 hình PGN-LM mơ hình PGN-LM Chúng PGN-LM mơ hình Base-Seq2Seq bigelow Base-Seq2Seq Hình 19 -LM 5.3 tồn tốn khơng có nhãn rõ ràng 29 Bilingual Evaluation Understudy (BLEU), ROUGE, Summary Assessment by Relevancy Analysis PGN-LM i BLUEscore Covered (16) Total 30 5.4 K t qu th c nghi m Base-Seq2Seq Base-Seq2Seq Smooth-Seq2Seq Base-Seq2Seq Language Model PGN-LM hình chúng tơi Pointer Generator Network Language Model BaseSeq2Seq SmoothSeq2Seq PGN-LM I have tried several other brands of roasted salted almonds available via the internet and in stores the blue diamond brand is definitely the best tasting of all Best Best roasted almonds almonds ever Best almonds ever Simply the best roasted almonds that i have found good Delicious alternative pretzel for those who cannot eat wheat with just the right amount of crunch and salt You not have to give up pretzels Pretzels Pretzels Excellent snack food for those who can not eat wheat Awesome flavor awesome crunch it almost melts in your mouth could not ask for a better granola bar Best tasting granola bar Best tasting granola bar Best tasting granola bar ever Sweet crunch and perfect for a better gluten free granol A rich satisfying flavor i recommend this chocolate tea above all others So good Excellent so good Excellent Rich flavor without too much bitter smell and very good The coffee was fresh and had a good taste at a very good price Very coffee Good taste 31 Good coffee Great taste at a great price and fast shipping excellent Base-Seq2Seq hình Base-Seq2Seq Smooth-Seq2Seq Base-Seq2Seq Language Model PGN-LM ô Pointer Generator Network Language Model Smooth-Seq2Seq Base-Seq2Seq PGN-LM pretzel for those who cannot eat wheat with just the right amount of crunch and bánh q Excellent PGN-LM 5.5 5.5.1 ng PGN-LM 32 Hình 20:Ví - - This fruit is very healthy This fruit is so healthy 4/5 = 0.8 Hình 21 Base-Seq2Seq Smooth-Seq2Seq PGN-LM 35,23% 35,97% 42,88% - 33 BLEU, mơ hình PGN-LM Base-Seq2Seq mơ hình Smooth-Seq2Seq 5.5.2 i Base-Seq2Seq PGN-LM mơ hình Smooth-Seq2Seq Hình 22 34 lý mơ hình Hình 23 Hình 24 Base-Seq2Seq Smooth-Seq2Seq PGN-LM 64,73% 79,02% 99.49% 35 Base-Seq2Seq 26,64% Smooth-Seq2Seq PGN-LM 43,37% 99,49% PGN-LM Base-Seq2Seq Smooth-Seq2Seq 99,49 % 36 T NG K T 6.1 K t lu n PGN-LM -Of- 6.2 H ng m r ng c tài 37 - 38 ách TÀI LI U THAM KH O [1] [2] Information Processing and Management, Elsevier, vol 43, no 6, Nov 2007 N P C Khatri, and G summarization using document context vector and recurrent neural Computation and Language (cs.CL), arXiv:1807.08000, Jul 2018 [3] K L Computation and Language (cs.CL), arXiv:1512.01712, Dec 2015 [4] summarization using document context vector and recurrent neural Computation and Language (cs.CL), arXiv:1807.08000, Jul 2018 [5] Journal of the ACM, vol 16, iss 2, pp 264-285, Apr 1969 [6] -based lexical centrality as salience in Computation arXiv:1109.2128, Sep 2011 and Language (cs.CL), [7] Journal of Emerging Technologies in Web Intelligence 2, 3, Aug 2010 [8] C D M Bigail See, and P pointer- Computation and Language (cs.CL), arXiv:1704.04368, Apr 2017 [9] Z Liu, Angela Ng, S Lee, Ai Ti Aw, and N pointer- -aware Computation and Language (cs.CL), arXiv:1910.01335, Oct 2019 [10] Y Tay, S Wang, L A Tuan, J Fu, M C Phan, X Yuan, J Rao, Siu C Hui, and A ffective curriculum pointer-generator Computation and Language (cs.CL), arXiv:1905.10847, May 2019 [11] W Li, R Peng, Y Wang, and Z language generation with adapted pointerNeurocomputing, vol 382, pp 174 187, Mar 2020 39 [12] M Ihori, A Takashima, and R -context pointer- generator networks for spoken-to-written sty ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, May 2020 [13] T Mikolov, K Chen, G Corrado, and J Computation and Language (cs.CL), arXiv:1301.3781, Jan 2013 [14] A Galassi, M Lippi, and P Computation and Language (cs.CL), arXiv:1902.02181, Feb 2019 [15] A Vaswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, A N Gomez, L Kaiser, and I Polosukhin 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA , arXiv: 1706.03762, Jun 2017 [16] V P T Cherian, and A -cell lstm based neural language Neural and Evolutionary Computing (cs.NE), arXiv:1811.06477, Nov 2018 [17] F Workshop on Pattern Recognition in Practice, pp 381 397, Jan 1980 [18] Chris Manning and Hinrich Schütze, Foundations of Statistical Natural Language Processing, Cambridge, MA: MIT Press, May 1999 [19] T W Kishore Papineni, Salim Roukos and W.ACL-2002: 40th Annual meeting of the Association for Computational Linguistics, pp 311 318, Jul 2002 [20] comprehensive and diagnostic mt evaluation: Initial arabic, chinese, Proceedings of Human Language Technology, pp 132 137, Mar 2002 [21] M O Chris Callison- -evaluating the role of bleu 1th Conference of the European Chapter of the Association for Computational Linguistics: EACL, Association for Computational Linguistics, pp 249 256, Apr 2006 40 41 PH N LÝ L CH TRÍCH NGANG Thành Nhân - Sinh viên khoa KH & KT Máy tính (khóa 2014 2018) (khóa 2018 nay) - Q TRÌNH CƠNG TÁC - 42 ... editors during compilation With the goal of coming up with impressive titles, often the author will choose words that never appear in the original passage, and this is a huge challenge for previous... Out-Of-Vocabulary UNK Unknown Token Seq2Seq Sequence-to-Sequence x GI I THI U 1.1 Gi i thi tài Hình 1 1.2 M c tiêu c 1.3 Ph tài tài CƠNG TRÌNH LIÊN QUAN - 2.1 pháp tóm t t c abstraction method n extraction... reviewerID, Text Language Model Language Model 27 5.2 K t qu hu n luy n Base-Seq2Seq PGN-LM Hình 18 cho Base-Seq2Seq the bigelow Base-Seq2Seq Hình 18 -Seq2Seq 28 hình PGN-LM mơ hình PGN-LM Chúng