Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language)

Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language) Nghiên cứu tổng hợp tiếng nói cho ngôn ngữ ít nguồn tài nguyên theo hướng thích nghi, ứng dụng với tiếng Mường (Speech Synthesis for LowResourced Languages based on Adaptation Approach Application to Muong Language)

DECLARATION OF AUTHORSHIP I, Pham Van Dong, declare that the dissertation titled “Speech Synthesis for LowResourced Languages based on Adaptation Approach: Application to Muong Language” has been entirely composed by myself I assure you of some points as follows:  This work was done wholly or mainly while in candidature for a Ph.D research degree at Hanoi University of Science and Technology  The work has not been submitted for any other degree or qualifications at Hanoi University of Science and Technology or any other institution  Appropriate acknowledgment has been given within this dissertation, where reference has been made to the published work of others  The dissertation submitted is my own, except where work in the collaboration has been included The collaborative contributions have been indicated Hanoi, December 8, 2023 Ph.D Student Pham Van Dong ADVISORS Dr Mac Dang Khoa Assoc Prof Tran Do Dat i ACKNOWLEDGMENT Foremost, I would like to express my most sincere and deepest gratitude to my thesis advisors Dr Mạc Đăng Khoa (Speech Communication Department, MultiLab at MICA), Prof TRẦN Đỗ Đạt (The Ministry of Science and Technology, Vietnam) for their continuous support and guidance during my Ph.D program, and for providing me with such a severe and inspiring research environment I am grateful to Dr Mạc Đăng Khoa for his excellent mentorship, caring, patience, and immense Text-To-Speech (TTS) knowledge His advice helped me in all the research and writing of this thesis I am very thankful to Prof Đạt for shaping my thesis at the beginning and for their enthusiasm and encouragement Prof Trần Đỗ Đạt substantially facilitated my Ph.D research, especially when I was a freshman on speech processing and TTS, with his valuable comments on Vietnamese and Muong TTS I thank all MICA members for their help during my Ph.D study My sincere thanks to Dr Nguyen Viet Son, Assoc Prof Dao Trung Kien and Dr Do Thi Ngoc Diep for giving me much support and valuable advice Thanks to Nguyen Van Thinh, Nguyen Tien Thanh, Dang Thanh Mai, and Vu Thi Hai Ha for their help I want to thank my Hanoi University of Mining and Geology colleagues for all their support during my Ph.D study Special thanks to my family for understanding my hours glued to the computer screen Hanoi, December 8, 2023 Ph.D Student ii ABSTRACT Text-to-speech (TTS) synthesis is the automatic conversion of text into speech Typically, building high-quality voiceovers requires collecting tens of hours of the voice of a professional speaker with a high-quality microphone There are about 7,000 languages spoken worldwide, but only a few languages, such as English, Spanish, Mandarin, and Japanese, are used in good TTS With so-called "low-resourced languages" or even languages that are not yet written, these languages not have TTS Thus, to apply TTS technology to low-resourced language, it is necessary to study other TTS methods In Vietnam, Vietnamese is the mother tongue and is used the most The Muong is a group of the language spoken by the Muong people of Vietnam They are in the Austroasiatic language family and are closely related to Vietnamese, and Muong is also one of the five ethnic groups with the largest population However, Muong still needs an official script, a typical representative of the low-resourced language in Vietnam Therefore, researching TTS technologies to create TTS for the Muong language is challenging In the first part of this thesis, we an overview of TTS Researching the phonetics of Vietnamese and Muong languages, the thesis has also researched and published some tools to support TTS technology for Vietnamese and Muong languages In the rest of the thesis, we conduct various experiments in creating TTS for low-resourced language; specifically, we experiment with the Muong language We focus on two main low-resourced language groups:  Written: We use emulating to simulate the reading of the Muong language using Vietnamese TTS and cross-lingual adaptation transfer-learning  Unwritten: We experiment with adaptation in two directions The first is to create Muong speech synthesis directly from Vietnamese Text and Muong voice The second is to create Muong speech synthesis from translation through intermediate representation We hope our findings can serve as an impetus to develop speech synthesis for low-resourced languages worldwide and contribute to the basis for speech synthesis development for 53 ethnic minority languages in Viet Nam Hanoi, December 8, 2023 Ph.D Student iii CONTENT DECLARATION OF AUTHORSHIP .I ACKNOWLEDGMENT II ABSTRACT III CONTENT IV ABBREVIATIONS VIII LIST OF TABLES X LIST OF FIGURES XII INTRODUCTION PART : BACKGROUND AND RELATED WORKS CHAPTER OVERVIEW OF SPEECH SYNTHESIS AND SPEECH SYNTHESIS FOR LOW-RESOURCED LANGUAGE 1.1 Overview of speech synthesis 1.1.1 Overview 1.1.2 TTS architecture 1.1.3 Evolution of TTS methods over time 1.1.3.1 TTS using unit-selection method 10 1.1.3.2 Statistical parameter speech synthesis 11 1.1.3.3 Speech synthesis using deep neural networks 13 1.1.3.4 Neural speech synthesis 14 1.2 Speech synthesis for low-resourced languages 19 1.2.1 TTS using emulating input approach 20 1.2.2 TTS using the polyglot approach 22 1.2.3 Speech synthesis for low-resourced language using the adaptation approach 25 1.3 Machine translation 27 1.3.1 Neural translation model 28 1.3.2 Attention in neural machine translation 29 1.3.3 Statistical machine translation based on phrase 30 1.3.3.1 Statistical machine translation problem based on phrase 30 1.3.3.2 Translation model and language model 31 1.3.3.3 Decode the input sentence in the translation system 32 1.3.3.4 Model for building a statistical translation system 34 1.3.4 Machine translation through intermediate representation 34 1.3.5 Speech translation for unwritten low-resourced languages 36 1.4 Speech synthesis evaluation metrics 38 1.4.1 Mean Opinion Score (MOS) 38 1.4.1.1 Definition 38 1.4.1.2 Formula 38 1.4.1.3 Significance 38 1.4.1.4 Confidence Interval (CI) 39 1.4.2 Mel Cepstral Distortion (MCD) 39 iv 1.4.2.1 Concept 39 1.4.2.2 Formula 39 1.4.2.3 Significance 40 1.4.2.4 MCD with Dynamic Time Warping (MCD – DTW) 40 1.4.3 Analysis of variance (Anova) 40 1.4.4 Intelligibility 42 1.5 Conclusion 42 CHAPTER VIETNAMESE AND MUONG LANGUAGE 44 2.1 Vietnamese language 44 2.1.1 History of Vietnamese 44 2.1.2 Vietnamese phonetic system 45 2.1.2.1 Vietnamese syllabus structure 46 2.1.2.2 Vietnamese phonetic system 47 2.1.2.3 Vietnamese tone system 49 2.2 Muong language 50 2.2.1 Overview of Muong people and Muong language 50 2.2.1.1 Muong history 50 2.2.1.2 Viet Muong group 51 2.2.1.3 Muong dialects 53 2.2.1.4 Muong written script 54 2.2.2 Muong phonetics system 55 2.2.2.1 Muong syllable structure 55 2.2.2.2 Muong phoneme system 55 2.2.2.3 Muong tone system 57 2.3 Comparison between Vietnamese and Muong 57 2.4 Dicussion and proposal approach 60 PART : SPEECH SYNTHESIS FOR MUONG AS A WRITTEN LANGUAGE 61 CHAPTER EMULATING OF THE MUONG TTS BASED ON INPUT TRANSFORMATION OF THE VIETNAMESE TTS 62 3.1 Proposed method 63 3.1.1 Muong G2P module 64 3.1.2 Muong emulating IPA module 65 3.2 Experiment 65 3.2.1 Testing materials 66 3.2.2 Experiment protocol 67 3.2.3 Results 68 3.2.4 Analysis by ANOVA method 72 3.2.4.1 MOS analysis by ANOVA 72 3.2.4.2 Intelligibility analysis by ANOVA 75 3.3 Conclusion 77 v CHAPTER CROSS-LINGUAL TRANSFER LEARNING FOR MUONG SPEECH SYNTHESIS 78 4.1 Proposed method 78 4.2 Experiment 82 4.2.1 Dataset 82 4.2.1.1 Vietnamese data 82 4.2.1.2 Muong Project‘s data 84 4.2.1.3 Muong fine-tuning data 84 4.2.2 Graphemes to phonemes 85 4.2.3 Training the pretrained model using Vietnamese dataset 86 4.2.4 Finetuned TTS model on Muong datasets 87 4.3 Evaluation 88 4.4 MOS analysis by ANOVA 91 4.5 Conclusion 94 PART : SPEECH SYNTHESIS FOR MUONG AS AN UNWRITTEN LANGUAGE 96 CHAPTER GENERATE UNWRITTEN LOW-RESOURCED LANGUAGE’S SPEECH DIRECTLY FROM RICH-RESOURCE LANGUAGE’S TEXT 97 5.1 Introduction 97 5.2 Proposed method 98 5.2.1 Model architecture 98 5.2.2 Database 99 5.2.3 Training the speech synthesis system 100 5.2.4 Evaluation 100 5.2.5 MOS analysis by ANOVA 105 5.2.5.1 ANOVA analysis in Muong Bi speech synthesis 105 5.2.5.2 ANOVA analysis in Muong Tan Son speech synthesis 108 5.3 Conclusion 111 CHAPTER SPEECH SYNTHESIS FOR UNWRITTEN LOW-RESOURCED LANGUAGE USING INTERMEDIATE REPRESENTATION 112 6.1 Proposal Method 112 6.2 Experiment 114 6.2.1 Database building 114 6.2.2 System development 114 6.2.2.1 Text to phone translation 115 6.2.2.2 Phone to Sound Conversion 117 6.3 Evaluation 119 6.3.1 Evaluation in Muong Bi and Muong Tan Son 119 6.3.2 MOS analysis by ANOVA 122 6.3.2.1 ANOVA analysis in Muong Bi speech synthesis 122 6.3.2.2 ANOVA analysis in Muong Tan Son speech synthesis 125 6.4 Conclusion and comparison 128 CONCLUSION AND FUTURE WORKS 135 vi Conclusions 135 Future work 136 PUBLICATIONS 138 REFERENCES 139 APPENDIX A A.1 Vietnamese and Muong phonetic A.2 Muong G2P A.3 Muong Vietnamese phone mapping A.4 Information of Muong volunteers who participated in the assessment A.5 Speech signal samples of the Muong TTS in chapter 12 vii ABBREVIATIONS Abbreviation CART F0 HMM HTK HTS IPA MARY (TTS) MFCC ML MLSA MOS MSDHMM NLP OCR POS PP PSOLA SAMPA SPTK SSML TDPSOLA TTS VNSP WEKA X-SAMPA XML PLP Expansion Explanation Classification And Regression Tree Fundamental Frequency Hidden Markov Model Hidden markov model ToolKit HMM-based speech synthesis International Phonetic Alphabet Modular Architecture for Research on speech sYnthesis Mel Frequency Cepstral Coefficents Maximum Likelihood Mel Log Spectrum Approximation Mean Opinion Score Multi-Space probability Distribution HMM Natural Language Processing Optical Character Recognition Part-Of-Speech Prepositional Phrase Pitch Synchronous OverLap and Add Speech Assessment Methods Phonetic Alphabet Speech signal Processing ToolKit Speech Synthesis Markup Language Time-Domain Pitch Synchronous OverLap and Add Text-To-Speech VNSpeechCorpus for synthesis Waikato Environment for Knowledge Analysis Extended Speech Assessment Methods Phonetic Alphabet eXtensible Markup Language Perceptual Linear Prediction viii A portable toolkit for building and manipulating hidden Markov models Word class or a lexical category A collection of machine learning algorithms for data mining tasks: G2P ANOVA DNN ANN LPC EM MLE PTN CNN NMT SMT RNN GRU DTW MCD Argmax Log ̅ p(e | f)   H0 Grapheme to Phoneme Analysis of Variance Deep Neural Network Artificial Neural Network Linear Predictive Coding Expectation Maximization Algorithm Maximum Likelihood Network Phonetic Transformation Network Convolutional Neural Network Neural Machine Translation Statistical Machine Translation Recurrent Neural Network Gated Recurrent Unit Dynamic Time Warping Mel Ceptral Distortion Arguments of the maxima Logarith Sample mean Conditional Probability Pi Argmax is an operation that finds the argument that gives the maximum value from a target function Product of a sequence of numbers Sigma Factor Null Hypothesis ix LIST OF TABLES Table 2.1 Vietnamese syllabus structure [94] 46 Table 2.2 Vietnamese syllabus structure [96] 46 Table 2.3 Vietnamese syllables based on structure 47 Table 2.4 Hanoi Vietnamese inital consonants 48 Table 2.5 The letter of initial consonant 48 Table 2.6 Hanoi Vietnamese final consonant 49 Table 2.7 Tone of Hanoi Vietnamese [108] 49 Table 2.8 Muong syllabic structure 55 Table 2.9 Muong final sound system 56 Table 2.10 Muong Hoa Binh tone system [115] .57 Table 2.11 Muong Bi and Muong Tan Son Tone 57 Table 2.12 Muong and Vietnamese phonetic comparison (orthography in normal, IPA in italic; Vi: Vietnamese; Mb: Muong Bi ; Mts : Muong Tan Son) .59 Table 2.13 Comparing the tone of Vietnamese with Muong Tan Son and Muong Bi .60 Table 3.1 Muong G2P Result Sample .64 Table 3.2 Examples of applying transformation rules to convert the Muong text into input text for Vietnamese TTS 65 Table 3.3 Testing material for emulating tone 66 Table 3.4 Testing material for emulating phone (the concerning phonemes in bold) .67 Table 3.5 Testing material for remaining phonemes .67 Table 3.6 ANOVA Results for MOS Test 73 Table 3.7 ANOVA Results for Intelligibility Test 75 Table 4.1 Parameters of acoustic model 80 Table 4.2 Vietnamese dataset information 83 Table 4.3 Muong recorded data 85 Table 4.4 The Muong split data set 85 Table 4.5 Parameter for optimizer 86 Table 4.6 Value of parameters when training Hifigan model 86 Table 4.7 The specifications of the in-domain and out-domain test sets 89 Table 4.8 Test set samples 89 Table 4.9 Evaluation results .90 Table 4.10 ANOVA Results for in-domain MOS Test .92 Table 4.11 ANOVA Results for out-domain MOS Test .93 Table 4.12 ANOVA Results for in/out domain MOS Test 94 Table 5.1 Evaluation Score .102 Table 5.2 TTS evaluation with in-domain test set 103 Table 5.3 TTS evaluation with out-domain test set 104 Table 5.4 ANOVA Results for in-domain MOS Test for Muong Bi 106 Table 5.5 ANOVA Results for out-domain MOS Test for Muong Bi 107 Table 5.6 ANOVA Results for Muong Bi in/out domain MOS Test 107 Table 5.7 ANOVA Results for in-domain MOS Test for Muong Tan Son .109 Table 5.8 ANOVA Results for out-domain MOS Test for Muong Tan Son .110 Table 5.9 ANOVA Results for Muong Tan Son in/out domain MOS Test .110 Table 6.1 Examples of labeling Vietnamese text into an intermediate representation of Muong Bi and Muong Tan Son phonemes 117 Table 6.2 Text information of Muong language datasets .118 Table 6.3 TTS evaluation with in-domain test set 119 Table 6.4 TTS evaluation with out-domain test set 120 x Labial Aspirate Labiodental laminal – dental - flat Laminal dorsal Tongue root c-ch cha – vườn; k-k|c ka – gà Voiceless p-p pa – ba t-t ta – da Aspirate ph – th-th thăi – tay kh – kh kha – vợt cá tl-tl tlôốcđầu kl-kl klời – trời ph phew – tre Lateral approxi mant Lateral approxim ant Nasal Nasal Nasal Frica tive Fricativ e Voiced Fricativ e Voiceless Fricativ e Lateral approxim ant Laminal Laminal m-m mâl – mây n-n nu – củ nâu v - v: vêl z-z zamày ɲ-nh nhuúc – thịt r-r rô - điên Tlottal ŋ-ng ngoóc – ngẩng g-g gả - gả s-x xó – gió h-h – ta l-l lom – gan hr-hr hroóch – ruột Table A.3 Muong vowels system NO Vowel name Writing i i Tongue Open position mouth Front Close Length Lip shape Long Unrounded ɯ ưư trước âm cuối nh, ch, ng, c Back Close Long Unrounded For example ti- đi, pílquả bí chỉlchỉ chưừngvách núi, khưừngsừng, khưức- sức NO 10 Vowel name u e ɤ ɤ̆ o ɛ a ă Writing – trường hợp lại uu trước âm cuối ch, ng, nh, c u – trường hợp lại êê trước âm cuối ch, nh, ng, c ê – trường hợp cịn lại â ơơ trước âm cuối nh, ch, ng, c ô – trường hợp lại e Tongue Open position mouth Back Close Length Lip shape Short Rounded Back Close Long Back Close Long Front Closemid Long Front Closemid Short Back Closemid Long Closemid Short Closemid Long Closemid Long Closemid Long Open Long Back Back Back Front aa trước âm cuối ch, nh Back a – trường hợp lại ă Back Unrounded Unrounded Unrounded Unrounded Unrounded Long Unrounded Front Open Short Rounded 11 ɔ oo trước âm cuối nh, ch, ng, c Back Closemid khứl- lợi, từ- nhiều huunh- hôn, buúch- mút, Unrounded kướngsướng pul- vun, pùn- bùn, Unrounded kùmchuồng khêênhgần, têếnhUnrounded đến, têếchđứt tê- dê, pềnbền, tlê, Rounded đếp- nếp Unrounded Open For example Long Unrounded tlớngtrứng, tởđể cho, kởigửi chấy- giấy, kâl- gỗ, mâl- mây thôốc- đẹp, hôốc- gọi, tlôốngtrống thốl- tối, thôm- tôm, tlốn- trốn tẻ- đẻ, enhanh, pénbán taanh- đan, paánh- bán, kaách- cát káng- cằm, nàw- cào, mal- rắn cạp nong tắng- đắng, tắnh- đánh, tăw- đau nhoọnhnhọn, poóch- vót, oóng- uống NO Vowel name 12 iə Writing Tongue Open position mouth o – trường hợp lại iê Back Front Length Closemid Short Close Diphthong Lip shape Unrounded Rounded 13 ɯɤ ươ Central Close Diphthong Unrounded 14 uə uô Back Close Diphthong Unrounded For example non- trẻ, chól- gà sống, bóimuối tiể- đỉa, iếng- nghe, ká chiếc- cá diếc lươnhlươn, mườlmười, khướngsân puốchvuốt, ká chuốl- cá chuối, kuốn- Table A.4 The correspondences between Vietnamese and Muong in 12 words refer to the human body parts [137] No English Vietnamese (IPA) Muong (IPA) head ɗɤ̆wA2, ʈokD1 ʈokD1 hair tɔkD1 tʰak eyes mătD1 mătD1 ear tayA1 tʰai nose muyB1 mui mouth miəŋB2 mɛŋ tooth răŋA1 not mentioned tongue lɯəyC2 laːi neck koC1 kel, kok 10 lip moyA1 not mentioned 11 chin kămA2 kăŋ 12 arm (or hand) tăyA1 tʰai A.2 Muong G2P Table A.5 Muong G2P Type Onsets Graphemes b t th Phonemes b t th Type OnGlides OffGlides Graphemes đ, d, gi ch kh g, gh l m n ngh, ng nh ph v x, s h p qu q, k, c tr r hr kl tl oa, ố, ồ, oả, ỗ, oạ, óa, ịa, ỏa, õa, ọa, ua, , , uả, uã, uạ oă, oắ, oằ, oẳ, oẵ, oặ, uă, uắ, uằ, uẳ, uẵ, uặ oe, oé, oè, oẻ, oẽ, oẹ, óe, òe, ỏe, õe, ọe, ue, ué, uè, uẻ, uẽ, uẹ uâ, uấ, uầ, uẩ, uẫ, uậ uê, uế, uề, uể, uễ, uệ uơ, uớ, uờ, uở, uỡ, uợ uy, uý, uỳ, uỷ, uỹ, uỵ, ùy uya, uýa, uỳa, uỷa, uỹa, uỵa, uyê, uyế, uyề, uyể, uyễ, uyệ uyu, uyú, uyù, uyủ, uyũ, uyụ, uýu, uỳu, uỷu, uỹu, uỵu oen, oén, oèn, oẻn, oẽn, oẹn oet, oét, oèt, oẻt, oẽt, oẹt ai, ái, ài, ải, ãi, ại ay, áy, ày, ảy, ãy, ạy ao, áo, ào, ảo, ão, ạo au, áu, àu, ảu, ãu, ạu ây, ấy, ầy, ẩy, ẫy, ậy âu, ấu, ầu, ẩu, ẫu, ậu eo, éo, èo, ẻo, ẽo, ẹo iu, íu, ìu, ỉu, ĩu, ịu oi, ói, ịi, ỏi, õi, ọi ơi, ối, ồi, ổi, ỗi, ội ui, úi, ùi, ủi, ũi, ụi uy, úy, ùy, ủy, ũy, ụy ơi, ới, ời, ởi, ỡi, ợi ưi, ứi, ừi, ửi, ữi, ựi ưu, ứu, ừu, ửu, ữu, ựu iêu, iếu, iều, iểu, iễu, iệu, yêu, yếu, yều, yểu, yễu, yệu Phonemes d c kh g l m n ng nh ph v s h p kw k tr r hr kl tl a aw e aa ee ow i ie i uz e nz e tc a iz aw iz a uz aw uz aa iz aa uz e uz i uz o iz oo iz u iz wi ow iz uw iz uw uz ie uz Type OnOffGlides Nuclides Codas Tones Graphemes uôi, uối, uồi, uổi, uỗi, uội ươi, ưới, ười, ưởi, ưỡi, ượi ươu, ướu, ườu, ưởu, ưỡu, ượu oe, oé, oè, oẻ, oẽ, oẹ oai, ối, ồi, oải, ỗi, oại, uai, i, i, uải, i, uại oay, ốy, ồy, oảy, ỗy, oạy, uay, y, y, uảy, y, uạy oao, ốo, ồo, oảo, ỗo, oạo oeo, o, o, oẻo, oẽo, oẹo, óeo, ịeo, ỏeo, õeo, ọeo, ueo, uéo, uèo, uẻo, uẽo, uẹo uây, uấy, uầy, uẩy, uẫy, uậy a, á, à, ả, ã, â, ấ, ầ, ẩ, ẫ, ậ ă, ắ, ằ, ẳ, ẵ, ặ e, é, è, ẻ, ẽ, ẹ ê, ế, ề, ể, ễ, ệ i, í, ì, ỉ, ĩ, ị, y, ý, ỳ, ỷ, ỹ, ỵ o, ó, ị, ỏ, õ, ọ, oo, óo, ịo, ỏo, õo, ọo, , , oỏ, õ, oọ, ơơ, ốơ, ồơ, ổơ, ỗơ, ộơ, ơố, ôồ, ôổ, ôỗ, ôộ ô, ố, ồ, ổ, ỗ, ộ ơ, ớ, ờ, ở, ỡ, ợ u, ú, ù, ủ, ũ, ụ ư, ứ, ừ, ử, ữ, ự eo, éo, èo, ẻo, ẽo, ẹo êu, ếu, ều, ểu, ễu, ệu ia, ía, ìa, ỉa, ĩa, ịa, iá, ià, iả, iã, iạ, ya, ýa, ỳa, ỷa, ỹa, ỵa, yá, yà, yả, yã, yạ, iê, iế, iề, iể, iễ, iệ, yê, yế, yề, yể, yễ, yệ yu, ýu, ỳu, ỷu, ỹu, ỵu, yú, yù, yủ, yũ, yụ ua, úa, ùa, ủa, ũa, ụa, uô, uố, uồ, uổ, uỗ, uộ, uơ, uớ, uờ, uở, uỡ, uợ ưa, ứa, ừa, ửa, ữa, ựa, ươ, ướ, ườ, ưở, ưỡ, ượ p t c, ch m n ng, nh l á, ấ, ắ, é, ế, í, ó, ố, ớ, ú, ứ, ý à, ầ, ằ, è, ề, ì, ị, ồ, ờ, ù, ừ, ỳ ả, ẩ, ẳ, ẻ, ể, ỉ, ỏ, ổ, ở, ủ, ử, ỷ ã, ẫ, ẵ, ẽ, ễ, ĩ, õ, ỗ, ỡ, ũ, ữ, ỹ ạ, ậ, ặ, ẹ, ệ, ị, ọ, ộ, ợ, ụ, ự, ỵ A.3 Muong Vietnamese phone mapping Phonemes uo iz wa iz wa uz we a iz aw iz a uz e uz aa iz a aa aw e ee i o oo ow u uw e uz ee uz ie i uz uo wa pc tc kc mz nz ngz l Table A.6 Muong Vietnamese phone mapping Type Onsets OnGlides OffGlides Graphemes b c, k ch d đ g, gh gi h kh l m n ng, ngh nh ph p qu r s t th tr v x hr kl tl w oa, ố, ồ, oả, ỗ, oạ, óa, ịa, ỏa, õa, ọa, ua, uá, uà, uả, uã, uạ oă, oắ, oằ, oẳ, oẵ, oặ, uă, uắ, uằ, uẳ, uẵ, uặ oe, , , oẻ, oẽ, oẹ, óe, ịe, ỏe, õe, ọe, ue, ué, uè, uẻ, uẽ, uẹ uâ, uấ, uầ, uẩ, uẫ, uậ uê, uế, uề, uể, uễ, uệ uy, uý, uỳ, uỷ, uỹ, uỵ uya, uyá, uyà, uyả, uyã, uyạ, uyê, uyế, uyề, uyể, uyễ, uyệ oen, oén, oèn, oẻn, oẽn, oẹn oet, oét, oèt, oẻt, oẽt, oẹt ai, ái, ài, ải, ãi, ại ay, áy, ày, ảy, ãy, ạy ao, áo, ào, ảo, ão, ạo au, áu, àu, ảu, ãu, ạu ây, ấy, ầy, ẩy, ẫy, ậy âu, ấu, ầu, ẩu, ẫu, ậu eo, éo, èo, ẻo, ẽo, ẹo êu, ếu, ều, ểu, ễu, ệu iu, íu, ìu, ỉu, ĩu, ịu Phonemes Muong (IPA) ɓ k c j ɗ ɣ z h x l m n ŋ ɲ f p kʷ r ʂ t tʰ ʈ v s hr kl tl w aː a ɛ ə e i iə ɛn ɛt aːj aj aːw aw əj əw ɛw ew iw Phonemes Viet (IPA) ɓ k c j ɗ ɣ z h x l m n ŋ ɲ f p kʷ r ʂ t tʰ ʈ v s aː a ɛ ə e i iə ɛn ɛt aːj aj aːw aw əj əw ɛw ew iw Type OnOffGlides Nuclides Codas Graphemes oi, ói, ịi, ỏi, õi, ọi ôi, ối, ồi, ổi, ỗi, ội ui, úi, ùi, ủi, ũi, ụi, uy, úy, ùy, ủy, ũy, ụy ơi, ới, ời, ởi, ỡi, ợi ưi, ứi, ừi, ửi, ữi, ựi ưu, ứu, ừu, ửu, ữu, ựu iêu, iếu, iều, iểu, iễu, iệu, yêu, yếu, yều, yểu, yễu, yệu uôi, uối, uồi, uổi, uỗi, uội ươi, ưới, ười, ưởi, ưỡi, ượi ươu, ướu, ườu, ưởu, ưỡu, ượu oai, ối, ồi, oải, oãi, oại, uai, uái, uài, uải, uãi, uại oay, oáy, oày, oảy, oãy, oạy, uay, uáy, uày, uảy, uãy, uạy oao, ốo, ồo, oảo, ỗo, oạo oeo, o, o, oẻo, oẽo, oẹo, óeo, ịeo, ỏeo, õeo, ọeo, ueo, o, o, uẻo, uẽo, uẹo uêu, uếu, uều, uểu, uễu, uệu uyu, uyú, uyù, uyủ, uyũ, uyụ, uýu, uỳu, uỷu, uỹu, uỵu uây, uấy, uầy, uẩy, uẫy, uậy a, á, à, ả, ã, â, ấ, ầ, ẩ, ẫ, ậ ă, ắ, ằ, ẳ, ẵ, ặ e, é, è, ẻ, ẽ, ẹ ê, ế, ề, ể, ễ, ệ i, í, ì, ỉ, ĩ, ị, y, ý, ỳ, ỷ, ỹ, ỵ o, ó, ò, ỏ, õ, ọ ô, ố, ồ, ổ, ỗ, ộ ơ, ớ, ờ, ở, ỡ, ợ u, ú, ù, ủ, ũ, ụ ư, ứ, ừ, ử, ữ, ự ia, ía, ìa, ỉa, ĩa, ịa, iá, ià, iả, iã, iạ, iê, iế, iề, iể, iễ, iệ, yê, yế, yề, yể, yễ, yệ oo, óo, ịo, ỏo, õo, ọo, , , oỏ, oõ, oọ ôô, ốô, ồô, ổô, ỗô, ộô, ôố, ôồ, ôổ, ôỗ, ôộ ua, úa, ùa, ủa, ũa, ụa, uô, uố, uồ, uổ, uỗ, uộ, uơ, uở, uờ, uỡ, uợ ưa, ứa, ừa, ửa, ữa, ựa, ươ, ướ, ườ, ưở, ưỡ, ượ c, k ch m n ng nh p Phonemes Muong (IPA) ɔj oj Phonemes Viet (IPA) ɔj oj uj əːj ɨj ɨw uj əːj ɨj ɨw iəw uəj ɨəj ɨəw iəw uəj ɨəj ɨəw aːj aːj aj aw aj aw ɛw ew ɛw ew iw əj aː ə a ɛ e i ɔ o əː u ɨ iw əj aː ə a ɛ e i ɔ o əː u ɨ iə iə ɔː ɔː oː oː uə uə ɨə k c m n ŋ ɲ p ɨə k c m n ŋ ɲ p Type Tones Graphemes t l w á, ấ, ắ, é, ế, í, ó, ố, ớ, ú, ứ, ý à, ầ, ằ, è, ề, ì, ò, ồ, ờ, ù, ừ, ỳ ả, ẩ, ẳ, ẻ, ể, ỉ, ỏ, ổ, ở, ủ, ử, ỷ ã, ẫ, ẵ, ẽ, ễ, ĩ, õ, ỗ, ỡ, ũ, ữ, ỹ ạ, ậ, ặ, ẹ, ệ, ị, ọ, ộ, ợ, ụ, ự, ỵ a, â, ă, e, ê, i, o, ô, ơ, u, ư, y Phonemes Muong (IPA) t l w None Phonemes Viet (IPA) t A.4 Information of Muong volunteers who participated in the assessment Table A.7 Muong Hoa Binh volunteers No Full name Nguyễn Thị Sâm Birthplace Hoa Binh Year old 22 Academic level University Dương Phương Anh Hoa Binh 21 University Phạm Thị Hòa Hoa Binh 20 University Nguyễn Phương Nhung Hoa Binh 20 University Bùi Thanh Chúc Hoa Binh 20 University Đặng Minh Thư Hoa Binh 20 University Bùi Linh Hương Hoa Binh 20 University Bùi Thị Hương Hoa Binh 36 University Bùi Thị Thanh Thảo Hoa Binh 20 University 10 Nguyễn Phương Thúy Hoa Binh 20 University 11 Hà Thị Thu Thủy Hoa Binh 20 University 12 Hồng Thị Hình Hoa Binh 34 University 13 Nguyễn Hồng Như Hoa Binh 20 University 14 Trịnh Thị Hiền Hoa Binh 20 University 15 Vũ Phương Anh Hoa Binh 20 University 16 Nguyễn Thị Ánh Nguyệt Hoa Binh 17 Hight school 17 Nguyễn Thúy Hảo Hoa Binh 51 University 18 Vũ Hương Giang Hoa Binh 17 Hight school 19 Bùi Trang Nhung Hoa Binh 17 Hight school 20 Phùng Thị Chuyên Hoa Binh 35 University 21 Bùi Nhật Hạ Hoa Binh 17 Hight school 22 Tạ Minh Ngọc Hoa Binh 35 University Birthplace No Full name 23 Trịnh Thị Thể Hoa Binh Year old 63 Academic level University 24 Hà Công Đạt Hoa Binh 36 University 25 Phạm Đức Minh Hoa Binh 22 University 26 Đoàn Quang Trường Hoa Binh 21 University 27 Phạm Tuấn Đạt Hoa Binh 20 Hight school 28 Bùi Văn Niên Hoa Binh 37 University 29 Nguyễn Hoài Nam Hoa Binh 20 Hight school 30 Dư Quốc Huy Hoa Binh 20 Hight school 31 Đỗ Ngọc Thiện Hoa Binh 20 Hight school 32 Hồ Quang Anh Hoa Binh 20 Hight school 33 Lương Quang Khải Hoa Binh 20 Hight school 34 Vũ Hà Huy Tuấn Hoa Binh 20 Hight school 35 Nguyễn Mạnh Tú Hoa Binh 20 Hight school 36 Bùi Trọng Quý Hoa Binh 20 Hight school 37 Lại Khánh Duy Hoa Binh 20 Hight school 38 Vũ Thanh Tùng Hoa Binh 20 Hight school 39 Nguyễn Thành Đạt Hoa Binh 20 Hight school 40 Vũ Thanh Tùng Hoa Binh 20 Hight school 41 Kiều Bình Hồng Hoa Binh 20 Hight school 42 Nguyễn Đình Thi Hoa Binh 20 Hight school 43 Dương Minh Trường Hoa Binh 20 Hight school 44 Lê Trung Hiếu Hoa Binh 20 Hight school 45 Tống Hải Long Hoa Binh 20 Hight school 46 Đinh Văn Hiếu Hoa Binh 20 Hight school 47 Trần Duy Minh Hoa Binh 20 Hight school 48 Bùi Trung Sơn Hoa Binh 19 Hight school 49 Trần Văn Thanh Hoa Binh 24 University 50 Bùi Thị Hoa Hoa Binh 22 University Table A.8 Muong Phu Tho volunteers No Full name Hà Thị Bích Birthplace Phu Tho Year old 16 Academic level Hight school Đinh Ngọc Diệp Phu Tho 17 Hight school Hà Thị Bích Huệ Phu Tho 45 University Đinh Thị Huyền Phu Tho 17 Hight school Hà Thị Phương Phu Tho 16 Hight school 10 No Full name Hạ Thị Thu Phương Birthplace Phu Tho Year old 16 Academic level Hight school Phạm Đình Mạnh Phu Tho 41 University Dương Phương Anh Phu Tho 21 University Vũ Thị Linh Chi Phu Tho 22 University 10 Bùi Văn Thái Phu Tho 23 University 11 Hà Văn Năm Phu Tho 16 Hight school 12 Hà Chí Kiên Phu Tho 17 Hight school 13 Đinh Đức Độ Phu Tho 17 Hight school 14 Hà Phú Duy Phu Tho 17 Hight school 15 Mai Thị Quyên Phu Tho 44 University 16 Hà Thị Khánh Linh Phu Tho 17 Hight school 17 Hà Thị Thanh Thúy Phu Tho 17 Hight school 18 Hà Huỳnh Anh Phu Tho 17 Hight school 19 Hà Thị Kiều Phu Tho 17 Hight school 20 Nguyễn Hà Nhật Vy Phu Tho 16 Hight school 21 Đinh Thị Thu Hằng Phu Tho 17 Hight school 22 Hà Thị Khánh Huyền Phu Tho 17 Hight school 23 Trần Thị Việt Hà Phu Tho 18 Hight school 24 Đinh Thị Hoài Phu Tho 18 Hight school 25 Phùng Thị Thanh Phu Tho 18 Hight school 26 Hà Thị Hồng Dương Phu Tho 36 Hight school 27 Hà Thị Bạch Dương Phu Tho 36 Hight school 28 Hà Thị Mai Thuyên Phu Tho 36 Hight school 29 Hà Tú Anh Phu Tho 36 Hight school 30 Phùng Thị Thùy Linh Phu Tho 17 Hight school 31 Hà Quang Trường Phu Tho 17 Hight school 32 Đinh Quang Hùng Phu Tho 19 University 33 Vũ Hải Dương Phu Tho 19 University 34 Nguyễn Tài Thu Phu Tho 19 University 35 Nguyễn Quang Lâm Phu Tho 19 University 36 Hồ Việt Hưng Phu Tho 19 University 37 Nguyễn Tuấn Khanh Phu Tho 19 University 38 Nguyễn Bá Quý Phu Tho 19 University 39 Đào Quang Tùng Phu Tho 20 University 40 Phan Văn Khải Phu Tho 20 University 41 Nguyễn Công Minh Phu Tho 20 University 11 No Full name 42 Nguyễn Xuân Đức Birthplace Phu Tho Year old 20 Academic level University 43 Hoàng Thái Dương Phu Tho 20 University 44 Đỗ Tuấn Anh Phu Tho 20 University 45 Vương Trung Hiếu Phu Tho 20 University 46 Nguyễn Minh Hiếu Phu Tho 20 University 47 Nguyễn Quý Trọng Phu Tho 20 University 48 Phạm Tiến Thành Phu Tho 20 University 49 Trần Xuân Bách Phu Tho 20 University 50 Lê Đức Huy Phu Tho 20 University A.5 Speech signal samples of the Muong TTS in chapter Below are some pictures of the synthesized signal in the sets of in-domain compared with the original signal of two Muong dialects Synthetic sentence: “ban vận động thành lập hội trí thức tỉnh mắt” 12 Figure A.1 Raw Muong Hoa Binh: ban vận động thành lập hội trí thức tỉnh mắt Figure A.2 Muong Hoa Binh synthesis: ban vận động thành lập hội trí thức tỉnh mắt 13 Figure A.3 Muong Phu Tho raw: ban vận động thành lập hội trí thức tỉnh mắt Figure A.4 Muong Phu Tho synthesis: ban vận động thành lập hội trí thức tỉnh mắt Below are some pictures of the synthesized signal in the sets of out-domain compared with the original signal of two Muong dialects Synthetic sentence: “Bố cháu nhà hay đâu?” 14 Figure A.5 Muong Hoa Binh raw - Bố cháu nhà hay đâu Figure A.6 Muong Hoa Binh synthesis: Bố cháu nhà hay đâu 15 Figure A.7 Muong Phu Tho raw: Bố cháu nhà hay đâu Figure A.8 Muong Phu Tho synthesis: Bố cháu nhà hay đâu 16