... essential for automaticevaluationofChinesetranslation output. Our analysis suggests several key reasons behind this finding. 2 ChineseTranslationEvaluation Automatic MT evaluation aims ... adopted as the smallest unit in most tasks ofChinese language processing. However, for automaticevaluationof the quali-ty ofChinesetranslationoutput when translat-ing from other languages, ... Linguistics Automatic EvaluationofChineseTranslation Output: Word-Level or Character-Level? Maoxi Li Chengqing Zong Hwee Tou Ng National Laboratory of Pattern Recognition Institute of Automation,...
... present the evaluations of ROUGE-L, ROUGE-S, and compare their per-formance with other automaticevaluation meas-ures. 5 Evaluations One of the goals of developing automatic evalua-tion ... averages of the adequacy and fluency scores of each system assigned by human evalua-tors. For the input ofautomaticevaluation meth-ods, we created three evaluation sets from the MT outputs: ... Quality Measure for Machine Translation System. In Proceedings of COLING-92, Nantes, France. Thompson, H. S. 1991. AutomaticEvaluationof Translation Quality: Outline of Methodology and Report...
... Evaluationof Ma-chine Translation. Technical Report RC22176, IBM.Joseph Turian, Luke Shen, and I. Dan Melamed. 2003. Evalua-tion of Machine Translation and its evaluation. In Proceedings of MT ... grammar, rhythm and flow,appropriateness of tone, and several other specificcharacteristics of good text.In terms ofautomatic evaluation, we are not aware of any technique that measures only fluency ... source contentwith fluency of the resulting text. In thispaper we develop an automatic evaluation metric to estimate fluency alone, by examin-ing the use of parser outputs as metrics, andshow...
... highlights parts of the MT output that it suspects are non-grammatical.Even though the main purpose of BLAST is forerror annotation of machine translation output, thefreedom in the use of error typologies ... ref-erence translation. 1 IntroductionMachine translationevaluation is a difficult task,since there is not only one correct translationof asentence, but many equally good translation options.Often, ... developers.2 MT Evaluation and Error AnalysisHovy et al. (2002) discussed the complexity of MT evaluation, and stressed the importance of adjusting evaluation to the purpose and context of the trans-lation....
... trans./word4.21Rewritten TranslationsOriginal TranslationsTable 1: Comparison of TDMT Training Translations and Original Translationsthe rule writers rewrote translations to make tar-get ... words in the target languageand the mean number oftranslation words, bothvalues of the rewritten translations are lower thanthose of the original translations. This is because1When TDMT translates ... number of word links When itis calculated only from the sentences that containfour word links, the value of the rewritten transla-tions is higher than that of the original translations.4 Translation...
... year of evaluation to the next.5.2 Evaluationof rankingsWe examine the predictive power of our featuresfor each of the five linguistic quality questions intwo settings. In system-level evaluation, ... of systems,ignoring the linguistic quality of the system out-put. Part of the reason for this imbalance is theexistence of ROUGE (Lin and Hovy, 2003; Lin,2004), the system for automaticevaluation ... row of Table 1 thatnone of the system rankings based on these mea-sures of linguistic quality are significantly posi-tively correlated with system rankings of content.The development of automatic...
... 2003. Appli-cations ofautomaticevaluation methods to measuringa capability of speech translation system. In Proceed-ings of the 10th Conference of the European Chap-ter of the Association ... improve further.7 Discussion7.1 Other AutomaticEvaluation MethodsThe idea of feedback cleaning is independent of BLEU. Some automaticevaluation methods of MTquality other than BLEU have been ... rules.3 AutomaticEvaluationof MT QualityWe utilize BLEU (Papineni et al., 2002) for the au-tomatic evaluationof MT quality in this paper.BLEU measures the similarity between MT re-sults and translation...
... chal-lenge is the evaluation: manual evaluation is adifficult, time-consuming process and not ap-plicable within efficient development of sys-tems. Automaticevaluation requires a cor-pus of questions ... Y are known enemies.The problem of several possible answers and, inconsequence, automaticevaluation has been tackledfor years within another field of study: automatic summarisation (Hori et ... slow a rate of data acquisition. Therefore it is of- ten necessary to raise the reward and rely on efficient automatic validation of the data.We have looked into the answer agreement of the workers...
... collaboration of Professor Anthony Oettinger and of his other colleagues at the Harvard Computation Laboratory. semiautomatic synthesis oftranslation algorithms from empirical data, a means of formula ... Synthesis ofTranslation Algorithms by Vincent E. Giuliano*, Computation Laboratory of Harvard University A system of procedures and computer programs is proposed for the semi -automatic synthesis of ... predicates. Information enabling the automatic specifi- cation of each of these variables is present in the form of grammatical codes in the entries of the Harvard Automatic Dictionary. The indicated...
... of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 522–530,Suntec, Singapore, 2-7 August 2009.c2009 ACL and AFNLP Automatic Adaptation of Annotation Standards: Chinese ... efficacy of thismethod in the context ofChinese wordsegmentation and part -of- speech tagging,where no segmentation and POS taggingstandards are widely accepted due to thelack of morphology in Chinese. ... latter problem often implies the former,as in our case study.To test the efficacy of our method we choose Chinese word segmentation and part -of- speechtagging, where the problem of incompatible...
... Assessments of Machine Translation Quality. In Proc. of MT Summit IX, 63–70.Hiroshi Echizen-ya and Kenji Araki. 2007. Auto-matic Evaluationof Machine Translation basedon Recursive Acquisition of an ... Stephen Wan andRobert Dale. 2007. GLEU: Automatic Eval-uation of Sentence-Level Fluency. In Proc. of ACL’07, 344–351.NIST. 2002. Automatic Evaluation of Machine Translation Quality Us-ing N-gram ... with Applications to Machine Translation Evaluation. In Proc. of MT Summit IX, 240–247.Chin-Yew Lin and Franz Josef Och. 2004. Auto-matic Evaluationof Machine Translation Qual-ity Using Longest...