automatic evaluation of text coherence models and representations

... section, we present the evaluations of ROUGE-L, ROUGE-S, and compare their per- formance with other automatic evaluation meas- ures. 5 Evaluations One of the goals of developing automatic evalua- tion ... (Stem) Table 1. Pearson’s ρ and Spearman’s ρ correlations of automatic evaluation measures vs. adequacy and fluency: B LEU1, 4, and 12 are BLEU with maximum of 1, 4, and 12 grams, NIST is the NIST ... cognate candi- dates during construction of N-best translation lexicons from parallel text. Melamed (1995) used the ratio (LCSR) between the length of the LCS of two words and the length of the...

Ngày tải lên: 20/02/2014, 16:20

... Introduction Research and development on automatic and man- ual evaluation of summarization systems have been mainly focused on content coverage (Lin and Hovy, 2003; Nenkova and Passonneau, 2004; ... research on automatic evaluation of summary readability, the Text Analysis Conference (TAC) (Owczarzak and Dang, 2011) introduced a new subtask on readability to its Automatically Evaluating Summaries of ... 1006–1014, Jeju, Republic of Korea, 8-14 July 2012. c 2012 Association for Computational Linguistics Combining Coherence Models and Machine Translation Evaluation Metrics for Summarization Evaluation Ziheng...

Ngày tải lên: 07/03/2014, 18:20

... are. Belz and Reiter (2006) and Reiter and Belz (2009) describe com- parison experiments between the automatic eval- uation of system output and human (expert and non-expert) evaluation of the same ... involves string comparisons between the output of the sys- tem and some gold standard set of strings. Typi- cally automatic metrics from the fields of Machine Translation (e.g. BLEU) or Summarisation ... paper 99 Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 97–100, Suntec, Singapore, 4 August 2009. c 2009 ACL and AFNLP Correlating Human and Automatic Evaluation of a German Surface Realiser Aoife...

Ngày tải lên: 23/03/2014, 17:20

Ngày tải lên: 05/09/2013, 16:10

Ngày tải lên: 11/09/2013, 11:44

Ngày tải lên: 18/02/2014, 11:20

... spelling, good grammar, rhythm and flow, appropriateness of tone, and several other specific characteristics of good text. In terms of automatic evaluation, we are not aware of any technique that measures ... choosing the parsers and the metrics derived from them; generating some texts for human and parser evaluation; and, the key part, getting human judgements on these texts and corre- lating them ... faithfulness and to fluency. In addition, the need for reference texts for an evaluation metric can be problematic, and intuitively seems unneces- sary for characterising an aspect of text quality...

Ngày tải lên: 20/02/2014, 12:20

... intense band of 61 kDa (NrfA) and a band of weak intensity of 19 kDa (NrfH), confirming its hetero-oligomeric nature (Fig. 1, lane 1). However, in the absence of boiling (Fig. 1A, lanes 2 and 4) ... and 4) high molecular mass bands of approximately 110 kDa and > 200 kDa were visible, as well as a faint band at 37 kDa, suggesting the presence of dimers. All of the bands stained positively ... B.H., Scheidt, R. & Osvath, S.R. (1986) Models of the cytochromes b6. The effect of axial ligand plane orientation on the EPR and Mo ¨ ssbauer spectra of low-spin fer- rihemes. J. Am. Chem. Soc....

Ngày tải lên: 21/02/2014, 00:20

... and my family—my wife Wei, our kids, our parents Donna and Robert and Pei and Robert, and my sibs Anne, Beth, and Mikhael, whose contributions of friendship, love, understanding of my professional ... Department of Molecular and Medical Pharmacology, David Geffen School of Medicine, University of California, Los Angeles, CA Sung-Cheng Huang, D.Sc. Professor, Department of Molecular and Medical ... Section, Department of Molecular and Medical Pharmacology, David Geffen School of Medicine, University of California, Los Angeles, CA xiii 1 Clinical Evaluation of Dementia and When to Perform...

Ngày tải lên: 05/03/2014, 23:20

... compression. Artificial Intelligence, 139(1):91–107. M. Lapata and R. Barzilay. 2005. Automatic evalua- tion of text coherence: Models and representations. In International Joint Conference On Artificial ... entity coherence, sentence fluency and lan- guage models are the most powerful classes of fea- tures that should be used in automation of evalu- ation and against which novel predictors of text quality ... Predicting the fluency of text with shallow structural features: case studies of machine translation and human-written text. In Proceedings of EACL, pages 139–147. E. Charniak and M. Elsner. 2009....

Ngày tải lên: 16/03/2014, 23:20

... Chinese Translation Evaluation Automatic MT evaluation aims at formulating au- tomatic metrics to measure the quality of MT out- put. Compared with human assessment, automatic evaluation metrics ... Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/ or ... I. Dan Melamed, Ryan Green and Joseph P. Turian, 2003. Precision and Recall of Machine Translation. Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational...

Ngày tải lên: 17/03/2014, 00:20

... QARLA, for the evaluation of text summarisation systems. The in- put of the framework is a set of man- ual (reference) summaries, a set of base- line (automatic) summaries and a set of similarity ... Metrics. In Proceedings of MT Summit IX, New Orleans,LA. Luke Shen Joseph P. Turian and I. Dan Melamed. 2003. Evaluation of Machine Translation and its Evaluation. In In Proceedings of MT Summit IX, New ... discussion 6.1 Application of similarity metrics to evaluate summaries Both in Text Summarisation and Machine Trans- lation, the automatic evaluation of systems con- sists of computing some similarity...

Ngày tải lên: 17/03/2014, 05:20

... the BLEU score and the monolingual group. Of particular interest is the accuracy of BLEU’s esti- mate of the small difference between S2 and S3 and the larger difference between S3 and H1. The figure also ... and monitored by SPAWAR under contract No. N66001-99-2-8916. The views and findings contained in this material are those of the authors and do not necessarily reflect the position of pol- icy of ... Proceedings of the Eagles Workshop on Standards and Evaluation, Pisa, Italy. Kishore Papineni, Salim Roukos, Todd Ward, John Hen- derson, and Florence Reeder. 2002. Corpus-based comprehensive and diagnostic...

Ngày tải lên: 23/03/2014, 20:20

Ngày tải lên: 31/03/2014, 04:20

Ngày tải lên: 31/03/2014, 21:20

Ngày tải lên: 20/06/2014, 23:20

... the evaluation: manual evaluation is a difficult, time-consuming process and not ap- plicable within efficient development of sys- tems. Automatic evaluation requires a cor- pus of questions and ... problem of several possible answers and, in consequence, automatic evaluation has been tackled for years within another field of study: automatic summarisation (Hori et al., 2003; Lin and Hovy, 2003). ... selection of natural questions. The articles varied in topic, degree of formality and the amount of details; from ”Horror film” and ”Christ- mas worldwide” to ”G-Man (Half-Life)” and ”His- tory of London”....

Ngày tải lên: 20/02/2014, 09:20

... will illustrate the ad- vantages and disadvantages of these parsers and rep- resentations, leading us to better parsing models and a better design for parse representations. 4.4 Comparison with ... dependency parsers (Mc- Donald and Pereira, 2006; Nivre and Nilsson, 2005; Sagae and Tsujii, 2007) and deep parsers (Kaplan et al., 2004; Clark and Curran, 2004; Miyao and Tsujii, 2008). However, ... Dependency-based evaluation of MINI- PAR. In LREC Workshop on the Evaluation of Parsing Systems. M. Marcus, B. Santorini, and M. A. Marcinkiewicz. 1994. Building a large annotated corpus of En- glish:...

Ngày tải lên: 08/03/2014, 01:20

Ngày tải lên: 24/03/2014, 02:20