regression for sentencelevel mt evaluation

Ngày tải lên: 31/03/2014, 01:20

... word alignment information. 3 Experiments 3.1 PORT as an Evaluation Metric We studied PORT as an evaluation metric on WMT data; test sets include WMT 2008, WMT 2009, and WMT 2010 all-to-English, ... and 22.0% ties). 1 Introduction Automatic evaluation metrics for machine translation (MT) quality are a key part of building statistical MT (SMT) systems. They play two 1 PORT: Precision-Order-Recall ... An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of ACL Workshop on Intrinsic & Extrinsic Evaluation Measures for Machine Translation...

Ngày tải lên: 19/02/2014, 19:20

... a good MT- evaluation metric. Second, we analyze the resource requirement for regression models for different sizes of feature sets through learning curves. Finally, we show that SVM -regression ... datasets: NIST 2002 Chinese MT Evaluation (3 systems, 2634 sentences total), NIST 2003 Arabic MT Evaluation (2 systems, 1326 sentences total), and NIST 2004 Chinese MT Evalu- ation (10 systems, ... in- valuable resource for measuring the reliability of au- tomatic evaluation metrics. In this paper, we show that they are also informative in developing better metrics. 3 MT Evaluation with Machine...

Ngày tải lên: 08/03/2014, 02:21

... Association for Computational Linguistics, pages 139–144, Jeju, Republic of Korea, 8-14 July 2012. c 2012 Association for Computational Linguistics A Graphical Interface for MT Evaluation and ... rich set of metrics and meta-metrics for assessing MT quality (Gim ´ enez and M ` arquez, 2010a). Although automatic MT evaluation is still far from manual evaluation, it is indeed necessary to ... existing evaluation measures and to support the development of further improve- ments or even totally new evaluation metrics. This information can be gathered both from the experi- 139 Figure 1: MT...

Ngày tải lên: 16/03/2014, 20:20

Ngày tải lên: 31/03/2014, 00:20

Ngày tải lên: 07/12/2013, 11:41

... seems that there are people willing to work on them even for free. MTurk requesters cannot however rely on this voluntary workforce. From MTurk Forums it is clear that some of the workers rely on ... significant chal- lenge is the evaluation: manual evaluation is a difficult, time-consuming process and not ap- plicable within efficient development of sys- tems. Automatic evaluation requires a cor- pus ... contains information re- lated to the corpus collection process. We be- lieve this additional information can be used to post-process the data, and to develop an auto- matic approval system for further...

Ngày tải lên: 20/02/2014, 09:20

... were translated (Akiba et al., 2004). For purposes of au- tomatic evaluation, 16 reference translations and outputs by 20 different MT systems are available for each sentence. Moreover, each of ... table for nine o’clock 7: my name is endo and i reserved a table with you for nine o’clock 8: i ’ve booked a table under endo for nine o’clock 9: my name is endo and i have a table reserved for ... reservation for a table at nine o’clock 11: my name is endo and i reserved a table for nine o’clock 12: the name is endo and i have a reservation for nine 13: i have a table reserved for nine under...

Ngày tải lên: 20/02/2014, 12:20

... used in the vec- tor-space model for Information Retrieval (Salton and Leck, 1968) and the S-score proposed for evaluating MT output corpora for the purposes of Information Extraction (Babych ... for translation: MT systems that have no means for prioritising this information often in- troduce excessive information noise into the tar- get text by literally translating structural information, ... translation variation for automatic evaluation of MT quality. In: Proceedings of LREC 2004 (forthcoming). Papineni, K., S. Roukos, T. Ward, W J. Zhu. 2002 BLEU: a method for automatic evaluation of...

Ngày tải lên: 20/02/2014, 16:20

... for evaluation General statistics for the AdjN and PNV base sets are given in Table 1. Manual annotation was performed for AdjN pairs with frequency and PNV triples with only (see section 5 for ... best measure for identifying AdjN collocations, except for - coordinates between 15% and 20% where t-test outperforms log-likelihood. For PNV data, the curves of all measures (ex- cept for frequency) ... around the best-performing measure for the AdjN data with (log-likelihood). There is no significant difference between log- likelihood and t-test. And only for -best lists with , frequency performs marginally significantly...

Ngày tải lên: 20/02/2014, 18:20

... that are used for disambiguation and parse evaluation. After some experimentation, the evaluation feature weights were set in the following way. As previously described, the penalty for a skipped ... CMU-CMT-88-MEMO, 1988. [Tomita, 1986] M. Tomita. Efficient Parsing for Nat. nral Language. Kluwer Academic Publishers, Hing- ham, Ma., 1986. 318 AN INTEGRATED HEURISTIC SCHEME FOR PARTIAL ... bad parses. Our results indicate that our full integrated heuris- tic scheme for selecting the best parse out-performs the simple heuristic, that considers only the number of words skipped....

Ngày tải lên: 20/02/2014, 21:20

... it is well-formed or ill-formed. • maximal iff it is well-formed and its parent (if it has one) is ill-formed. In other words, a maxi- mal TNCB is a largest well-formed component of a TNCB. ... time algorithm for lexicalist MT generation pro- vided that sufficient information can be transferred to ensure more determinism. 1 Introduction Lexicalist approaches to MT, particularly ... dominating the new well- formed node are disrupted. By dominance monotonicity, all nodes which were disrupted by the adjunction must become well-formed af- ter re -evaluation. And nodes dominating...

Ngày tải lên: 20/02/2014, 22:20

... Roles (Gimenez and Marquez, 2007) proposed using deeper linguistic information to evaluate MT per- formance. For evaluation in the ACL-07 MT work- shop, the authors used the metric which they termed as ... auto- matic metric for MT evaluation with improved corre- lation with human judgments. In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Mea- sures for MT and/or Summarization, ... Correlations on the News Commentary dataset. MT 2003 evaluation exercise. 5.1 ACL-07 MT Workshop The ACL-07 MT workshop evaluated the transla- tion quality of MT systems on various translation tasks,...

Ngày tải lên: 08/03/2014, 01:20

... N SC ORE (mt, M, ref, N)  Compute the alignment score of the MT output mt with length M and the reference ref with length N for i = 1; i ≤ M; i = i +1 do for j = 1; j ≤ N; j = j +1 do for k = ... means. While sentence-level evaluation is useful if we are interested in a confidence measure on MT out- puts, syste-x level evaluation is more useful for comparing MT systems and guiding their ... (+07.8%) Table 4: 95% significance intervals for sentence- level adequacy evaluation cult for one metric to significantly outperform an- other metric in sentence-level evaluation. The re- sults show that...

Ngày tải lên: 08/03/2014, 02:21

... Translating Genomic-Based Research for Health Board on Health Sciences Policy SyStemS for reSearch and evaluation for tranSlating Genome-Based discoveries for health W o r k s h o p s u m m ... horizon, understand what the public is pushing for, and determine at 2 SYSTEMS FOR RESEARCH AND EVALUATION Research for Health identified a need for a workshop to examine existing systems that ... acceptability, and other contextual issues before making a decision. Quantitative Information for Decision Making Quantitative information needed for decision making includes data on effectiveness,...

Ngày tải lên: 15/03/2014, 15:20

... summary for a given document, something that a key approach to evaluation does not capture. Brandow et al. (1995) found this to be the case for some documents where all ANES-genemted and ... news. Leading text extracts such as the LEAD field are appealing for commercial use as summaries for a number of reasons. For general news documents, they are usually acceptable as summaries. ... this investigation, we found that to be the case for most list and newsbrief documents and for many transcripts. Beyond news data, this holds for case law documents, many types of financial...

Ngày tải lên: 17/03/2014, 07:20

... automatic evaluation measures and human judgment. 1 Introduction Research in machine translation (MT) depends heavily on the evaluation of its results. Espe- cially for the development of an MT system, an ... a method for evaluation automatic evaluation metrics for machine translation. COLING 2004, pages 501– 507, Geneva, Switzerland, Aug. D. Lopresti and A. Tomkins. 1997. Block edit models for approximate ... tedious and cost-intensive, automatic evaluation measures are used in most MT research tasks. A high correlation between these automatic evaluation measures and human evaluation is thus desirable. State-of-the-art...

Ngày tải lên: 17/03/2014, 22:20

... 3: for serum calcium and phos- phorus, every 6–12 months; and for PTH, based on baseline level and CKD progression. K In CKD stage 4: for serum calcium and phos- phorus, every 3–6 months; and for ... follow-up, and the quality grade for the respective outcome. Conceptually, information on the left upper corner shows high-quality evidence for outcomes of high importance. Information on the right lower ... low-quality evidence for outcomes of lesser impor- tance. Evidence for AEs was not graded for quality, but still tabulated in one column in the matrices. An evidence matrix was not generated for a systematic review...

Ngày tải lên: 22/03/2014, 09:20

Ngày tải lên: 23/03/2014, 13:20

Ngày tải lên: 26/03/2014, 14:07