All, A reminder that registration is ongoing for NIST MetricsMATR10, our second Metrics for Machine Translation Challenge, joined this year with the Fifth Workshop on Statistical Machine Translation (WMT10).
You can register until March 26. Metrics are due at NIST on May 14. Please see the original announcement below. NEW: The source data of the NIST MetricsMATR dev set is now available to registered participants in addition to the system translations, reference translations, and human assessments. NEW: the MetricsMATR10 test set will be enriched with a portion of MT output in Iraqi Arabic and Farsi in addition to the NIST MetricsMATR test sets in English and French and the WMT10 output languages Czech, English, French, German, and Spanish. We hope to receive your registration soon. Best, Kay / NIST Open MT Team. From: Peterson, Kay Sent: Tuesday, January 26, 2010 4:02 PM To: '[email protected]'; '[email protected]' Subject: Announcing NIST MetricsMATR10 Dear MT and MT metric community, We're pleased to announce the second NIST Metrics for Machine Translation Challenge, MetricsMATR10, which will continue the quest for improved automated machine translation metrology. MetricsMATR10 will join efforts with the Workshop on Statistical Machine Translation (WMT) as the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR10<http://www.statmt.org/wmt10>, a workshop of ACL 2010<http://www.acl2010.org/>. MetricsMATR10 will represent the "evaluation task" of the workshop. MetricsMATR10 will be implemented similarly to the last MetricsMATR challenge, MetricsMATR08. Participants will submit their metric(s) to NIST for installation. NIST will run them on an evaluation set of machine translations that remains at NIST, and report correlations with human assessments. Because of the collaboration with WMT, NIST will also be able to evaluate the metrics against human assessments of the machine translations from the WMT10 translation task. The MetricsMATR10 website<http://www.nist.gov/itl/iad/mig/metricsmatr10.cfm> will be up very soon. Important dates: Immediately: MetricsMATR08 development data available as a re-release for MetricsMATR10 March 26 2010: Metric submission commitment due at NIST March 26 - May 14 2010: Metric submission period; metrics must be installed and operational at NIST by May 14, 5pm EDT June 18 2010: Informal metric descriptions due at NIST July 15 - 16 2010: Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR10 (at ACL 2010 in Uppsala, Sweden) Feel free to pass this announcement on to those outside of the immediate MT community who might be interested in participating. Please contact [email protected]<mailto:[email protected]> if you have any questions. Best, Kay / NIST Open MT Team. -- Kay Peterson NIST ITL IAD Multimodal Information Group http://www.nist.gov/itl/iad/mig (+1) 301-975-2960 Phone (+1) 301-670-0939 Fax
_______________________________________________ Mt-list mailing list
