This might be what Miles is referring too http://www.statmt.org/wmt09/pdf/WMT-0939.pdf
There was some progress towards getting this into moses http://lium3.univ-lemans.fr/mtmarathon2010/ProjectFinalPresentation/MERT/StabilizingMert.pdf On Friday 25 March 2011 13:02, Miles Osborne wrote: > There is work published on making mert more stable (on the train so can't > easily dig it up) > > Miles > > sent using Android > > On 25 Mar 2011 12:49, "Lane Schwartz" <dowob...@gmail.com> wrote: > > We know that there is nondeterminism during optimization, yet virtually all > papers report results based on a single MERT run. We know that results can > very widely based on language pair and data sets, but a large majority of > papers report results on a single language pair, and often for a single > data set. > > While these issues are widely known at the informal level, I think that > Suzy's point is well taken. I think there would be value in published > studies showing just how wide the gap due to nondeterminism can be expected > to be. It may be that such studies already exist, and I'm just not aware of > them. Does anyone know of any? > > Cheers, > Lane > > On Fri, Mar 25, 2011 at 7:03 AM, Barry Haddow <bhad...@inf.ed.ac.uk> wrote: > > Hi > > > > This is an is... -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
