that's very worrying. This is a major reason we're running end-to-end training as part of the releases, to make sure the whole pipeline hasn't degraded.
a few questions: 1. is the test or tuning data in any way unstable? eg. it's really small. Can you run the entire training again and see what the bleu it get again. 2. with the same weights as the 2011 version, does the decoder get the same model and bleu score? 3. Can you make the 2011 working data available for download and i'll take a look On 29 July 2013 17:45, Eleftherios Avramidis <[email protected]>wrote: > Hi, > > I have build some models some time ago, using moses and training code > which had been checked out in 2011. Now I need to improve these models, > so I decided to rerun the old configuration with the freshly checked-out > Moses. I realize that my BLEU scores, on the exact same configuration, > are down by 0.5 to 1.2 BLEU score. > Both were trained/tested with the same version of experiment.perl. > > I guess I will have to either dig to find what exactly made the > difference, or just use the old version if I need improvements. But > would you think an obvious reason why this happened? > > cheers > Lefteris > > -- > MSc. Inf. Eleftherios Avramidis > DFKI GmbH, Alt-Moabit 91c, 10559 Berlin > Tel. +49-30 238 95-1806 > > Fax. +49-30 238 95-1810 > > > ------------------------------------------------------------------------------------------- > Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH > Firmensitz: Trippstadter Strasse 122, D-67663 Kaiserslautern > > Geschaeftsfuehrung: > Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender) > Dr. Walter Olthoff > > Vorsitzender des Aufsichtsrats: > Prof. Dr. h.c. Hans A. Aukes > > Amtsgericht Kaiserslautern, HRB 2313 > > ------------------------------------------------------------------------------------------- > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
