Thanks Marcin. Its for a new resource-poor language so I only trained it with what I could collect so far (i.e. only 190,630 words of parallel data). I retrained the entire system each time without any tuning.
On 22 June 2015 at 01:00, Marcin Junczys-Dowmunt <junc...@amu.edu.pl> wrote: > Hi, > I think the average is OK, your variance is however quite high. Did you > retrain the entire system or just optimize parameters a couple of times? > > Two useful papers on the topic: > > https://www.cs.cmu.edu/~jhclark/pubs/significance.pdf > http://www.mt-archive.info/MTS-2011-Cettolo.pdf > > > On 22.06.2015 02:37, Hokage Sama wrote: > > Hi, > > > > Since MT training is non-convex and thus the BLEU score varies, which > > score should I use for my system? I trained my system three times > > using the same data and obtained the three different scores below. > > Should I take the average or the best score? > > > > BLEU = 17.84, 49.1/22.0/12.5/7.5 (BP=1.000, ratio=1.095, hyp_len=3952, > > ref_len=3609) > > BLEU = 16.51, 48.4/20.7/11.4/6.5 (BP=1.000, ratio=1.093, hyp_len=3945, > > ref_len=3609) > > BLEU = 15.33, 48.2/20.1/10.3/5.5 (BP=1.000, ratio=1.087, hyp_len=3924, > > ref_len=3609) > > > > Thanks, > > Hilton > > > > > > _______________________________________________ > > Moses-support mailing list > > Moses-support@mit.edu > > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
