Hi, On Tue, Nov 22, 2011 at 10:18 PM, somayeh bakhshaei <s.bakhsh...@gmail.com> wrote: > Hello, > > Thanks for all answers. > > Also thanks Jehan. > As you might follow moses emails there is an inconsistency problem about > tuning in mert (expressed by Neda) > For reducing this problem everyone offered to tune the system repeatedly > then choosing the best answer.
Thanks for this explication. Reading Tom Hoar's email, yours and researching and finding the original discussion, I am not sure to have understood what is the proposed solution: - should we average all the weights in the various moses.ini generated during these tunings? Would weights really still make sense doing so? - should we compare the BLEU values of the various tuning and take as-is (without modifying it) the moses.ini whose BLEU was the closer to the average of all the BLEUs? > It is a way of getting rid of local maxima but not exactly catching the > global Maxima but instead trapping in another local one :) > So I think a better solution is needed! So if I get it, the logics is that we may get very good BLEU (as from what I read, the closer to 1, the better) on some tuning, but they are actually local maxima (hence may be in fact terrible against real life data). Hence in order to counter this, we prefer to use a tuning which made an average BLEU on our data because it would be more robust on the long term? Also, my mathematics are far, but from what I recall, when we want to get away from local maxima/minima, one would prefer to use median rather than the average (even more on short samples like here), which is also very influence by local maxima. Shouldn't it also be the case here? Regards, Jehan > > On Tue, Nov 22, 2011 at 3:12 PM, Jehan Pages <je...@mygengo.com> wrote: >> >> Hi, >> >> On Tue, Nov 22, 2011 at 5:57 PM, somayeh bakhshaei >> <s.bakhsh...@gmail.com> wrote: >> > Hello all, >> > >> > Salam, >> > >> > I am using moses in this way: >> > >> > train, >> > for i=1 to 3 >> > tune >> > end for >> >> Sorry for not answering your problem (I don't have the solution though >> I saw others did answer with a possible resolution). I just note that >> you tune 3 times. Do you mean you re-tune using the exact same data >> set these 3 times? Does it bring better results to tune several times >> like this? >> Thanks! >> >> Jehan >> >> > decode >> > evaluate >> > >> > in the above loop for something unexpected happens, in large execution >> > sometime the weights produced in moses.ini are wrong. For example once >> > it >> > produce 3 in the other case produce 4, take a look hear: >> > >> > # translation model weights >> > [weight-t] >> > 0.0106455 >> > 0.036391 >> > 0.0453815 >> > 0.0716856 >> > 0.0271838 >> > >> > # translation model weights >> > [weight-t] >> > 0.0705978 >> > 0.0652413 >> > 0.100475 >> > 0.00356951 >> > >> > in the case in the previous iteration nothing is wrong. >> > Did anyone can tell me what is happening here please? >> > >> > >> > >> > >> > --------------------- >> > Best Regards, >> > S.Bakhshaei >> > >> > After All you will come .... >> > And will spread light on the dark desolate world! >> > O' Kind Father! We will be waiting for your affectionate hands ... >> > >> > >> > _______________________________________________ >> > Moses-support mailing list >> > Moses-support@mit.edu >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> > > > > > -- > > > > --------------------- > Best Regards, > S.Bakhshaei > > After All you will come .... > And will spread light on the dark desolate world! > O' Kind Father! We will be waiting for your affectionate hands ... > > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
