Hi Mark, I have checked that the weight of the laguage model is -0.000345 in the tuned moses.ini, which is almost the same as you anaylzed :-)
I have replaced the source langauge model in moses.ini file generated by the training step with the in-domain target langauge model, and then went on to do the tuning and evaluation steps again. Hope this can get its quality to be better. I have read the Moses manual again, it seems the training steps actually has no releationship with the langauge model, so it should be OK if I replace the language model with the right one in the moses.ini to proceed the tuning steps, right? I also used the in-domain language model this time (the traning step was performed by using both in-domain and out-domain training data), because I saw one paper said that the in-domain LM will help to improve the quality for our domain, hope this helps. Thank you Mark ;-) 2010/8/11, Mark Fishel <fis...@ut.ee>: > Hi, > > I would guess that your tuning (i.e. MERT) would push the weight of > the language model towards 0, thus kind of translating without a > language model. You can check the LM weight from the resulting > moses.ini file. > > As for the quality of the engine, during translation the Spanish > tokens would mostly be unknown to the language model, which means that > for most n-grams the language model would give you the same weight > (i.e. the weight of the p(UNK|UNK, ..., UNK) n-gram), which again means > that this turns into translation without a language model. > > I'd suggest simply to train a correct language model and repeat the > MERT and evaluation steps, unless you are interested in your setup as > a researcher :) > > Best, > Mark > > On Tue, Aug 10, 2010 at 5:04 PM, Wenlong Yang <welons...@gmail.com> wrote: >> Hi, >> >> I have trained a Moses Engine for the language pair English to >> Spanish, but for the training script, during below step: >> "nohup nice >> tools/moses-scripts/scripts-20100203-0505/training/train-factored-phrase-model.perl >> -scripts-root-dir tools/moses-scripts/scripts-20100203-0505 -root-dir >> Work-SP -corpus Work-SP/corpus/news-commentary.lowercased -f en -e sp >> -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm >> 0:3:/home/user/Work-SP/lm/news-commentary.lm >& Work-SP/training.out >> &" >> >> The "news-commentary.lm" in above command line should be the language >> model of the target language, but by mistake, I was using the language >> model for the source language, then I went on to finish the tuning and >> evaluation steps for this engine. >> >> The weird thing is that this trained engine still can work for English >> to Spanish, but I am doubting that its quality is getting lower >> because of the wrong language model used. >> >> Can any of you help to analyze what will be influenced by the wrongly >> used language model? Will it influence the engine's quality? >> >> Thanks. >> _______________________________________________ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
