Hi Angli, Here is an excerpt of Hieu's answers regarding this topic when I was doing research in factored models, might be of some help:
On 30/06/2016 21:44, Sašo Kuntaric wrote: Hi all, I would like to ask one more question. When you say that my reference only has the surface form, are you talking about the "tuning corpus", which in the case of my command ~/mosesdecoder/scripts/training/mert-moses.pl ~/working/IT_corpus/TMX/txt/ factored_corpus/singles/tuning_corpus.tagged.clean.en ~/working/IT_corpus/TMX/txt/factored_corpus/singles/tuning _corpus.tagged.clean.sl ~/mosesdecoder/bin/moses ~/working/IT_corpus/TMX/txt/factored_corpus/singles/test/model/moses.ini --mertdir ~/mosesdecoder/bin/ --decoder-flags="-threads all" are tuning_corpus.tagged.clean.en and tuning_corpus.tagged.clean.sl? Can tuning be done with files that only contains surface forms? it's usual that the reference tuning data does not have factors, even if there are factors in the phrase table. After all, you don't care if the output surface form is correct but the other factors are wrong. Will the results be compatible with tuning done with a factored tuning corpus? yes Best regards, Sašo 2016-12-04 1:37 GMT+01:00 Hieu Hoang <hieuho...@gmail.com>: > > > Hieu > Sent while bumping into things > > On 1 Dec 2016 07:01, "Angli Liu" <ang...@cs.washington.edu> wrote: > > Hi, what's the major difference between the tuning process for a factored > phrase based system (i.e., surface+pos data) and a simple baseline phrase > based system? > > > Nothing, the tuning just optimise weights for feature functions. > > If you decompose your translation so that it has multiple phrase tables > and generation models, then they are just extra feature functions with > weights to be tuned > > Do I need to organize the dev set the same way as the training set (i.e., > surface|pos)? > > Yes > > Is there a tutorial on the moses website on this topic? > > Maybe this > http://www.statmt.org/moses/?n=FactoredTraining.FactoredTraining > > > Thanks! > > -Angli > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > > -- lp, Sašo
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
