Re: [Moses-support] Tuning with no language model

The following command works fine for when a language model is specified. Deleting the line KENLM lazyken=0 name=LM0 factor=0 path=$PWD/blm.en order=3 causes the script to fail with error: ERROR: Failed to run '/pathtomoses/mosesdecoder/bin/moses -config /pathtodata/binarised/moses.ini

[Moses-support] Tuning with no language model

Returning to an old discussion we once had on this list. I've tried to tune some systems with no reference in the configuration file to a language model so that I can do Moses justice when comparing it to a number of lines of work I have done that use only translation models and various

Re: [Moses-support] Tuning with no language model

Can you please send me the moses.ini file you're using and the exact command you're executing. You should be able to tune without a LM. I've done it myself a couple of years ago. There's no distinction between a LM and any other feature function On 13/01/16 13:22, Read, James C wrote:

Re: [Moses-support] Tuning with no language model

there lies your problem. There's a weight for a non-existant feature function On 13/01/16 13:57, Read, James C wrote: no *From:* Hieu Hoang *Sent:* Wednesday, January 13, 2016 1:56 PM *To:* Read,

Re: [Moses-support] Tuning with no language model

So now I get the following error: Loading UnknownWordPenalty0 Loading WordPenalty0 Loading PhrasePenalty0 Loading LexicalReordering0 Loading Distortion0 Loading TranslationModel0 LM0 The following weights have no feature function. Maybe incorrectly spelt weights: LM0,Exit code: 1 I didn't

Re: [Moses-support] Tuning with no language model

The following command works fine for when a language model is specified. Deleting the line KENLM lazyken=0 name=LM0 factor=0 path=/pathtodata/blm.en order=3 causes the script to fail with error: ERROR: Failed to run '/pathtomoses/mosesdecoder/bin/moses -config

Re: [Moses-support] Tuning with no language model

no From: Hieu Hoang Sent: Wednesday, January 13, 2016 1:56 PM To: Read, James C; Moses Support Subject: Re: [Moses-support] Tuning with no language model did you delete the line LM0= 0.5 too? On 13/01/16 13:53, Read, James C wrote: The

Re: [Moses-support] Tuning with no language model

please show me the EXACT moses.ini and command that you use. Hieu Hoang http://www.hoang.co.uk/hieu On 13 January 2016 at 14:04, Read, James C wrote: > So now I get the following error: > > > > Loading UnknownWordPenalty0 > Loading WordPenalty0 > Loading PhrasePenalty0 >

Re: [Moses-support] Tuning with no language model

looks fine to me. This is what I get when I used it to run # $MOSES_DIR/bin/moses -f moses.ini -show-weights LexicalReordering0= 0.30 0.30 0.30 0.30 0.30 0.30 Distortion0= 0.30 UnknownWordPenalty0 UNTUNEABLE WordPenalty0= -1.00 PhrasePenalty0= 0.20

Re: [Moses-support] Tuning with no language model

ok. The mert script create a temporary directory every time you run it. By default it's named mert-work Since you ran mert with an incorrect moses.ini previously, it may have polluted the temporary directory and cause problem now. You should find this temporary directory and delete it before

Re: [Moses-support] Tuning with no language model

Thanks, and if I wanted Moses to use one feature of TM alone I'm guessing there's no way to do this other than zeroing out the undesired TM features? From: Hieu Hoang Sent: Wednesday, January 13, 2016 3:05 PM To: Read, James C Cc: Moses

[Moses-support] MosesCore Europarl-trained models

Hey all, Just wondering if the Europarl-trained models discussed at http://www.statmt.org/moses/?n=Moses.SampleData are still around. The link on that page goes to the Moses 2.1 release, but I don't see any models there. Thanks, Lane ___ Moses-support

Re: [Moses-support] Tuning with no language model

This is what I get when I run the same command as you: LexicalReordering0= 0.30 0.30 0.30 0.30 0.30 0.30 Distortion0= 0.30 UnknownWordPenalty0 UNTUNEABLE WordPenalty0= -1.00 PhrasePenalty0= 0.20 TranslationModel0= 0.20 0.20 0.20 0.20 Looks

Re: [Moses-support] Tuning with no language model

Defined parameters (per moses.ini or switch): config: /media/bigdata/jcread/llv/data/europarlv7/raw/aligned/bg-en/training_data/binarised/moses-tm.ini Found this in the output so I'm sure it's using the right config file. From: Read, James C Sent:

Re: [Moses-support] Tuning with no language model

OK, looks like it's running. Probably won't be able to see if it generates useful weights until tomorrow. Thanks. As a side line of thought. I was wondering how many lines of configuration I could get away with deleting? How would Moses behave if I were to delete the lexical reordering lines?

Re: [Moses-support] Tuning with no language model

you can delete all feature functions except for the UnknownWordPenalty. In the current moses, thats hardcoded into decoding. So be my guest and delete away! Hieu Hoang http://www.hoang.co.uk/hieu On 13 January 2016 at 15:02, Read, James C wrote: > OK, looks like it's

Re: [Moses-support] Error on lmplz

Thanks, Kenneth. Here's what I get now. $ ~/mosesdecoder.multisource.git/bin/lmplz -o 2 <<< "that is what happens ? > cssd has nothing more or voldemort or pastries in prague ." > === 1/5 Counting and sorting n-grams === > Reading /tmp/sh-thd-1452698150 (deleted) > >

Re: [Moses-support] Tuning with no language model

# ### MOSES CONFIG FILE ### # # input factors [input-factors] 0 # mapping steps [mapping] 0 T 0 [distortion-limit] 6 # feature functions [feature] UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryCompact name=TranslationModel0

Re: [Moses-support] Tuning with no language model

Hi, If you don't need all score components of a phrase table, the easiest way to get rid of them is to set the scaling factors for the undesired phrase table feature function components to 0 before tuning, and ask the optimizer to ignore them. The feature function configuration parameter

Re: [Moses-support] Error on lmplz

Works as intended. On 01/13/2016 02:59 PM, Lane Schwartz wrote: > Thanks, Kenneth. Here's what I get now. > > $ ~/mosesdecoder.multisource.git/bin/lmplz -o 2 <<< "that is what > happens ? cssd has nothing more or voldemort or pastries in prague ." > === 1/5 Counting and sorting

Re: [Moses-support] Error on lmplz

Thanks! On Wed, Jan 13, 2016 at 11:44 AM, Kenneth Heafield wrote: > Works as intended. > > On 01/13/2016 02:59 PM, Lane Schwartz wrote: > > Thanks, Kenneth. Here's what I get now. > > > > $ ~/mosesdecoder.multisource.git/bin/lmplz -o 2 <<< "that is what > > happens