[Moses-support] Experimenter: giving an aligned corpus

Hi, I am using experimenter to run Moses. In the configuration file, I provide word alignments by specifying it as"word-alignment =" in TRAINING. I could not find a way to specify cleaned corpus. I am using the latest branch of Moses. In the previous one, I used to specify cleaned corpus in the TR

[Moses-support] New parameter "const std::string &line" in FeatureFunction

I'm doing some LM tinkering, and I noticed that there's a new parameter in FeatureFunction and its descendents. What is the following parameter represent? "const std::string &line" Thanks, Lane ___ Moses-support mailing list Moses-support@mit.edu http:

[Moses-support] Fwd: Re: Error with processPhaseTableMin

Hi, Might be two reasons: 1) are you running this on a 32-bit machine or Cygwin? Then the maximum size phrase table that can be built is about 3GB, which can be hit quite quickly. 2) do you have only little free space in your /tmp directory? You can change the directory used for temporary fil

[Moses-support] Error with processPhaseTableMin

Hello, I am trying to create a compact version of the phrase table based on the pre-trained models of release 1. I get the following error when I run the processPhraseTableMin on a ubuntu Vagrant machine. Thanks for your help, João vagrant@precise64:/vagrant/mt-models/en-es$ ~/mosesdecoder/bin

Re: [Moses-support] Error when attempting to translate: fails with " StrayFactorException "

did you escape your training and input data? There must not be | characters in your data unless you are using factored models the moses tokenizer script does it, as well as the specific escape script. scripts/tokenizer/tokenizer.perl scripts/tokenizer/escape-special-chars.perl On 26/08/2

Re: [Moses-support] Decoding with word lattice

you mean your lattice only has 1 path? ie. it is a sentence encoded as a lattice? Did you retune your model? I am surprise, but no-one has done that experiment. if you can make the model files available, i can debug it and let you know why there is a difference On 26/08/2013 14:42, Wei Qiu

Re: [Moses-support] Segmentation fault on generateSequences while running OSM-Train.sh

Hi Nadir, Thanks a million, it really helped! After switching e and f positions at the command line, it worked fine on my test corpus. Also worked on the new Perl script. Thanks again, Jian On Tue, Aug 27, 2013 at 1:50 PM, Nadir Durrani wrote: > Hi Jian, > > Try out > > ${MOSES_DIR}/script

Re: [Moses-support] Did the configuration files change

Hi Hieu and Phillip, Thanks a lot for your help. Hieu, sorry for the q problem. I was tired, should have payed more attention. Best regards, João Graça On Tue, Aug 27, 2013 at 2:26 PM, Hieu Hoang wrote: > internally, the decoder also does this conversion. > > however, it might be better t

Re: [Moses-support] Did the configuration files change

internally, the decoder also does this conversion. however, it might be better to use philipp's script to be more transparent On 27 August 2013 13:58, Philipp Koehn wrote: > Hi, > > note that the script > scripts/training/convert-moses-ini-to-v2.perl < IN > OUT > allows you to easily convert

Re: [Moses-support] Did the configuration files change

Hi, note that the script scripts/training/convert-moses-ini-to-v2.perl < IN > OUT allows you to easily convert an old config file to the new format. -phi On Tue, Aug 27, 2013 at 1:47 PM, Hieu Hoang wrote: > there's a character >q > at the beginning of the file. > > Delete it > > > On 26 A

[Moses-support] Segmentation fault on generateSequences while running OSM-Train.sh

Hi Jian, Try out ${MOSES_DIR}/scripts/OSM/OSM-Train.sh ${TARGET} ${SOURCE} ${ALIGN} ${ORDER} ${WORKDIR} ${MOSES_DIR} ${SRILM_BINDIR} the script takes target before source. Btw we made some changes. There's a new script OSM-Train.perl. So use that now. I have also uploaded the source code for ge

Re: [Moses-support] lmplz: BadDiscountException: Is this small or artificial data?

OK, I was hoping for a dirty method to bypass that. No pressure :) W dniu 2013-08-27 14:38, Kenneth Heafield napisał(a): > Hi, > > I look forward to working with you at MT Marathon on adding features to > lmplz. > > Kneser-Ney smoothing is not well defined when there is no singleton. > You

Re: [Moses-support] Did the configuration files change

there's a character q at the beginning of the file. Delete it On 26 August 2013 22:34, João Graça wrote: > Dear Hieu, > > I am trying to use the new version of moses with the old format moses.ini > from the pre-trained models. > > In attach is the configuration file I am currently trying to

Re: [Moses-support] lmplz: BadDiscountException: Is this small or artificial data?

Hi, I look forward to working with you at MT Marathon on adding features to lmplz. Kneser-Ney smoothing is not well defined when there is no singleton. You'll get a similar error message from SRILM on this data with KN. I suggest you consider a different smoothing method. Curren

[Moses-support] lmplz: BadDiscountException: Is this small or artificial data?

Hi Kenneth, I am getting the following error from lmplz on training data built from POS-tags: /home/m.junczys/kenlm/lm/builder/adjust_counts.cc:42 in void lm::builder::{anonymous}::StatCollector::CalculateDiscounts() threw BadDiscountException because `s.n[j] == 0'. Could not calculate Knese