Re: [Moses-support] experiment.perl with IRSTLM only (no SRILM installed)
> I looked at the experiment.perl script, and it's not set up to pass the > correct arguments to irstlm, so what you're suggesting above shouldn't work. > You could probably get it to work by messing with experiment.meta - look for > [LM], then look for train, and edit the template. That worked nicely - experiment.meta was really smooth to modify. Now I can train with experiment.perl using IRSTLM only: problem solved, thanks. > I'm guessing you're avoiding srilm due to licensing restrictions? Yes. I will of course distribute my simple changes (even though they are quick and dirty). I'll post them here on request, too (or email me to make sure I see the request). Thanks again /Greg ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] experiment.perl with IRSTLM only (no SRILM installed)
Hi Greg On Friday 27 May 2011 09:47, Greg Wilson wrote: >lm-training = path-to-irstlm/build-lm.sh >order = 5 >settings = ...irstlm arguments... and figure out how order above is > passed to -n here... > I looked at the experiment.perl script, and it's not set up to pass the correct arguments to irstlm, so what you're suggesting above shouldn't work. You could probably get it to work by messing with experiment.meta - look for [LM], then look for train, and edit the template. But it may be easier just to train the lm separately. I'm guessing you're avoiding srilm due to licensing restrictions? You can also use kenlm at run time (it doesn't do training) but again it's not integrated into experiment.perl - yet. cheers - Barry -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] experiment.perl with IRSTLM only (no SRILM installed)
Thanks for your mail, I missed that experiment.perl was not designed for training with IRSTLM. I also just found the LM-setting: lm-training = $srilm-dir/ngram-count obviously this will fail. Perhaps I can somehow make that lm-training = path-to-irstlm/build-lm.sh order = 5 settings = ...irstlm arguments... and figure out how order above is passed to -n here... Again, thanks! /Greg On Fri, May 27, 2011 at 10:42 AM, Barry Haddow wrote: > Hi Greg > > The instructions that you refer to are for when you want to use irstlm in > decoding. As far as I can see, the only options for training language models > in experiment.perl are srilm and randlm > (http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc14). > > If you want to train using irstlm, then you can train outside of > experiment.perl (see the irstlm manual for details) then specify the trained > language model in experiment.perl using something like: > > [LM] > order = 5 > [LM;mylm] > lm = path/to/my/lm > > You should also remove all mentions of srilm in your config file. > > best regards - Barry > > On Friday 27 May 2011 09:16, Greg Wilson wrote: >> Hi, first let me thank the people who are making Moses available, your >> work is very appreciated! >> >> I am trying to run experiment.perl on an installation with only IRSTLM >> (SRILM is not installed). This works perfectly fine if I do the >> experiments manually. >> >> I followed this instruction for configuring the experiment to only use >> IRSTLM: http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc13 >> The instruction is quite clear: uncomment lm-binarizer and >> lm-quantizer, which I did. >> >> The problem is that it seems like experiment.perl still try to use SRILM: >> >> perl $m/scripts-20110520-1542/ems/experiment.perl -config config.toy >> Use of implicit split to @_ is deprecated at >> /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 2145. >> STARTING UP AS PROCESS 14381 ON liveserver0 AT Fri May 27 06:45:00 UTC 2011 >> LOAD CONFIG... >> find: `/usr/local/srilm/bin/i686/ngram-count*': No such file or directory >> LM:lm-training: file /usr/local/srilm/bin/i686/ngram-count does not exist! >> find: `/usr/local/srilm/bin/i686*': No such file or directory >> GENERAL:srilm-dir: file /usr/local/srilm/bin/i686 does not exist! >> Died at /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 360. >> >> Is it possible to do what I want; to configure an >> experiment.perl-experiment to only use IRSTLM, or are there hardwired >> calls to SRILM somewhere in there? >> >> Thankful for any advice, >> /Greg >> ___ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] experiment.perl with IRSTLM only (no SRILM installed)
Hi Greg The instructions that you refer to are for when you want to use irstlm in decoding. As far as I can see, the only options for training language models in experiment.perl are srilm and randlm (http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc14). If you want to train using irstlm, then you can train outside of experiment.perl (see the irstlm manual for details) then specify the trained language model in experiment.perl using something like: [LM] order = 5 [LM;mylm] lm = path/to/my/lm You should also remove all mentions of srilm in your config file. best regards - Barry On Friday 27 May 2011 09:16, Greg Wilson wrote: > Hi, first let me thank the people who are making Moses available, your > work is very appreciated! > > I am trying to run experiment.perl on an installation with only IRSTLM > (SRILM is not installed). This works perfectly fine if I do the > experiments manually. > > I followed this instruction for configuring the experiment to only use > IRSTLM: http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc13 > The instruction is quite clear: uncomment lm-binarizer and > lm-quantizer, which I did. > > The problem is that it seems like experiment.perl still try to use SRILM: > > perl $m/scripts-20110520-1542/ems/experiment.perl -config config.toy > Use of implicit split to @_ is deprecated at > /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 2145. > STARTING UP AS PROCESS 14381 ON liveserver0 AT Fri May 27 06:45:00 UTC 2011 > LOAD CONFIG... > find: `/usr/local/srilm/bin/i686/ngram-count*': No such file or directory > LM:lm-training: file /usr/local/srilm/bin/i686/ngram-count does not exist! > find: `/usr/local/srilm/bin/i686*': No such file or directory > GENERAL:srilm-dir: file /usr/local/srilm/bin/i686 does not exist! > Died at /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 360. > > Is it possible to do what I want; to configure an > experiment.perl-experiment to only use IRSTLM, or are there hardwired > calls to SRILM somewhere in there? > > Thankful for any advice, > /Greg > ___ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] experiment.perl with IRSTLM only (no SRILM installed)
Hi, thanks for your response. No, I run a new experiment from scratch, based on the config.toy that comes with moses. I follow the Quick start instructions (http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc3) and customise: working-dir, data-dir, moses-script-dir, moses-src-dir, decoder, (and don't touch srilm-dir, as I don't have it installed, but experiment.perl requires it to be set to something) then also: lm-binarizer, lm-quantizer So no LM has been created when I execute experiment.perl. /Greg On Fri, May 27, 2011 at 10:24 AM, Miles Osborne wrote: > is this after running with SRILM? > > if so, then look for the script which creates the LM and delete it. > that should force it to be re-created, using IRSLM > > Miles > > On 27 May 2011 09:16, Greg Wilson wrote: >> Hi, first let me thank the people who are making Moses available, your >> work is very appreciated! >> >> I am trying to run experiment.perl on an installation with only IRSTLM >> (SRILM is not installed). This works perfectly fine if I do the >> experiments manually. >> >> I followed this instruction for configuring the experiment to only use >> IRSTLM: >> http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc13 >> The instruction is quite clear: uncomment lm-binarizer and >> lm-quantizer, which I did. >> >> The problem is that it seems like experiment.perl still try to use SRILM: >> >> perl $m/scripts-20110520-1542/ems/experiment.perl -config config.toy >> Use of implicit split to @_ is deprecated at >> /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 2145. >> STARTING UP AS PROCESS 14381 ON liveserver0 AT Fri May 27 06:45:00 UTC 2011 >> LOAD CONFIG... >> find: `/usr/local/srilm/bin/i686/ngram-count*': No such file or directory >> LM:lm-training: file /usr/local/srilm/bin/i686/ngram-count does not exist! >> find: `/usr/local/srilm/bin/i686*': No such file or directory >> GENERAL:srilm-dir: file /usr/local/srilm/bin/i686 does not exist! >> Died at /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 360. >> >> Is it possible to do what I want; to configure an >> experiment.perl-experiment to only use IRSTLM, or are there hardwired >> calls to SRILM somewhere in there? >> >> Thankful for any advice, >> /Greg >> ___ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] experiment.perl with IRSTLM only (no SRILM installed)
is this after running with SRILM? if so, then look for the script which creates the LM and delete it. that should force it to be re-created, using IRSLM Miles On 27 May 2011 09:16, Greg Wilson wrote: > Hi, first let me thank the people who are making Moses available, your > work is very appreciated! > > I am trying to run experiment.perl on an installation with only IRSTLM > (SRILM is not installed). This works perfectly fine if I do the > experiments manually. > > I followed this instruction for configuring the experiment to only use IRSTLM: > http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc13 > The instruction is quite clear: uncomment lm-binarizer and > lm-quantizer, which I did. > > The problem is that it seems like experiment.perl still try to use SRILM: > > perl $m/scripts-20110520-1542/ems/experiment.perl -config config.toy > Use of implicit split to @_ is deprecated at > /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 2145. > STARTING UP AS PROCESS 14381 ON liveserver0 AT Fri May 27 06:45:00 UTC 2011 > LOAD CONFIG... > find: `/usr/local/srilm/bin/i686/ngram-count*': No such file or directory > LM:lm-training: file /usr/local/srilm/bin/i686/ngram-count does not exist! > find: `/usr/local/srilm/bin/i686*': No such file or directory > GENERAL:srilm-dir: file /usr/local/srilm/bin/i686 does not exist! > Died at /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 360. > > Is it possible to do what I want; to configure an > experiment.perl-experiment to only use IRSTLM, or are there hardwired > calls to SRILM somewhere in there? > > Thankful for any advice, > /Greg > ___ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] experiment.perl with IRSTLM only (no SRILM installed)
Hi, first let me thank the people who are making Moses available, your work is very appreciated! I am trying to run experiment.perl on an installation with only IRSTLM (SRILM is not installed). This works perfectly fine if I do the experiments manually. I followed this instruction for configuring the experiment to only use IRSTLM: http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc13 The instruction is quite clear: uncomment lm-binarizer and lm-quantizer, which I did. The problem is that it seems like experiment.perl still try to use SRILM: perl $m/scripts-20110520-1542/ems/experiment.perl -config config.toy Use of implicit split to @_ is deprecated at /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 2145. STARTING UP AS PROCESS 14381 ON liveserver0 AT Fri May 27 06:45:00 UTC 2011 LOAD CONFIG... find: `/usr/local/srilm/bin/i686/ngram-count*': No such file or directory LM:lm-training: file /usr/local/srilm/bin/i686/ngram-count does not exist! find: `/usr/local/srilm/bin/i686*': No such file or directory GENERAL:srilm-dir: file /usr/local/srilm/bin/i686 does not exist! Died at /usr/local/bin/scripts-20110520-1542/ems/experiment.perl line 360. Is it possible to do what I want; to configure an experiment.perl-experiment to only use IRSTLM, or are there hardwired calls to SRILM somewhere in there? Thankful for any advice, /Greg ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
