Re: [Moses-support] experiment.perl, processPhraseTableMin and threads option
Hi Philipp, Shame on me, you are right. Thanx for help! Regards, TG From: phko...@gmail.com [mailto:phko...@gmail.com] On Behalf Of Philipp Koehn Sent: Wednesday, September 23, 2015 3:30 PM To: Tomasz Gawryl Cc: moses-support@mit.edu Subject: Re: [Moses-support] experiment.perl, processPhraseTableMin and threads option Hi, I ran the command as you provided it, and it worked. The "-threads all" functionality was added in August - did you compile the latest version of the code? -phi On Tue, Sep 22, 2015 at 12:16 PM, Tomasz Gawryl wrote: Hi, I'm trying to add this line to config: ttable-binarizer = "$moses-bin-dir/processPhraseTableMin -threads all" This option is supported by processPhraseTableMin command(https://github.com/moses-smt/mosesdecoder/blob/master/misc/processPh raseTableMin.cpp line 24). /home/moses/src/mosesdecoder/scripts/training/binarize-model.perl /home/moses/working/experiments/model/moses.ini.1 /home/mose s/working/experiments/model/moses.bin.ini.3 -Binarizer /home/moses/src/mosesdecoder/bin/processPhraseTableMin -threads all But it produces error "Unknown option: threads" in file TRAINING_binarize-config.3.STDERR (and stops training). I removed this option but it seems that such case it uses only one thread: moses 1470 113 10.8 4255200 3448392 pts/13 Sl 11:05 66:03 /home/moses/src/mosesdecoder/bin/processPhraseTableMin -in /home/moses/working/experiments/model/moses.bin.ini.5.tables/phrase-table.0- 0.1.1.gz.sorted -out /home/moses/working/experiments/model/moses.bin.ini.5.tables/phrase-table.0- 0.1.1 -nscores 4 -threads 1 I know that my server is able to run around 16 threads (and indeed does during former steps). What can I do to improve this step to use more threads? Regards, TG ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] experiment.perl, processPhraseTableMin and threads option
Hi, I ran the command as you provided it, and it worked. The "-threads all" functionality was added in August - did you compile the latest version of the code? -phi On Tue, Sep 22, 2015 at 12:16 PM, Tomasz Gawryl wrote: > Hi, > > I'm trying to add this line to config: > > ttable-binarizer = "$moses-bin-dir/processPhraseTableMin -threads all" > > This option is supported by processPhraseTableMin > command( > https://github.com/moses-smt/mosesdecoder/blob/master/misc/processPh > raseTableMin.cpp line 24). > > /home/moses/src/mosesdecoder/scripts/training/binarize-model.perl > /home/moses/working/experiments/model/moses.ini.1 /home/mose > s/working/experiments/model/moses.bin.ini.3 -Binarizer > /home/moses/src/mosesdecoder/bin/processPhraseTableMin -threads all > > But it produces error "Unknown option: threads" in file > TRAINING_binarize-config.3.STDERR (and stops training). > > I removed this option but it seems that such case it uses only one thread: > > moses 1470 113 10.8 4255200 3448392 pts/13 Sl 11:05 66:03 > /home/moses/src/mosesdecoder/bin/processPhraseTableMin -in > > /home/moses/working/experiments/model/moses.bin.ini.5.tables/phrase-table.0- > 0.1.1.gz.sorted -out > > /home/moses/working/experiments/model/moses.bin.ini.5.tables/phrase-table.0- > 0.1.1 -nscores 4 -threads 1 > > I know that my server is able to run around 16 threads (and indeed does > during former steps). > > What can I do to improve this step to use more threads? > > Regards, > TG > > > > ___ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] experiment.perl, processPhraseTableMin and threads option
Hi, I'm trying to add this line to config: ttable-binarizer = "$moses-bin-dir/processPhraseTableMin -threads all" This option is supported by processPhraseTableMin command(https://github.com/moses-smt/mosesdecoder/blob/master/misc/processPh raseTableMin.cpp line 24). /home/moses/src/mosesdecoder/scripts/training/binarize-model.perl /home/moses/working/experiments/model/moses.ini.1 /home/mose s/working/experiments/model/moses.bin.ini.3 -Binarizer /home/moses/src/mosesdecoder/bin/processPhraseTableMin -threads all But it produces error "Unknown option: threads" in file TRAINING_binarize-config.3.STDERR (and stops training). I removed this option but it seems that such case it uses only one thread: moses 1470 113 10.8 4255200 3448392 pts/13 Sl 11:05 66:03 /home/moses/src/mosesdecoder/bin/processPhraseTableMin -in /home/moses/working/experiments/model/moses.bin.ini.5.tables/phrase-table.0- 0.1.1.gz.sorted -out /home/moses/working/experiments/model/moses.bin.ini.5.tables/phrase-table.0- 0.1.1 -nscores 4 -threads 1 I know that my server is able to run around 16 threads (and indeed does during former steps). What can I do to improve this step to use more threads? Regards, TG ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support