[Moses-support] Fwd: EMS problems - asking for help
-- Forwarded message -- From: Despina Mouratidi <dmourat...@gmail.com> Date: 2017-07-27 10:38 GMT+03:00 Subject: Re: EMS problems - asking for help To: Rico Sennrich <rico.sennr...@ed.ac.uk> Dear Rico, It works!!! But now I have errors with step EVALUATION:test:nist-bleu-c crashed step EVALUATION:test:nist-bleu crashed both of the STDERR files said FATAL INPUT ERROR: no 'src_set' tag in src_file '/home/despina/mosesdecoder/scripts/ems/example/data/test- src.en.sgm'. I am searching in the mailing list but I can't find the solution. Thank you very much! Despina 2017-07-21 19:21 GMT+03:00 Rico Sennrich <rico.sennr...@ed.ac.uk>: > Hello Despina, > > Assuming that you installed moses via 'git clone > https://github.com/moses-smt/mosesdecoder.git', you can update it with > 'git pull'. If you installed it via a RELEASE version or package, I > recommend that you switch to using the git version, which is the most > up-to-date. After 'git pull', you will need to recompile moses with the > './compile.sh' script. > > You will have to look at the log files (STDERR and STDOUT) to find out why > these other steps crashed. > > best wishes, > Rico > > > On 19/07/17 13:33, Despina Mouratidi wrote: > > Hello Rico, > > thank you for the quick answer. You were right about missing file > moses.ini from /home/despina/working/experime > nts/evaluation/test.filtered.11directory. So this error is gone but I > have the old one error and one new > step EVALUATION:test:reference-from-sgm crashed > step EVALUATION:test:decode crashed > > For the first one you said to update my moses version, how can i do that? > I think that I have the latest version of moses. > Concerning to the EVALUATION:test:decode it said > can't read > output file /home/despina/working/experiments/evaluation/test.output.17 > is empty. > > Thanks you > Best > Despina > > 2017-07-17 19:14 GMT+03:00 Rico Sennrich <rico.sennr...@ed.ac.uk>: > >> Hello Despina, >> >> I think the error message indicates that the pseudo-config is missing, >> which I think is one of these two files: >> /home/despina/working/experiments/evaluation/test.filtered.11/moses.ini >> /home/despina/working/experiments/model/moses.ini.11 >> >> can you check if both files exist? I'm not sure why it is missing, but >> maybe you're trying to resume an interrupted training run, or there is a >> small bug in the EMS computation graph. Either creating the config >> manually, copying it from a previous run (if nothing relevant has changed), >> or ensuring that the step that produces this file is re-run, should fix the >> problem. >> >> regarding the perl compatibility, updating your moses version is the most >> likely fix. >> >> best wishes, >> Rico >> >> >> On 17/07/17 14:58, Despina Mouratidi wrote: >> >> Dear Rico, >> hello, how are you? >> >> I am sending this email because I have some problems with ems. Now >> actually I am trying to run test with ems and theano. At this email i would >> like to ask for help about two errors that I have. >> >> I did all the steps from the manual page 107, 3.5.3 (quick start) but >> When I am typing from ~/mosesdecoder/scripts/ems directoyry >> >> $ nohup nice ~/mosesdecoder/scripts/ems/experiment.perl -config >> config.toy -exec OUT.[RUN] & >> >> I came with this error in EVALUATION_test_apply-filter.STDERR >> >> filteredPath=/home/despina/working/experiments/evaluation/test.filtered.11/moses.ini >> >> origPath=/home/despina/working/experiments/model/moses.ini.11 >> tunedPath=/home/despina/working/experiments/tuning/moses.tuned.ini.11 >> outPath=/home/despina/working/experiments/evaluation/test.filtered.ini.13 >> >> tempPath=/tmp/moses.ini.8735 >> cmd=/home/despina/mosesdecoder/scripts/ems/support/substitute-filtered-tables.perl >> /home/despina/working/experiments/evaluation/test.filtered.11/moses.ini >> < /home/despina/working/experiments/model/moses.ini.11 > >> /tmp/moses.ini.8735 >> ERROR: could not find pseudo-config with filtered tables at >> /home/despina/mosesdecoder/scripts/ems/support/substitute-filtered-tables.perl >> line 12. >> cmd=/home/despina/mosesdecoder/scripts/ems/support/substitute-weights.perl >> /tmp/moses.ini.8735 >> /home/despina/working/experiments/tuning/moses.tuned.ini.11 >> /home/despina/working/experiments/evaluation/test.filtered.ini.13 >> >> and also this error in EVALUATION_test_reference-from-sgm.STDERR >> >> defined(@array) is deprecated at /home/despina/mosesdecoder/scr >> ipts/ems/support/
[Moses-support] EMS - EVALUATION_test_reference-from-sgm - perl version?
Hello guys, When i am typing from ~/mosesdecoder/scripts/ems directoyry $ nohup nice ~/mosesdecoder/scripts/ems/experiment.perl -config config.toy -exec OUT.[RUN] & I came with this error in EVALUATION_test_reference-from-sgm.STDERR defined(@array) is deprecated at /home/despina/mosesdecoder/scripts/ems/support/reference-from-sgm.perl line 63. (Maybe you should just omit the defined()?) Died at /home/despina/mosesdecoder/scripts/ems/support/reference-from-sgm.perl line 33, line 2. Maybe i must to update my perl version? I have the v5.18.2. Thanks Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] EMS - EVALUATION_test_apply-filter
Hello guys, When i am typing from ~/mosesdecoder/scripts/ems directoyry $ nohup nice ~/mosesdecoder/scripts/ems/experiment.perl -config config.toy -exec OUT.[RUN] & I came with this error in EVALUATION_test_apply-filter.STDERR filteredPath=/home/despina/working/experiments/evaluation/test.filtered.11/moses.ini origPath=/home/despina/working/experiments/model/moses.ini.11 tunedPath=/home/despina/working/experiments/tuning/moses.tuned.ini.11 outPath=/home/despina/working/experiments/evaluation/test.filtered.ini.13 tempPath=/tmp/moses.ini.8735 cmd=/home/despina/mosesdecoder/scripts/ems/support/substitute-filtered-tables.perl /home/despina/working/experiments/evaluation/test.filtered.11/moses.ini < /home/despina/working/experiments/model/moses.ini.11 > /tmp/moses.ini.8735 ERROR: could not find pseudo-config with filtered tables at /home/despina/mosesdecoder/scripts/ems/support/substitute-filtered-tables.perl line 12. cmd=/home/despina/mosesdecoder/scripts/ems/support/substitute-weights.perl /tmp/moses.ini.8735 /home/despina/working/experiments/tuning/moses.tuned.ini.11 /home/despina/working/experiments/evaluation/test.filtered.ini.13 Any idea about this error? Thanks Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] SOS- EMS - steps crased (.STDEER FILES)
Hello everyone... while I was trying to run experiment.perl EXECUTE STEPS number of steps doable or running: 2 at Τρι 04 Ιούλ 2017 01:58:51 μμ EEST doable: CORPUS:toy:clean doable: EVALUATION:test:reference-from-sgm executing /home/despina/working/experiments/steps/6/CORPUS_toy_clean.6 via sh (1 active) executing /home/despina/working/experiments/steps/6/ EVALUATION_test_reference-from-sgm.6 via sh (2 active) step CORPUS:toy:clean crashed step EVALUATION:test:reference-from-sgm crashed *CORPUS.STDERR :* clean-corpus.perl: processing /home/despina/working/ experiments/corpus/toy.tok.1.en & .fr to /home/despina/working/ experiments/corpus/toy.clean.6, cutoff 1-80, ratio 9 /home/despina/working/experiments/corpus/toy.tok.1.fr is too long! at /home/despina/mosesdecoder/scripts/training/clean-corpus-n.perl line 154, line 1001. *EVALUATION.STDERR :* defined(@array) is deprecated at /home/despina/mosesdecoder/ scripts/ems/support/reference-from-sgm.perl line 63. (Maybe you should just omit the defined()?) Died at /home/despina/mosesdecoder/scripts/ems/support/reference-from-sgm.perl line 33, line 2. Thank you in advance ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] steps crased (.STDEER FILES) while running EMS
Hello everyone... while I was trying to run experiment.perl EXECUTE STEPS number of steps doable or running: 2 at Τρι 04 Ιούλ 2017 01:58:51 μμ EEST doable: CORPUS:toy:clean doable: EVALUATION:test:reference-from-sgm executing /home/despina/working/experiments/steps/6/CORPUS_toy_clean.6 via sh (1 active) executing /home/despina/working/experiments/steps/6/EVALUATION_test_reference-from-sgm.6 via sh (2 active) step CORPUS:toy:clean crashed step EVALUATION:test:reference-from-sgm crashed *CORPUS.STDERR :* clean-corpus.perl: processing /home/despina/working/experiments/corpus/toy.tok.1.en & .fr to /home/despina/working/experiments/corpus/toy.clean.6, cutoff 1-80, ratio 9 /home/despina/working/experiments/corpus/toy.tok.1.fr is too long! at /home/despina/mosesdecoder/scripts/training/clean-corpus-n.perl line 154, line 1001. *EVALUATION.STDERR :* defined(@array) is deprecated at /home/despina/mosesdecoder/scripts/ems/support/reference-from-sgm.perl line 63. (Maybe you should just omit the defined()?) Died at /home/despina/mosesdecoder/scripts/ems/support/reference-from-sgm.perl line 33, line 2. Thank you in advance Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] problem with Experiment.perl
Then I try the other way but stell error : could not find config file at ./experiment.perl line 69. despina@despina-Satellite-L855:~/mosesdecoder/scripts/ems$ chmod +x experiment.perl despina@despina-Satellite-L855:~/mosesdecoder/scripts/ems$ ./experiment.perl STARTING UP AS PROCESS 4813 ON despina-Satellite-L855 AT Παρ 30 Ιούν 2017 04:17:35 μμ EEST error: could not find config file at ./experiment.perl line 69. Best Despina 2017-06-30 10:40 GMT+03:00 Despina Mouratidi <dmourat...@gmail.com>: > Hello again, > > thank you Mathias, > > it seems that i have the same error. But I want to ask, all the above I > have to run from my new experiment directory? not from the > /home/despina/mosesdecoder/scripts/ems/? So the files experiment.perl, > config and config.toy must be in the new directory that I created > (home/despina/working/experiments)? Oh, I'm confused. > > Thank you in advance > > Despina > > 2017-06-26 22:14 GMT+03:00 Mathias Müller <mmuel...@ifi.uzh.ch>: > >> Hi Despina >> >> Seems like you are in the wrong directory. Run >> >> /path/to/moses/scripts/ems/experiment.perl >> >> instead of only “experiment.perl”. If that does not help, then perhaps >> “experiment.perl” is not executable. In that case, run >> >> perl /path/to/experiment.perl >> >> or >> >> chmod +x /path/to/experiment.perl >> >> and then >> >> /path/to/experiment.perl >> >> Regards >> Mathias >> >> >> On 26 Jun 2017, at 20:57, Despina Mouratidi <dmourat...@gmail.com> wrote: >> >> Hello everyone again! >> After a long time typing in all the commands from moses user manual, I am >> trying the Experiment.perl. >> >> 1. I made a file in ~/working/experiments >> 2. I find experiment.perl in scripts/ems >> 3. I got a sample configuration file from scripts/ems/example/config.toy >> 4. I made all the neccesary changes but >> how can I run it? >> I type >> experiment.perl -config config -exec but experiment.perl: command not >> found >> >> Thank you! >> >> >> >> 2017-06-26 21:53 GMT+03:00 Despina Mouratidi <dmourat...@gmail.com>: >> >>> Hello everyone again! >>> After a long time typing in all the commands from moses user manual, I >>> am trying the Experiment.perl. >>> >>> 1. I made a file in ~/working/experiments >>> 2. I find experiment.perl in scripts/ems >>> 3. I got a sample configuration file from scripts/ems/example/config.toy >>> 4. I made all the neccesary changes but >>> how can I run it? >>> >>> >>> Thank you! >>> Despina >>> >> >> ___ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] problem with Experiment.perl
Hello everyone again! After a long time typing in all the commands from moses user manual, I am trying the Experiment.perl. 1. I made a file in ~/working/experiments 2. I find experiment.perl in scripts/ems 3. I got a sample configuration file from scripts/ems/example/config.toy 4. I made all the neccesary changes but how can I run it? I type experiment.perl -config config -exec but experiment.perl: command not found Thank you! 2017-06-26 21:53 GMT+03:00 Despina Mouratidi <dmourat...@gmail.com>: > Hello everyone again! > After a long time typing in all the commands from moses user manual, I am > trying the Experiment.perl. > > 1. I made a file in ~/working/experiments > 2. I find experiment.perl in scripts/ems > 3. I got a sample configuration file from scripts/ems/example/config.toy > 4. I made all the neccesary changes but > how can I run it? > > > Thank you! > Despina > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] problem with Experiment.perl
Hello everyone again! After a long time typing in all the commands from moses user manual, I am trying the Experiment.perl. 1. I made a file in ~/working/experiments 2. I find experiment.perl in scripts/ems 3. I got a sample configuration file from scripts/ems/example/config.toy 4. I made all the neccesary changes but how can I run it? Thank you! Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] SOS-Problem with filter-model-given-input.pl
Firstly, thank you, I did that but still the same error. I cant understand why working/working/mert/moses has appeared. despina@despina-Satellite-L855:~/working$ ~/mosesdecoder/scripts/training/ filter-model-given-input.pl \ > /home/despina/working/filtered-newstest2011 > /home/despina/working/mert-work/moses.ini \ > ~/working/corpus/newstest2011.true.fr \ > -Binarizer ~/mosesdecoder/bin/processPhraseTableMin Executing: mkdir -p /home/despina/working/filtered-newstest2011 Stripping XML... Executing: /home/despina/mosesdecoder/scripts/training/../generic/strip-xml.perl < /home/despina/working/corpus/newstest2011.true.fr > /home/despina/working/ filtered-newstest2011/input.3741 Can't write /home/despina/working/filtered-newstest2011 working/working/ mert-work/moses.ini at /home/despina/mosesdecoder/scripts/training/ filter-model-given-input.pl line 135. 2017-06-23 15:46 GMT+03:00 Despina Mouratidi <dmourat...@gmail.com>: > Firstly, thank you, > > I did that but still the same error. I cant understand why > working/working/mert/moses has appeared. > > > despina@despina-Satellite-L855:~/working$ ~/mosesdecoder/scripts/training/ > filter-model-given-input.pl \ > > /home/despina/working/filtered-newstest2011 > > /home/despina/working/mert-work/moses.ini > \ > > ~/working/corpus/newstest2011.true.fr \ > > -Binarizer ~/mosesdecoder/bin/processPhraseTableMin > Executing: mkdir -p /home/despina/working/filtered-newstest2011 > Stripping XML... > Executing: > /home/despina/mosesdecoder/scripts/training/../generic/strip-xml.perl > < /home/despina/working/corpus/newstest2011.true.fr > > /home/despina/working/filtered-newstest2011/input.3741 > Can't write /home/despina/working/filtered-newstest2011 working/working/ > mert-work/moses.ini at /home/despina/mosesdecoder/scripts/training/ > filter-model-given-input.pl line 135. > > > 2017-06-23 14:34 GMT+03:00 Hieu Hoang <hieuho...@gmail.com>: > >> you must use absolute paths. Not >> >> filtered-newstest2011 >> >> mert-work/moses.ini >> >> but >> >>/home/despina/whatever/filtered-newstest2011 >> >>/home/despina/whatever/mert-work/moses.ini >> >> On 23/06/2017 07:04, Despina Mouratidi wrote: >> >> Hello everyone, >> I am trying to filtered my set as follows >> >> cd ~/working >> ~/mosesdecoder/scripts/training/filter-model-given-input.pl \ >> filtered-newstest2011 mert-work/moses.ini ~/corpus/newstest2011.true.fr \ >> -Binarizer ~/mosesdecoder/bin/processPhraseTableMin >> >> But after i got this. >> >> Can't read /working/mert-work/moses.ini at /home/despina/mosesdecoder/scr >> ipts/training/filter-model-given-input.pl line 136. >> >> This is my mert-work/moses.ini >> >> # MERT optimized configuration >> # decoder /home/despina/mosesdecoder/bin/moses >> # BLEU 0.186802 on dev /home/despina/working/corpus/news-test2008.true.fr >> # We were before running iteration 9 >> # finished Τετ 14 Ιούν 2017 07:54:12 μμ EEST >> ### MOSES CONFIG FILE ### >> # >> >> # input factors >> [input-factors] >> 0 >> >> # mapping steps >> [mapping] >> 0 T 0 >> >> [distortion-limit] >> 6 >> >> # feature functions >> [feature] >> UnknownWordPenalty >> WordPenalty >> PhrasePenalty >> PhraseDictionaryMemory name=TranslationModel0 num-features=4 >> path=/home/despina/working/train/model/phrase-table.gz input-factor=0 >> output-factor=0 >> LexicalReordering name=LexicalReordering0 num-features=6 >> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >> path=/home/despina/working/train/model/reordering-table.wbe- >> msd-bidirectional-fe.gz >> Distortion >> KENLM name=LM0 factor=0 path=/home/despina/lm/news-commentary-v8.fr-en.blm.en >> order=3 >> >> # dense weights for feature functions >> [weight] >> >> LexicalReordering0= 0.061536 0.0312366 0.0679776 0.100544 0.0571078 >> 0.0366389 >> Distortion0= 0.0914135 >> LM0= 0.0877283 >> WordPenalty0= -0.0882794 >> PhrasePenalty0= 0.0850377 >> TranslationModel0= 0.0215993 0.108041 0.102662 0.0601976 >> UnknownWordPenalty0= 1 >> >> Best >> Despina >> >> >> ___ >> Moses-support mailing >> listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> -- >> Hieu Hoanghttp://moses-smt.org/ >> >> > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] SOS-Problem with filter-model-given-input.pl
Hello everyone, I am trying to filtered my set as follows cd ~/working ~/mosesdecoder/scripts/training/filter-model-given-input.pl \ filtered-newstest2011 mert-work/moses.ini ~/corpus/newstest2011.true.fr \ -Binarizer ~/mosesdecoder/bin/processPhraseTableMin But after i got this. Can't read /working/mert-work/moses.ini at /home/despina/mosesdecoder/ scripts/training/filter-model-given-input.pl line 136. This is my mert-work/moses.ini # MERT optimized configuration # decoder /home/despina/mosesdecoder/bin/moses # BLEU 0.186802 on dev /home/despina/working/corpus/news-test2008.true.fr # We were before running iteration 9 # finished Τετ 14 Ιούν 2017 07:54:12 μμ EEST ### MOSES CONFIG FILE ### # # input factors [input-factors] 0 # mapping steps [mapping] 0 T 0 [distortion-limit] 6 # feature functions [feature] UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/despina/working/train/model/phrase-table.gz input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/home/despina/working/train/model/reordering-table.wbe-msd-bidirectional-fe.gz Distortion KENLM name=LM0 factor=0 path=/home/despina/lm/news-commentary-v8.fr-en.blm.en order=3 # dense weights for feature functions [weight] LexicalReordering0= 0.061536 0.0312366 0.0679776 0.100544 0.0571078 0.0366389 Distortion0= 0.0914135 LM0= 0.0877283 WordPenalty0= -0.0882794 PhrasePenalty0= 0.0850377 TranslationModel0= 0.0215993 0.108041 0.102662 0.0601976 UnknownWordPenalty0= 1 Best Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Experiment.perl vs classical moses tutorial
Hello everyone, I want to use moses smt for experiments, so I am typing all the comands of the baseline moses smt. At the end of all the 'long' procegure i find out about experiment.perl. Wich of these two procegures are better? Thank you Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Using gpu nvidia card for speed during the procces
Hello guys, I am turning my model right now, which is a long - lasting procces. So I was wondering if I can use my gpu nvidia card wich I bought for theano. If I can do so, what i have to type? Thanks in advance Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Unable to translate the in file became out file!
Thank you for the quik response. Unfortunatelly it happens the same thing when I am typing ./moses -f ~/Desktop/moses/train_ken/model/moses.ini in ~Desktop/moses/moses/bin directory. All the words that I am using are from the phrase table. For excample I am typing this is Translating: this is Line 2: Initialize search took 0.000 seconds total Line 2: Collecting options took 0.024 seconds at moses/Manager.cpp Line 141 Line 2: Search took 0.000 seconds this is BEST TRANSLATION: this|UNK|UNK|UNK is|UNK|UNK|UNK [11] [total=-208.787] core=(-200.000,-2.000,2.000,0.000,0.000,0.000,0.000,-0.234,0.000,0.000,-0.201,0.000,0.000,0.000,-80.590) Line 2: Decision rule took 0.000 seconds total Line 2: Additional reporting took 0.000 seconds total Line 2: Translation took 0.024 seconds total reset caches 2017-04-27 14:09 GMT+03:00 Hieu Hoang <hieuho...@gmail.com>: > it looks like the words in your input is are not in your phrase-table. By > default they will just be copied to the output as-is. > > > > * Looking for MT/NLP opportunities * > Hieu Hoang > http://moses-smt.org/ > > > On 27 April 2017 at 10:41, Despina Mouratidi <dmourat...@gmail.com> wrote: > >> Hello guys... >> >> I am trying to translate with moses but when I am typing >> ./moses -f ~/Desktop/moses/train_ken/model/moses.ini < ~/Desktop/in > >> ~/Desktop/out in my directory the out file became the in file. >> >> Thus, >> Defined parameters (per moses.ini or switch): >> config: /home/despina/Desktop/moses/train_ken/model/moses.ini >> distortion-limit: 6 >> feature: UnknownWordPenalty WordPenalty PhrasePenalty >> PhraseDictionaryCompact name=TranslationModel0 num-features=4 >> path=/home/despina/Desktop/moses/train_ken/model/phrase-table >> LexicalReordering name=LexicalReordering0 num-features=6 >> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >> path=/home/despina/Desktop/moses/train_ken/model/reordering Distortion >> IRSTLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/ >> corpus.en-fr.blm.fr order=3 >> input-factors: 0 >> mapping: 0 T 0 >> weight: LexicalReordering0= 0.0693229 0.0693229 0.0693229 0.0693229 >> 0.0693229 0.0693229 Distortion0= 0.00636699 LM0= 0.115539 WordPenalty0= >> -0.231077 PhrasePenalty0= 0.046216 TranslationModel0= 0.046216 0.046216 >> 0.046216 0.046216 UnknownWordPenalty0= 1 >> line=UnknownWordPenalty >> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0 >> line=WordPenalty >> FeatureFunction: WordPenalty0 start: 1 end: 1 >> line=PhrasePenalty >> FeatureFunction: PhrasePenalty0 start: 2 end: 2 >> line=PhraseDictionaryCompact name=TranslationModel0 num-features=4 >> path=/home/despina/Desktop/moses/train_ken/model/phrase-table >> FeatureFunction: TranslationModel0 start: 3 end: 6 >> line=LexicalReordering name=LexicalReordering0 num-features=6 >> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >> path=/home/despina/Desktop/moses/train_ken/model/reordering >> Initializing Lexical Reordering Feature.. >> FeatureFunction: LexicalReordering0 start: 7 end: 12 >> line=Distortion >> FeatureFunction: Distortion0 start: 13 end: 13 >> line=IRSTLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/ >> corpus.en-fr.blm.fr order=3 >> FeatureFunction: LM0 start: 14 end: 14 >> Loading UnknownWordPenalty0 >> Loading WordPenalty0 >> Loading PhrasePenalty0 >> Loading LexicalReordering0 >> Loading Distortion0 >> Loading LM0 >> Language Model Type of /home/despina/Desktop/moses/lm2/ >> corpus.en-fr.blm.fr is 1 >> Language Model Type is 1 >> starting to use OOV words [] >> OOV code is 0 >> OOV code is 0 >> LM0 LanguageModelIRST::Load() m_unknownId=0 >> Loading TranslationModel0 >> Created input-output object : [0.172] seconds >> Translating: hello >> Line 0: Initialize search took 0.000 seconds total >> Line 0: Collecting options took 0.000 seconds at moses/Manager.cpp Line >> 141 >> Line 0: Search took 0.000 seconds >> BEST TRANSLATION: hello|UNK|UNK|UNK [1] [total=-105.310] >> core=(-100.000,-1.000,1.000,0.000,0.000,0.000,0.000,0.000,0. >> 000,0.000,0.000,0.000,0.000,0.000,-48.354) >> Line 0: Decision rule took 0.000 seconds total >> Line 0: Additional reporting took 0.000 seconds total >> Line 0: Translation took 0.000 seconds total >> reset caches >> reset mmap >> Name:mosesVmPeak:565984 kBVmRSS:49036 kBRSSMax:370304 kB >> user:0.136sys:0.104CPU:0.240real:0.238 >> >> >> Thanks in advance >> Despina >> >> >> >> ___ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Unable to translate the in file became out file!
Hello guys... I am trying to translate with moses but when I am typing ./moses -f ~/Desktop/moses/train_ken/model/moses.ini < ~/Desktop/in > ~/Desktop/out in my directory the out file became the in file. Thus, Defined parameters (per moses.ini or switch): config: /home/despina/Desktop/moses/train_ken/model/moses.ini distortion-limit: 6 feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryCompact name=TranslationModel0 num-features=4 path=/home/despina/Desktop/moses/train_ken/model/phrase-table LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/home/despina/Desktop/moses/train_ken/model/reordering Distortion IRSTLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/ corpus.en-fr.blm.fr order=3 input-factors: 0 mapping: 0 T 0 weight: LexicalReordering0= 0.0693229 0.0693229 0.0693229 0.0693229 0.0693229 0.0693229 Distortion0= 0.00636699 LM0= 0.115539 WordPenalty0= -0.231077 PhrasePenalty0= 0.046216 TranslationModel0= 0.046216 0.046216 0.046216 0.046216 UnknownWordPenalty0= 1 line=UnknownWordPenalty FeatureFunction: UnknownWordPenalty0 start: 0 end: 0 line=WordPenalty FeatureFunction: WordPenalty0 start: 1 end: 1 line=PhrasePenalty FeatureFunction: PhrasePenalty0 start: 2 end: 2 line=PhraseDictionaryCompact name=TranslationModel0 num-features=4 path=/home/despina/Desktop/moses/train_ken/model/phrase-table FeatureFunction: TranslationModel0 start: 3 end: 6 line=LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/home/despina/Desktop/moses/train_ken/model/reordering Initializing Lexical Reordering Feature.. FeatureFunction: LexicalReordering0 start: 7 end: 12 line=Distortion FeatureFunction: Distortion0 start: 13 end: 13 line=IRSTLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/ corpus.en-fr.blm.fr order=3 FeatureFunction: LM0 start: 14 end: 14 Loading UnknownWordPenalty0 Loading WordPenalty0 Loading PhrasePenalty0 Loading LexicalReordering0 Loading Distortion0 Loading LM0 Language Model Type of /home/despina/Desktop/moses/lm2/corpus.en-fr.blm.fr is 1 Language Model Type is 1 starting to use OOV words [] OOV code is 0 OOV code is 0 LM0 LanguageModelIRST::Load() m_unknownId=0 Loading TranslationModel0 Created input-output object : [0.172] seconds Translating: hello Line 0: Initialize search took 0.000 seconds total Line 0: Collecting options took 0.000 seconds at moses/Manager.cpp Line 141 Line 0: Search took 0.000 seconds BEST TRANSLATION: hello|UNK|UNK|UNK [1] [total=-105.310] core=(-100.000,-1.000,1.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,-48.354) Line 0: Decision rule took 0.000 seconds total Line 0: Additional reporting took 0.000 seconds total Line 0: Translation took 0.000 seconds total reset caches reset mmap Name:mosesVmPeak:565984 kBVmRSS:49036 kBRSSMax:370304 kB user:0.136sys:0.104CPU:0.240real:0.238 Thanks in advance Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] wrap-xml.perl HELP
Hi all, I want to convert a txt file to .sgm. I know that i can do it through wrap-xml.perl script and I already see the excample, $ scripts/wrap-xml.perl wmt08/devtest/devtest2006-ref.en.sgm en < > working-dir/evaluation/devtest2006.output.detokenized > > working-dir/evaluation/devtest2006.output.sgm but its still confused. The text that I want to convert is 1000.en in Desktop folder. Do you know how the command will be like? Thanks in advance Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] The out file it became as the in file. HELP
Helloo, unfortunatelly my problem remains. I am trying to translate the file in to a file out and I am typing ~/Desktop/moses/moses/bin$ ./moses -f ~/Desktop/moses/train_ken/model/moses.ini < ~/Desktop/in > ~/Desktop/out I am in the bin directory. My moses.in file is # ### MOSES CONFIG FILE ### # # input factors [input-factors] 0 # mapping steps [mapping] 0 T 0 [distortion-limit] 6 # feature functions [feature] UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryCompact name=TranslationModel0 num-features=4 path=/home/despina/Desktop/moses/train_ken/model/phrase-table LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/home/despina/Desktop/moses/train_ken/model/reordering Distortion KENLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/ corpus.en-fr.arpa.fr order=3 # dense weights for feature functions [weight] # The default weights are NOT optimized for translation quality. You MUST tune the weights. # Documentation for tuning is here: http://www.statmt.org/moses/?n=FactoredTraining.Tuning UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3 Distortion0= 0.3 LM0= 0.5 the results...nothing happens. The out file it became as the in file. Thanks in advance Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Compiling with ./bjam problem
Thank you so much! Finally SUCCESS with boost 1_60_0 Thank you all!!! Best Despina 2016-03-12 16:27 GMT+02:00 Santanu Pal <santanu.pal...@gmail.com>: > I followed 4 steps in the link : > http://www.statmt.org/moses/?n=Development.GetStarted > and then, before training/decoding > I export > > export DYLD_LIBRARY_PATH=.:/Volumes/disk0s5/MT/mosesdecoder/opt/lib64 > > > my moses installation path: /Volumes/disk0s5/MT/mosesdecoder/ > > It works for me (both ubuntu and mac.) > > May this will also work for you. > > > Best > > Santanu > > > On Sat, Mar 12, 2016 at 2:57 PM, Despina Mouratidi <dmourat...@gmail.com> > wrote: > >> Hello again, >> i installed boost_1_60_0 and ERROR again. >> >> ... >> Data::m_score_type BLEU >> Data::Scorer type from Scorer: BLEU >> Data::m_score_type BLEU >> Data::Scorer type from Scorer: BLEU >> Data::m_score_type BLEU >> Data::Scorer type from Scorer: BLEU >> Data::m_score_type BLEU >> Data::Scorer type from Scorer: BLEU >> Data::m_score_type BLEU >> Data::Scorer type from Scorer: BLEU >> >> *** No errors detected >> gcc.link >> mert/bin/gcc-4.8/release/link-static/threading-multi/mira_feature_vector_test >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/mira_feature_vector_test.passed >> Running 1 test case... >> >> *** No errors detected >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/ForestRescoreTest.o >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/HypergraphTest.o >> gcc.link >> mert/bin/gcc-4.8/release/link-static/threading-multi/forest_rescore_test >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/forest_rescore_test.passed >> Running 2 test cases... >> >> *** No errors detected >> gcc.link >> mert/bin/gcc-4.8/release/link-static/threading-multi/hypergraph_test >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/hypergraph_test.passed >> Running 1 test case... >> >> *** No errors detected >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/OptimizerFactoryTest.o >> gcc.link >> mert/bin/gcc-4.8/release/link-static/threading-multi/optimizer_factory_test >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/optimizer_factory_test.passed >> Running 2 test cases... >> >> *** No errors detected >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/ReferenceTest.o >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/SingletonTest.o >> gcc.link >> mert/bin/gcc-4.8/release/link-static/threading-multi/reference_test >> ...on 700th target... >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/reference_test.passed >> Running 5 test cases... >> >> *** No errors detected >> gcc.link >> mert/bin/gcc-4.8/release/link-static/threading-multi/singleton_test >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/singleton_test.passed >> Running 1 test case... >> >> *** No errors detected >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/NgramTest.o >> gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/ngram_test >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/ngram_test.passed >> Running 3 test cases... >> >> *** No errors detected >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/PointTest.o >> gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/point_test >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/point_test.passed >> Running 1 test case... >> >> *** No errors detected >> gcc.link mert/mert >> gcc.link mert/extractor >> gcc.link mert/evaluator >> gcc.link mert/pro >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/TimerTest.o >> gcc.link mert/kbmira >> gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/timer_test >> testing.unit-test >> mert/bin/gcc-4.8/release/link-static/threading-multi/timer_test.passed >> Running 1 test case... >> >> *** No errors detected >> gcc.link mert/sentence-bleu >> gcc.link mert/sentence-bleu-nbest >> gcc.link mert/hgdecode >> gcc.compile.c++ >> mert/bin/gcc-4.8/release/link-static/threading-multi/VocabularyTest.o >> gcc.link >
[Moses-support] Compiling with ./bjam problem
Hello again, i am trying to compile moses throught ./bjam as follows ./bjam --with-boost=~/boost_1_55_0 -j7 and it gives error. Best Despina build.log.gz Description: GNU Zip compressed data ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Fwd: Problem with bjam
-- Forwarded message -- From: Despina Mouratidi <dmourat...@gmail.com> Date: 2016-03-08 20:57 GMT+02:00 Subject: Re: [Moses-support] Problem with bjam To: Hieu Hoang <hieuho...@gmail.com> Hello, after that at the end of this proccess it said ERROR. I am confused because it said ***no errors detected and then ERROR thanks Despina 2016-03-04 22:47 GMT+02:00 Hieu Hoang <hieuho...@gmail.com>: > There doesn't seem to be an error. It's not building the server 'cos the > xmlrpc-c library was not found. > > You don't need the server unless you want to deploy moses as a service > > Hieu Hoang > http://www.hoang.co.uk/hieu > > On 4 March 2016 at 11:29, Despina Mouratidi <dmourat...@gmail.com> wrote: >> >> Hello everyone, >> I am trying to install Moses on Ubuntu 14 (i have >> studied the instructions of the site (statmt.org) about Moses). I >> installed boost and then I wrote ./bjam but it showed like this >> (bjam.png). What can i do? >> >> Thanks in advance! >> Despina >> >> ___ >> Moses-support mailing list >> Moses-support@mit.edu >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Problem with bjam
Hello everyone, I am trying to install Moses on Ubuntu 14 (i have studied the instructions of the site (statmt.org) about Moses). I installed boost and then I wrote ./bjam but it showed like this (bjam.png). What can i do? Thanks in advance! Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Eclipse for moses smt PROBLEM
Hello, i was follow the instructions of the video for using eclipse but i dont have eclipse luna. I have eclipse 3.8 which hasnt c++ extension. I downloaded eclipse mars but i cant use it through command line for moses. Any ideas? Thanks in advance Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
[Moses-support] Installing Eclipse for moses
Hello, i am a beginner in Moses (debugging and development) and i was wondering if its better to use Moses through Eclipse. Thanks in advance Despina ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
