you're gonna have to do some debugging yourself. Perhaps look inside train_Moses.job and training.out
Hieu Hoang http://statmt.org/hieu On Tue, 30 Jul 2019 at 11:48, Aaron Poet <hanlifengaa...@gmail.com> wrote: > Hi Hieu > Thanks for the reply. But, after I added the colon there, it still have > some problem in running. > *When I tried to run directly on the note of server it shows below:* > > (tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice > /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl > -root-dir train -corpus > /home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en > -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm > 0:3:$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir > /home/aaronhan/Moses/mosesdecoder/tools/ >& training.out & > > [1] 26177 > > (tensorflow) aaronhan@grove2:~/Moses/working$ > > It produced a folder 'train' and sub-folder 'corpus' and the empty files > "en.vcb.classes > en.vcb.classes.cats zh.vcb.classes zh.vcb.classes.cats" > > (tensorflow) aaronhan@grove2:~/Moses/working$ ls > > slurm-23199.out slurm-23201.out *train* *train_Moses.job* > > slurm-23200.out slurm-23202.out training.out > > (tensorflow) aaronhan@grove2:~/Moses/working$ cd train/ > > (tensorflow) aaronhan@grove2:~/Moses/working/train$ ls > > *corpus* > > (tensorflow) aaronhan@grove2:~/Moses/working/train$ ls corpus/ > > en.vcb.classes en.vcb.classes.cats zh.vcb.classes zh.vcb.classes.cats > > (tensorflow) aaronhan@grove2:~/Moses/working/train$ > > > *When I submitted this command as job file to server, it does not appear > in the job list/ doesn't to look like running.* > > Do you have any clue for this, please? > > Kind regards, > > > > On Tue, Jul 30, 2019 at 8:45 AM Hieu Hoang <hieuho...@gmail.com> wrote: > >> >> >> On Mon, 29 Jul 2019, 19:58 Aaron Poet, <hanlifengaa...@gmail.com> wrote: >> >>> Hi Moses-Support >>> >>> I firstly build the lm as below: >>> >>> (base) aaronhan@grove2:~/Moses$ head -20 >>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en >>> >>> \data\ >>> >>> ngram 1=553697 >>> >>> ngram 2=8790963 >>> >>> ngram 3=28838697 >>> >>> >>> \1-grams: >>> >>> -6.993945 <unk> 0 >>> >>> 0 <s> -1.6929523 >>> >>> -2.0611665 </s> 0 >>> >>> -2.3942525 the -0.9461817 >>> >>> -3.6470015 show -0.6162386 >>> >>> -4.1808434 stars -0.5792577 >>> >>> -4.1823444 X -0.53191173 >>> >>> -5.0213814 Girls -0.33374956 >>> >>> -2.5064495 - -0.6326866 >>> >>> -2.5754578 a -0.9741243 >>> >>> -5.1265607 troupe -0.3527482 >>> >>> -2.2843323 of -0.91076595 >>> >>> -4.663717 talented -0.3983 >>> >>> -5.2061567 topless -0.27003714 >>> >>> Then, I binary it as bellow: >>> >>> /home/aaronhan/Moses/mosesdecoder/bin/build_binary >>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en >>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en >>> >>> (base) aaronhan@grove2:~/Moses$ ls -lh >>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en >>> >>> -rw-r--r-- 1 aaronhan users 715M Jul 29 18:38 >>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en >>> >>> when I test it, it looks working out: >>> >>> (tensorflow) aaronhan@grove2:~/Moses/working$ echo "is this an English >>> sentence ?" \ >>> >>> > | /home/aaronhan/Moses/mosesdecoder/bin/query >>> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en >>> >>> is=23 2 -2.5766807 this=321 3 -1.0483611 an=108 3 -2.1032705 >>> English=1731 2 -2.8970845 sentence=3266 3 -2.8317716 ?=815 2 >>> -2.0824053 </s>=2 3 -0.035550244 Total: -13.575124 OOV: 0 >>> >>> Perplexity including OOVs: 86.9567697150713 >>> >>> Perplexity excluding OOVs: 86.9567697150713 >>> >>> OOVs: 0 >>> >>> Tokens: 7 >>> >>> Name:query VmPeak:748300 kB VmRSS:4052 kB RSSMax:503644 >>> kB user:0.008 sys:0.652 CPU:0.66 real:61.9797 >>> >>> However, when I run the overall Moses training, it produced error as >>> below: *Wrong format of --lm. Expected: --lm factor:order:filename* >>> >>> (tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice >>> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl >>> -root-dir train -corpus >>> /home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en >>> -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm >>> 0:3$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir >>> /home/aaronhan/Moses/mosesdecoder/tools/ >& training.out & >>> >>> [1] 23346 >>> >>> You forgot a colon between 3 and $HOME >> >>> >>> GNU nano 2.7.4 >>> File: training.out >>> >>> >>> >>> nohup: ignoring input >>> >>> Using SCRIPTS_ROOTDIR: /home/aaronhan/Moses/mosesdecoder/scripts >>> >>> Using single-thread GIZA >>> >>> using gzip >>> >>> ERROR: Wrong format of --lm. Expected: --lm factor:order:filename at >>> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl line >>> 605. >>> >>> >>> >>> Can you let me know how to solve this, please? >>> >>> Kind regards, >>> Aaron >>> >>> -- >>> >>> PhD student, Msc. (excellent), Bsc. (Maths) >>> >>> Google-Scholar >>> <https://scholar.google.nl/citations?user=_vf3E2QAAAAJ&hl=en> , >>> Presentation <https://www.slideshare.net/AaronHanLiFeng>-slides, >>> Research-Gate <https://www.researchgate.net/profile/Aaron_L-F_Han> >>> >>> Google-site <https://sites.google.com/view/poetgarden/home> Linkedin >>> <https://www.linkedin.com/in/aaronhan/>, >>> >>> ADAPT <https://www.adaptcentre.ie/> ResearchCentre & >>> >>> School of Computing, DCU, >>> >>> Dublin, Ireland >>> >>> >>> _______________________________________________ >>> Moses-support mailing list >>> Moses-support@mit.edu >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> > > -- > > PhD student, Msc. (excellent), Bsc. (Maths) > > Google-Scholar > <https://scholar.google.nl/citations?user=_vf3E2QAAAAJ&hl=en> , > Presentation <https://www.slideshare.net/AaronHanLiFeng>-slides, > Research-Gate <https://www.researchgate.net/profile/Aaron_L-F_Han> > > Google-site <https://sites.google.com/view/poetgarden/home> Linkedin > <https://www.linkedin.com/in/aaronhan/>, > > ADAPT <https://www.adaptcentre.ie/> ResearchCentre & > > School of Computing, DCU, > > Dublin, Ireland > > >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
