Hello Sir,
I'm trying to execute command to train the model as given in manual
provided onofficial website.
But I'm getting this error.
COMMAND:
hieu@hieu-VirtualBox:~/workspace/working$ nohup nice
/home/hieu/workspace/mosesdecoder/scripts/training/train-model.perl
-root-dir train -corpus /home/hieu/workspace/corpus/clean -f fr -e en
-alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm
0:3:/home/hieu/workspace/lm/blm.en:8 -external-bin-dir
/home/hieu/workspace/working/bin/ -mgiza >& training.out &
OUTPUT:
nohup: ignoring input
Using SCRIPTS_ROOTDIR: /home/hieu/workspace/mosesdecoder/scripts
Using multi-thread GIZA
using gzip
(1) preparing corpus @ Sat Apr 16 19:42:06 BST 2016
Executing: mkdir -p /home/hieu/workspace/working/train/corpus
(1.0) selecting factors @ Sat Apr 16 19:42:06 BST 2016
(1.1) running mkcls @ Sat Apr 16 19:42:06 BST 2016
/home/hieu/workspace/working/bin/mkcls -c50 -n2
-p/home/hieu/workspace/corpus/clean.fr
-V/home/hieu/workspace/working/train/corpus/fr.vcb.classes opt
/home/hieu/workspace/working/train/corpus/fr.vcb.classes already in
place, reusing
(1.1) running mkcls @ Sat Apr 16 19:42:06 BST 2016
/home/hieu/workspace/working/bin/mkcls -c50 -n2
-p/home/hieu/workspace/corpus/clean.en
-V/home/hieu/workspace/working/train/corpus/en.vcb.classes opt
/home/hieu/workspace/working/train/corpus/en.vcb.classes already in
place, reusing
(1.2) creating vcb file /home/hieu/workspace/working/train/corpus/fr.vcb @
Sat Apr 16 19:42:06 BST 2016
(1.2) creating vcb file /home/hieu/workspace/working/train/corpus/en.vcb @
Sat Apr 16 19:42:06 BST 2016
(1.3) numberizing corpus
/home/hieu/workspace/working/train/corpus/fr-en-int-train.snt @ Sat Apr 16
19:42:06 BST 2016
/home/hieu/workspace/working/train/corpus/fr-en-int-train.snt already in
place, reusing
(1.3) numberizing corpus
/home/hieu/workspace/working/train/corpus/en-fr-int-train.snt @ Sat Apr 16
19:42:06 BST 2016
/home/hieu/workspace/working/train/corpus/en-fr-int-train.snt already in
place, reusing
(2) running giza @ Sat Apr 16 19:42:06 BST 2016
(2.1a) running snt2cooc fr-en @ Sat Apr 16 19:42:06 BST 2016
Executing: mkdir -p /home/hieu/workspace/working/train/giza.fr-en
Executing: /home/hieu/workspace/working/bin/snt2cooc
/home/hieu/workspace/working/train/giza.fr-en/fr-en.cooc
/home/hieu/workspace/working/train/corpus/en.vcb
/home/hieu/workspace/working/train/corpus/fr.vcb
/home/hieu/workspace/working/train/corpus/fr-en-int-train.snt
/home/hieu/workspace/working/bin/snt2cooc
/home/hieu/workspace/working/train/giza.fr-en/fr-en.cooc
/home/hieu/workspace/working/train/corpus/en.vcb
/home/hieu/workspace/working/train/corpus/fr.vcb
/home/hieu/workspace/working/train/corpus/fr-en-int-train.snt
END.
(2.1b) running giza fr-en @ Sat Apr 16 19:42:06 BST 2016
/home/hieu/workspace/working/bin/mgiza -CoocurrenceFile
/home/hieu/workspace/working/train/giza.fr-en/fr-en.cooc -c
/home/hieu/workspace/working/train/corpus/fr-en-int-train.snt -m1 5 -m2 0
-m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 -ncpus 4
-nodumps 1 -nsmooth 4 -o
/home/hieu/workspace/working/train/giza.fr-en/fr-en -onlyaldumps 1 -p0
0.999 -s /home/hieu/workspace/working/train/corpus/en.vcb -t
/home/hieu/workspace/working/train/corpus/fr.vcb
/home/hieu/workspace/working/train/giza.fr-en/fr-en.A3.final.gz seems
finished, reusing.
(2.1a) running snt2cooc en-fr @ Sat Apr 16 19:42:06 BST 2016
Executing: mkdir -p /home/hieu/workspace/working/train/giza.en-fr
Executing: /home/hieu/workspace/working/bin/snt2cooc
/home/hieu/workspace/working/train/giza.en-fr/en-fr.cooc
/home/hieu/workspace/working/train/corpus/fr.vcb
/home/hieu/workspace/working/train/corpus/en.vcb
/home/hieu/workspace/working/train/corpus/en-fr-int-train.snt
/home/hieu/workspace/working/bin/snt2cooc
/home/hieu/workspace/working/train/giza.en-fr/en-fr.cooc
/home/hieu/workspace/working/train/corpus/fr.vcb
/home/hieu/workspace/working/train/corpus/en.vcb
/home/hieu/workspace/working/train/corpus/en-fr-int-train.snt
END.
(2.1b) running giza en-fr @ Sat Apr 16 19:42:07 BST 2016
/home/hieu/workspace/working/bin/mgiza -CoocurrenceFile
/home/hieu/workspace/working/train/giza.en-fr/en-fr.cooc -c
/home/hieu/workspace/working/train/corpus/en-fr-int-train.snt -m1 5 -m2 0
-m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 -ncpus 4
-nodumps 1 -nsmooth 4 -o
/home/hieu/workspace/working/train/giza.en-fr/en-fr -onlyaldumps 1 -p0
0.999 -s /home/hieu/workspace/working/train/corpus/fr.vcb -t
/home/hieu/workspace/working/train/corpus/en.vcb
/home/hieu/workspace/working/train/giza.en-fr/en-fr.A3.final.gz seems
finished, reusing.
(3) generate word alignment @ Sat Apr 16 19:42:07 BST 2016
Combining forward and inverted alignment from files:
/home/hieu/workspace/working/train/giza.fr-en/fr-en.A3.final.{bz2,gz}
/home/hieu/workspace/working/train/giza.en-fr/en-fr.A3.final.{bz2,gz}
Executing: mkdir -p /home/hieu/workspace/working/train/model
Executing: /home/hieu/workspace/mosesdecoder/scripts/training/giza2bal.pl
-d "gzip -cd
/home/hieu/workspace/working/train/giza.en-fr/en-fr.A3.final.gz" -i "gzip
-cd /home/hieu/workspace/working/train/giza.fr-en/fr-en.A3.final.gz"
|/home/hieu/workspace/mosesdecoder/scripts/../bin/symal -alignment="grow"
-diagonal="yes" -final="yes" -both="yes" >
/home/hieu/workspace/working/train/model/aligned.grow-diag-final-and
symal: computing grow alignment: diagonal (1) final (1)both-uncovered (1)
skip=<0> counts=<993>
(4) generate lexical translation table 0-0 @ Sat Apr 16 19:42:07 BST 2016
(/home/hieu/workspace/corpus/clean.fr
,/home/hieu/workspace/corpus/clean.en,/home/hieu/workspace/working/train/model/lex)
reusing: /home/hieu/workspace/working/train/model/lex.f2e and
/home/hieu/workspace/working/train/model/lex.e2f
(5) extract phrases @ Sat Apr 16 19:42:07 BST 2016
/home/hieu/workspace/mosesdecoder/scripts/generic/extract-parallel.perl 2
split "sort " /home/hieu/workspace/mosesdecoder/scripts/../bin/extract
/home/hieu/workspace/corpus/clean.en /home/hieu/workspace/corpus/clean.fr
/home/hieu/workspace/working/train/model/aligned.grow-diag-final-and
/home/hieu/workspace/working/train/model/extract 7 orientation --model
wbe-msd --GZOutput
Executing:
/home/hieu/workspace/mosesdecoder/scripts/generic/extract-parallel.perl 2
split "sort " /home/hieu/workspace/mosesdecoder/scripts/../bin/extract
/home/hieu/workspace/corpus/clean.en /home/hieu/workspace/corpus/clean.fr
/home/hieu/workspace/working/train/model/aligned.grow-diag-final-and
/home/hieu/workspace/working/train/model/extract 7 orientation --model
wbe-msd --GZOutput
MAX 7 1 0
Started Sat Apr 16 19:42:07 2016
using gzip
isBSDSplit=0
Executing: mkdir -p /home/hieu/workspace/working/train/model/tmp.6041; ls
-l /home/hieu/workspace/working/train/model/tmp.6041
total=993 line-per-split=497
split -d -l 497 -a 7 /home/hieu/workspace/corpus/clean.en
/home/hieu/workspace/working/train/model/tmp.6041/target.split -d -l 497 -a
7 /home/hieu/workspace/working/train/model/aligned.grow-diag-final-and
/home/hieu/workspace/working/train/model/tmp.6041/align.split -d -l 497 -a
7 /home/hieu/workspace/corpus/clean.fr
/home/hieu/workspace/working/train/model/tmp.6041/source.merging extract /
extract.inv
gunzip -c
/home/hieu/workspace/working/train/model/tmp.6041/extract.0000000.gz
/home/hieu/workspace/working/train/model/tmp.6041/extract.0000001.gz |
LC_ALL=C sort -T /home/hieu/workspace/working/train/model/tmp.6041 2>>
/dev/stderr | gzip -c >
/home/hieu/workspace/working/train/model/extract.sorted.gz 2>> /dev/stderr
gunzip -c
/home/hieu/workspace/working/train/model/tmp.6041/extract.0000000.inv.gz
/home/hieu/workspace/working/train/model/tmp.6041/extract.0000001.inv.gz |
LC_ALL=C sort -T /home/hieu/workspace/working/train/model/tmp.6041 2>>
/dev/stderr | gzip -c >
/home/hieu/workspace/working/train/model/extract.inv.sorted.gz 2>>
/dev/stderr
gzip: /home/hieu/workspace/working/train/model/tmp.6041/extract.0000000.gz:
No such file or directory
gzip: /home/hieu/workspace/working/train/model/tmp.6041/extract.0000001.gz:
No such file or directory
gzip:
/home/hieu/workspace/working/train/model/tmp.6041/extract.0000000.inv.gz:
No such file or directory
gzip:
/home/hieu/workspace/working/train/model/tmp.6041/extract.0000001.inv.gz:
No such file or directory
Finished Sat Apr 16 19:42:07 2016
(6) score phrases @ Sat Apr 16 19:42:07 BST 2016
(6.1) creating table half
/home/hieu/workspace/working/train/model/phrase-table.half.f2e @ Sat Apr 16
19:42:07 BST 2016
/home/hieu/workspace/mosesdecoder/scripts/generic/score-parallel.perl 2
"sort " /home/hieu/workspace/mosesdecoder/scripts/../bin/score
/home/hieu/workspace/working/train/model/extract.sorted.gz
/home/hieu/workspace/working/train/model/lex.f2e
/home/hieu/workspace/working/train/model/phrase-table.half.f2e.gz 0
Executing:
/home/hieu/workspace/mosesdecoder/scripts/generic/score-parallel.perl 2
"sort " /home/hieu/workspace/mosesdecoder/scripts/../bin/score
/home/hieu/workspace/working/train/model/extract.sorted.gz
/home/hieu/workspace/working/train/model/lex.f2e
/home/hieu/workspace/working/train/model/phrase-table.half.f2e.gz 0
using gzip
Started Sat Apr 16 19:42:07 2016
/home/hieu/workspace/mosesdecoder/scripts/../bin/score
/home/hieu/workspace/working/train/model/tmp.6079/extract.0.gz
/home/hieu/workspace/working/train/model/lex.f2e
/home/hieu/workspace/working/train/model/tmp.6079/phrase-table.half.0000000.gz
2>> /dev/stderr
/home/hieu/workspace/working/train/model/tmp.6079/
run.0.sh/home/hieu/workspace/working/train/model/tmp.6079/run.1.shmv
/home/hieu/workspace/working/train/model/tmp.6079/phrase-table.half.0000000.gz
/home/hieu/workspace/working/train/model/phrase-table.half.f2e.gzmv: cannot
stat
'/home/hieu/workspace/working/train/model/tmp.6079/phrase-table.half.0000000.gz':
No such file or directory
Exit code: 1
ERROR: Scoring of phrases failed at
/home/hieu/workspace/mosesdecoder/scripts/training/train-model.perl line
1784.
(6.3) creating table half
/home/hieu/workspace/working/train/model/phrase-table.half.e2f @ Sat Apr 16
19:42:07 BST 2016
/home/hieu/workspace/mosesdecoder/scripts/generic/score-parallel.perl 2
"sort " /home/hieu/workspace/mosesdecoder/scripts/../bin/score
/home/hieu/workspace/working/train/model/extract.inv.sorted.gz
/home/hieu/workspace/working/train/model/lex.e2f
/home/hieu/workspace/working/train/model/phrase-table.half.e2f.gz --Inverse
1
Executing:
/home/hieu/workspace/mosesdecoder/scripts/generic/score-parallel.perl 2
"sort " /home/hieu/workspace/mosesdecoder/scripts/../bin/score
/home/hieu/workspace/working/train/model/extract.inv.sorted.gz
/home/hieu/workspace/working/train/model/lex.e2f
/home/hieu/workspace/working/train/model/phrase-table.half.e2f.gz --Inverse
1
using gzip
Started Sat Apr 16 19:42:07 2016
/home/hieu/workspace/mosesdecoder/scripts/../bin/score
/home/hieu/workspace/working/train/model/tmp.6097/extract.0.gz
/home/hieu/workspace/working/train/model/lex.e2f
/home/hieu/workspace/working/train/model/tmp.6097/phrase-table.half.0000000.gz
--Inverse 2>> /dev/stderr
/home/hieu/workspace/working/train/model/tmp.6097/
run.0.sh/home/hieu/workspace/working/train/model/tmp.6097/run.1.shgunzip -c
/home/hieu/workspace/working/train/model/tmp.6097/phrase-table.half.*.gz
2>> /dev/stderr| LC_ALL=C sort -T
/home/hieu/workspace/working/train/model/tmp.6097 | gzip -c >
/home/hieu/workspace/working/train/model/phrase-table.half.e2f.gz 2>>
/dev/stderr rm -rf /home/hieu/workspace/working/train/model/tmp.6097
Finished Sat Apr 16 19:42:07 2016
(6.6) consolidating the two halves @ Sat Apr 16 19:42:07 BST 2016
Executing: /home/hieu/workspace/mosesdecoder/scripts/../bin/consolidate
/home/hieu/workspace/working/train/model/phrase-table.half.f2e.gz
/home/hieu/workspace/working/train/model/phrase-table.half.e2f.gz
/dev/stdout | gzip -c >
/home/hieu/workspace/working/train/model/phrase-table.gz
/home/hieu/workspace/mosesdecoder/scripts/../bin/consolidate: error while
loading shared libraries: libboost_serialization.so.1.59.0: cannot open
shared object file: No such file or directory
Executing: rm -f
/home/hieu/workspace/working/train/model/phrase-table.half.*
(7) learn reordering model @ Sat Apr 16 19:42:07 BST 2016
(7.1) [no factors] learn reordering model @ Sat Apr 16 19:42:07 BST 2016
(7.2) building tables @ Sat Apr 16 19:42:07 BST 2016
Executing:
/home/hieu/workspace/mosesdecoder/scripts/../bin/lexical-reordering-score
/home/hieu/workspace/working/train/model/extract.o.sorted.gz 0.5
/home/hieu/workspace/working/train/model/reordering-table. --model "wbe msd
wbe-msd-bidirectional-fe"
Lexical Reordering Scorer
scores lexical reordering models of several types (hierarchical,
phrase-based and word-based-extraction
terminate called after throwing an instance of 'util::ErrnoException'
what(): util/file.cc:76 in int util::OpenReadOrThrow(const char*) threw
ErrnoException because `-1 == (ret = open(name, 00))'.
No such file or directory while opening
/home/hieu/workspace/working/train/model/extract.o.sorted.gz
Aborted (core dumped)
Exit code: 134
ERROR: Lexical reordering scoring failed at
/home/hieu/workspace/mosesdecoder/scripts/training/train-model.perl line
1922.
I guess the error starts with step5. Please help.
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
