Hi all,
I built a factored model following the guideline on Moses web page. In order to
faster the decoding speed, I’m trying to use the binarised phrase table.
The binaring progress is finished, when trying to decode with the binarised
phrase table, the translation got failed. The input and output are the same.
Dose Moses support binarised translation table for factored model? Does anybody
also meet this issue?
Below are the outputs of the decoding process:
1.decoding with binarised phrase-table:
[root@Redhat-252 binarised-model]# echo 'the_DT' |
/data/moses/moses-smt-mosesdecoder/bin/moses -f moses.ini
Defined parameters (per moses.ini or switch):
config: moses.ini
distortion-limit: 6
factor-delimiter: _
input-factors: 0
lmodel-file: 0 0 3
/data/english-chinese_POS_tag/chinese-lm/english-chinese.lm.cn 0 1 3
/data/english-chinese_POS_tag/chinese-pos-lm/english-chinese.lm.cn
mapping: 0 T 0
ttable-file: 1 0 0,1 5
/data/english-chinese_POS_tag/binarised-model/phrase-table
ttable-limit: 20
weight-d: 0.6
weight-l: 0.2500 0.2500
weight-t: 0.20 0.20 0.20 0.20 0.20
weight-w: -1
/data/moses/moses-smt-mosesdecoder/bin
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/data/english-chinese_POS_tag/chinese-lm/english-chinese.lm.cn : [0.001] seconds
/data/english-chinese_POS_tag/chinese-lm/english-chinese.lm.cn: line 125679:
warning: non-zero probability for <unk> in closed-vocabulary LM
Start loading LanguageModel
/data/english-chinese_POS_tag/chinese-pos-lm/english-chinese.lm.cn : [7.148]
seconds
/data/english-chinese_POS_tag/chinese-pos-lm/english-chinese.lm.cn: line 46:
warning: non-zero probability for <unk> in closed-vocabulary LM
Finished loading LanguageModels : [7.214] seconds
Start loading PhraseTable
/data/english-chinese_POS_tag/binarised-model/phrase-table : [7.214] seconds
filePath: /data/english-chinese_POS_tag/binarised-model/phrase-table
Finished loading phrase tables : [7.214] seconds
IO from STDOUT/STDIN
Created input-output object : [7.214] seconds
Translating line 0 in thread id 140249033144064
Translating: the
reading bin ttable
size of OFF_T 8
binary phrasefile loaded, default OFF_T: -1
Line 0: Collecting options took 0.000 seconds
Line 0: Search took 0.000 seconds
the
BEST TRANSLATION: the_UNK_UNK_UNK [1] [total=-111.439] <<0.000, -1.000,
-100.000, -23.206, -26.549, 0.000, 0.000, 0.000, 0.000, 0.000>> 0-0
Line 0: Translation took 0.894 seconds total
2．Normal decoding
[root@Redhat-252 english-chinese_POS_tag]# echo 'the_DT' |
/data/moses/moses-smt-mosesdecoder/bin/moses -f train/model/moses.ini
Defined parameters (per moses.ini or switch):
config: train/model/moses.ini
distortion-limit: 6
factor-delimiter: _
input-factors: 0
lmodel-file: 0 0 3
/data/english-chinese_POS_tag/chinese-lm/english-chinese.lm.cn 0 1 3
/data/english-chinese_POS_tag/chinese-pos-lm/english-chinese.lm.cn
mapping: 0 T 0
ttable-file: 0 0 0,1 5
/data/english-chinese_POS_tag/train/model/phrase-table.0-0,1.gz
ttable-limit: 20
weight-d: 0.6
weight-l: 0.2500 0.2500
weight-t: 0.20 0.20 0.20 0.20 0.20
weight-w: -1
/data/moses/moses-smt-mosesdecoder/bin
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/data/english-chinese_POS_tag/chinese-lm/english-chinese.lm.cn : [0.000] seconds
/data/english-chinese_POS_tag/chinese-lm/english-chinese.lm.cn: line 125679:
warning: non-zero probability for <unk> in closed-vocabulary LM
Start loading LanguageModel
/data/english-chinese_POS_tag/chinese-pos-lm/english-chinese.lm.cn : [4.239]
seconds
/data/english-chinese_POS_tag/chinese-pos-lm/english-chinese.lm.cn: line 46:
warning: non-zero probability for <unk> in closed-vocabulary LM
Finished loading LanguageModels : [4.254] seconds
Start loading PhraseTable
/data/english-chinese_POS_tag/train/model/phrase-table.0-0,1.gz : [4.254]
seconds
filePath: /data/english-chinese_POS_tag/train/model/phrase-table.0-0,1.gz
Finished loading phrase tables : [4.254] seconds
Start loading phrase table from
/data/english-chinese_POS_tag/train/model/phrase-table.0-0,1.gz : [4.254]
seconds
Reading /data/english-chinese_POS_tag/train/model/phrase-table.0-0,1.gz
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Finished loading phrase tables : [422.886] seconds
IO from STDOUT/STDIN
Created input-output object : [422.895] seconds
Translating line 0 in thread id 139991742867200
Translating: the
Line 0: Collecting options took 0.061 seconds
Line 0: Search took 0.185 seconds
在
BEST TRANSLATION: 在_P [1] [total=-6.025] <<0.000, -1.000, 0.000, -12.496,
-9.723, -1.545, -1.590, -2.312, -2.906, 1.000>>
Line 0: Translation took 0.247 seconds total
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
