There may be non printing characters in the phrase table. Are you sure your training data is clean?
On 26 May 2017 11:06 p.m., "刘焰" <liuyan1...@163.com> wrote: > Defined parameters (per moses.ini or switch): > config: /home/liuyan/new_moses/translate-model/zh-ar/model/ > moses.ini > distortion-limit: 6 > feature: UnknownWordPenalty WordPenalty PhrasePenalty > PhraseDictionaryMemory name=TranslationModel0 num-features=4 > path=/home/liuyan/new_moses/translate-model/zh-ar/model/phrase-table.gz > input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 > num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 > output-factor=0 path=/home/liuyan/new_moses/translate-model/zh-ar/model/ > reordering-table.wbe-msd-bidirectional-fe.gz Distortion SRILM name=LM0 > factor=0 path=/home/liuyan/new_moses/language-model/u130.arpa.ar order=3 > input-factors: 0 > mapping: 0 T 0 > weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= > 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 > 0.3 0.3 Distortion0= 0.3 LM0= 0.5 > line=UnknownWordPenalty > FeatureFunction: UnknownWordPenalty0 start: 0 end: 0 > line=WordPenalty > FeatureFunction: WordPenalty0 start: 1 end: 1 > line=PhrasePenalty > FeatureFunction: PhrasePenalty0 start: 2 end: 2 > line=PhraseDictionaryMemory name=TranslationModel0 num-features=4 > path=/home/liuyan/new_moses/translate-model/zh-ar/model/phrase-table.gz > input-factor=0 output-factor=0 > FeatureFunction: TranslationModel0 start: 3 end: 6 > line=LexicalReordering name=LexicalReordering0 num-features=6 > type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 > path=/home/liuyan/new_moses/translate-model/zh-ar/model/ > reordering-table.wbe-msd-bidirectional-fe.gz > Initializing Lexical Reordering Feature.. > FeatureFunction: LexicalReordering0 start: 7 end: 12 > line=Distortion > FeatureFunction: Distortion0 start: 13 end: 13 > line=SRILM name=LM0 factor=0 path=/home/liuyan/new_moses/language-model/ > u130.arpa.ar order=3 > FeatureFunction: LM0 start: 14 end: 14 > Loading UnknownWordPenalty0 > Loading WordPenalty0 > Loading PhrasePenalty0 > Loading LexicalReordering0 > Loading table into memory...done. > Loading Distortion0 > Loading LM0 > Loading TranslationModel0 > Start loading text phrase table. Moses format : [707.706] seconds > Reading /home/liuyan/new_moses/translate-model/zh-ar/model/phrase-table.gz > ----5---10---15---20---25---30---35---40---45---50---55--- > 60---65---70---75---80---85---90---95--100 > *** Segmentation fault > Register dump: > > RAX: 0000000000000000 RBX: 0000000293f9b200 RCX: 0000000000000000 > RDX: 0000000000000000 RSI: 00007ffde232ce30 RDI: 00007ffde232cd70 > RBP: 0000000000000000 R8 : 0000000000000000 R9 : 0000000000000000 > R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 > R13: 0000000000000000 R14: 0000000001992090 R15: 00000000019920a8 > RSP: 00007ffde232cd30 > > RIP: 0000000000461b0f EFLAGS: 00010246 > > CS: 0033 FS: 0000 GS: 0000 > > Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 00000020 > > FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000 > RIP: 00000000 RDP: 00000000 > > ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000 > ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000 > ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000 > ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000 > mxcsr: 1fb6 > XMM0: 00000000000000000000000000000000 XMM1: > 00000000000000000000000000000000 > XMM2: 00000000000000000000000000000000 XMM3: > 00000000000000000000000000000000 > XMM4: 00000000000000000000000000000000 XMM5: > 00000000000000000000000000000000 > XMM6: 00000000000000000000000000000000 XMM7: > 00000000000000000000000000000000 > XMM8: 00000000000000000000000000000000 XMM9: > 00000000000000000000000000000000 > XMM10: 00000000000000000000000000000000 XMM11: > 00000000000000000000000000000000 > XMM12: 00000000000000000000000000000000 XMM13: > 00000000000000000000000000000000 > XMM14: 00000000000000000000000000000000 XMM15: > 00000000000000000000000000000000 > > Backtrace: > /home/liuyan/new_moses/md3/bin/moses[0x461b0f] > /home/liuyan/new_moses/md3/bin/moses[0x71c7e2] > /home/liuyan/new_moses/md3/bin/moses[0x5f3de0] > /home/liuyan/new_moses/md3/bin/moses[0x4533bb] > /home/liuyan/new_moses/md3/bin/moses[0x459a23] > /home/liuyan/new_moses/md3/bin/moses[0x41b30f] > /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fdf00839b35] > /home/liuyan/new_moses/md3/bin/moses[0x41904c] > > Memory map: > > 00400000-0097b000 r-xp 00000000 fd:02 2156843667 > /home/liuyan/new_moses/md3/bin/moses > 00b7b000-00b7c000 r--p 0057b000 fd:02 2156843667 > /home/liuyan/new_moses/md3/bin/moses > 00b7c000-00b7e000 rw-p 0057c000 fd:02 2156843667 > /home/liuyan/new_moses/md3/bin/moses > 00b7e000-00b83000 rw-p 00000000 00:00 0 > 01107000-29404c000 rw-p 00000000 00:00 0 > [heap] > 7fdf00818000-7fdf009ce000 r-xp 00000000 fd:00 201352300 > /usr/lib64/libc-2.17.so > 7fdf009ce000-7fdf00bce000 ---p 001b6000 fd:00 201352300 > /usr/lib64/libc-2.17.so > 7fdf00bce000-7fdf00bd2000 r--p 001b6000 fd:00 201352300 > /usr/lib64/libc-2.17.so > 7fdf00bd2000-7fdf00bd4000 rw-p 001ba000 fd:00 201352300 > /usr/lib64/libc-2.17.so > 7fdf00bd4000-7fdf00bd9000 rw-p 00000000 00:00 0 > 7fdf00bd9000-7fdf00bf0000 r-xp 00000000 fd:00 201352328 > /usr/lib64/libpthread-2.17.so > 7fdf00bf0000-7fdf00def000 ---p 00017000 fd:00 201352328 > /usr/lib64/libpthread-2.17.so > 7fdf00def000-7fdf00df0000 r--p 00016000 fd:00 201352328 > /usr/lib64/libpthread-2.17.so > 7fdf00df0000-7fdf00df1000 rw-p 00017000 fd:00 201352328 > /usr/lib64/libpthread-2.17.so > 7fdf00df1000-7fdf00df5000 rw-p 00000000 00:00 0 > 7fdf00df5000-7fdf00e0a000 r-xp 00000000 fd:00 208532472 > /usr/lib64/libgcc_s-4.8.5-20150702.so.1 > 7fdf00e0a000-7fdf01009000 ---p 00015000 fd:00 208532472 > /usr/lib64/libgcc_s-4.8.5-20150702.so.1 > 7fdf01009000-7fdf0100a000 r--p 00014000 fd:00 208532472 > /usr/lib64/libgcc_s-4.8.5-20150702.so.1 > 7fdf0100a000-7fdf0100b000 rw-p 00015000 fd:00 208532472 > /usr/lib64/libgcc_s-4.8.5-20150702.so.1 > 7fdf0100b000-7fdf01030000 r-xp 00000000 fd:00 201630920 > /usr/lib64/libgomp.so.1.0.0 > 7fdf01030000-7fdf0122f000 ---p 00025000 fd:00 201630920 > /usr/lib64/libgomp.so.1.0.0 > 7fdf0122f000-7fdf01230000 r--p 00024000 fd:00 201630920 > /usr/lib64/libgomp.so.1.0.0 > 7fdf01230000-7fdf01231000 rw-p 00025000 fd:00 201630920 > /usr/lib64/libgomp.so.1.0.0 > 7fdf01231000-7fdf01331000 r-xp 00000000 fd:00 201352308 > /usr/lib64/libm-2.17.so > 7fdf01331000-7fdf01531000 ---p 00100000 fd:00 201352308 > /usr/lib64/libm-2.17.so > 7fdf01531000-7fdf01532000 r--p 00100000 fd:00 201352308 > /usr/lib64/libm-2.17.so > 7fdf01532000-7fdf01533000 rw-p 00101000 fd:00 201352308 > /usr/lib64/libm-2.17.so > 7fdf01533000-7fdf0161c000 r-xp 00000000 fd:00 201352677 > /usr/lib64/libstdc++.so.6.0.19 > 7fdf0161c000-7fdf0181c000 ---p 000e9000 fd:00 201352677 > /usr/lib64/libstdc++.so.6.0.19 > 7fdf0181c000-7fdf01825000 r--p 000e9000 fd:00 201352677 > /usr/lib64/libstdc++.so.6.0.19 > 7fdf01825000-7fdf01827000 rw-p 000f2000 fd:00 201352677 > /usr/lib64/libstdc++.so.6.0.19 > 7fdf01827000-7fdf0183c000 rw-p 00000000 00:00 0 > 7fdf0183c000-7fdf01843000 r-xp 00000000 fd:00 201352334 > /usr/lib64/librt-2.17.so > 7fdf01843000-7fdf01a42000 ---p 00007000 fd:00 201352334 > /usr/lib64/librt-2.17.so > 7fdf01a42000-7fdf01a43000 r--p 00006000 fd:00 201352334 > /usr/lib64/librt-2.17.so > 7fdf01a43000-7fdf01a44000 rw-p 00007000 fd:00 201352334 > /usr/lib64/librt-2.17.so > 7fdf01a44000-7fdf01a48000 r-xp 00000000 fd:00 201352297 > /usr/lib64/libSegFault.so > 7fdf01a48000-7fdf01c47000 ---p 00004000 fd:00 201352297 > /usr/lib64/libSegFault.so > 7fdf01c47000-7fdf01c48000 r--p 00003000 fd:00 201352297 > /usr/lib64/libSegFault.so > 7fdf01c48000-7fdf01c49000 rw-p 00004000 fd:00 201352297 > /usr/lib64/libSegFault.so > 7fdf01c49000-7fdf01c86000 r-xp 00000000 fd:02 16708066 > /home/liuyan/new_moses/opt/lib/libboost_serialization.so.1.64.0 > 7fdf01c86000-7fdf01e85000 ---p 0003d000 fd:02 16708066 > /home/liuyan/new_moses/opt/lib/libboost_serialization.so.1.64.0 > 7fdf01e85000-7fdf01e88000 r--p 0003c000 fd:02 16708066 > /home/liuyan/new_moses/opt/lib/libboost_serialization.so.1.64.0 > 7fdf01e88000-7fdf01e89000 rw-p 0003f000 fd:02 16708066 > /home/liuyan/new_moses/opt/lib/libboost_serialization.so.1.64.0 > 7fdf01e89000-7fdf01eaf000 r-xp 00000000 fd:00 203961964 > /usr/lib64/libtcmalloc_minimal.so.4.2.6 > 7fdf01eaf000-7fdf020af000 ---p 00026000 fd:00 203961964 > /usr/lib64/libtcmalloc_minimal.so.4.2.6 > 7fdf020af000-7fdf020b0000 r--p 00026000 fd:00 203961964 > /usr/lib64/libtcmalloc_minimal.so.4.2.6 > 7fdf020b0000-7fdf020b1000 rw-p 00027000 fd:00 203961964 > /usr/lib64/libtcmalloc_minimal.so.4.2.6 > 7fdf020b1000-7fdf020d7000 rw-p 00000000 00:00 0 > 7fdf020d7000-7fdf020f1000 r-xp 00000000 fd:02 16796028 > /home/liuyan/new_moses/opt/lib/libxmlrpc.so.3.39 > 7fdf020f1000-7fdf022f0000 ---p 0001a000 fd:02 16796028 > /home/liuyan/new_moses/opt/lib/libxmlrpc.so.3.39 > 7fdf022f0000-7fdf022f1000 r--p 00019000 fd:02 16796028 > /home/liuyan/new_moses/opt/lib/libxmlrpc.so.3.39 > 7fdf022f1000-7fdf022f2000 rw-p 0001a000 fd:02 16796028 > /home/liuyan/new_moses/opt/lib/libxmlrpc.so.3.39 > 7fdf022f2000-7fdf02312000 r-xp 00000000 fd:02 16796055 > /home/liuyan/new_moses/opt/lib/libxmlrpc++.so.8.39 > 7fdf02312000-7fdf02511000 ---p 00020000 fd:02 16796055 > /home/liuyan/new_moses/opt/lib/libxmlrpc++.so.8.39 > 7fdf02511000-7fdf02512000 r--p 0001f000 fd:02 16796055 > /home/liuyan/new_moses/opt/lib/libxmlrpc++.so.8.39 > 7fdf02512000-7fdf02513000 rw-p 00020000 fd:02 16796055 > /home/liuyan/new_moses/opt/lib/libxmlrpc++.so.8.39 > 7fdf02513000-7fdf0252a000 r-xp 00000000 fd:02 16796008 > /home/liuyan/new_moses/opt/lib/libxmlrpc_abyss.so.3.39 > 7fdf0252a000-7fdf02729000 ---p 00017000 fd:02 16796008 > /home/liuyan/new_moses/opt/lib/libxmlrpc_abyss.so.3.39 > 7fdf02729000-7fdf0272a000 r--p 00016000 fd:02 16796008 > /home/liuyan/new_moses/opt/lib/libxmlrpc_abyss.so.3.39 > 7fdf0272a000-7fdf0272b000 rw-p 00017000 fd:02 16796008 > /home/liuyan/new_moses/opt/lib/libxmlrpc_abyss.so.3.39 > 7fdf0272b000-7fdf02730000 r-xp 00000000 fd:02 16796032 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server.so.3.39 > 7fdf02730000-7fdf02930000 ---p 00005000 fd:02 16796032 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server.so.3.39 > 7fdf02930000-7fdf02931000 r--p 00005000 fd:02 16796032 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server.so.3.39 > 7fdf02931000-7fdf02932000 rw-p 00006000 fd:02 16796032 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server.so.3.39 > 7fdf02932000-7fdf0293c000 r-xp 00000000 fd:02 16796058 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server++.so.8.39 > 7fdf0293c000-7fdf02b3c000 ---p 0000a000 fd:02 16796058 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server++.so.8.39 > 7fdf02b3c000-7fdf02b3d000 r--p 0000a000 fd:02 16796058 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server++.so.8.39 > 7fdf02b3d000-7fdf02b3e000 rw-p 0000b000 fd:02 16796058 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server++.so.8.39 > 7fdf02b3e000-7fdf02b44000 r-xp 00000000 fd:02 16796035 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server_abyss.so.3.39 > 7fdf02b44000-7fdf02d43000 ---p 00006000 fd:02 16796035 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server_abyss.so.3.39 > 7fdf02d43000-7fdf02d44000 r--p 00005000 fd:02 16796035 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server_abyss.so.3.39 > 7fdf02d44000-7fdf02d45000 rw-p 00006000 fd:02 16796035 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server_abyss.so.3.39 > 7fdf02d45000-7fdf02d4d000 r-xp 00000000 fd:02 16796070 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server_abyss++.so.8.39 > 7fdf02d4d000-7fdf02f4c000 ---p 00008000 fd:02 16796070 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server_abyss++.so.8.39 > 7fdf02f4c000-7fdf02f4d000 r--p 00007000 fd:02 16796070 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server_abyss++.so.8.39 > 7fdf02f4d000-7fdf02f4e000 rw-p 00008000 fd:02 16796070 > /home/liuyan/new_moses/opt/lib/libxmlrpc_server_abyss++.so.8.39 > 7fdf02f4e000-7fdf02f53000 r-xp 00000000 fd:02 16796000 > /home/liuyan/new_moses/opt/lib/libxmlrpc_util.so.3.39 > 7fdf02f53000-7fdf03152000 ---p 00005000 fd:02 16796000 > /home/liuyan/new_moses/opt/lib/libxmlrpc_util.so.3.39 > 7fdf03152000-7fdf03153000 r--p 00004000 fd:02 16796000 > /home/liuyan/new_moses/opt/lib/libxmlrpc_util.so.3.39 > 7fdf03153000-7fdf03154000 rw-p 00005000 fd:02 16796000 > /home/liuyan/new_moses/opt/lib/libxmlrpc_util.so.3.39 > 7fdf03154000-7fdf0315d000 r-xp 00000000 fd:02 16796004 > /home/liuyan/new_moses/opt/lib/libxmlrpc_util++.so.8.39 > 7fdf0315d000-7fdf0335c000 ---p 00009000 fd:02 16796004 > /home/liuyan/new_moses/opt/lib/libxmlrpc_util++.so.8.39 > 7fdf0335c000-7fdf0335d000 r--p 00008000 fd:02 16796004 > /home/liuyan/new_moses/opt/lib/libxmlrpc_util++.so.8.39 > 7fdf0335d000-7fdf0335e000 rw-p 00009000 fd:02 16796004 > /home/liuyan/new_moses/opt/lib/libxmlrpc_util++.so.8.39 > 7fdf0335e000-7fdf0336e000 r-xp 00000000 fd:02 16796016 > /home/liuyan/new_moses/opt/lib/libxmlrpc_xmlparse.so.3.39 > 7fdf0336e000-7fdf0356d000 ---p 00010000 fd:02 16796016 > /home/liuyan/new_moses/opt/lib/libxmlrpc_xmlparse.so.3.39 > 7fdf0356d000-7fdf0356e000 r--p 0000f000 fd:02 16796016 > /home/liuyan/new_moses/opt/lib/libxmlrpc_xmlparse.so.3.39 > 7fdf0356e000-7fdf0356f000 rw-p 00010000 fd:02 16796016 > /home/liuyan/new_moses/opt/lib/libxmlrpc_xmlparse.so.3.39 > 7fdf0356f000-7fdf03589000 r-xp 00000000 fd:02 16796020 > /home/liuyan/new_moses/opt/lib/libxmlrpc_xmltok.so.3.39 > 7fdf03589000-7fdf03789000 ---p 0001a000 fd:02 16796020 > /home/liuyan/new_moses/opt/lib/libxmlrpc_xmltok.so.3.39 > 7fdf03789000-7fdf0378b000 r--p 0001a000 fd:02 16796020 > /home/liuyan/new_moses/opt/lib/libxmlrpc_xmltok.so.3.39 > 7fdf0378b000-7fdf0378c000 rw-p 0001c000 fd:02 16796020 > /home/liuyan/new_moses/opt/lib/libxmlrpc_xmltok.so.3.39 > 7fdf0378c000-7fdf0379b000 r-xp 00000000 fd:00 201423374 > /usr/lib64/libbz2.so.1.0.6 > 7fdf0379b000-7fdf0399a000 ---p 0000f000 fd:00 201423374 > /usr/lib64/libbz2.so.1.0.6 > 7fdf0399a000-7fdf0399b000 r--p 0000e000 fd:00 201423374 > /usr/lib64/libbz2.so.1.0.6 > 7fdf0399b000-7fdf0399c000 rw-p 0000f000 fd:00 201423374 > /usr/lib64/libbz2.so.1.0.6 > 7fdf0399c000-7fdf039c1000 r-xp 00000000 fd:00 201357033 > /usr/lib64/liblzma.so.5.2.2 > 7fdf039c1000-7fdf03bc0000 ---p 00025000 fd:00 201357033 > /usr/lib64/liblzma.so.5.2.2 > 7fdf03bc0000-7fdf03bc1000 r--p 00024000 fd:00 201357033 > /usr/lib64/liblzma.so.5.2.2 > 7fdf03bc1000-7fdf03bc2000 rw-p 00025000 fd:00 201357033 > /usr/lib64/liblzma.so.5.2.2 > 7fdf03bc2000-7fdf03be2000 r-xp 00000000 fd:00 201352293 > /usr/lib64/ld-2.17.so > 7fdf03da7000-7fdf03db4000 rw-p 00000000 00:00 0 > 7fdf03de0000-7fdf03de1000 rw-p 00000000 00:00 0 > 7fdf03de1000-7fdf03de2000 r--p 0001f000 fd:00 201352293 > /usr/lib64/ld-2.17.so > 7fdf03de2000-7fdf03de3000 rw-p 00020000 fd:00 201352293 > /usr/lib64/ld-2.17.so > 7fdf03de3000-7fdf03de4000 rw-p 00000000 00:00 0 > 7ffde230e000-7ffde232f000 rw-p 00000000 00:00 0 > [stack] > 7ffde23a1000-7ffde23a3000 r-xp 00000000 00:00 0 > [vdso] > ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 > [vsyscall] > Segmentation fault (core dumped) > > 153M -rw-rw-r--. 1 liuyan liuyan 153M May 26 23:40 > aligned.grow-diag-final-and > 548M -rw-rw-r--. 1 liuyan liuyan 548M May 27 00:30 extract.inv.sorted.gz > 482M -rw-rw-r--. 1 liuyan liuyan 482M May 27 00:29 extract.o.sorted.gz > 547M -rw-rw-r--. 1 liuyan liuyan 547M May 27 00:30 extract.sorted.gz > 101M -rw-rw-r--. 1 liuyan liuyan 101M May 26 23:44 lex.e2f > 101M -rw-rw-r--. 1 liuyan liuyan 101M May 26 23:44 lex.f2e > 4.0K -rw-rw-r--. 1 liuyan liuyan 1.2K May 27 09:05 moses.ini > 1.2G -rw-rw-r--. 1 liuyan liuyan 1.2G May 27 01:15 phrase-table.gz > 495M -rw-rw-r--. 1 liuyan liuyan 495M May 27 01:25 > reordering-table.wbe-msd-bidirectional-fe.gz > > > > > _______________________________________________ > Moses-support mailing list > Moses-support@mit.edu > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
